This looks like the Mame2016 core, you might have enabled one of core options that start the MAME OSD menu.