I took @jonnykesh advice and did a fresh install to a new SD card - and everything works - pressing COIN and SELECT exits MAME2003 as expected.
The only thing I can think of which I did differently was this time I set up the hotkey as soon as I started up the new install - I think last time I left the hotkey unassigned, and then tried to assign it later. However I'd probably messed about with retroarch.cfg too much by then.
Thanks for the help.