Well, the OP should just move to MAME current tbh, not only it has support for TATE in both directions, but it probably also has better support for cocktail mode (i highly doubt all cocktail fixes from the past 22 years were backported to MAME2003+).

As for FBNeo, it is not suitable at all for cocktail setups : some games had their 2P mode and cocktail mode tied together, and ended up being un-cocktail-ified so that 2 players could play them without requiring a cocktail setup.

Tate mode doesn't auto-rotate. TATE mode adjusts the screen to fill your monitor. What you need to do is is specify the exact rotation you want for each game you need to and make a game-specific config file for it. If you have auto-rotation on and auto-rotate on, you will get some weird results. Auto-rotation should be turned off when using TATE mode (setting allow_video_rotate = false). Some games will need 180 degree rotation. You'll have to decide which end of your monitor you will consider your base (the one that will be the bottom edge when you spin it).

For each game you want to have in cocktail mode, you will likely need a cfg file with the setting
video_rotation = 2
for games that need to be flipped. Set it to 0 (or don't write a cfg) for games that sit the right way from the get go.

This post was greatly helpful for me when I was doing my cocktail setup: https://retropie.org.uk/forum/post/39024

@jdrassa thanks, I was looking for screen offset also. Perfect

@langest said in Possible to rotate screen per rom file?:

Now I am only wondering, where should I put the rom specific file?

As the posts above said - and it's also mentioned in the Docs - , next to the ROM file.

bump, sorry