@goestohollywood Don't worry about the order in the file: ES is always shuffling that around anyway if you have 'save metadata' enabled. The list SHOULD be in alphabetical order inside ES either way (or else it's conforming to any <sortname> tags you might have).
You might consider using <sortname/>  in the gamelist instead of modifying the display name of the ROM upon preprocessing.
What duration adds the preprocessing to the ES start?
Do you rewrite every extra information to the gamelist every time (video, thumbnail, ... and hiscores)? I would assume only rewriting the hiscores would be sufficient as they change more often than the "static" scraped data.
Last but not least: What are your plans to publish the sources?
Are you running SS through the setup script or from the command-line? If script, do you have generate options >> remove bracket info (enabled)? (This is global, but maybe if 32x is the only system you've scraped since it changed or something.)
Beyond that, the contents of your /home/pi/.skyscraper/config.ini file might shed some light on what's happening. You can post it here if you can't make sense of it.
Use code blocks (start and end
with three backticks on a new
line) so special symbols aren't
interpreted as formatting.
Or upload it to (edit:spelling)
pastebin.com if it's very long.
Yeah having games inside subfolders should be no problem, as long as their path falls inside the "system path" (~/RetroPie/roms/snes, etc.) defined in es_systems.cfg. If you try to point a gamelist for a system at a rom file outside of this system path, ES won't like it and will skip that entry.
EmulationStation will use the gamelist.xml ROMs folder, if found, otherwise it looks in /opt/retropie/configs/all/emulationstation/gamelists/<system>. The 2 files are not combined, just one of them is used, in that order.
If you enabled the 'Use Local Art' option in EmulationStation's settings, it will try find a video matching the game by looking under roms/<system>/media/<ROM_NAME>-video.mp4, so even if there's no video declared in gamelist.xml, it might still find a video associated with it.
In fact, what would be the best way to ensure that I get both videos and correct information into my setup?
Don't mix the scrapers if you want video - the internal scraper doesn't support videos so it might overwrite the video tags if you re-scrape with it. Use the external scraper and see which videos have issues - some arcade videos are incompatible with RPI's hardware video codecs and might not play correctly.