Update: I did clean the code a bit

What version of EmulationStation are you using ?

@Nathan1031982 Skyscraper (and perhaps Sselph's Scraper too, but I haven't used it) will write gamelists in alphabetical order when it processes them.

After you play a game, ES updates the last-played and play count data. It removes the old enty with the old data, and writes the new entry with updated data at the end of the file.

You can re-write your gamelist with Skyscraper if you want an alphabetized snapshot of it to use for another project but you can't make it stay alphabetized if you play any games.

@Folly said in Moving art/video-snaps/gamelists/and-other-settings to USB Drive ALONG with ROMs?:

you also have to copy the media and change the locations in the gamelist.xml otherwise your media isn't found.
You can simply do that with find and replace in a text editor.

Thank you for that information, but TBH the entire point of me making this post is basically to NOT have to manually edit .XML files (and like 30+ of them for all the systems I run). Just trying to see how in RetroPie you can turn on the feature to have the gamelist.xml's and the media folder w/ artwork switched over to the individual rom folders. Skyscraper seems to do this, but that process can take days or WEEKS even (when it's acting up and timing out 99.99% of the time) – so it is possible, just don't know how...

@cdaters said in Moving art/video-snaps/gamelists/and-other-settings to USB Drive ALONG with ROMs?:

What you are proposing is very cool, you are looking for a portable RetroPie set-up much in the same vein as a Portable Apps USB flashdrive. I don't know if this is realistic in the sense that your emulators on each Pi that you run that USB drive on would have to be configured the same way. This includes your controller configs, etc.

Right now I'll be happy if I can get my first step figured out: how to get my games + scraped art onto the USB drive, and have a fresh system read from the USB for those two things (nobody yet has been able to tell me a good way to point RetroPie to the USB drive for those things...).

My premise to do this is simple: scraping is STILL a nightmare, ~SIX years later (since my 1st build). Granted these last two spats have been the best ever, but with a six year history of it timing out forever (even with extra threads on Screenscraper.fr), and taking me literally, no joke, days to even WEEKS to scrape my 30+ systems properly, I have very high doubts it'll continue to be good.

So, when I can get an entire Pi setup, ALL my customizations, all roms working, everything – in a couple hours, minus artwork... but then it can STILL take anywhere from like 6-12ish hours (best case scenario), to DAYS to WEEKS to scrape the art for 30+ systems. This is why I want a USB drive with all games, then in each individual rom directory have the gamelist.xml + 'media' folder with all the art/videos so I never, ever, have to deal with a scraping nightmare ever again.

...I'll deal with the settings later (which BTW, the gamelists have NOTHING to do with any of my other tweaks - they literally just say the game name + where to find the art/videos-snaps... so they work on ANY emulator on ANY model of Pi!).

@winterelf 3rd party builds are not supported by this forum.

If this were a build direct from this forum, the above fix would have worked. The file extension list in the es_system.cfg is the only change you need to make to allow a file to be shown in the game list of each system. It will just show without meta data until it is scrapped. This is also how it shows any images or videos. Once ES reads that file extension in that list, it knows what to do.

So who knows why your build is not doing that. You'll have to ask the team that created it.

@Folly said in Creating ES gamelists with media for MAME/lr-mess:

Screenscraper doesn't add a picture to it.

It works with screenscraper. You have to add a system and customize the rom path in SKRAPER

@sleve_mcdichael Exactly, I couldn't override any rules that screenscraper may set in terms of limits or others in their APIs

@mitu

Wow! At some point I did disable the save on exit setting. Thank you.

@waitingmoon I've adapted the changes submitted by @Zigurana and eagle0wl and submitted a new PR, which was accepted recently.

If you're using RetroPie, you can install the emulationstation-dev package (from the experimental packages section) and try it out.

@mitu said in EmulationStation Marquee Bug (with work-around):

EDIT: Try the Magazine Madness theme, it has support for marquees in Detailed view.

You're correct @mitu. Just tried in both Detailed and Automatic views and without videos referenced, the marquees do indeed appear.

For each theme I used that exhibits the problem, the marquee section is under <view name = "video">. Updating the themes, including Carbon, by moving the marquee section under <view name = "Detailed"> works (and would seem to be a more common scenario).

Seems that many themes used Carbon as a baseline and have marquee under the video view section. Easy to fix (or change, if it was what the author intended).

Thank you, as always.

Please add some info about your setup, as detailed in https://retropie.org.uk/forum/topic/3/read-this-first.

Did you run a scrape before trying to generate the gamelist ? Do you have any amiga data (art/text) in your cache ?

@johnodon PR created: https://github.com/RetroPie/EmulationStation/pull/767

johnodon created this issue in RetroPie/EmulationStation closed Update FileSystemUtil.cpp - Allow for multilevel absolute paths in gamelist.xml #767

Most likely the .xml you created is not correct. If you look in the es_log.txt logfile (you can find it in \\retropie\configs\all\emulationstation) you'll see any errors from parsing the gamelist.xml (careful, not gamelink.xml).

Post the gamelist.xml on pastebin.com to take a look.

@EctoOne Oh, that's a real bummer. I thought the package with the "-Linux-64Bit.tar.gz" in the filename from the releases page, which is build with PyInstaller would solve issues like that in the future. I get a fontconfig error with it on my current system and it can be a local issue on my machine. Not very encouraging, but maybe you could try to run that.

But well, it contains binary files, so from security standpoint I can understand when you don't even want try that.

@muldjord
Thank you for your answer. And congratulations on your fantastic work on the Skyscraper, the best multi-platform scraping tool, in my opinion.

Actually, I'm not very comfortable with the process of adding a .m3u file to each of my 2500+ individual Amstrad CPC games (or to every game in my other retro-collections).

And, @mitu, I'm not comfortable neither with the idea of adding a non-standard extension to my games, just to have ES showing the .m3u playlists only. I understand that ".cd1", ".cd2", etc. are like a standard now in PSX multi-disc games, but I suspect a home-made ".dsm", ".disk1" or whatever I could invent in my room for Amstrad CPC/MSX/other Libretro would drive me to a lot of issues when I update/scrape/configure something in the future. The same applies to hidden folders: if I want to update the scraping info, I would have to rename the folder before, re-hide the folder after... Even the idea to manage ".disk1" or ".disk2" sounds laborious when you have hundreds of folders, thousands of files...

But, I still think one scrapped entry in ES for a multi-disc game sounds like the most elegant solution.

So, I share here the solution I took:
1- As mentioned before, I create a .m3u playlist for each multi-disc game and put all the files, including the .m3u, inside a folder with the same basename (I do this with the script in my first post)

2- As @mitu told me, I edit the Skyscraper's config.ini to add the .m3u extension at the platform I want to scrap.

3- I scrap everything. So regular files, regular files in subfolders, and .m3u files get an image+description in gamelist.xml.

4- I duplicate the information related to the .m3u files in the gamelist.xml to create new <folder> sections with the same scraped information. Here is the command lines I used to do this:

# extract the XML sections about .m3u files to a new tmp.xml file perl -ne 'BEGIN{$/="</game>\n";} print m|(<game>.*m3u.*$/)|ms' gamelist.xml >> tmp.xml # change any <game> tag to a <folder> tag sed -i .bak "s/<game>/<folder>/g" tmp.xml # change any </game> tag to a </folder> tag sed -i .bak "s/<\/game>/<\/folder>/g" tmp.xml # crop any .m3u filename to keep the path only sed -i .bak 's/\/[[:alnum:][:blank:]\.\+\&,\(\);_-]*.m3u//g' tmp.xml # delete the closing tag at the end of gamelist.xml sed -i .bak '/^<\/gameList>/d' gamelist.xml # add the content of tmp.xml to the end of gamelist.xml cat tmp.xml >> gamelist.xml # add the closing tag at the end of gamelist.xml echo "</gameList>" >> gamelist.xml

And I must say, I'm quite happy with the result : now I have in ES only one scraped entry for any mono-disc game as I have one scraped entry for any subfolder (multi-disc game). When I select a game that is a multi-disc game, It opens the sublist - that is also scraped with the same information - and I just have to select the first entry to run the game.

I hope all this sounds useful.

EmulationStation will use the gamelist.xml ROMs folder, if found, otherwise it looks in /opt/retropie/configs/all/emulationstation/gamelists/<system>. The 2 files are not combined, just one of them is used, in that order.

If you enabled the 'Use Local Art' option in EmulationStation's settings, it will try find a video matching the game by looking under roms/<system>/media/<ROM_NAME>-video.mp4, so even if there's no video declared in gamelist.xml, it might still find a video associated with it.

In fact, what would be the best way to ensure that I get both videos and correct information into my setup?

Don't mix the scrapers if you want video - the internal scraper doesn't support videos so it might overwrite the video tags if you re-scrape with it. Use the external scraper and see which videos have issues - some arcade videos are incompatible with RPI's hardware video codecs and might not play correctly.

@mitu

Thanks Mitu. I ended up using locate to find any corresponding gamelist.xml for the Naomi folder(s) and wiped them, then I just started from scratch, adding one game at a time and it looks like it finally took the information. I must have missed something in there, but just copy/pasting by hand finally nixed the issue.

It was so weird. Before that it was even naming stuff totally wrong. Like I had MegaMan 3 for NES Playchoice listed in the Naomi folder. And a LOT of the games were being listed as Power Instinct, which is not on the Naomi. No idea why it was generating games that weren't in the folder at all.

As @YFZdude mentioned, the genesis folder is a symlink (shortcut) to the megadrive folder, so there's no need to remove it - it's not actually taking any space. You can leave it as is, it's ok - the scriptmodules expect a megadrive ROM folder and updating the MD emulators will re-create the folder.