How to generate gamelist.xml

Dal1980

Hi guys

I'm not new to RetroPie but I am new to ES as I have always just switched to Attract Mode for the front end.

So since I'm a novice user of ES I just wanted to ask a couple of really simple questions that don't seem to be so obvious to me.

I've added a bunch of ROMs into a couple of the preset folders for gb, nes and snes

I've also researched where I need to place my box art "/opt/retropie/configs/all/emulationstation/downloaded_images/nes" for nes for instance.

Nothing showed up after restarting and I thought it was perhaps a ".png" issue. I couldn't find anywhere for setting this to png instead of jpg but I seem to have found the root of the problem. My "/opt/retropie/configs/all/emulationstation/gamelists" directory was empty. I've created the "gb", "nes" and "snes" directories inside here but there isn't any option I can find to get ES to generate a gamelist.xml for each collection.

Baring in mind this is a retropie build I can't run any windows tools to generate this file, secondly, since this is part of the core functionality why isn't there an option to do this build right into the menu?

Many thanks

mitu

@dal1980 I'm sorry, but I don't understand what you're trying to do. When you say:

Nothing showed up after restarting...

what are you referring to ? The artwork didn't show up or the list of ROMs in Emulationstation ?
Emulationstation will scan the systems' folder and generate the gamelist.xml file for each system, you don't need to create the folders or the gamelist file manually.

Dal1980

Hi mitu, thanks for the reply

That's strange.

Ok, let me try explain/confirm some things

For NES as an example
I placed the roms into "/home/pi/RetroPie/roms/nes"

rebooted

Nintendo Entertainment System then showed on the main wheel/menu and inside were the roms. These also started and where playable.

I then tried to add my own personal box art files to "/opt/retropie/configs/all/emulationstation/downloaded_images/nes"

and rebooted (probably didn't need to but I thought I'd better)

Then the problem described above regarding trying to track down why no box art was being shown. There are no gamelist.xml's so I created "/opt/retropie/configs/all/emulationstation/gamelists/nes" folder then couldn't figure out how to get ES to generate the gamelist.xml file inside

Does that help explain things?

Thanks

mitu

@dal1980 said in How to generate gamelist.xml:

Does that help explain things?

Yes. I don't think ES automatically detects metadata (artwork), but the gamelist.xml should be generated when exiting/restart ES - after the ROMs are detected - so you shouldn't create it manually.
How are your images names - do they have the same name (without the extension) as the ROMs ?

Dal1980

Yes

example:
"/home/pi/RetroPie/roms/nes/10-Yard Fight (USA, Europe).zip"
"/opt/retropie/configs/all/emulationstation/downloaded_images/nes/10-Yard Fight (USA, Europe).png"

mitu

@dal1980 Try adding an -image after the name and before the extension, i.e.
ROM - /home/pi/RetroPie/roms/nes/10-Yard Fight (USA, Europe).zip
Image - /opt/retropie/configs/all/emulationstation/downloaded_images/nes/10-Yard Fight (USA, Europe)-image.png
and then restart ES.

If it doesn't work, you can modify the gamelist.xml and add the paths for <image> for each game (make sure ES is stopped when doing those modifications).

Dal1980

adding -image "10-Yard Fight (USA, Europe)-image.png" didn't work

my gamelist.xml contains only 1 entry from when I tested that a rom loads and works

<?xml version="1.0"?>
<gameList>
<game>
<path>./Flying Warriors (USA).zip</path>
<name>Flying Warriors (USA)</name>
<playcount>1</playcount>
<lastplayed>20180405T095850</lastplayed>
</game>
</gameList>

You want me to add all these entries by hand?

mitu

@dal1980 said in How to generate gamelist.xml:

You want me to add all these entries by hand?

It's up to you, you can test with one image and if it works, you could automate this if you want to have the artwork appearing.

Dal1980

Thanks mitu

Is there any better scrappers other than gamesdb. I have found issues with this not grabbing the correct details. If I've at least scrapped to populate the xml file I can begin editing the stupid thing.

AM has an option to just produce the equivalent of ES gamelist.xml through the main menu, shame ES hadn't thought of this.

Many thanks for sticking with me through this issue.

mitu

@dal1980 said in How to generate gamelist.xml:

AM has an option to just produce the equivalent of ES gamelist.xml through the main menu, shame ES hadn't thought of this.

ES does produce the gamelist.xml file, what it does not do is automatic scraping and artwork retrieval until you use its built-in scraper.
For alternate scrapers, try Skyscraper, Universal XML Scraper, Skraper or Steven Selph's scraper.

ChuckyP

If you have the artwork, but you don’t have the “image” tag to give you the artwork location in your gamelist then you need to add that in. Scraping will add this value but since you already have the image files try “Gengal”.

It is a fairly simple utility that will add the proper “image” or “video” tags. You should be able to work through the process once you launch it.

akafox

@dal1980 this has nothing to do with your xml file at all. It has everything to do with the naming convention of your snaps.

you MUST have the "tag" on the end of your artwork...thus

name-marquee.png/.jpg
name-video.mp4
name-flyer.png./jpg

ect ect...

I ran into the same problem...I batched renamed the artwork with the right "tag".
note that your artwork files can be ANYWHERE on the system...you just have to tell es where it is at....the default you are correct on...

the other "default" is the rom folder then in the "images" folder. Example:

/home/pi/roms/nes/images
or
./images in the xml file because your gamelist.xml is inside the rom folder.

you CAN change this info and make es see it without the "tags" AFTER you make your list..then you can switch between ES and AM (AM does not understand the "tags" on the images so they do not show up in AM.)

easy fix..re-scrape all your artwork using stelph's scrapper... or
use a batch re-namer.

Yes this works and that is your problem most likely. I have custom made and or edited most of my lists..this how i know this.

Note here that "-flyer" means arcade flyer...however for consoles the "-flyer" is the boxart.

EctoOne

@akafox but doesn't the xml needs to have the path to the image as well with the <image> tag or was it added that ES can find images on its own now?

I would go with @ChuckyP to use Gengal, I used that when video view was added to update my gamelists with marquees.

Dal1980

Thanks guys

I took about 10 hours to finish scraping for the 3 systems. I have better information & graphic stored locally but it seems from what is explained ES has it's own convention to everything.

Images should really be stored in folders that describe the artwork as adding "tags" to filenames has got to be the dumbest way. I've never seen this being used for any other frontends.

If it took 10 hours for 3 systems then thank god I only have approximately 160 hours to complete all 50 collections... (not that I am using 50 but you get my point).

I spotted Gengal on my journey here to the forum, the two things that put me off using that is firstly the Raspberry Pi doesn't run Windows :P and so I'd need to setup my collection on my PC first. Secondly, I read that it's old (WindowsXP era) and not been touched in years and some people were having some issues... not to mention I need to install some more crap on my system to get it working.

I'm working on a theme which was the reason why I'm hanging in ES. Apart from the obviously poor asset management, I don't think it is a bad system. In-fact I thought it was the cleanest one around when I first ran into it back in 2014-2015ish when I was on the hunt for a good frontend.

I appreciate the help and I think once I get past the initial setup/scrapes I will save everything so I don't have to ever do this again. I see other people sharing their gamelist.xml files which makes perfect sense.

ChuckyP

Not trying to force you to use a utility you don’t want to. But Gengal saved me a ton of time adding in the Video tags (which are part of the XML data, not added to the file name).

You can copy the gamelist XML to a PC and run them through Gengal there and it is super fast. Use Find and Replace commands to change any of the locations of the files (I store mine in “downloaded_images” & “downloaded_videos” folders like you mention above.

EctoOne

I agree that generating the perfect gamelist.xml took me more time that I spend gaming, especially when you want all data (images/videos) but don't want to actually use them (I wanted the videos scraped but don't use them). It even was more work, because of all the additions (video support and fixing the thumbnail tag).
Then there is the issue that ES deletes stuff from the xml that it can't find, in my case it deletes the video tag because i don't have them on my pi. Setting ES to parse gamelists only fixes that though but might create issues that I have not noticed yet.

One other problem I had before <thumbnail> was fixed, was that I wanted to use screenshots instead of marquees and that I had either to replace all links in the gamelist or rename all files. What I went with was, that I separated all media into it's own subfolders and removed the suffixes from the names. Now all my media has the same name as the Rom and if I ever want to change what image i want to use for a certain element, I can simply rename the folders. Still not a perfect solution but better then messing around with the gamelist or by renaming a lot of files.

I'm not sure if you want to use a different frontend but if not, I also suggest to keep your gamelist and media in your rom folder. I scraped/edited all my data on a windows pc and use relative paths in my gamelists. That let's me keep an easy backup of all data which can be easily be transferred to my Pi if I ever need to do a clean install again. I don't have to move the media files to the default path and have direct access to the folders with Samba and/or without the need for root. It also makes adding games easier, just scrape the data on the PC for the new games only, and copy the new data into the old gamelist and move the media files into the various folders.

Riverstorm

If it's helpful. In the RetoPie Docs - Take and Scrape Your Own Screenshots they have a few ways to use your own screenshots while generating a gamelist.xml or create new screenshots and generate one.

Used2BeRX

@dal1980 you may want to look at the script that's been worked on for the last 6 or so months if you've got your own stuff you'd rather use than what's on another database.

https://retropie.org.uk/forum/topic/10406/making-premade-gamelist-xml-with-xtra-media-collections/

Since NES seems to be your thing, you might also be interested in the NES project that I've been working on. It won't be done any time soon, but it will be the best NES setup out there with nothing coming remotely close to it when it is.

https://retropie.org.uk/forum/topic/14261/nes-ressurctionxtras-2-0-project-thread-previously-titled-does-anybody-work-on-emulators-to-fix-broken-games-here/4

Clyde

@dal1980 said in How to generate gamelist.xml:

Images should really be stored in folders that describe the artwork as adding "tags" to filenames has got to be the dumbest way. I've never seen this being used for any other frontends.

Sselph's Scraper supports this via command line arguments. See Sselphs Scraper Advanced Configuration or type /opt/retropie/supplementary/scraper/scraper --help in Retropie's command console for a list of options. With them, you can set alternative paths or suffixes for the various media files.