Unable to get scraper to update gamelist.xml

jvetter

I have a simple use case for gamelist.xml that I cannot seem to get to work:

Adding new roms to an existing working system (e.g. ~/RetroPie/roms/psx)
Including all of the box art, videos, etc...
Trying to use scraper to update the gamelist.xml to include the new ROM and media.
Emulationstation will detect the new ROM on startup, but gamelist.xml never gets updated.

My setup:
Hardware: Raspberry Pi 4 Model B
Image: CRT Blast (from 2020). I know old, but I like the simplicity of it. I updated Debian and RetroPie to 4.8.9, removed some legacy source packages and installed scraper
Status: Everything boots and runs normal after the updates
ROMS Folder: Symlink from /home/pi/RetroPie/roms to /mnt/ssd1/RetroPie/roms (i.e. a USB SSD)

I have tried many combination of options, but what I settled on is the following:

pi@retropie:~/RetroPie/roms/psx $ /opt/retropie/supplementary/scraper/scraper -add_not_found=true -download_images=false -img_format=png -video_dir=snap -marquee_dir=wheel -image_dir=boxart

Part of the confusion for me is that the location of the media is different across images, for example:

Newer Images          My Image
------------------    --------
media/screenshots --> boxart
media/videos      --> snap
media/marquees    --> wheel

So trying to standardize on what my image has.

No matter what I do or what options I use, scraper will not add my new rom to the gamelist.xml. I have tried using add_not_found=[false|true], -append=[false|true], -download_images=[false|true]. Nothing works. Yes, I know I can edit it manually but that gets tedious when adding lots of ROMs.

If the gamelist.xml doesn't exist it says:

2025/01/08 03:07:52 ERR: Can't open gamelist.xml, creating new file. error "open gamelist.xml: no such file or directory"

If It exists it doesn't do anything at all. Sometimes it will touch the file but not add my rom.

If I use -append it will list the ones already there as skipped but still not add my new one. Most runs only print:

2025/01/08 03:12:32 arcade  srcs:
2025/01/08 03:12:32 console srcs: gdb
2025/01/08 03:12:32 INFO: Checking for new hash.csv.
2025/01/08 03:12:32 INFO: hash.csv "d9eedf6e622a8ab6a1a14486bb90ffb8" up to date.
No gamesdb api key provided, using default

My current default scraper settings are below, but I am using the commandline:

Other oddities, when I run /opt/retropie/supplementary/scraper/scraper -version it prints an empty line with no version.

What am I doing wrong?

mitu

@jvetter said in Unable to get scraper to update gamelist.xml:

Image: CRT Blast (from 2020). I know old, but I like the simplicity of it. I updated Debian and RetroPie to 4.8.9, removed some legacy source packages and installed scraper

We don't support 3rd party images. If you wish to get support here in the forums, please use the RetroPie official image.

jvetter

@mitu the majority of people using RetroPie are using it as the basis of their custom images and then adding on top of it. I am not asking for support of CRT Blast, but asking for insight into the tools that are integrated with retropie. Your readme asks for context so I provided it. Then using that context as the basis to reject requests for help is ridiculous.

Iff you dont't know why its behaving the way it is then just say so. I am just looking for some insight, Like:

What I am doing should work, or
My use case is not supported by scraper., or
It appears you have an old version, or
Try applying these changes to get more logs, or
Scraper doesnt support symlinks, or
Your using the wrong settings, or
Scraper skips xml that it doesn't know how to parse, or
*Scraper acts this way when the file permissions are wrong, or
Etc....literally anything that could explain the behavior other than just saying bugger off you useless newb.

Not asking you to solve all my problems. I can spend some time looking at the source, putting in bugs reports, or just writing my own tools, thats easy. But before I go down that route figured someone had seen this behavior before.

I have searched and everything I read indicates what I am doing should work. So please, ANY insight is would help.

Thanks!

mitu

@jvetter said in Unable to get scraper to update gamelist.xml:

. Then using that context as the basis to reject requests for help is ridiculous.

No, it's not. You may disagree, but your previous topic just showed that issues reported against images produced by 3rd party have modifications that we cannot support and reporting them here will not solve them. This topic is done.