Very special case. But you can save the import part. However, some manual steps are required:

Copy the pq1-amiga.svm to the ~/RetroPie/roms/amiga/ folder.

Find the Amiga video (I guess you already did that).

e.g, https://www.screenscraper.fr/gameinfos.php?gameid=34415&action=onglet&zone=gameinfosmedias

Navigate to the ROM/ISOs latch and pick a checksum and use it in the query parameter.

Run Skyscraper -p amiga -d /tmp --flags videos -s screenscraper --query="md5=2c4827513db0e45eed77d20d290fa17d" ~/RetroPie/roms/amiga/pq1-amiga.svm.

Find the video for "Police Quest" from screenscraper in the scummvm Skyscraper DB file, usually ~/.skyscraper/cache/scummvm/db.xml (type="video" source="screenscraper" with the matching resource id for Policy Quest). Note the resource id.

Copy the amiga video into the scummvm cache, e.g.:

cp /tmp/videos/screenscraper/494148112002ee474500cc10cbd212b1dedc5bd0.mp4 ~/.skyscraper/cache/scummvm/videos/<just-noted-resource-id>.mp4

It could be that the resource id is identical between Amiga and ScummVM, as the Skyscraper resource id is calculated from the filename and not from the file content for *.svm files. I did not verify this, but I assume it will be the case.

The single file launcher for both scummvm variants (native/libretro) can be achieved like this:

Make a folder with the ScummVM Game Id with the extension, thus sword1.svm in the roms/scummvm/ folder. Put the gamefiles into that folder, and add no *.svm in here Run scummvm from ES, select Add Game in ScummVM, select the folder sword1.svm you just created Remember / note the gameid provided by ScummVM autodetection (maybe sword1-win-gb, e.g containing some variants information)

Now, if the game id from autodetect is different than your created game id folder, there are two options to rectify the situation:

Either edit the scummvm.ini (in /opt/retropie/configs/scummvm) and change the ini section name from [sword1-win-gb] to [sword1]

Or rename the game folder from sword1.svm/ to sword1-win-gb/ in the filesystem.

You may encounter files (not directories) with the extension *.svm directly below the roms/scummvm/ folder. These are caused by the lines in +Start ScummVM.sh:

while read id desc; do echo "$desc" > "/home/pi/RetroPie/roms/scummvm/$id.svm" done < <(/opt/retropie/emulators/scummvm/bin/scummvm --list-targets | tail -n +3)

Disable/remove them to avoid having these re-appear after each ScummVM run.
Remove also the files *.svm with find $HOME/RetroPie/roms/scummvm/ -name "*.svm" -type f -exec rm -f {}

Important: Key is to have the game id folder name match the entry in the scummvm.ini for the native ScummVM the game id derived from the folder is passed to ScummVM.

In contrast lr-scummvm does not rely on *.svm files/folders at all. (Technical details: When launching lr-scummvm the absolute path of the game is passed to the libretro core. The core then searches for a *.scummvm (sic!) file with the game id in that path, if that is not found it falls back to autodetection.)

With this setup Skyscraper will detect and scrape the games properly.

Epilogue:
Also keep in mind that ScummVM and lr-scummvm are using scummvm.ini files in different locations.

ScummVM: /opt/retropie/configs/scummvm
lr-scummvm: $HOME/RetroPie/BIOS/scummvm

This setup works but may be a awkward (imposing file/foldernames, many manual steps in the right order, possibility of diversion of scummvm.ini files). Thus I came up with a more user-friendly approach, you may read the gory details in this thread

@sith-lord-goz You may try with Skyscraper 3.16 which adds --query="id=<IGDB-id>". However, the Game filename must have some similarity with the IGDB Game Title. Also the GameBase DB import from the SpeccyMania GameBase may come in handy.

"The real world is a special case."

FTR, the RetroPie version 4.8.6 onwards deploys the configurable platform information (peas.json). The upcoming Skyscraper release 3.14 will assess the RetroPie Setup version and will provide the hint on how to proceed if a older version of RetroPie Setup is found.

The scriptmodule is thought as a "ease of use" / for default setups, if you want to unleash the full potential of Skyscraper use the command line.

Within the scriptmodule you can only alter the (output) gamelist (-g) folder and media folder (-o).

With Use ROM folder (Enabled) (see generate options in the menu) the gamelist location is
/home/pi/RetroPie/roms/<platform> and /home/pi/RetroPie/roms/<platform>/media.

With Use ROM folder (Disabled) the gamelist location is
/home/pi/.emulationstation/gamelists/<platform> and /home/pi/.emulationstation/gamelists/<platform>/media.

The inputFolder is in any case picked up from the config.ini, however, the scriptmodule only uses $romdir which is currently taken from the default RetroPie value (which is /home/pi/RetroPie/roms). See _list_systems_skyscraper() in the scriptmodule skyscraper.sh.

TL;DR: Using a symlink as you did is the right way.

@sleve_mcdichael - ah thanks for the info! Last time I set up a pi system (3b) was about 6 years ago so I'm out of touch :)

@Lolonois - thanks for replying so quickly. I was reading the MobyGames section on https://gemba.github.io/skyscraper/SCRAPINGMODULES/ and it sounded like it could be something cool, but maybe i'm trying to solve a problem that doesn't exist. So far screenscraper, openretro, etc are covering things really well. I might look into it more and report back if there is anything worthwhile there. Cheers!

@Flexo55 Good you did find the cause. Please, as a courtesy to other users of this forum, do edit your first post and add to the subject something like [solved].

It’s working fine, thank you!

Thank you very much for the detail answer! i will check all options carefully.

Even if it has been around ever since, the scriptmodule opens a separate config path outside the config ini and it is not transparent to the user that some flags are hardwired in the scriptmodule.

I understand that this may cause some confusion, but as I said, I'm not aware of any reported issues related to this.

I ditched the initial idea to have exactly one config source thus, to have any config switch set in/get from the scriptmodule as it would be a royal annoyance to rewrite the evaluation/precedence logic of the config ini in bash.

I don't see why would be necessary. Ultimately the dialog based UI maintained in RetroPie is just a wrapper over the CLI options, meant to be used by users who don't need many options besides maybe turning on 'video' downloads. Avanced/experienced users are using the CLI to scrape and can (and do) ignore the UI. For the former users, the current .cfg is (ubenknownst to them) the config.ini, since they just use a couple of options to scrape and don't seek or need other parameters.

As a compromise I would see that the all applied switches from the scriptmodule are prompted to the user before he/she starts the scraping and/or have the skipped flag also configurable.

Yes, I can see this working - start anew without any .cfg so the user would always need to set the parameters and those parameters (including the skipped flag if we don't want to default to a safe value) are not persisted to any configuration. We can make also make it more clear that options set in the UI override the config.ini options.

Okay I think I sorted it out.

I edited my xml files to remove any absolute paths and replace them with the old /home/<user>/RetroPie/roms/... equivalent line.
Then I deleted the <folder> lines from the bottom of the file.

Everything works again.

Yes, you can modify the theme and remove it from the view. I don't think it has anything to do with Skyscraper.

@sleve_mcdichael said in Skyscraper - generate gamelist only once in the end?:

If you want to re-download data from a source again (or gather additional data such as marquees or videos that weren't enabled the first time), use --cache refresh.

Almost: --refresh / --cache refresh does update every enabled property (ie. cacheCovers=true and others in config.ini) of the scraped ROM(s).

By using only --flags video (or videos=true in config.ini) or setting in the ini: cacheMarquees=true (if it was false before) without --refresh will scrape the missing properties for the ROM(s), unless they are present in the cache for that scraper module.

Thank you both for catching and fixing this. It escaped me.

Thanks that you deployed a fix for this too 😃. I havent tested the fix but I will also mark this thread as solved 😊

@m33ts4k0z Thanks for reporting back. Please add to the subject [solved] or similar.

I just had a convo on discord with the screenscraper guys and it looks that version 3.9.1 is broken.
This command yields no results on 3.9.1, but it does work on 3.7.7

Skyscraper -p switch -s screenscraper --query "romnom=Super%20Mario%20Odyssey%20(World)%20(En,Ja,Fr,De,Es,It,Nl,Zh,Ru)%20(Rev%203).xci" /home/retropie/RetroPie/roms/switch/super-mario-odyssey.xci

[...]
My sd-card die recently. i transfered my metadata, download_images and gamelists folders, to a new setup and everything looks fine except that, if i run again skyscraper, it starts to download the pics that i have already download. My question: Is this a normal behavior or i missing something? Thanks!

Yes, it's normal - Skyscraper downloads files into it's cache first, then it copies them to the final location. If you haven't transferred your cache (from $HOME/.skyscraper/cache), then it's normal that the artwork/videos are re-downloaded.

@Lolonois said in Skyscraper is back! (v3.15):

no ETA for 3.16 we are talking at least about months.

Well, at least more than one month. :D But anyhow. 3.16 has just arrived.

With this release Skyscraper has gotten its 10th scraping module: This time for GameBase DB. Any GameBase DB is usually provided as Microsoft Access DB but Skyscraper provides a script to convert it to a more compliant format (Sqlite), which also saves you to install some MS blob DB drivers.

In general the other scrapers also did get some attention:

IGDB now scrapes Covers and Screenshots and supports querying for a id=... in the --query parameter, ArcadeDB does scrape HD-video for any title that provides it, various things put straight in Mobygames scraper, Import scraper with XML format now allows the closing XML element to be on a different line than the opening XML element (comes in handy for longer game descriptions).

The scraper documentation got also some (hopefully useful) overview tables.

See the details of changes with all links to the added and updated documentation.

Please update the RetroPie-Setup script before updating Skyscraper to have everything in place.

Happy scraping!

@mitu Is it so simple then? OK, thank you for your answer and for your time.