EmulationStation, ScummVM, and .svm files

ctrueden

I'm setting up RetroPie on a Raspberry Pi 4B, and for the most part it's going super smoothly. RetroPie is an awesome project. Thanks to everyone involved in developing and improving it. 🥰

I'm working through ScummVM right now, and I got everything working, but my experience with using .svm files to launch games directly from EmulationStation differs from the documentation, and I wanted to ask the community about it.

What the docs say

Place an .svm file in the ROM folder for each game which you want to appear in EmulationStation.

The .svm file contains the short name of the game. This can be used to directly launch that game without starting the ScummVM GUI first.

And the example it gives is:

/home/pi/RetroPie/roms/scummvm/Day of the Tentacle/Day of the Tentacle.svm
/home/pi/RetroPie/roms/scummvm/Day of the Tentacle/MONSTER.SOU
/home/pi/RetroPie/roms/scummvm/Day of the Tentacle/TENTACLE.000
/home/pi/RetroPie/roms/scummvm/Day of the Tentacle/TENTACLE.001

With contents:

pi@retropie:~/RetroPie/roms/scummvm $ cat 'Day of the Tentacle/Day of the Tentacle.svm'
tentacle

What goes wrong

The EmulationStation launcher feeds the .svm filename, minus extension, to the +Start ScummVM.sh launcher. The launch script then passes that name unquoted as arguments to ScummVM, resulting in an error:

pi@retropie:~/RetroPie/roms/scummvm $ cat /dev/shm/runcommand.log
Parameters:
Executing: bash /home/pi/RetroPie/roms/scummvm/+Start\ ScummVM.sh "Day of the Tentacle"
scummvm: Stray argument 'Day'

The +Start ScummVM.sh script, after launching ScummVM (whether successfully or not), tries to be smart and (re)generate .svm files in ~/roms/scummvm, with filenames matching the game IDs, and contents matching the description. But that's backwards: we want the filename to be the full game name, and the contents to be the game ID, no?
The documentation says to nest each .svmfile inside its respective directory, and EmulationStation sees that, but it exposes that folder structure in the GUI. E.g.: "Day of the Tentacle" is in the ScummVM list, and when you click on it, there is then another "Day of the Tentacle" alone, which if then clicked on suffers from (1) above.

To address these issues, I modified the shell script as follows:

diff --git a/+Start ScummVM.sh.original b/+Start ScummVM.sh
index 43aa227..4a44235 100755
--- a/+Start ScummVM.sh.original
+++ b/+Start ScummVM.sh
@@ -1,8 +1,15 @@
 #!/bin/bash
 game="$1"
 pushd "/home/pi/RetroPie/roms/scummvm" >/dev/null
+
+# Read game ID from .svm file, and pass it to scummvm.
+test -f "$game.svm" && game=$(cat "$game.svm")
+
 /opt/retropie/emulators/scummvm/bin/scummvm --fullscreen --joystick=0 --extrapath="/opt/retropie/emulators/scummvm/extra" $game
+
+# Write .svm files matching titles sans parentheses, not IDs.
 while read id desc; do
-    echo "$desc" > "/home/pi/RetroPie/roms/scummvm/$id.svm"
+    title=${desc% (*}
+    echo "$id" > "/home/pi/RetroPie/roms/scummvm/$title.svm"
 done < <(/opt/retropie/emulators/scummvm/bin/scummvm --list-targets | tail -n +3)
 popd >/dev/null

Questions for the community

Has anyone else needed to do something like this? Or did I miss something more fundamental here?
If people agree these changes to the launch script are good: to which git repository should I contribute them? I looked in RetroPie-Setup on GitHub, but did not immediately find where this code lives.

Thanks for your time and interest!

mitu

The RetroPie launch script was created when ScummVM was added, expecting the id of the game (the internal short name of the game in ScummVM). Down the road, support for lr-scummvm was added and I think the docs expand the instructions for the latter (with the additions of a .svm in the game folder).
Not sure how both could be supported, the +Start ScummVM.sh is created during installation here).

TonyUK

@ctrueden Hey having the exact same issue as you, you mention modifying the shell script. where do I find said shell script to add your adjustments?
Thanks for any advice.

ctrueden

@tonyuk said in EmulationStation, ScummVM, and .svm files:

where do I find said shell script to add your adjustments?

Many apologies for the late reply. If you didn't find it already, the shell script in question is at /home/pi/RetroPie/roms/scummvm/+Start ScummVM.sh.

@mitu Thanks for the pointer to the source. I finally found the time to file a PR with my changes:
https://github.com/RetroPie/RetroPie-Setup/pull/3430

Edit: I also filed a PR to tweak the documentation about the .svm file location:
https://github.com/RetroPie/RetroPie-Docs/pull/98

Lolonois

@ctrueden thanks for sending this over. While the sibling *.svm file to the game directory is working for ScummVM, it creates a regression in the libretro scummvm.

The latter requires a *.svm within the game folder. If solely the game directory is passed into lr-scummvm (via romdir-launcher.sh) it crashes with a segfault. If the sibling *.svm is passed in, the lr-scummvm can not detect the game, as a satisfied -p parameter (gamepath) is crucial for the launch. I haven't considered possible solutions to satisfy both emulator flavors with one folder structure yet.

Lolonois

I understand the legacy code was not in a proper state and diverted between ScummVM and it's Retroarch cousin. I had an extended look at the legacy state, the proposed PR and the constraints which the libretro ScummVM implementation has.

NB, used nomenclature: <gameid> equals "Game Short Name" from [1], <gamedesc> refers to the description shown in the ScummVM GUI and which is the same as the property description= in scummvm.ini. The <gamedesc> contains human readable game name and extra information e.g., Floppy, CDROM, Platform, in game languages used...

[1] https://www.scummvm.org/compatibility/

Current HEAD of master (04a74980)

Expects <gameid> passed in as first parameter (%BASENAME%), but gets "<gamedesc>" passed in. So this is clearly broken as @ctrueden reported.
Everytime the <gameid>.svm files are touched/recreated when ScummVM is exited
The <gamefolder>/<gamefolder>.svm is not used (as it was noted in the pre version in the Wiki, before [2]), nor the <gameid> within this *.svm file.
ScummVM is started in ~/RetroPie/roms/scummvm
User needs explicitly run once either Mass Add... or individual Add Game... in ScummVM GUI (as noted in the RP-Docs/Wiki). This mass add is problematic by itself because it may add more games (or demos) to the list as the user intented (=possible clutter in gamelist).

[2] RetroPie-Docs/pull/98

Proposed PR (ScummVM) [3]

Creates <title>.svm (<title> denotes the <gamedesc>, see above) with <gameid> inside, sibling to a <gamefolder>. This <gamefolder> may be different than <title>.svm: The <gameid> and <gamefolder> are linked via the scummvm.ini, which has to contain a section with [<gameid>] thus the initial Add Game... / Mass Add....
<title>.svm recreated every time when ScummVM is exited (legacy from current implementation, see above).

May create double/triple <title>.svm files for same <gamefolder> e.g.,

Little Big Adventure (DOS English (GB)).svm
Little Big Adventure (DOS French).svm
Little Big Adventure (DOS German).svm

Problematic to use generated <title> for Samba: Guess which game denotes MDWOZH~3.SVM?. Tick (') is not a valid CIFS/Samba character for filenames for example. Gets displayed as shortened, somewhat cryptic filenamne (Spoiler: It is Monkey Island 2: LeChuck's Revenge (DOS English).svm).
Minor: Current PR contains a typo. Note the missing \ before $(cat ... at [4]

[3] RetroPie-Setup/pull/3430
[4] https://github.com/RetroPie/RetroPie-Setup/blob/7b6218d6d68268b48e99d0b7c7a76c0bf65816ed/scriptmodules/emulators/scummvm.sh#L85

Constraints by Retroarch ScummVM

Retroarch lr-scummvm is started in /home/pi but uses absolute paths for the ROM/game (refer to /opt/retropie/libretrocores/lr-scummvm/romdir-launcher.sh, gets %ROM% (=absolute path) passed in).
Some file, from within has to be passed into (%ROM% parameter may not be NULL). However the denoted file passed in does not have to be present in the filesystem. But: The basename is used for *.srm and *.state files of retroarch.
If the file is a directory instead of an *.svm file a *.svm is searched within that <gamefolder>
If a *.svm from within the <gamefolder> is passed (either by search or explicitly) into romdir-launcher.sh this *.svm file is ignored (sic!) (see [5])
However, if a file (not a directory) ends on *.scummvm the contained <gameid> is used, in all other cases autodetection is used in the <gamefolder>.
If a *.scummvm is present in the <gamefolder> it has to contain a valid <gameid>.

Tl;DR: Proposed Changes for ScummVM and lr-scummvm interoperability

I propose thess changes for seamless interoperability of ScummVM / Retroarch ScummVM and also to apply the principle of least astonishment for the enduser. The scriptmodules I keep currently here for scummvm.sh and here for lr-scummvm.sh.

Keep the <gamefolder>.svm file sibling to its <gamefolder> directory and not inside, as @ctrueden suggests and I do support.
Document expected workflow when deploying games (--> To be noted in RP-Docs/Wiki): Create *.svm file, and testdrive each game. This is a needed smoketest anyway as coping game files is a manual task.
Use scummvm --detect instead of scummvm --list-targets as the former is more accurate.
Accept userdefined <gamefolder>.svm and user defined <gamefolder> directory sibling to each other. Thus no overly long autogenerated *.svm files and/or no strange filenames in Samba/CIFS (unless the user uses deliberately Samba/CIFS unsupported filename characters).
Patch libretro.cpp to accept files with extension *.svm instead of *.scummvm, pull request [5].
*.svm can be empty (=without <gameid>), the <gameid> would be added on first launch of *.svm with ScummVM. Thus no need more to run Mass Add... (or Add Game... when one game is put into the scummvm/ folder).
When lr-scummvm / romdir-launcher.sh is used before running at least once ScummVM on the game: <gameid> must be present in *.svm (--> Document in RP-Docs/Wiki).
On juvenile launch of a game with lr-scummvm the romdir-launcher.sh creates a symlink to parent *.svm (from ScummVM launcher) to explicitly pass in the <gameid> to lr-scummvm.
For bulk creation of *.svm files the scummvm --detect should be used on every <gamefolder>, could be an extra script or --> to be documented in the RP-Docs/Wiki.

[5] RetroPie-Setup/pull/3351

Long post, but hopefully my text and ideas are understandable.
What is the stance of the maintainers on the proposed changes?

mitu

@lolonois said in EmulationStation, ScummVM, and .svm files:

Thank for the analysis. My comments below.

Keep the <gamefolder>.svm file sibling to its <gamefolder> directory and not inside, as @ctrueden suggests and I do support.

I think this is how it's done right now. But, with the changes proposed by @ctrueden in https://github.com/RetroPie/RetroPie-Setup/pull/3430, the format may change.

Document expected workflow when deploying games (--> To be noted in RP-Docs/Wiki): Create *.svm file, and testdrive each game. This is a needed smoketest anyway as coping game files is a manual task.

Agreed.

Use scummvm --detect instead of scummvm --list-targets as the former is more accurate.

Since the detection of games should be handled by starting ScummVM, I don't think --detect is necessary and it might take longer. Once the games are known to ScummVM, then --list-targets will show them and they'll be able to be stored as .svm - we don't want to run a detection.

Accept userdefined <gamefolder>.svm and user defined <gamefolder> directory sibling to each other. Thus no overly long autogenerated *.svm files and/or no strange filenames in Samba/CIFS (unless the user uses deliberately Samba/CIFS unsupported filename characters).

I see the CIFS issue as secondary, it's not a ScummVM or RetroPie problem. I also think the issue is not with the ' character, but with : (which is not a valid Windows filename character).

I don't see an issue with the overly long filenames.
If the game has different versions (i.e. DOS/Amiga, En/De/It, etc.), the option for which game to add is up to the user when the game is added from ScummVM. This should be up to the user.

Patch libretro.cpp to accept files with extension *.svm instead of *.scummvm, pull request [5].

IMHO this should go upstream first. They might accept it or not, but I forgot how the libretro core handles this so it might not be suited for their use. If we can't find a middle ground here, maybe add it as a local patch.

*.svm can be empty (=without <gameid>), the <gameid> would be added on first launch of *.svm with ScummVM. Thus no need more to run Mass Add... (or Add Game... when one game is put into the scummvm/ folder).

Does this work with standalone ? Haven't tried it yet.

When lr-scummvm / romdir-launcher.sh is used before running at least once ScummVM on the game: <gameid> must be present in *.svm (--> Document in RP-Docs/Wiki).

No objection to documenting this, but I think we might need the ID of the game if we want to generate a compatible libretro core launch file.

On juvenile launch of a game with lr-scummvm the romdir-launcher.sh creates a symlink to parent *.svm (from ScummVM launcher) to explicitly pass in the <gameid> to lr-scummvm.

I think this should be handled automatically, similar to how the .svm are automatically created.

For bulk creation of *.svm files the scummvm --detect should be used on every <gamefolder>, could be an extra script or --> to be documented in the RP-Docs/Wiki.

That could be added in a configure action for the scriptmodule and documented.

okorz001

@ctrueden said in EmulationStation, ScummVM, and .svm files:

The documentation says to nest each .svmfile inside its respective directory, and EmulationStation sees that, but it exposes that folder structure in the GUI. E.g.: "Day of the Tentacle" is in the ScummVM list, and when you click on it, there is then another "Day of the Tentacle" alone

Forgive me if this post is too old, but I just wanted to share that I was very confused by this behavior when I installed lr-scummvm yesterday. In fact, I was certain that I must have found a bug in EmulationStation, but I couldn't find any open issues about other games disappearing when launching. Through luck, I eventually found this thread and finally understood what was actually happening -- I have not previously used directories for any of my other systems so I had no idea EmulationStation behaved this way.

If anyone else stumbles upon this, you can workaround the issue for lr-scummvm by also naming the ROM directory itself with a .svm extension.

Using the example from the documentation:

/home/pi/RetroPie/roms/scummvm/Day of the Tentacle.svm/Day of the Tentacle.svm
/home/pi/RetroPie/roms/scummvm/Day of the Tentacle.svm/MONSTER.SOU
/home/pi/RetroPie/roms/scummvm/Day of the Tentacle.svm/TENTACLE.000
/home/pi/RetroPie/roms/scummvm/Day of the Tentacle.svm/TENTACLE.001

I am not sure if this works for regular scummvm though.