lr-scummvm: request for comments and testing

hhromic

Hi all,
I noticed that the lr-scummvm RetroArch core is not in the RetroPie library. I did some research and there wasn't much about it so I decided to give it a try myself with the current version from the libretro repository. It turns out to be working quite decently in the current state! I tested the core with the following games and so far so good, they launch fine and are playable very well:

Beneath a Steel Sky (CD VGA)
Broken Sword 1 The Shadow of the Templar (CD Windows)
Day Of The Tentacle (CD Dos)
Full Throttle (CD DOS)
Indiana Jones and the Fate of Atlantis (CD DOS VGA)
Loom (FM Towns)
Monkey Island 2 LeChuck's Revenge (Floppy DOS VGA)
Sam and Max Hit the Road (CD DOS)
The Curse Of Monkey Island (CD Windows)
The Dig (CD DOS)
The Secret of Monkey Island (CD DOS VGA)

Before attempting to submit the scriptmodule to to the RetroPie main repository, I would like some testing and feedback from the community. Please see below.

Thanks all in advance for the testing and feedback!

NOTES / CAVEATS

I only have tested the core on my stock, non-overclocked RPI 3B+. I would highly appreciate people with other platforms (ODROID, PC, etc) to give it a try and report back your experience.
The scriptmodule enables MT-32 emulation during compilation, however the RPI proved to be not very powerful and it crackles with some games. You may still try it anyway by enabling it in the ScummVM options, specially for other platforms.
The scriptmodule automatically downloads and installs necessary extra data for the core from the official libretro repository, e.g. soundfont, game data, default theme, etc.
The scriptmodule automatically configures the core to be usable out-of-the-box with standard settings, e.g. enable fluidsynth and the SC-55 soundfont, enable subtitles, etc.
Retroarch savestates are not implemented by the core, you must load/save games using the ScummVM load/state internal functions via its menu (normally pressing the START key).
In some cases trying to exit directly, e.g. using HOTKEY+START, freezes RetroArch (not the system). A way to avoid this is to exit the core first (via the ScummVM menu -> Quit) and then exit RetroArch normally. [A FIX IS INCLUDED NOW]

BENEFITS OF SCUMMVM UNDER RETROARCH

Running ScummVM as a RetroArch core has interesting/handy features:

Joypad autoconfiguration (see below for mappings)
Shaders support
Bezels support

INSTALLING/UNINSTALLING THE LR-SCUMMVM LIBRETRO CORE

The scriptmodule is now part of official RetroPie. To install or uninstall the lr-scummvm libretro core, go to the "Manage Packages" section, then to experimental and you will find the lr-scummvm package there.

If you want to tinker with the core's configuration, the scummvm.inifile is located in the BIOS directory: $HOME/RetroPie/BIOS/scummvm.ini.

ADDING SCUMMVM GAMES

For adding your ScummVM games, you simply need to drop the uncompressed data files into their own directories under the $romdir/scummvm directory (typically /home/pi/RetroPie/roms/scummvm).

Important: each game directory must contain .svm at the end of the name! This is not an "extension" but is just part of the directory's name. This is for EmulationStation to be able to launch the games directly when selecting the folder, without the need to "open" it first. Also it is useful for scraping. ES will put the images and metadata directly for each directory. In summary, ES will not consider the directory as directory but as a game.

Now, inside each game directory, you need to create a SHORT_NAME.svm file with SHORT_NAME as the content. No other .svm files should exist in the game directory!

The following is an example for the "Indiana Jones and the Fate of Atlantis" game:

$ cd $HOME/RetroPie/roms/scummvm/
$ cd Indiana\ Jones\ and\ the\ Fate\ of\ Atlantis.svm/  # note the "extension" at the end
$ ls -la
total 70060
drwxr-xr-x  2 pi pi     4096 Oct 20 17:32 .
drwxr-xr-x 13 pi pi     4096 Oct 20 18:54 ..
-rw-r--r--  1 pi pi    12035 Dec 24  1996 ATLANTIS.000
-rw-r--r--  1 pi pi  9823920 Dec 24  1996 ATLANTIS.001
-rw-r--r--  1 pi pi    65536 Dec 24  1996 MT32_CONTROL.ROM
-rw-r--r--  1 pi pi   524288 Dec 24  1996 MT32_PCM.ROM
-rw-r--r--  1 pi pi       25 Oct 20 17:32 atlantis.iq
-rw-r--r--  1 pi pi        9 Oct 20 12:06 atlantis.svm
-rw-r--r--  1 pi pi 61294074 Dec 24  1996 monster.sof
$ cat atlantis.svm
atlantis

You can check the SHORT_NAME for all compatible games from the ScummVM compatbility website: https://www.scummvm.org/compatibility/

After adding your games with the above layout, simply restart EmulationStation and they should appear inside the ScummVM system and ready to launch. No further configuration needed.

RETROARCH JOYPAD MAPPINGS

For the ScummVM core, the default joypad mappings can be seen in https://docs.libretro.com/library/scummvm/#joypad.
Also read that section for other handy mappings such as virtual numpad input and other functions with the gamepad.

BuZz

Interesting. I have a module for this but last time I tested there were major problems (on the RPI). Will retest. Thanks.

hhromic

Yes, seems the core has improved a lot recently. These games I listed run pretty well for me, some even with MIDI synth via fluidsynth/MT32 emulation. However the RPI can't cope with some of them with these settings. Hence I left MT32 disabled by default, but can be enabled on per-game basis.
The speed hack option when enabled (done by the scriptmodule by default) also makes a solid 60fps for me.
The gamepad via retroarch works very nicely, the mouse cursor even goes faster/slower according to your analog stick position, handy!
Hope to hear some feedback soon and see if we can integrate it to RetroPie.

Edit: I also run the core with the zfast-crt shader for added nostalgia, works flawlessly.

BuZz

@hhromic I'm happy to accept a PR for your module as it sounds more developed than mine. Can go in experimental and get more testing. Appreciated!

hhromic

Happy to contribute! It is possible to have a read of your old scriptmodule? Just to study and incorporate anything I might have missed, specially compiler flags.
There are some minor issues like RA freezing when exiting on certain conditions (only on exiting, when running is fine), but I'm investigating.
About doing a PR, sure can do, however I thought of testing from my own branch so I can iron-out things faster than sending more PRs to the main repo.
I will allow some time of feedback before sending the PR, the module is simple enough anyway, the majority of issues probably will be with the core itself.

hhromic

Update
After some ironing of the scriptmodule, I've been happily playing ScummVM games flawlessly on my stock, non overclocked RPI 3B+ for some time now. Playing ScummVM with a gamepad (PS3 in my case) through RA is much more comfortable than I could have expected. Saves states from within the ScummVM menu work fine too.

There is an evolving RetroArch-specific compatibility list being maintained here (not by me!): https://docs.google.com/spreadsheets/d/1a87BnQJc7_fTD5zF55JXcm79d5_l_d1ghDxtiubTnDE, which is very promising about many games.

I'm currently in the process of finishing Indiana Jones and The Fate of Atlantis as a test-case. Afterwards I think I will submit the scriptmodule for inclusion to experimental in RetroPie.

There is a minor issue when trying to exit RA using the Hotkey + Start combination, that freezes RA. It happens rarely and in some games only. If you close the core from within the ScummVM menu it never freezes. This is def a core issue and I will try to track how to reproduce to send upstream. The scriptmodule shouldn't need any changes regardless.

Edit: I've been playing using the zfast crt shader too, great look for ScummVM :)

hhromic

Update
I successfully completed Indiana Jones and The Fate of Atlantis now with the different endings. All is working nice and smooth.
I think the scriptmodule should be ready to submit as any game-glitch should be handled in the core code, not this scriptmodule. I will wait some days in case someone come with new feedback.

EctoOne

I have not tested your script because I'm in the process of switching to play on an Android device. But if the core is the same, and it seems to be because the crashing bug you've mentioned also happens for me, I really like it. I have not finished a game yet but I tried both Discworld games and had no issues.

hhromic

@EctoOne yes the script is pulling the source for the core from the official libretro github repository, so should be the same for Android if you are using official RA.

The crash-on-exit bug is the only issue I have seen so far so I hope a fix is found upstream. I still don't know exactly how to reproduce it. If you do, it would be really useful if you can share the steps :)

Thanks for reporting, I also really like the lr-based core of scummvm and would love to have it included in official RetroPie soon.

EctoOne

@hhromic the crash always happens when I use my hot key combo in game to open the retroarch gui and then select Quit RetroArch. It's just that simple. Quitting ScummVM first from its own Menu and then quitting RetroArch works fine.
Yet again, that's on Android without any Frontend, just RetroArch.

hhromic

@EctoOne ah, for me the crash is not always but random. It happens both, when I use the hotkey combo to go to the RGUI menu -> quit retroarch, or if I use the hotkey combo to quit directly. Like you, if I quit ScummVM from its own menu it never crashes. If it happens so "reproducible" in your Android setup, probably is easier to debug there. Hope the devs update soon.

Edit: I found an issue in their github repository reporting the same problem in WiiU. I sent a comment reporting that it also happens in RPI and Android. Hopefully we get some attention :). Issue: https://github.com/libretro/scummvm/issues/116

pjft

@hhromic what do you mean by crashing, just it of curiosity? Will the pi freeze/the process hang, or will it return to emulation station albeit with a segfault or another exception?

Looking forward to this!

hhromic

@pjft it just freezes retroarch, not the system. When crashed, you can login via ssh and kill the retroarch process (skill -KILL retroarch) and you will get back to emulationstation. When frozen, retroarch also doesn't respond to the SIGTERM signal (skill retroarch or skill -TERM retroarch).
No segfaults nor exceptions reported, just freezes.

The verbose logging is not very informative either. When it freezes it just stops in the "unloading game" operation:

  :
[INFO] [GL]: VSync => on
[INFO] [EGL]: eglSwapInterval(1)
[INFO] Unloading game..

And when it doesn't freeze, it gracefully finishes the shutdown:

  :
[INFO] [GL]: VSync => on
[INFO] [EGL]: eglSwapInterval(1)
[INFO] Unloading game..
[INFO] Unloading core..
[INFO] Unloading core symbols..
[INFO] Saved core options file to "/opt/retropie/configs/all/retroarch-core-options.cfg"

So it looks it crashes when trying to "unload the game". Also I found out that it happens more often if you are starting the game than after running it for a while.

Furthermore, apparently it doesn't happen when you send a SIGTERM signal to quit retroarch, which also triggers the unload game, core, etc. normally. Therefore, it seems to happen only when you quit either via Hotkey+Start exit combo or the "close content" option in the RGUI, and early in the game.

Edit: again, this only happens (sometimes) when trying to quit from RA, if you quit from the ScummVM's internal menu, it never crashes.

marus

@hhromic Hi, I've just started trying lr-scumm as you suggested from another thread.
It seems very interesting, I really appreciate your effort, but i fail to start games.
I followed exactly your instructions about ADDING SCUMM GAMES but when I select one the only thing happen is the opening of lr-scumm GUI empty (no games at all). May it be a language issue? I'm trying to start italian language versions....

hhromic

@marus thanks for testing! your feedback is appreciated.

Just to be clear, you have the following dir/files structure for your games:

/home/pi/RetroPie/roms/scummvm/Your Game Name  # <- game directory
/home/pi/RetroPie/roms/scummvm/Your Game Name/file1.sof  # <- game files inside dir
/home/pi/RetroPie/roms/scummvm/Your Game Name/file2.001  # <- game files inside dir
  :
/home/pi/RetroPie/roms/scummvm/Your Game Name/gameid.svm # <- game id file inside dir with "gameid" (no quotes) also inside the file as text

Make sure the gameidis correct for the game your trying to load. Check the ScummVM compatbility list, where they list the "short game name" (game id) for each game. See the list here: https://www.scummvm.org/compatibility/

The game language shouldn't be an issue. Can you let me know exactly which game are you trying to run? I can give it a try.

marus

@hhromic said in lr-scummvm: request for comments and testing:

with "gameid" (no quotes) also inside the file

That was the trick! I had .svm files empty because other scumm emulators simply does not check them inside.

Now I'am really happy with it! my controller wii u pro is finally ok, configurations and sound are great, i can play Indy 3 without numeric pad for the fight, everything seems perfect at the moment. Many thanks for your efforts.

The only thing hurting to me, other than random crash when exit by select+start that I hope will be fixed soon, is the gamelist. Because of the need to place .svm file into game folder, they are not in the root of ES scumm section as it happen with other scumm emulators (sdl1 & sdl2) so you have to select folder first then inside the game to run.
Any way to avoid it and let scraped games ready to launch directly from the root ?

hhromic

@marus I'm glad it worked now for you! good news. Yes I'm also very happy on how the core was developed for RA, very comfy controls and runs very smooth. The shaders are also a treat, make sure to try out the zfast crt shader! it looks great.

Regarding the random crash, yes I'm trying to figure out how to deterministically reproduce it so we can either report it upstream or try to find the culprit. Unfortunately I have little experience with debugging RA cores so I go slow.

Regarding the folder-based selection in emulationstation, yes that also bothers me a bit :). It is actually a limitation of EmulationStation, the way the core is implemented has to use that structure on the filesystem. Recalbox has a trick specifically for scummvm games to show directly instead of the folders, I will research how they do it. However, I suspect they modified EmulationStation for this to work, in this case we can try to push that to the RetroPie ES branch.

Cheers and thanks again for tesitng! the more you test and report (specially if you manage to easily and precisely reproduce the exit crashes), the better :)

marus

@hhromic said in lr-scummvm: request for comments and testing:

@marus I'm glad it worked now for you! good news. Yes I'm also very happy on how the core was developed for RA, very comfy controls and runs very smooth. The shaders are also a treat, make sure to try out the zfast crt shader! it looks great.

Where can I find more about trying zfast crt shader?

Regarding the random crash, yes I'm trying to figure out how to deterministically reproduce it so we can either report it upstream or try to find the culprit. Unfortunately I have little experience with debugging RA cores so I go slow.

Regarding the folder-based selection in emulationstation, yes that also bothers me a bit :). It is actually a limitation of EmulationStation, the way the core is implemented has to use that structure on the filesystem. Recalbox has a trick specifically for scummvm games to show directly instead of the folders, I will research how they do it. However, I suspect they modified EmulationStation for this to work, in this case we can try to push that to the RetroPie ES branch.

Ok, let me know please :-)

Cheers and thanks again for tesitng! the more you test and report (specially if you manage to easily and precisely reproduce the exit crashes), the better :)

Ok, thank you, cheers!

hhromic

@marus in case you haven't read about shaders in general and what they do, take a look here: https://github.com/RetroPie/RetroPie-Setup/wiki/Shaders-and-Smoothing

In summary, shaders allow to "enhance" the image via simulating certain aspects of old consoles/devices. In the case of ScummVM, these games were meant for CRT "tube" monitors, so we can simulate a scanline effect with shaders. It is true that CRT monitor is not the same as CRT TV, however I think the effect still looks nice for ScummVM.

To try out, in the guide I linked above follow the steps to enable shaders support and then select the shader you want. There are many shaders, I particularly like "zfast crt" normal (not the curvature version). But you can try others, for example the "crt-pi" one. Beware that most of these shaders are intensive and won't work very well on the RPI. The zfast* and crt/lcd-pi ones are optimised.

Also another tip: do NOT use smoothing from the guide, it will make the emulation slower probably and the results aren't that good.

EctoOne

with the standalone scummvm version i had an extra scummvm-data folder for the game files/folders and the svm files in the default scummvm folder. i don't know if that would still work with the lr core and i currently can't test it myself.
the only other solution would be to use that setup i mentioned and create symlinks in the default folder to the svm files.