lr-scummvm: request for comments and testing

BiZzAr721

Great work @hhromic . I want to share with you some information you may find useful.

During the evolution of OpenBOR on Retropie (now compiled from latest source, fully functional) with great efforts by @cyperghost, @darknior and @zanac, who has been paramount, & made us aware of GL4ES, which is a wrapper to provide OpenGL 2.x functionalty.

@zanac has compiled and shared his gl4es lr-scummvm binary, as well as recommendations for compiling with GL4ES, which retroarch can use natively!

https://retropie.org.uk/forum/topic/13784/openbor-finally-working-fine-on-retropie-with-es/299?page=15
(you can find the link to his lr-ScummVm core in post #307)

hhromic

@BiZzAr721 thanks for the support and for the information you shared.

I checked the post you referenced however I got a bit confused. The scriptmodule I'm proposing is pulling lr-scummvm from the official libretro repository here: https://github.com/libretro/scummvm. If I understood the post correctly, @zanac already mentioned that same repository for his build, so i guess his contributions are already there upstream? In that case his contributions to the core are already being pulled and used in this scriptmodule :)

Just to clarify, I'm not trying to port/adapt ScummVM to libretro on my own, but I am creating a retropie scriptmodule for building, configuring and installing the official upstream libretro core mentioned above to be as out-of-the-box experience as possible.

hhromic

Update
Good news about the annoying Random-Freeze-on-Quit bug of the core! I dedicated some time to try to debug the problem and I think I found a solution. At least for me, lr-scummvm is not crashing anymore when quiting games using the HOTKEY+START combo :)

I already updated the scriptmodule now to add a patch with the fix, I would really appreciate if testers can check if the freezing is gone when quiting directly from RetroArch (e.g. using the exit combo) instead of the ScummVM internal menu.
If the patch is confirmed to be a suitable solution, I will send the patch upstream and hopefully can be merged (this means for example Android builds would also be fixed cc @EctoOne ).

@marus , I hope you have the time to test the fix and report back on your side.

Edit don't forget to uninstall and re-install the scriptmodule (see OP instructions) to make sure you get the patch and the new scriptmodule.
Edit2 I tested with all my current games (see OP) and they all seem to be quiting now fine with the fix :)

hhromic

@BuZz , I would like some advice from you to improve user experience :)

I've been studying on supporting launching scummvm games directly from the game folder in EmulationStation (single operation) instead of having to enter the folder and launch the svm file in there (two operations). Also, by having ES recognising the folder itself, it's easier to scrape the games too.

I managed to make this work by means of a hack in the emulator entry of the scriptmodule in this way:

local -r emulator_cmd=(
    "$emudir/retroarch/bin/retroarch" -L "$md_inst/scummvm_libretro.so"
    --config "$md_conf_root/scummvm/retroarch.cfg" %ROM%/*.svm
)
addEmulator 0 "$md_id" "scummvm" "${emulator_cmd[*]}"

The method is very simple as you can see. When the game folder has a known extension for EmulationStation (.svm for example) then at launch %ROM% will be the folder itself in runcommand. Therefore, by adding /*.svm, runcommand will build the final cmdline with any .svm FILE inside the rom "folder" to pass to RA.

This works fine as long as there is a single .svm file in the game folder. However I think this can be done more reliable and elegant in runcommand itself.

My proposal is then to make a convention and make runcommand detect if the passed %ROM% is a folder, then search if there is a single file with the same extension as the passed folder inside, and if so, use the folder + file found as the final rom in the cmdline. This change is trivial and should be non-intrusive.

The most important benefit is that this would not force users to rename their scummvm game folders (keeping the two-clicks to launch if they wish), and users that do so would get the "one-click launch" feature. Currently, the hack in the scriptmodule requires that the folders be renamed with the extension of scummvm to be used as roms by ES. I believe some users might not get this at first.

If you believe this is a worthy argument/use case, I can make the modifications to runcommand, test them and send a PR for you to review. Thanks!

EctoOne

@hhromic Would your folder hack work for other systems as well?
I was always a little bit disappointed that I wasn't able to use the same subfolder structure which Kodi has (all game/movie related files in its own folder). That would make rom/asset management so much easier.

pjft

@hhromic I believe Daphne uses folders as well, if I recall correctly, so it might be good for inspiration to see how it is set up? Maybe there's something you can adapt from it.

BuZz

@hhromic I've only just scanned over your msg. Please can you do a PR and we can discuss it there so I don't forget. Need to check some code in regards to your idea.

Btw sorry I never posted my module. I couldn't find it. It's in a backup for sure, but don't think it's much use now anyway. It's pretty basic.

BiZzAr721

@hhromic From my understanding, those are just optimizations made to the make file which was made using make -j 4, and could possibly be applied using a patch, still building from source. Script-Fu master, @cyperghost may be able to give further insight. Maybe there is a simple solution to get games to load using the same files/location as the stand alone ScummVM.

I looked into the GL4ES, and I think the libretro ScummVM is already using it, therefore can use some of the advantages of OpenGL 2 (Where-as the standalone ScummVM runs off SDL). I don't know if additional dependencies are required.

Great to hear you may have solved the Freeze-on-Quit bug! I thought I finally had everything all set up just right, but now I'll have to give lr-scummVM a try!
Thank you.

hhromic

@EctoOne said in lr-scummvm: request for comments and testing:

@hhromic Would your folder hack work for other systems as well?
I was always a little bit disappointed that I wasn't able to use the same subfolder structure which Kodi has (all game/movie related files in its own folder). That would make rom/asset management so much easier.

What I'm proposing is not strictly tied to lr-scummvm, yes. However I'm not sure what you meant. To clarify, I'm proposing for runcommand to be able to do the following:

If the passed rom is a folder and the name has an extension, then search inside the passed folder for a single file with the same extension. If found, then use the combination of "passed folder/found file" as the rom to launch.
Otherwise, just behave like it is doing currently and do not transform the rom passed.

This would allow everything to work as it is doing currently, but in addition will allow for lr-scummvm games to be launched directly with one click from ES if the user suffixes the game folder names with the ".svm" extension. If not, then it will still work but the user will need to enter the folder first.
If other emulators support running roms in folders in this same way, then it should work too.

@pjft said in lr-scummvm: request for comments and testing:

@hhromic I believe Daphne uses folders as well, if I recall correctly, so it might be good for inspiration to see how it is set up? Maybe there's something you can adapt from it.

I will take a look, thanks for the pointer!

hhromic

@BiZzAr721 using make -j 4 is not a binary optimisation, just a build optimisation to compile faster. Currently, lr-scummvm compiles in less than an hour, so I just use the defaults from retropie. I don't know if GL4ES is used upstream in lr-scummvm but so far it is running very efficiently on my stock RPI 3B+, and others seem to be having good results too.

I'm already proposing an enhancement for runcommand to be able to launch games directly from folders, that should be robust, extensible and harmless enough to be integrated in RP.

I hope you can test the Freeze-on-Quit fix, I'm looking very forward to confirm it works to submit upstream :) Thanks!

hhromic

@BuZz said in lr-scummvm: request for comments and testing:

@hhromic I've only just scanned over your msg. Please can you do a PR and we can discuss it there so I don't forget. Need to check some code in regards to your idea.

Will do.

Btw sorry I never posted my module. I couldn't find it. It's in a backup for sure, but don't think it's much use now anyway. It's pretty basic.

No worries, the scriptmodule seems working fine so far :) Hope to send it soon to RP.

cyperghost

@hhromic @BiZzAr721 My knowledge about bash scripting is little. I'm stuck in building GLES out from the scriptmodule ;) Maybe I can use your script and rebuild it to this one openbor-6xxx

BiZzAr721

@hhromic Well, good news is I haven't had any freezing when exiting using HOTKEY + START! Navigating using joypad worked great, but my mouse would not work. (neither Trackpad incorporated in logitech KB, or corded USB mouse) Is there an option to allow joypad and KB & Mouse simultaneously? That's how the regular ScummVM core works. Mostly I will use the KB & Mouse combo, but I use joypad as well when I do not want to bother getting the keyboard.
Also, Speed Hack was disabled by default.

hhromic

@BiZzAr721 thanks for testing! I'm happy the freeze bug seems to be gone for you too.
Regarding using a real mouse/keyboard to play in ScummVM, I haven't tested that myself yet but will do to see how it works. Because RetroArch is focused on gamepad gaming, I suspect mouse/keyboard are mapped as gamepads by default. Will take a look and report back.

Regarding the speed hack disabled by default, as suggested by the official documentation of the core, it is only enabled automatically if you are installing on an ARM device. What platform are you using?

BiZzAr721

I am using raspberry pi 3b, running RetroPie 4.4 (Stretch)

hhromic

@BiZzAr721 for RPIs the speed hack should be automatically enabled. I tested on my RPI3 and RPI2 with stock RetroPie installed and in both it is enabled just fine.

By design of RetroPie, the speed hack core option will not be set if you already had it configured to any value, for example "disabled". This is to not overwrite any previous custom-configuration. I suspect that is what happening for you.

You can verify this by removing the core option scummvm_speed_hack in /opt/retropie/configs/all/retroarch-core-options.cfg and re-configuring lr-scummvm using sudo ./retropie_packages.sh lr-scummvm configure. You will then see that the speed hack is now enabled automatically.

BiZzAr721

I will try this when I have an opportunity. Also, this is just an image of the microSD I normally use - a testing ground of sorts. So any unistalling/reinstalling/changing configs ect that can help just let me know.

EDIT
@hhromic I was going through the regular ScummVM setup script (trying to change from version 2.0.1pre to latest/master - no luck yet) Anyway I noticed 01_rpi_enable_scalers.diff, and as I looked into it some more I noticed it was created by you. So, hats off to you for your work. I'm hoping that means I can change - build_scalers=yes to enable scanlines or something of the sort!

hhromic

@BiZzAr721 thanks but I'm not the author of that patch. I just refactored it out of the scriptmodule into its own external file for easier maintenance :)
The purpose of that patch is precisely to enable scalers, so you shouldn't need to do anything. What it does is to remove build_scalers=no so it defaults to yes.
I'm not currently using/experimenting with the standalone ScummVM, so maybe would be best to open a new topic if you are having issues with enabling scanlines there. Cheers!

darknior

Hi
I really love this project, i'm a fan of ScummVM and using it with overlay is a dream for me :p

I have try your version and using your tutorial in first post.

This lr-scummvm working for me.
Overlay is working, with joystick, and menu access key on START and START + SELECT to exit.
But it is less smooth than the original one :(
I can't understand how to launch a game from ES :(

I have add the directory of the game with the complete game in /home/pi/RetroPie/roms/scummvm/freddi-win-fr.svm
When i try to launch it it show me the ScummVM menu empty.
I add it like with the old ScummVM official version and i can see it on the INI file

[freddi-win-fr]
description=Freddi Fish 1: The Case of the Missing Kelp Seeds (Windows/French)
talkspeed=85
path=/home/pi/RetroPie/roms/scummvm/freddi-win-fr.svm
subtitles=false
gameid=freddi
language=fr
LastBottleLocations=2
TextOn=0
platform=windows
guioptions=sndNoMIDI launchNoLoad noAspect lang_French

If i try to launch it again i come back to the ScummVM menu and i must click start

Then like you write i add and create in the game directory a file freddi-win-fr.svm
When i try to launch it, i have a black screen and go back to ES.

In runcommand.log i have

Parameters: 
Executing: /opt/retropie/emulators/retroarch/bin/retroarch -L /opt/retropie/libretrocores/lr-scummvm/scummvm_libretro.so --config /opt/retropie/configs/scummvm/retroarch.cfg "/home/pi/RetroPie/roms/scummvm/freddi-win-fr.svm"/*.svm --appendconfig /dev/shm/retroarch.cfg
scummvm: Stray argument 'Freddi'
Usage: scummvm [OPTIONS]... [GAME]

Try 'scummvm --help' for more options.

If i launch the same game with the old version of ScummVM it works fine.

hhromic

@darknior thanks for testing! and thanks for the support on this scriptmodule, I hope it gets ready to merge upstream soon!

Beware that this core doesn't use the very latest upstream scummvm, so do not expect the exact same compatibility. It is not (yet) intended as replacement of the standalone scummvm.

However, I just tried the game you mention (Freddi Fish 1: The Case of the Missing Kelp Seeds (Windows/French) and worked fine for me on first try.
Did you add the string freddie inside the freddi-win-fr.svm file? If still no luck, let me help you get things clear/correct so you can try again.

First, make sure you are using the current version of the scriptmodule. Uninstall lr-scummvm, and remove the scriptmodule and associated files as instructed on the OP. Also make sure to remove your ~/RetroPie/BIOS/scummvm folder and ~/RetroPie/BIOS/scummvm.ini file. This should put things back to the beginning. Then, re-install the scriptmodule and re-build/re-install the core. I know this can take some time, but is just to make sure you are on the right state. This is a summary of commands for accomplishing all this:

cd $HOME/RetroPie-Setup  # go to the RetroPie Setup directory
sudo ./retropie_packages.sh lr-scummvm remove  # remove lr-scummvm core
rm -rf scriptmodules/libretrocores/{lr-scummvm.sh,lr-scummvm}  # remove scriptmodule and old scriptmodule data
rm -rf $HOME/RetroPie/BIOS/{scummvm.ini,scummvm}  # remove config file and extras folder
patch -p1 < <(wget https://github.com/hhromic/RetroPie-Setup/compare/master...add-lr-scummvm.diff -qO-)  # re-patch with new version of the script
sudo ./retropie_packages.sh lr-scummvm clean  # clean any old build directory
sudo ./retropie_packages.sh lr-scummvm  # re-build lr-scummvm, configure and re-install

Also make sure there are no scummvm options in /opt/retropie/configs/all/retroarch-core-options.cfg before re-installing/re-configuring.

At this point you should have a freshly built and configured lr-scummvm core. Do not modify scummvm.ini. Please also consider that the lr-scummvm core doesn't need to add games to the internal ScummVM launcher, doing so might confuse the loading of games from the core.

Make sure your games are in the scummvm roms folder using the following structure:

roms/scummvm/freddi-win-fr.svm     <-- this is a directory not a file
    FREDDI.HE0
    FREDDI.HE1
    FREDDI.HE2
    FREDDI.HE3
    FREDDI.HE4
    freddi-win-fr.svm   <-- this is a file with the string "freddi" inside

~~Make sure there is no *.svm file with the same name outside the game folder (I think standalone scummvm does this?), EmulationStation might get confused with this layout.~~ I confirmed now that having standalone svm files does not affect.

At this point, the game should appear as a single entry in the scummvm system in EmulationStation and should launch fine. It can even be scraped (I did and got a nice box art for the game).

Regarding the speed, what devices are you using? on my non-overclocked RPI3B+ all seems to be working fine and smooth. I agree the mouse pointer speed (not the game itself) can be a bit slow in some games, you can adjust this in the RetroArch core options of the core (Gamepad Cursor Speed). Also check there that the Speed Hack option is enabled (should be done automatically if you cleaned the core options before re-installed.

Let me know if this works for you. ~~If this folder layout is not compatible with the standalone scummvm let me know, we can work out a co-existing system so both the standalone and lr-core versions work with the same game files/folders.~~ I also confirmed now that they can co-exist, but standalone scummvm will create svm files in the roms/scummvm folder, you still need the ones inside each folder for lr-scummvm with the gameid as content.

Thanks again for testing, much appreciated! Your feedback helps making the core integration better!