lr-scummvm: request for comments and testing

marus

@hhromic said in lr-scummvm: request for comments and testing:

@EctoOne @marus , guys I got a bit confused with what are you trying to do with that.

@EctoOne , I understand now, you mean the actual game files in a separate folder than the "scummvm" and the symlink the svm files so EmulationStation doesn't see the actual folders of each game?

After checking how recalbox does it, I think I got an idea on how to make it for EmulationStation to display just the game and not the folder, and still make it work as intended. However I need a bit of time to figure out a proper solution. When I have it done I will update the patch and you can test it.

Thanks for all the feedback/testing, very helpful!

Happy to read it, anyway it should be the same that Retropie standard Scumm emulator do.
I have Scumm games in subfolders but standard Scumm emulator, once added a game, put .svm file into roms/scummvm so the gamelist does not show subfolders.
For the same games lr-scumm need the .svm files into game folders so now my gamelist got two entries for every game: one for standard scumm and one for lr-scumm, but the one for lr-scumm is a folder with the game inside.
I'll wait for your patch, in the meanwhile i could try ectoone solution to give a little contribute.
Cheers ;)

hhromic

@marus I just made a new version of the scriptmodule, see my post above. With this approach, both the standalone and libretro emulators can live alongside. The only modification to your game directories is to add .svm to the directory name. The standalone ScummVM should be still happy with this (ignores the internal svm file), and the libretro core is happy (uses the internal svm file). EmulationStation hides the .svm "extension" of the directory name anyway, so looks lik a "game".
Try the newest iteration and let me know.

marus

@hhromic said in lr-scummvm: request for comments and testing:

Update

Good news! Making EmulationStation open the game by its folder directly was easier than I thought :) I implemented this new feature now in the proposed scriptmodule. I updated the instructions in the first post accordingly. To summarise for existing testers:
Your game directories must have .svm at the end of the name. For example: Indiana Jones and the Fate of Atlantis.svm. Please note that this looks like an extension but is actually just part of the directory name. This is what tricks EmulationStation to consider the directory itself as the "game" and you can then launch it directly without opening it first.
You must first update the lr-scummvm scriptmodule (recommended is to uninstall it and re-patch the scriptmodule, see instructions). Recompiling/reinstalling lr-scummvm is not necessary. Then you need to re-configure the lr-scummvm installation. You can do this using the following commands:
cd $HOME/RetroPie-Setup
sudo ./retropie_packages.sh lr-scummvm configure
After you re-configure lr-scummvm and rename your game directories to have .svm at the end of the name, you should be able to launch the games from EmulationStation without entering the directories first, and also able to scrape them like any console rom. Let me know how it goes!

Great! I've just replied to prior post :-D
Are you sure that changing folder name does not affect how the game work?
In addiction, there are some games, like Full Throttle, that have 2 subfolders, I actually have .svm file for this game into roms/scummvm/ft/resources: not sure your workaround work if I rename resurces in full throttle.svm, i'll try it.

hhromic

@marus yes I tested it :)
Regarding the subfolders, do not rename internal subfolders!
You just need to rename the main/top folder containing all game files. In your case, just rename the main/top ft folder to ft.svm. Everything else inside (including subfolders) should remain intact.
Also, you don't need to create internal svm files for each subfolder, only in the main/top folder of the game.

marus

@hhromic I'm going to try, just a clarification: after renaming game folder have I to create .svm file (whit short name inside) inside the folder itself?

hhromic

@marus yes, exactly, like you did before. In summary you should have a directory structure like in this example with two games:

roms/scummvm/
       ┣ Indiana Jones and The Fate of Atlantis.svm/
       ┃   ┣ atlantis.svm   <- "atlantis" inside
       ┃   ┣ MONSTER.SOU
       ┃   ┣ ATLANTIS.000
       ┃   ┕ ATLANTIS.001
       ┕ Day of the Tentacle.svm/
            ┣ tentacle.svm   <- "tentacle" inside
            ┣ MONSTER.SOU
            ┣ TENTACLE.000
            ┕ TENTACLE.001

Remember that if the game has more subdirectories, just leave them there untouched, the only thing that matters is the main directory having ".svm" at the end of the name and the game ".svm" file inside with the game id.
Let me know how how it goes for you. Thanks for testing!

hhromic

@BuZz I have a curious question. How strong is the requirement for modules to compile using -O2? The Makefile of the core defaults to -O3 and the resulting binary seems to be working fine. If we allow this optimisation I can get rid of the small patch this scriptmodule is applying to force -O2. What do you think?

BuZz

@hhromic can leave it as O3. O2 is just our defaults

hhromic

@BuZz great, I will test more with the default O3 from the core. In general the scriptmodule seems to be evolving good and getting good reception and ideas, should be ready to submit soon :)

marus

@hhromic I tried to follow your instruction about repatching script but something is gone wrong, now i have to reinstall lr-scumm. Maybe you should write step by step how to uninstall and patch the script, at least for no experts 😉
No more time for testing these days, Once reinstalled i’ll give you some feedback, thanks again for your work!

hhromic

@marus oh, sorry to hear. Anyway is not such bad news as I now am no longer forcing O2 optimisation for the binary building, but the default O3 from upstream. This should create, in some cases, a better optimised/faster binary.

To uninstall, update the patch, and restart from a clean state (rebuild the binary), execute the following commands step-by-step:

cd $HOME/RetroPie-Setup  # go to the RetroPie Setup directory
sudo ./retropie_packages.sh lr-scummvm remove  # remove lr-scummvm core
rm -rf scriptmodules/libretrocores/{lr-scummvm.sh,lr-scummvm}  # remove scriptmodule and old scriptmodule data
patch -p1 < <(wget https://github.com/hhromic/RetroPie-Setup/compare/master...add-lr-scummvm.diff -qO-)  # re-patch with new version of the script
sudo ./retropie_packages.sh lr-scummvm clean  # clean any old build directory
sudo ./retropie_packages.sh lr-scummvm  # re-build lr-scummvm, configure and re-install

Then make sure your games are in the new format: .svm at the end of each folder name and a unique .svm file with the corresponding gameid inside each game folder.

Finally, restart EmulationStation and you should see your games under the scummvm system as single entries and not folders. If you click on any, it should immediately launch the game. You can also scrape the system too to get art and metadata for the games.

Take your time to test and let me know of your feedback when you can :), much appreciated!

BiZzAr721

Great work @hhromic . I want to share with you some information you may find useful.

During the evolution of OpenBOR on Retropie (now compiled from latest source, fully functional) with great efforts by @cyperghost, @darknior and @zanac, who has been paramount, & made us aware of GL4ES, which is a wrapper to provide OpenGL 2.x functionalty.

@zanac has compiled and shared his gl4es lr-scummvm binary, as well as recommendations for compiling with GL4ES, which retroarch can use natively!

https://retropie.org.uk/forum/topic/13784/openbor-finally-working-fine-on-retropie-with-es/299?page=15
(you can find the link to his lr-ScummVm core in post #307)

hhromic

@BiZzAr721 thanks for the support and for the information you shared.

I checked the post you referenced however I got a bit confused. The scriptmodule I'm proposing is pulling lr-scummvm from the official libretro repository here: https://github.com/libretro/scummvm. If I understood the post correctly, @zanac already mentioned that same repository for his build, so i guess his contributions are already there upstream? In that case his contributions to the core are already being pulled and used in this scriptmodule :)

Just to clarify, I'm not trying to port/adapt ScummVM to libretro on my own, but I am creating a retropie scriptmodule for building, configuring and installing the official upstream libretro core mentioned above to be as out-of-the-box experience as possible.

hhromic

Update
Good news about the annoying Random-Freeze-on-Quit bug of the core! I dedicated some time to try to debug the problem and I think I found a solution. At least for me, lr-scummvm is not crashing anymore when quiting games using the HOTKEY+START combo :)

I already updated the scriptmodule now to add a patch with the fix, I would really appreciate if testers can check if the freezing is gone when quiting directly from RetroArch (e.g. using the exit combo) instead of the ScummVM internal menu.
If the patch is confirmed to be a suitable solution, I will send the patch upstream and hopefully can be merged (this means for example Android builds would also be fixed cc @EctoOne ).

@marus , I hope you have the time to test the fix and report back on your side.

Edit don't forget to uninstall and re-install the scriptmodule (see OP instructions) to make sure you get the patch and the new scriptmodule.
Edit2 I tested with all my current games (see OP) and they all seem to be quiting now fine with the fix :)

hhromic

@BuZz , I would like some advice from you to improve user experience :)

I've been studying on supporting launching scummvm games directly from the game folder in EmulationStation (single operation) instead of having to enter the folder and launch the svm file in there (two operations). Also, by having ES recognising the folder itself, it's easier to scrape the games too.

I managed to make this work by means of a hack in the emulator entry of the scriptmodule in this way:

local -r emulator_cmd=(
    "$emudir/retroarch/bin/retroarch" -L "$md_inst/scummvm_libretro.so"
    --config "$md_conf_root/scummvm/retroarch.cfg" %ROM%/*.svm
)
addEmulator 0 "$md_id" "scummvm" "${emulator_cmd[*]}"

The method is very simple as you can see. When the game folder has a known extension for EmulationStation (.svm for example) then at launch %ROM% will be the folder itself in runcommand. Therefore, by adding /*.svm, runcommand will build the final cmdline with any .svm FILE inside the rom "folder" to pass to RA.

This works fine as long as there is a single .svm file in the game folder. However I think this can be done more reliable and elegant in runcommand itself.

My proposal is then to make a convention and make runcommand detect if the passed %ROM% is a folder, then search if there is a single file with the same extension as the passed folder inside, and if so, use the folder + file found as the final rom in the cmdline. This change is trivial and should be non-intrusive.

The most important benefit is that this would not force users to rename their scummvm game folders (keeping the two-clicks to launch if they wish), and users that do so would get the "one-click launch" feature. Currently, the hack in the scriptmodule requires that the folders be renamed with the extension of scummvm to be used as roms by ES. I believe some users might not get this at first.

If you believe this is a worthy argument/use case, I can make the modifications to runcommand, test them and send a PR for you to review. Thanks!

EctoOne

@hhromic Would your folder hack work for other systems as well?
I was always a little bit disappointed that I wasn't able to use the same subfolder structure which Kodi has (all game/movie related files in its own folder). That would make rom/asset management so much easier.

pjft

@hhromic I believe Daphne uses folders as well, if I recall correctly, so it might be good for inspiration to see how it is set up? Maybe there's something you can adapt from it.

BuZz

@hhromic I've only just scanned over your msg. Please can you do a PR and we can discuss it there so I don't forget. Need to check some code in regards to your idea.

Btw sorry I never posted my module. I couldn't find it. It's in a backup for sure, but don't think it's much use now anyway. It's pretty basic.

BiZzAr721

@hhromic From my understanding, those are just optimizations made to the make file which was made using make -j 4, and could possibly be applied using a patch, still building from source. Script-Fu master, @cyperghost may be able to give further insight. Maybe there is a simple solution to get games to load using the same files/location as the stand alone ScummVM.

I looked into the GL4ES, and I think the libretro ScummVM is already using it, therefore can use some of the advantages of OpenGL 2 (Where-as the standalone ScummVM runs off SDL). I don't know if additional dependencies are required.

Great to hear you may have solved the Freeze-on-Quit bug! I thought I finally had everything all set up just right, but now I'll have to give lr-scummVM a try!
Thank you.

hhromic

@EctoOne said in lr-scummvm: request for comments and testing:

@hhromic Would your folder hack work for other systems as well?
I was always a little bit disappointed that I wasn't able to use the same subfolder structure which Kodi has (all game/movie related files in its own folder). That would make rom/asset management so much easier.

What I'm proposing is not strictly tied to lr-scummvm, yes. However I'm not sure what you meant. To clarify, I'm proposing for runcommand to be able to do the following:

If the passed rom is a folder and the name has an extension, then search inside the passed folder for a single file with the same extension. If found, then use the combination of "passed folder/found file" as the rom to launch.
Otherwise, just behave like it is doing currently and do not transform the rom passed.

This would allow everything to work as it is doing currently, but in addition will allow for lr-scummvm games to be launched directly with one click from ES if the user suffixes the game folder names with the ".svm" extension. If not, then it will still work but the user will need to enter the folder first.
If other emulators support running roms in folders in this same way, then it should work too.

@pjft said in lr-scummvm: request for comments and testing:

@hhromic I believe Daphne uses folders as well, if I recall correctly, so it might be good for inspiration to see how it is set up? Maybe there's something you can adapt from it.

I will take a look, thanks for the pointer!