moonlight: request for comments and testing

Silent

Now the only outstanding issue is : character being present in generated filenames and Windows not liking that :)

hhromic

@Silent yes I have that written down on my TODO list :)

I've been thinking on the idea of having an option to slugify the filenames when autogenerating the game configs for these users that can't use the full names. I already have a prototype bash function to do this, and this is how your list of games would look like using it:

Spintires MudRunner ==> spintires-mudrunner.ml
Run PCSX2 ==> run-pcsx2.ml
Grand Theft Auto Vice City Stories PS2 ==> grand-theft-auto-vice-city-stories-ps2.ml
Grand Theft Auto Vice City PS2 ==> grand-theft-auto-vice-city-ps2.ml
Grand Theft Auto San Andreas PS2 ==> grand-theft-auto-san-andreas-ps2.ml
Grand Theft Auto Liberty City Stories PS2 ==> grand-theft-auto-liberty-city-stories-ps2.ml
Gran Turismo 4 ==> gran-turismo-4.ml
Crash Bandicoot N. Sane Trilogy ==> crash-bandicoot-n-sane-trilogy.ml
BattleBlock Theater ==> battleblock-theater.ml
American Truck Simulator ==> american-truck-simulator.ml
Worms W.M.D ==> worms-w-m-d.ml
Portal 2 ==> portal-2.ml
GRID 2 ==> grid-2.ml
Steam ==> steam.ml
Grand Theft Auto V ==> grand-theft-auto-v.ml
Fallout 4 ==> fallout-4.ml
Sid Meier's Civilization V (DX 11) ==> sid-meier-s-civilization-v-dx-11.ml
GRID ==> grid.ml
Driver San Francisco ==> driver-san-francisco.ml
DiRT Showdown ==> dirt-showdown.ml
Left 4 Dead 2 ==> left-4-dead-2.ml
Saints Row: The Third (DX 11) ==> saints-row-the-third-dx-11.ml
Saints Row IV (DX 11) ==> saints-row-iv-dx-11.ml
Deus Ex: Human Revolution ==> deus-ex-human-revolution.ml
Hitman™ ==> hitman-tm.ml
Double Dragon Neon ==> double-dragon-neon.ml
DiRT 3 Complete Edition ==> dirt-3-complete-edition.ml
Hitman 2 ==> hitman-2.ml
Hotline Miami 2: Wrong Number ==> hotline-miami-2-wrong-number.ml
Mafia III ==> mafia-iii.ml
Overcooked ==> overcooked.ml
Broforce ==> broforce.ml
DiRT Rally ==> dirt-rally.ml
L.A. Noire ==> l-a-noire.ml
I am Bread ==> i-am-bread.ml
Rocket League ==> rocket-league.ml
The Expendabros ==> the-expendabros.ml
The King Of Fighters XIII ==> the-king-of-fighters-xiii.ml
Max Payne 3 ==> max-payne-3.ml
Euro Truck Simulator 2 ==> euro-truck-simulator-2.ml
DiRT 4 ==> dirt-4.ml
Agents of Mayhem ==> agents-of-mayhem.ml

Note how things like the trademark symbol are transliterated to the string tm. This approach has the benefit of using only characters that are valid almost everywhere, including network shares and Windows filesystems. It also has the benefit of being scraper-friendly. I searched most of the slugified versions of your games in TheGameDB for example and almost everything is found successfully.

What do you think?

Edit: here is another list of games from another user:

Walking Dead ==> walking-dead.ml
Stardew Valley ==> stardew-valley.ml
Sonic and All-Stars Racing Transformed ==> sonic-and-all-stars-racing-transformed.ml
The Elder Scrolls V: Skyrim ==> the-elder-scrolls-v-skyrim.ml
Broforce ==> broforce.ml
Crawl ==> crawl.ml
Steam ==> steam.ml
Hotline Miami 2: Wrong Number ==> hotline-miami-2-wrong-number.ml
Trine 2 ==> trine-2.ml
Terraria ==> terraria.ml
Lara Croft and the Guardian of Light ==> lara-croft-and-the-guardian-of-light.ml
Rogue Legacy ==> rogue-legacy.ml
Metro: Last Light ==> metro-last-light.ml
Soma ==> soma.ml
Steam ==> steam.ml
Warframe ==> warframe.ml
Prey ==> prey.ml
Helldivers ==> helldivers.ml
Resident Evil HD REMASTER ==> resident-evil-hd-remaster.ml
Dishonored ==> dishonored.ml
Minecraft ==> minecraft.ml
Renegade Ops ==> renegade-ops.ml
Day of the Tentacle Remastered ==> day-of-the-tentacle-remastered.ml
Metro: Last Light ==> metro-last-light.ml
Dark Souls ==> dark-souls.ml
Rayman Legends ==> rayman-legends.ml
DuckTales: Remastered ==> ducktales-remastered.ml
Overcooked ==> overcooked.ml
The Witness ==> the-witness.ml
The Expendabros ==> the-expendabros.ml
7 Days to Die ==> 7-days-to-die.ml

Silent

@hhromic said in moonlight: request for comments and testing:

I searched most of the slugified versions of your games in TheGameDB for example and almost everything is found successfully.

If that's the case, then it's fine I guess! What about IGDB and Mobygames? Those are the most likely to be used with PC games.

Also PSA - with a few custom apps added to GFE and moonlight added as a ps2 "emulator" I'm successfully using PCSX2 over moonlight as a PS2 "emulator" :D Thankfully Gamestream allows specifying commandline arguments to shortcuts it's executing, so it's possible to just add multiple PCSX2 shortcuts with different parameters!

I only need to hook up PowerShell's Restart-Service to those shortcuts as my PS3 controller driver on PC doesn't handle disconnecting gracefully.

hhromic

@Silent said in moonlight: request for comments and testing:

If that's the case, then it's fine I guess! What about IGDB and Mobygames? Those are the most likely to be used with PC games.

You can give them a try on their websites search engine. I don't have much time nowadays, hence the delay implementing these things :( If you can confirm these other databases also handle slugified names properly, then I guess this is a solid idea. I don't think they would have any issues as dashes are most of the times ignored by text-based search engines. Beware that these scrapers that rely on checksums, will never work as the ml files never will have the correct checksum as the game they launch.

@Silent said in moonlight: request for comments and testing:

Also PSA - with a few custom apps added to GFE and moonlight added as a ps2 "emulator" I'm successfully using PCSX2 over moonlight as a PS2 "emulator" :D Thankfully Gamestream allows specifying commandline arguments to shortcuts it's executing, so it's possible to just add multiple PCSX2 shortcuts with different parameters!

that sounds great! I'm trying to do sometihng similar with CEMU for the WiiU, however in recent versions CEMU is not working with Moonlight in full screen (all I get is a black screen). I have yet to investigate what's going on, seems to be an OpenGL issue with GFE/Nvidia drivers, not Moonlight itself. DirectX games work fine (e.g. Dolphin for Gamecube/Wii games).

MightyHandy

This is super minor. But, are the uninstall instructions correct with respect to the config directory? Are the config settings in ‘config/all/moonlight’ not ‘config/moonlight’?

hhromic

@MightyHandy you are right! I mispelled the config directory, indeed it is $configdir/all/moonlight. Will correct the OP, apologies!
I hope this didn't get you too many headaches :)

Silent

@hhromic Continuing the discussion about DS3 here:

You're right about the mapping, I don't know why I haven't noticed that! I looked up and this button-based mapping seems to be in sync with an official SDL mapping, so I don't know if the maintainers would be happy with changing it to be axis based. Weird.

EDIT:

This project includes all mappings from the latest SDL_gamecontrollerdb.h. The community db adds a superset of mappings to the official ones, it does not modify them in any way.

If you ever wish to modify one of these mappings, you will need to send a PR directly to the SDL project. The changes will get pulled in the community db when a new SDL release occurs.

EDIT2:
Come to think of it, this specifically would probably be better posted in sixaxis topic, I'm stupid -.-

hhromic

@Silent said in moonlight: request for comments and testing:

so I don't know if the maintainers would be happy with changing it to be axis based. Weird.

In my honest opinion, the database should describe the actual capabilities of the controllers as close as possible and the application decide how to use them (treat axes as buttons for example). In this case, the DS3 controller does have native analog triggers and restricting them in the controller database is just crippling the device for no reason.

Perhaps you can open an issue in gabomdq's repository to discuss this? If eventually they consider this situation and merge it there, then it would be very useful to make Moonlight pull the database from there directly.

Btw, if Steam Link generates a controllerdatabase file for all the controllers you configure with it, then that is good news for you as you should be able to use -mapping /path/to/steamlink/database.txt in the global.conf file of Moonlight and then Moonlight will be always in sync automatically with whatever controllers you configure in Steam Link :)

Come to think of it, this specifically would probably be better posted in sixaxis topic, I'm stupid -.-

Don't punish you too harsh, IMHO you are a very capable user in this forum and provide very useful/insightful feedback and ideas. I still have the slugify for ML filenames idea in the TODO list! Did you have a chance to test the scrapers with these filenames? I hope it works because I don't have better ideas for that :D

Silent

I created an issue to discuss this:
https://github.com/gabomdq/SDL_GameControllerDB/issues/247

Good idea about linking to Steam Link's database! My two PS3 pads are slightly different (one is from a fat PS3, the other is from a SuperSlim one) so I may have to repeat the process for the other pad too.

I have not tested scrapers with these names yet, no. It's worth a try though! It indeed seems to be possible to locate most, and in the worst case Skyscraper's aliasMap can be utilized to provide non-slugified names for games.

I'd say go for it :D

Also about mappings - I removed mappings from /usr/local/ (before I pointed towards my own mappings via mapping) and controllers stopped working. Is this something you updated in scriptmodule and mine predates that change? Because it seems like the one from scriptmodule directory is not being used implicitly by my setup. Do note though that before using Moonlight via a scriptmodule I had a self-compiled version, so there may still be stuff lingering in my pi.

Also, you may want to know that Moonlight gets properly terminated by Multi Switch Shutdown Script, but terminating it this way does not send quit message to the host. I'm not sure if it's a bug or expected behaviour, but I'll just patch up the script on my end to do that for me!

EDIT:
I've been told to submit this as an issue for SDL, but honestly... I'm not sure where :D Any pointers?

hhromic

@Silent said in moonlight: request for comments and testing:

I'd say go for it :D

Done it now. See next post!

Because it seems like the one from scriptmodule directory is not being used implicitly by my setup.

You can actually verify which database is being loaded using the -debug command line switch of moonlight. For example in my setup:

$ /opt/retropie/supplementary/moonlight/moonlight.sh -debug stream
(..)
Loading mappingfile /opt/retropie/supplementary/moonlight/share//moonlight/gamecontrollerdb.txt
(..)

I would suggest you to start from scratch, clean any traces of moonlight and re-install this scriptmodule from zero.

Also, you may want to know that Moonlight gets properly terminated by Multi Switch Shutdown Script, but terminating it this way does not send quit message to the host. I'm not sure if it's a bug or expected behaviour, but I'll just patch up the script on my end to do that for me!

Umm this seems like a bug. If moonlight is terminated using SIGTERM it should gracefully finish (and send the quitapp request). I just tested and is not responding to the SIGTERM, I will debug and submit a PR upstream to fix this.

I've been told to submit this as an issue for SDL, but honestly... I'm not sure where :D Any pointers?

You mean the DS3 mapping correction? SDL indeed doesn't use GitHub and they have their own funky issue tracker (bugzilla). You might need to create an account there and have the patience to fill your issue using that system.

hhromic

(Big) Update

So, I finally got some time in my hands and decided to work on the received feedback to improve the scriptmodule. I re-designed the GUI menu to support most of your requests, including the management of the game config files generation and configuring global resolution and stream bitrate settings.

General Improvements to the GUI

Added support to slugify file names when generating game config files. Slugifying ensures the filenames use valid characters for windows-based filesystems such as FAT32 or remote SMB mounts.
Added support to configure global resolution settings, including standard 1080p30/60, 720p30/60 and also completely custom resolutions.
Added support to configure global stream bitrate settings, including standard 20000, 10000, 5000 Kbps and also completely custom bitrates.
The main menu now has three sub-menus (one for generating game config files, one for resolution settings and a third for stream bitrate settings).
All the menus remember your settings, e.g. the remote address and config files generation settings. This scriptmodule-specific settings are stored in $configdir/all/moonlight/scriptmodule.cfg.

New Main Menu

New Remote Apps Submenu

Overwrite existing files: NO=skip config files already existing, YES=always re-write game config files.
Wipe existing config files: NO=don't touch your files in $romdir/steam, YES=delete all config files before generating new config files. This is useful for a complete re-sync of the games configured in the remote computer.
Config filename mangling: NONE=use the app names directly for the config file names, SLUGIFY=slugify the app names and use that for the config file names, WINDOWS=only use Windows-compatible characters from the app names.

New Global Resolution Submenu

Unset (use default): this setting removes any custom/forced resolution setting.
1-4: set standard resolution setting.
Custom: set any custom resolution (WIDTH x HEIGHT / FPS).

The resolution settings are set in the global config file in $configdir/all/moonlight/global.conf using the width, height and fps moonlight options, therefore they are automatically loaded by all the generated config files. However, if you further re-fine resolutions per-game they take precedence over the global settings.

New Stream Bitrate Submenu

Unset (use default): this setting removes any custom/forced stream bitrate setting.
1-3: set standard stream bitrate setting.
Custom: set any custom stream bitrate (in Kbps).

The stream bitrate settings are set in the global config file in $configdir/all/moonlight/global.conf using the bitrate moonlight option, therefore it is automatically loaded by all the generated config files. However, if you further re-fine stream bitrate per-game they take precedence over the global settings.

To try these new improvements, simple re-patch the experimental scriptmodule. Let me know if these solve your usability suggestions @Silent, @seriously24 !

@buzz I think the scriptmodule is now quite mature and should be fine to integrate into RetroPie. We can finally retire the obsolete limelight module. I will give some extra time for users to test the improved GUI and then submit.

Also, I made two utility functions that maybe are worth promoting to the helpers module: _slugify_moonlight to slugify strings into safe filenames and _bfmt_moonlight to format boolean values into YES/NO for printing. Let me know if you have interest for these and I will move them to the helpers library.

Happy game streaming!

Edit: now that I was at it, added a new submenu to configure stream bitrate settings as well :)
Edit2: improved the filename mangling functionality, added "windows-compatible" option in addition to "slugify".

hhromic

@hhromic said in moonlight: request for comments and testing:

Also, you may want to know that Moonlight gets properly terminated by Multi Switch Shutdown Script, but terminating it this way does not send quit message to the host. I'm not sure if it's a bug or expected behaviour, but I'll just patch up the script on my end to do that for me!

Umm this seems like a bug. If moonlight is terminated using SIGTERM it should gracefully finish (and send the quitapp request). I just tested and is not responding to the SIGTERM, I will debug and submit a PR upstream to fix this.

@Silent I found out the problem now. Indeed it was a bug in the code. I submitted already a PR upstream and once merged, Moonlight will gracefully terminate when receiving SIGTERM signals, for example from the switch shutdown script.
This means it will send the quit-app request on this situation too.

Thanks for reporting this!

Silent

@hhromic Nice updates, thank you! I'll make sure I test them soon, after I've restored the SD card to pre-sixaxis changes (due to new changes I don't want to uninstall, but rather start fresh) and then will give those a spin.

hhromic

@Silent the SIGTERM issue is now fixed upstream, although not from my PR as a better solution was proposed by the author :D.
Anyway, recompile moonlight and it should now gracefully exit from the shutdown script.

Silent

Good! Will try to test it tonight.

Also FYI I also posted an issue about this DS3 mapping on SDL bugtracker, but it's hard to tell when will they react to that.

Silent

@hhromic I just gave the newest scriptmodule a try and slugify option is a bit disappointing - while of course it works as intended and ThegamesDB handles these names relatively well, MobyGames seems terrible at it (so far I have 0 matches for games which names consist of multiple words). Will try IGDB next and report.

I think I'll stick to non-slugified names for now.

BuZz

@hhromic thanks for your work on this. I'm happy to accept this when you are. The old limelight module is so out of date I'm keen to remove it.

In regards to splitting out functions. Certainly may be useful, but I reckon accepting first and can refactor out later if useful for other modules.

Thanks again! Looks excellent.

llamadestroyer

Can be used with steam link? I think they are using same rom folder. Your script works awesome! I asked just for curious.

hhromic

@llamadestroyer said in moonlight: request for comments and testing:

Can be used with steam link? I think they are using same rom folder. Your script works awesome! I asked just for curious.

Yes it can be used at the same time. Steam Link is installed under "ports" (and doesn't use any "roms") while Moonlight is installed as an "emulator" for the "steam" platform (using ".ml" "roms"). These two are 100% independent.

Thanks for the feedback, glad you like the scriptmodule!

hhromic

@Silent said in moonlight: request for comments and testing:

@hhromic I just gave the newest scriptmodule a try and slugify option is a bit disappointing - while of course it works as intended and ThegamesDB handles these names relatively well, MobyGames seems terrible at it (so far I have 0 matches for games which names consist of multiple words). Will try IGDB next and report.

I think I'll stick to non-slugified names for now.

Oh that's a boomer to hear about the slugify solution. May I ask how exactly are you scraping these? I might give it a try myself to sort out what can be further done. Perhaps after slugifying changing "-" to " " might help, but I don't want to introduce more noise to the filename mangling if not necessary. Thanks for testing!