moonlight: request for comments and testing

hhromic

@jml I see, then is very weird what is happening as both files seem normal. Can you please get the log in /dev/shm/runcommand.log for each case? Just start the session, try the game, close the streaming session and then grab the log.
Also, does this happen only with The Witcher 3? Thanks for testing!

seriously24

Hi,

So I just tried using the script today to redo a full installation of RetroPie 4.4 and Moonlight on Raspberry 3, and gotta say that everything worked like a charm.
The script is installed super-easily, and runs without issue. I got moonlight installed and (some of) my games automatically imported in a breeze. A few improvements/remarks though :

Only bug I am experiencing : I use moonlight to do some Remote Desktop too (using mstsc.exe as an app on available Shield applications), and once I quit it by doing Alt+Ctrl+Shift+Q, and when I go back to my pc, all the controls are messed up. My PC keyboard doesn't work anymore and the mouse buttons are acting weird. Also the sound doesn't work as it stays on "Nvidia Speakers", instead of my "Realtek high Definition" drivers. Maybe some controller mapping that is not correctly refreshed ?
Not sure if it is in the gui config, but it would be great to have a global.conf "autoconfigurator". By that I mean, at the end of the installation, or in the GUI it would be nice to have something where you can say "Ok I want 1920x10806fps" fast and easy. Not critical though since you can do this by editing the conf file manually. But still, would be cool to have.
Finally, one concern about the actual script. I don't know how the moonlight-embedded github is moderated in terms of merges in the default branch, but using it as the source of this script by default might be a quite "ballsy" move, as you may sometimes fetch buggy code. Also, it will make it really hard for anyone using this script to debug their bugs, as they will have no idea which version of moonlight-embedded they have fetched. Keeping an option in the script to compile from last sources known is super cool, and it is really helpful right now as there is no official release of moonlight-embedded to match the current Geforce Experience version. But I don't think it should be the default choice, or at least let people choose if they want the last release known from apt-get (or the last release on the git repo)

But really, awesome work man, it's a really great tool and works perfectly well ! Will surely continue using and keeping an eye on its development.

hhromic

@seriously24 first, thanks for testing! I'm glad the scriptmodule works good for you.

I got moonlight installed and (some of) my games automatically imported in a breeze.

You are having issues when autogenerating the config files for some games? Which ones? Can you describe what happens? Errors?

Only bug I am experiencing : I use moonlight to do some Remote Desktop too (using mstsc.exe as an app on available Shield applications), and once I quit it by doing Alt+Ctrl+Shift+Q, and when I go back to my pc, all the controls are messed up. My PC keyboard doesn't work anymore and the mouse buttons are acting weird. Also the sound doesn't work as it stays on "Nvidia Speakers", instead of my "Realtek high Definition" drivers. Maybe some controller mapping that is not correctly refreshed ?

Good you mention this! I also experience the same issue myself. The problem is that when quitting moonlight like that, it simply finishes the streaming session on its side immediately and terminates. Then, is up to the GFE server to clean-up the host computer and this is not happening gracefully.

To gracefully exit, the moonlight client instead has to send a request to quit packet to the server. This is what the quit option does in moonlight, for example with /opt/retropie/supplementary/moonlight/moonlight.sh quit.

Of course we don't want to be opening a console to write that command every time we want to quit, therefore I created a patch that adds a new key combo (Alt+Ctrl+Shift+X on keyboard or Play+Back+LThumb+RThumb on gamepad) that sends said request from the session. When GFE finishes processing it, including freeing the gamepads and sound redirection, then moonlight gets a remote disconnection and quits gracefully too.
So far my patch is working fine for me here locally, but I wanted to do some more code cleaning before putting into this scriptmodule for testing. So just a bit of patience and you will be able to test it yourself.
I already sent part of the patch to the moonlight repository, and eventually I want to send it entirely. However the maintainer doesn't seem very active so can take a while.

Not sure if it is in the gui config, but it would be great to have a global.conf "autoconfigurator". By that I mean, at the end of the installation, or in the GUI it would be nice to have something where you can say "Ok I want 1920x10806fps" fast and easy. Not critical though since you can do this by editing the conf file manually. But still, would be cool to have.

That is a good idea indeed, I will try to implement it in the GUI somehow, thanks!

Finally, one concern about the actual script. I don't know how the moonlight-embedded github is moderated in terms of merges in the default branch, but using it as the source of this script by default might be a quite "ballsy" move, as you may sometimes fetch buggy code. Also, it will make it really hard for anyone using this script to debug their bugs, as they will have no idea which version of moonlight-embedded they have fetched. Keeping an option in the script to compile from last sources known is super cool, and it is really helpful right now as there is no official release of moonlight-embedded to match the current Geforce Experience version. But I don't think it should be the default choice, or at least let people choose if they want the last release known from apt-get (or the last release on the git repo)

The upstream moonlight-embedded repository doesn't seem very active and I don't see it breaking easily as the maintainer doesn't merge things without reviewing. On the other hand, a lot of scriptmodules in RetroPie are designed to pull stuff from master branches, and things do break from time to time, yes.
I agree with you that this approach is fragile compared to keeping versioned artifacts. However RetroPie maintainers are pretty fast fixing up things and honestly puling from GitHub instead of installing debs is much easier to maintain.
Also, as you probably noticed, the moonlight maintainer seems to not be very active on refrshing the deb packages either, so another reason to pull from source instead. Having said all that, I strongly believe we should keep it like this.

However, If I notice that the moonlight repository starts to get unstable often, I can easily update the scriptmodule to pull from known-to-be-good commit hashes instead of HEAD in the future. Thanks for the suggestion!

Thanks again for testing and let me know of any issues you experience, the more feedback the more we can iron issues!

seriously24

@hhromic said in moonlight: request for comments and testing:

You are having issues when autogenerating the config files for some games? Which ones? Can you describe what happens? Errors?

I didn't get any errors or anything, I just don't understand the list of games it imported. On the Geforce Experience interface, the "discover games" button finds no more than 5 games, when your script finds 10ish. Though, I have 25-30 games installed on Steam. For example, Hotline Miami 2 was imported correctly, but Dragon Ball FighterZ wasn't.
It's not really a big deal, as I can add them manually the same way I added the Remote Desktop app, I just don't understand how it works :)

So far my patch is working fine for me here locally, but I wanted to do some more code cleaning before putting into this scriptmodule for testing. So just a bit of patience and you will be able to test it yourself.
I already sent part of the patch to the moonlight repository, and eventually I want to send it entirely. However the maintainer doesn't seem very active so can take a while.

Be sure that I can't wait to see and use it.

By playing a bit today with Retropie and Moonlight, testing things and stuff, I saw that the games in the Steam menu in Emulation Station are only added, never removed. I modified the name of my "Remote Desktop" app in the Geforce Experience, but it only got added as a new game when I relaunched a new "autodiscover of games" through the GUI. Is this on purpose ?

Finally, I got a bit overconfident about all this and tried to use my HORI Real Arcade Pro N on the RetroPie, and it surprisingly was recognized, but only as a Generic Xbox Controller. The input configuration and jstest works amazingly well, but on moonlight I get really messed up controls. My "R2" button becomes start button and things like this. Any advice on how I could investigate this further ? Or maybe an idea about how/why it fails ?

Edit : Doing some research on moonlight-embedded github, it looks like it uses a default mapping file which probably doesn't work for the stick, recognized as an Xbox controller. Will try to use the -mapping or a custom gamecontrollerdb.txt

Edit2 : It worked with a custom-made game controllerdb.txt. Tricky part was to find the GUID of the joystick, but it now works like a charm with this workaround.

hhromic

@seriously24 thanks for all the testing! sorry for the late response, I've been a bit busy nowadays.

I didn't get any errors or anything, I just don't understand the list of games it imported. On the Geforce Experience interface, the "discover games" button finds no more than 5 games, when your script finds 10ish. Though, I have 25-30 games installed on Steam. For example, Hotline Miami 2 was imported correctly, but Dragon Ball FighterZ wasn't.

It's not really a big deal, as I can add them manually the same way I added the Remote Desktop app, I just don't understand how it works :)

It is very simple actually. The script uses moonlight's list command to fetch available games from the host computer. Note that these games are the ones configured in GFE, not Steam. GFE should add supported games automatically and expose them to moonlight. That's all concerning the script.
If the script is adding games that are not in GFE's user interface, then there is something weird indeed. Just for debugging, can you execute:

/opt/retropie/supplementary/moonlight/moonlight.sh list

And verify the list you obtain there is the same list generated by the script? If the list doesn't match your installed games in GFE, then you have a further issue on the host computer. If the list doesn't match the generated games, please post the list here so I can take a look. Thanks.

Be sure that I can't wait to see and use it.

I will add it soon to the current script for you to test. Will update this topic when is there. Thanks for the patience!

By playing a bit today with Retropie and Moonlight, testing things and stuff, I saw that the games in the Steam menu in Emulation Station are only added, never removed. I modified the name of my "Remote Desktop" app in the Geforce Experience, but it only got added as a new game when I relaunched a new "autodiscover of games" through the GUI. Is this on purpose ?

Yes, this is on purpose. The reason is that users might have done customisations to these config files, hence it is best to not blindly overwrite them or delete them on game refreshes. Therefore, the game discovery mechanism only adds new games and does not change any existing config file. Currently, if you want to do a complete re-sync, you should manually delete all the files in the steam rom folder.
However I understand your use case too. Perhaps I can add a GUI option to do this so you don't need to get a console or external computer to do this. Will think about it.

Edit : Doing some research on moonlight-embedded github, it looks like it uses a default mapping file which probably doesn't work for the stick, recognized as an Xbox controller. Will try to use the -mapping or a custom gamecontrollerdb.txt
Edit2 : It worked with a custom-made game controllerdb.txt. Tricky part was to find the GUID of the joystick, but it now works like a charm with this workaround.

Yes, exactly. It uses a default shipped controller mappings file. This script installs it under /opt/retropie/supplementary/moonlight/share folder. However, I don't recommend changing that file because it will be re-installed on updates.

Instead, the recommended way is to create your own custom controller mappings file (as you already did) and place it somewhere handy (for example in /opt/retropie/configs/all/moonlight and then reference it in the global.conf file that is also in that recommended folder using the mapping option. When doing this, please use the full path to the mappings file in the option, even if the global conf file and the mapping are in the same directory.

Using this approach, all your autogenerated game files (current and future) will benefit from this configuration automatically as every config file loads the global conf by default.

Thanks for all the feedback and the testing! I will think about the suggestions and incorporate them in the next update. Happy game streaming!

LeSabotageur

I have a general question about moonlight : how can I prevent any controller plugged to the pi of being used ? As told earlier, I'm using VirtualHere and too often moonlight maps a controller on the pi, therefore screwing up my configuration. I've not found a reliable solution yet (commenting *.dll files in the Gamestream folder doesn't work), so I'm wondering if anyone here knows ?

hhromic

@LeSabotageur try adding input = /dev/null to /opt/retropie/configs/all/moonlight/global.conf.
This should make moonlight only use a null device as sole input and therefore should ignore all real controllers connected to your Pi locally.
I haven't tested it myself (not at home at the moment), but I suspect it could work. Please let us know if it worked.

Also, for this option to work, you should be using the autogenerated game config files that include the line config = /opt/retropie/configs/all/moonlight/global.conf to include the global conf.
As an alternative you can include the input line directly on your config files, but I think the global option makes it easier and more tidy to configure.

Silent

Generally it seems to work flawlessly! Moonlight itself is working OK but until now I've been using a different setup to get it running with RetroPie - that is, a modified retromoonlight. Your approach seems better however, most notably because it actually uses runcommand, unlike retromoonlight.

Two things to note:

Generated filenames still contain : in them, which greatly confuses Windows when previewing roms\steam directory from a network share. Maybe cut them, like retromoonlight does?
Some games listed by Moonlight have ™ (maybe also © or ®) in names and this confuses some other software. Maybe they also could be omitted from filenames.

LeSabotageur

@hhromic sadly, it doesn't work

Executing: /opt/retropie/supplementary/moonlight/moonlight.sh stream -config "/home/pi/RetroPie/roms/steam/Steam.ml"
No mapping available for  (00000000000000000000000000000000) on /dev/null
EVIOCGRAB failed with error 25
libevdev error in libevdev_next_event: BUG: device not initialized. call libevdev_set_fd() first
Error: Bad file descriptor

And then it goes back to retropie.
That's not a major issue, though, as I have some workaround for it (Steam allowing to change the order of the controller for instance. That would have been a quality of life improvement :)

seriously24

@LeSabotageur I am no specialist here, but the

No mapping available for  (00000000000000000000000000000000) on /dev/null

awefully looks like something similar than what I had a few posts ago. Have you tried the solution from @hhromic to use a custom controller mapping file ?

@hhromic said in moonlight: request for comments and testing:

Instead, the recommended way is to create your own custom controller mappings file (as you already did) and place it somewhere handy (for example in /opt/retropie/configs/all/moonlight and then reference it in the global.conf file that is also in that recommended folder using the mapping option. When doing this, please use the full path to the mappings file in the option, even if the global conf file and the mapping are in the same directory.

In this custom controller mappings file (source can be found here ) you can try adding a new line with ID 00000000000000000000000000000000, the name of your controller and finally the mappings of the buttons. Since you don't want to use this controller, I guess the mappings don't matter, so just copy the mapping of another controller.
Once the file is ready, to use it simply add a mapping = <yourfile> command in your .ml files.

Hope this helps :)

MightyHandy

@hhromic - this is awesome! Given that there is now a brand new experimental "steam-link" package... would it make more sense to store the ml files under:
/roms/moonlight
In order to eliminate the ambiguity.

Silent

@MightyHandy said in moonlight: request for comments and testing:

@hhromic - this is awesome! Given that there is now a brand new experimental "steam-link" package... would it make more sense to store the ml files under:
/roms/moonlight
In order to eliminate the ambiguity.

I don't agree. By using 'steam' you're playing nice with scrapers and ES themes (lots of them have a 'steam' theme). Moonlight is just an "emulator" used to play those, steam is "platform".

MightyHandy

Tested width, height and FPS options and worked great. Is there a list of other options? You listed examples of app, controller config too. Was gonna check vsync.

MightyHandy

@MightyHandy nevermind... found this:
https://github.com/irtimmer/moonlight-embedded/wiki/Usage
I assume you are just passing the options through.

hhromic

@Silent said in moonlight: request for comments and testing:

@MightyHandy said in moonlight: request for comments and testing:

@hhromic - this is awesome! Given that there is now a brand new experimental "steam-link" package... would it make more sense to store the ml files under:
/roms/moonlight
In order to eliminate the ambiguity.

I don't agree. By using 'steam' you're playing nice with scrapers and ES themes (lots of them have a 'steam' theme). Moonlight is just an "emulator" used to play those, steam is "platform".

I agree with @Silent , steam is a defacto standard and that's why I chose to use it :)
And yes indeed I designed the integration for Moonlight to just be an "emulator" for "steam".

hhromic

@MightyHandy said in moonlight: request for comments and testing:

@MightyHandy nevermind... found this:
https://github.com/irtimmer/moonlight-embedded/wiki/Usage
I assume you are just passing the options through.

Almost. The options are actually the ones that can be set in a moonlight.conf file.
The supported options (and their names) can be seen in the template config file from Moonlight here: https://github.com/irtimmer/moonlight-embedded/blob/master/moonlight.conf

hhromic

Hi all, I've been away for some time and I'm still busy nowadays :(
However I haven't forgotten about Moonlight and finishing the scriptmodule to be included in RetroPie officially. I will be working on some extra improvements after all the feedback you have given so far. Thanks for all the input guys, some ideas are very nice!

@Silent said in moonlight: request for comments and testing:

Two things to note:

Generated filenames still contain : in them, which greatly confuses Windows when previewing roms\steam directory from a network share. Maybe cut them, like retromoonlight does?
Some games listed by Moonlight have ™ (maybe also © or ®) in names and this confuses some other software. Maybe they also could be omitted from filenames.

I understand how this is problematic for your use case. However I think I have an idea on how to solve it nicely for both, you who can't use full names and also others (like me) that prefer the full names in filesystems that supports it. To help me (later) develop this, I would be very thankful if you can provide me with the your list of games so I can study what kind of characters are there.

Therefore, can you please send me the output of this command?

/opt/retropie/supplementary/moonlight/moonlight.sh list

Cheers!

Silent

@hhromic said in moonlight: request for comments and testing:

Therefore, can you please send me the output of this command?

1. Spintires MudRunner
2. Run PCSX2
3. Grand Theft Auto Vice City Stories PS2
4. Grand Theft Auto Vice City PS2
5. Grand Theft Auto San Andreas PS2
6. Grand Theft Auto Liberty City Stories PS2
7. Gran Turismo 4
8. Crash Bandicoot N. Sane Trilogy
9. BattleBlock Theater
10. American Truck Simulator
11. Worms W.M.D
12. Portal 2
13. GRID 2
14. Steam
15. Grand Theft Auto V
16. Fallout 4
17. Sid Meier's Civilization V (DX 11)
18. GRID
19. Driver San Francisco
20. DiRT Showdown
21. Left 4 Dead 2
22. Saints Row: The Third (DX 11)
23. Saints Row IV (DX 11)
24. Deus Ex: Human Revolution
25. Hitman™
26. Double Dragon Neon
27. DiRT 3 Complete Edition
28. Hitman 2
29. Hotline Miami 2: Wrong Number
30. Mafia III
31. Overcooked
32. Broforce
33. DiRT Rally
34. L.A. Noire
35. I am Bread
36. Rocket League
37. The Expendabros
38. The King Of Fighters XIII
39. Max Payne 3
40. Euro Truck Simulator 2
41. DiRT 4
42. Agents of Mayhem

Silent

Also something I am not sure which was brought up in the past, but I think it is worth to broadcast a moonlight quit message on closing Moonlight emulator, so killing the session on pi also shuts it down on host.

What I did was add this to runcommand_onend.sh:

#!/bin/bash
if [[ $2 == 'moonlight' ]] ; then
	/opt/retropie/supplementary/moonlight/moonlight.sh quit
fi

Also, Moonlight works great as a PS2 "emulator" too, by streaming PCSX2 games from the PC =) GeForce Experience preserves shortcut arguments for shortcuts you add, so it's just a matter of preparing proper ones for PCSX2 games and adding them.

hhromic

@Silent yes I'm very aware of that "issue" and it was briefly mentioned before. This happens to me as well and I understand how annoying it can be, specially when you stream the "desktop" and the computer is left with the audio unrestored. I have seen this reported multiple times in multiple places, so it is not an isolated case.

The "problem" is that when you quit the streaming session, Moonlight simply terminates without any further network messaging. Then, the GFE server on the remote computer never realises that the client vanished. That's why when you send a formal "quit" message (like you do in your "onend" script) it quits gracefully.

In my honest opinion this is an issue that GFE should address because it should be the responsability of the server to cleanup the session if the client dissappears. This can happen on network disruptions or any other cases too. However I don't really expect NVIDIA to change anything.

I've been working on a native solution since some time (I don't have much time nowadays so testing is going slow for me). My solution is implementing a new key combo to send a "quit" request to the server (like you do on your solution) from inside the streaming session but do not quit immediately. Once the server gracefully terminates the session on its side, Moonlight receives the termination packe tand simply quits itself cleanly too. This is the way it should work.

While the solution you post using the "onend" script in runcommand works (and is not a bad temporary fix), I don't think is the proper solution because a race condition is created between terminating the session and sending the quit request. I prefer to do it natively the other way around for robustness.

Currently I have it implemented, compiled and working on my system but I want to clean it a bit more before publishing here for wider testing. Let me know if you are interested on testing, that would speed-up things a lot on the development side! I intend to send this to upstream so it's official.

Thanks for the patience, and be sure I am definitively working on a solution.