Retropie Installation on Ubuntu Server x64 18.04.1

[johnodon]

@MisterB said in Retropie Installation on Ubuntu Server x64 18.04.1:

Take a look at the suggestions here:

https://askubuntu.com/questions/772874/how-to-turn-off-the-filesystem-check-message-which-occures-while-booting

I'm wondering if a line like this might suppress both messages:
GRUB_CMDLINE_LINUX_DEFAULT="quiet loglevel=3 fsck.mode=skip splash vt.global_cursor_default=0"

Thanks. I'll give that a try in a few mins.

[MisterB]

Bad copy/paste above...of course, remove the splash in the above line for your use case.

[johnodon]

@MisterB said in Retropie Installation on Ubuntu Server x64 18.04.1:

Bad copy/paste above...of course, remove the splash in the above line for your use case.

I saw that. :)

So, this works well. The only thing I had to do was set loglevel=0. If I set it to 3, I would see something flash quickly (in red)...to fast to read. Setting to 0 I see nothing.

Thanks for pointing me in the right direction!

John

[MisterB]

How about loglevel 2 (CRITICAL)? Curious if there is a minimum threshold that hides everything for 'normal' use, but can still show a truly critical system error if it occurs.

[johnodon]

@MisterB said in Retropie Installation on Ubuntu Server x64 18.04.1:

How about loglevel 2 (CRITICAL)? Curious if there is a minimum threshold that hides everything for 'normal' use, but can still show a truly critical system error if it occurs.

I'm reimaging right now. Give me 30 mins and I'll report back.

[johnodon]

@johnodon said in Retropie Installation on Ubuntu Server x64 18.04.1:

@MisterB said in Retropie Installation on Ubuntu Server x64 18.04.1:

How about loglevel 2 (CRITICAL)? Curious if there is a minimum threshold that hides everything for 'normal' use, but can still show a truly critical system error if it occurs.

I'm reimaging right now. Give me 30 mins and I'll report back.

OK...so it looks like loglevel 2 is hiding those messages. However, on the first reboot after running the script, I am seeing a single red 'FAILED' message. It happens so fast I can't read the rest and it doesn't happen on subsequent reboots. I tried looking in dmesg but didn't see anything I didn't already mention.

Where is the best place to look for that error after the first reboot? I'd like to see what it is about and if it is script related.

John

[johnodon]

Quick update...

I am seeing the same longer boot time behavior on my Lenovo T430 as I did on my other system. If I remove splash from the the grub line, boot time is shortened by about 20 seconds.

John

[johnodon]

@MisterB I think it may be close to time to start a new thread that is dedicated to your install scripts and just reference this one for posterity.

What do you think?

John

[etheling]

@MisterB Just wanted to chime in to say that I modified my pre/post scripts for Intel NUC to run from under optional_scripts/ and it worked out beautifully - in one swift run the pre-install part drops in new kernel, wpa_supplicant, and connects NUC to wifi and then proceeds with the rest of the install.

Couple of observations:

• I had edited my pre-install script under optional_scripts/pre_install, so I had preinstall.sh~ there as I executed main install script. Which then ran the preinstall script, and broken preinstall.sh~. Maybe filter out files with ~ at the end, or only run files with .sh extension?
• Would it make sense to run inxi -F at the end of the script just before reboot (and maybe some other utilities) to 'document' into install log hw etc. configuration?
• I think unrelated to your script, but I had to recompile xpad afterwards to fix button mappings at the controller. E.g. select, start and others were not reporting at correct values when looking /dev/input/js* with jstest. I'll look into this maybe next weekend to see if it reproduces.
• Unrelated to your script, but lr-puae doesn't install currently. It compiles ok, but there is no README file under /home/pi/RetroPie-Setup/tmp/build/lr-puae/ which breaks the install ('touch /home/pi/RetroPie-Setup/tmp/build/lr-puae/README' works around it - see here).

As for my post install scripts, I split my post install scripts to two parts, one that is specific to NUC and another that (I think, didn't test yet) will also work on Pi that sets up themes, overlays, emulator specific retroarch congifs, core overrides, sets up retroachievements.org account, etc.

Thanks again for all the work on the script. Very much appreciated!

[johnodon]

WIP... :)

I'm currently working out the code to separate pre-install and post-install package options.

(Radio buttons...only one selection allowed)

(Checklist...multiple selections allowed)

[etheling]

@etheling said in Retropie Installation on Ubuntu Server x64 18.04.1:

... but I had to recompile xpad afterwards to fix button mappings at the controller. E.g. select, start and others were not reporting at correct values when looking /dev/input/js* with jstest. I'll look into this maybe next weekend to see if it reproduces.

This reliably reproduced in different ways when my install order was: 1) pre-instal kernel , wpa_supplicant, start net, 2) main script, 3) post installs. I don't know what exactly is/was causing the problems but what seems to make things work smooth is to install custom kernel last (or after xpad has been installed, I suspect). So having modified my install order to 1) pre-instal wpa_supplicant & start net, 2) main script, 3) post installs (including install new kernel) installs/confiugres everything in a single smooth run. :)

[etheling]

@MisterB One thing I realized is that in function set_resolution_grub the script sets GRUB_GFXMODE=1920x1080x32,auto. It appears to me though that this (even when augmented with GRUB_GFXPAYLOAD_LINUX=keep) does not make kernel actually keep resolution that grub changes to (as per GRUB_GFXMODE), and at least with my NUC, kernel always switches back to 4k as it boots up.

I tried vga=ask, and vga=... kernel parameters as suggested by some not so recent docs, but alas, I think these are for some legacy use cases.

What worked for me though was video= kernel parameter. It's documentation leaves something to be desired, but ArchLinux had some useful documentation. Long story short: in my case with 8th Gen NUC, the kernel boot option for 5.8.x series kernel is just video=1920x1080which puts fb/console to that resolution during boot/before X is tarted (and especially after I exit X).

I think it might be useful to include video= to the script, but then again, I am not sure how well this works across different display graphics chipsets/hw and how portable that syntax is. Just wanted to mention this so it get's 'documented' here in case someone else wonders about this.

[etheling]

EDIT: See post 175 for most recent version of the script.

EDIT: See post 162 in this thread for updated version of the script. I updated this post to refer to updated script.

@MisterB , I did some changes to your script (see updated script here) that allows me to start ES and RetroArch in Linux KMS/DRM mode. This way everything (?) can be run without X (or Wayland). Not only this may be slightly more performant, it also seems to be a simpler approach with fewer moving parts (and is pretty close to how RetroPie runs on Pi).

I tested it to successfully boot to ES in KMS/DRM mode on 8th gen NUC, and in VM using 20.04 LTS default (5.4.0-42) kernel (e.g. I'm not thrilled about having to update to latest mainline with the NUC). With limited testing, retroarch cores and e.g. scummvm seem to work just as well as before.

There wasn't much information online that I could find about running RetroPie on PC/Linux this way. So maybe there is something I missed that makes this not such a great idea?

Anyways; here is a summary of changes I did in order of reading down the install script (in linked install script, search '# Eth:' to locate them).

1. Added fbset to RETROPIE_DEPENDS. This is used by /opt/retropie/supplementary/runcommand/runcommand.sh and was missing.

2. In install_retropie () patch RetroPie-Setup/scriptmodules/emulators/retroarch.sh (patch), RetroPie-Setup/scriptmodules/supplementary/sdl2.sh (patch), and RetroPie-Setup/scriptmodules/helpers.sh (patch) to enable KMS/DRM (on x11 platform). In particular I am curious about helpers .sh, as it appears that due to some conflict in Ubuntu 19.04, it wouldn't allow compiling (RetroPie copy) of SDL2 (recompiling is needed to enable KMS/DRM support). Also, is there a reason why kms/drm flags for retroarch and sdl2 have not been enabled for x11 platform?

3. In hide_boot_messages()set kernel command line to quiet vt.global_cursor_default=0 video=1920x1080. E.g. remove splash, add video. ES behaved poorly if I tried to start it with console at other (larger than 1080p) resolutions, even when using --resolution 1920 1080 parameter. See my earlier email about the video= parameter and that it maybe should use more complete syntax. But this seems to work, so... Also, I removed plymouth splash, as boot is now so fast that pacman pops up for a mere second or two (especially on NUC).

4. Add enable_autostart_emulationstation () to the script to create .bash_profile launch emulationstation from .bash_profile (instead of startx) and run it instead of enable_autostart_xwindows().

5. In fix_quirks() add $USER to the input group to allow /dev/input/* access (needed for keyboard to work in KMS/DRM mode)

6. In fix_quirks() allow fbset to be executed by non-root users. Fbset is used by /opt/retropie/supplementary/runcommand/runcommand.sh and it errors without this change.

7. In fix_quirks() add audio_driver = "alsa" to /opt/retropie/configs/all/retroarch.cfg to make audio work in KMS/DRM mode. I'm not sure about this one if it's absolutely required, but for me to get audio, I had to add that.

8. Comment out / don't run: enable_plymouth_theme "retropie-pacman", enable_autostart_xwindow, hide_openbox_windows, autostart_openbox_apps, set_resolution_xwindows "1920x1080" and add/run enable_autostart_emulationstation after enable_autologin_tty.

And that's it... I think. :)

Lastly some random notes:

• Even if RetroPie now runs without X, I think it's a good practice to install X for a few reasons and if for nothing else than to allow running multiple Emacses, or more seriously, I want to have ability to run X to launch gog.com games via wine / .xsession (I'll put how I am doing that to another post at later time), and it enables using browser via ports that @johnodon mentioned earlier.
• With vmware fusion 11.5.6 I had to use legacy bios to have 1920x1080 resolution available for grub / linux (e.g. /w UEFI videoinfo reported ~1024x768 etc. as a max)
• With vmware fusion, for some reason 1080p wasn't available for retroarch cores, so I had to hit 'a' to change default resolution to something thats available.. I thought this was odd.

It would be great if someone can test this on real hw and report back. I am curious about how 'portable' this is. Especially to non-Intel graphics chipsets.

What are your thoughts about this approach over using X? Maybe keep X as a default, but if starting the script with --kms etc. parameter, set-up RetroPie using KMS.

[praetorian55]

@etheling

Awesome work! I didn't test the entire script since I had already installed on my machine, but was eager to test the KMS/DRM support. I'm using an atom z8350 (Intel CherryTrail) and your changes to get emulation station to run in KMS/DRM mode definitely worked.
I'm fighting resolutions settings on a 4k TV right now so I hope to be able to go through some of your other settings as well to see if I can get that taken care of. Unfortunately I'm buried at work and probably won't be able to spend to much time on it anytime soon - I was just really excited someone figured out how to get KMS/DRM working!

Edit: was able to use your grub/kernel modification to tame the resolution to a perfect 1920x1080. That's a setting I've been struggling to find for awhile - thanks!

[etheling]

@praetorian55 Thanks! Great to hear! :)

One fragile part here is SDL2 because it looks like RetroPie installs/compiles it's own version and then holds the package (via echo "libsdl2-dev hold" | dpkg --set-selections) to prevent OS replace it. Alas, installed version is not new enough for e.g. winehq which then refuses to install. It looks like RetroPie's 'sdl2. sh' would allow installing a newer version of SDL2, but with a quick test it completely broke the install (presumably I either fat-fingered something or didn't take time to understand how the script logic works...I'll try again later).

I didn't really thought this through (like at all), but could RetroPie use local copy of SDL2, and statically link emulationstation and RetroArch against it when compiling? (whereas now it's creating and installing custom .deb package, that OS is trying to fight off). (edit: see post 162 in this thread, dynamic linker is way to go)

@MisterB - I'll experiment with SDL2 a bit and will post an updated script maybe over the weekend. I have couple of other improvements on my mind that I will add too (on which topic, we should add also GRUB_GFXPAYLOAD_LINUX="keep" to /etc/default/grub - it will avoid mode switch when Grub loads the kernel).

[etheling]

I spent some time trying to reduce video mode switching throughout the boot and until ES has loaded. Because when mode switches occur, my TV blanks out for a second or two displaying it's informational dialogs, which isn't particularly enjoyable (I know, I should get a better, and bigger one ;-) ).

Roughly speaking mode switches happen at 1) POST / BIOS initializes, 2) GRUB initializes, 3) grub exits and starts loading kernel, 4) kernel initializes graphics, 5) xinit/X starts loading, 6) X switches to user configured resolution (and 7) retroarch etc. changes to non-1080p custom resolution). Especially 1) and 5) appear to have tendency to try and switch to highest resolution supported by monitor. It looks like plymouth themes generally won't change resolution, but then again they are scriptable.

With Ubuntu 20.04 on PC, we have at least these mechanisms to control mode switches at above phases:

1. May or may not be configurable in BIOS settings. Most likely not.

2. set via /etc/default/grub:GRUB_GFXMODE= (and available modes found via vbe_info, or videoinfo in grub shell)

3. set via /etc/default/grub:GRUB_GFXPAYLOAD_LINUX="keep" - will retain display mode from 2)

4. setting video= kernel boot parameter in/etc/default/grub seems to do the trick here. Hard to tell how stable/portable this is.

5. Set resolution in '/etc/X11/xorg.conf' to control initial X resolution switch (instead of X selecting highest offered by monitor).

6. xrandr or other window manager specific mechanisms to set user preferred resolution

Ideally only one mode switch should happen, between 1) and 2), down from worst case six. However above mechanisms don't give exact control to available resolutions / bit depths / refresh rates / etc., so it's still not guaranteed that mode switches don't occur. Perceptually still, it feels nicer now. Also, this seems to work with Intel graphics chipsets (and with VMs), but I think especially 4) may need extra attention on e.g. AMD or nVidia cards.

As a side note - I have a 4k monitor that doesn't support 1080p for 2) and 3), so either I will get additional mode switch between 3) and 4), or I will keep initial 4k until 4) where I will switch to 1080p.

I'll add 3), and 5) to the script, but here is /etc/X11/xorg.conf I am using now:

## Set initial X startup resolution to 1080p                                                                                     
## URL: https://wiki.ubuntu.com/X/Config/Resolution                                                                                                                                                                                                                                             
Section "Monitor"                                                                                                                                   
        Identifier      "Configured Monitor"                                                                                                        
EndSection                                                                                                                                          
                                                                                                                                                    
Section "Screen"                                                                                                                                    
        Identifier      "Default Screen"                                                                                                            
        Monitor         "Configured Monitor"                                                                                                        
        Device          "Configured Video Device"                                                                                                   
           SubSection "Display"                                                                                                                     
                Virtual 1920 1080                                                                                                                   
           EndSubSection                                                                                                                            
EndSection                                                                                                                                          
                                                                                                                                                    
Section "Device"                                                                                                                                    
        Identifier      "Configured Video Device"                                                                                                   
EndSection                                                                                                                                                                      

[etheling]

EDIT: See post 175 for most recent version of the script.

I tinkered with the KMS/DRM setup and I think I am now at a pretty good place with it. Here is updated install script. It has few changes over the previous one, main goal being as few changes to RetroPie setup scripts as possible (and of course stable / working installation).

1. Added libdrm-dev libgbm-dev to RETROPIE_DEPENDS. These I think were previously installed by libsdl2 compile, but since libsdl2 is now compiled later these are needed to compile RetroArch with '--enable-kms --enable-egl' flags added.

2. Now we only needs to patch retroarch. sh under 'Retropie-Setup/' to enable KMS/DRM mode (e.g. installation now uses unmodified RetroPie helpers. sh, and sdl2. sh)

3. Added install_local_libsdl2_with_kmsdrm() - instead of RetroPie SDL-Mirror / package approach, compile a local copy of latest libsdl2 with '--enable-video-kmsdrm' to under /usr/local/lib and configure dynamic linker to make this copy available to ES, RetroArch and others by default (side benefit here is that it's now really easy to test impact of using different versions of libsdl2). There are some cleanups that could be done for this function, but it works for 20.04 so maybe in the future.

4. Added install_xorgconf() to install /etc/X11/xorg.conf that sets initial X resolution to 1080p

5. Added GRUB_GFXPAYLOAD_LINUX="keep" to /etc/default/grub in set_resolution_grub()

6. See post 158: changes 1...8 are still needed (sans patches to helper/sdl2 .sh).

Using local copy of libsdl2 seems to be the way to go here. That way there is no need to deal with a custom libsdl2 package that I felt OS tried to fight off. Furthermore, for some reason (didn't diff the sources) RP mirror of libsdl2 does something which causes gamepads not to work under EmulationStation on Ubuntu in KMS/DRM mode (e.g. evtest shows events coming in, but somehow they don't reach ES). Maybe someone familiar with what RetroPie does with libsdl2 can chime in here about this.

What's nice is that with these changes, RetroPie can be started in KMS/DRM mode, but it also doesn't prevent launching it via X! But if I may say so, I really like the snappiness of the setup when running in KMS/DRM mode - whether perceived or real! ;-)

@MisterB - I think this is it for the KMS/DRM updates from me. Other than small improvements to install_local_libsdl2_with_kmsdrm(), I don't think I'll send other updates that could go to your script soon. What's your thoughts of integrating these changes?

[MisterB]

While I haven't tested these changes, they look great, and I appreciate all the work here.

What we're evolving into here is a couple different flavors of the 'core' installation process, plus the ability to extend with optional feature scripts. I'm wondering if it makes sense to extend the optional script approach to the core functions as well. This would allow us to create "recipes" that package the individual core functions to create specific setups, such as KMS vs X11, rather than directly tweaking/branching the RetroPie_setup_ubuntu.sh script.

Not sure how quickly I can turn it around, but thoughts on this approach?

[etheling]

@MisterB Thanks :) My current feeling is that maybe try to avoid adding complexity to the script (which is what adding 'recipes' support sounded a bit like). It is very readable and editable in it's curent form. But let me sleep over it a bit. :)

I wonder if RetroPie could by default enable '--enable-kms --enable-egl' and libdrm-dev libgbm-dev depends for the retroarch. sh when building on x11 'arch'. If they would do that, then almost the only difference between X or KMS/DRM flavors would be what is written to .bash_profile (exec openbox vs. emulationstation). And of course if it would be X/openbox flavor, then there would be no need to compile local sdl2 and change audio device to alsa. But all the other changes should be ok for both I think.

I guess I could post and ask about enabling those flags to RetroPie Ideas and Development forum and ask - I'll do that tomorrow. edit: posted.

[johnodon]

@etheling said in Retropie Installation on Ubuntu Server x64 18.04.1:

I tinkered with the KMS/DRM setup and I think I am now at a pretty good place with it. Here is updated install script.

I'm having my first go at this and it is FANTASTIC! There is no gnome terminal window to speak of. Normally I would be able to ALT-TAB to the terminal but it isn't even there! :D