Retropie Installation on Ubuntu Server x64 18.04.1

johnodon

@johnodon said in Retropie Installation on Ubuntu Server x64 18.04.1:

@johnodon said in Retropie Installation on Ubuntu Server x64 18.04.1:

@MisterB said in Retropie Installation on Ubuntu Server x64 18.04.1:

Latest release is published:
https://github.com/MizterB/RetroPie-Setup-Ubuntu/tree/LTS-20.04

I'll take a snapshot of my system today and rebuild from scratch using the new scripts.

John

OK...just ran my first test and submitted my first issue. :)

https://github.com/MizterB/RetroPie-Setup-Ubuntu/issues/7

I'm combing through the logfile now to look for issues.

After the script finished running, I was dropped at a black screen. I had to ALT+F1 to get a prompt and startx manually. I'll keep investigating but this could be the kernel issue referenced earlier.

John

FYI...logfile is nice and clean. Also, upgrading kernel to 5.8.10 fixes my issue with X not starting.

MisterB

FYI - I have promoted all of the 20.04 code to the master branch, and have archived 18.04 to its own branch.

johnodon

@etheling @MisterB @movisman @ObiEric

I think I may have found another fix for the issue of the screen going black after the Plymouth theme (Pacman) and eventually dropping to a prompt. This one doesn't require mucking with kernel updates and is VERY simple...

All I did was add 'exec' before the 'startx' command in the .bash_profile file:

if [[ -z $DISPLAY ]] && [[ $(tty) = /dev/tty1 ]]; then
    exec startx -- >/dev/null 2>&1
fi

This works for me on a vanilla install of Ubuntu 20.04 mini + MisterB's installer script. The only ill effect I have seen so far is that it take 15 - 20 seconds for ES to finally load once past the Pacman theme. I'll continue to investigate to see what is causing the delay.

John

johnodon

OK...figured out what is causing the boot process to take longer than expected.

It is the Plymouth theme.

If I remove 'splash' from /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet vt.global_cursor_default=0"

boot time takes about 14 seconds from the last BIOS post message I see.

If I leave 'splash' in...

GRUB_CMDLINE_LINUX_DEFAULT="quiet vt.global_cursor_default=0"

...my boot time goes up to 30 seconds...even though the Pacman theme is gone in like 3 seconds. I think he eats about 10 dots before disappearing.

I can certainly do without the plymouth theme but when I disable it, I am now seeing 2 boot errors that I would need to figure out how to handle.

One is the typical 'clean' message on /dev/sda5 (like below):

dev/sda1: clean, 552599/6111232 files, 7119295/24414464 blocks

The other is a message I am not familiar with:

pnp 00:01 can’t evaluate _crs 12311

Any ideas?

John

MisterB

Take a look at the suggestions here:

https://askubuntu.com/questions/772874/how-to-turn-off-the-filesystem-check-message-which-occures-while-booting

I'm wondering if a line like this might suppress both messages:
GRUB_CMDLINE_LINUX_DEFAULT="quiet loglevel=3 fsck.mode=skip splash vt.global_cursor_default=0"

johnodon

@MisterB said in Retropie Installation on Ubuntu Server x64 18.04.1:

Take a look at the suggestions here:

https://askubuntu.com/questions/772874/how-to-turn-off-the-filesystem-check-message-which-occures-while-booting

I'm wondering if a line like this might suppress both messages:
GRUB_CMDLINE_LINUX_DEFAULT="quiet loglevel=3 fsck.mode=skip splash vt.global_cursor_default=0"

Thanks. I'll give that a try in a few mins.

MisterB

Bad copy/paste above...of course, remove the splash in the above line for your use case.

johnodon

@MisterB said in Retropie Installation on Ubuntu Server x64 18.04.1:

Bad copy/paste above...of course, remove the splash in the above line for your use case.

I saw that. :)

So, this works well. The only thing I had to do was set loglevel=0. If I set it to 3, I would see something flash quickly (in red)...to fast to read. Setting to 0 I see nothing.

Thanks for pointing me in the right direction!

John

MisterB

How about loglevel 2 (CRITICAL)? Curious if there is a minimum threshold that hides everything for 'normal' use, but can still show a truly critical system error if it occurs.

johnodon

@MisterB said in Retropie Installation on Ubuntu Server x64 18.04.1:

How about loglevel 2 (CRITICAL)? Curious if there is a minimum threshold that hides everything for 'normal' use, but can still show a truly critical system error if it occurs.

I'm reimaging right now. Give me 30 mins and I'll report back.

johnodon

@johnodon said in Retropie Installation on Ubuntu Server x64 18.04.1:

@MisterB said in Retropie Installation on Ubuntu Server x64 18.04.1:

How about loglevel 2 (CRITICAL)? Curious if there is a minimum threshold that hides everything for 'normal' use, but can still show a truly critical system error if it occurs.

I'm reimaging right now. Give me 30 mins and I'll report back.

OK...so it looks like loglevel 2 is hiding those messages. However, on the first reboot after running the script, I am seeing a single red 'FAILED' message. It happens so fast I can't read the rest and it doesn't happen on subsequent reboots. I tried looking in dmesg but didn't see anything I didn't already mention.

Where is the best place to look for that error after the first reboot? I'd like to see what it is about and if it is script related.

John

johnodon

Quick update...

I am seeing the same longer boot time behavior on my Lenovo T430 as I did on my other system. If I remove splash from the the grub line, boot time is shortened by about 20 seconds.

John

johnodon

@MisterB I think it may be close to time to start a new thread that is dedicated to your install scripts and just reference this one for posterity.

What do you think?

John

etheling

@MisterB Just wanted to chime in to say that I modified my pre/post scripts for Intel NUC to run from under optional_scripts/ and it worked out beautifully - in one swift run the pre-install part drops in new kernel, wpa_supplicant, and connects NUC to wifi and then proceeds with the rest of the install.

Couple of observations:

I had edited my pre-install script under optional_scripts/pre_install, so I had preinstall.sh~ there as I executed main install script. Which then ran the preinstall script, and broken preinstall.sh~. Maybe filter out files with ~ at the end, or only run files with .sh extension?
Would it make sense to run inxi -F at the end of the script just before reboot (and maybe some other utilities) to 'document' into install log hw etc. configuration?
I think unrelated to your script, but I had to recompile xpad afterwards to fix button mappings at the controller. E.g. select, start and others were not reporting at correct values when looking /dev/input/js* with jstest. I'll look into this maybe next weekend to see if it reproduces.
Unrelated to your script, but lr-puae doesn't install currently. It compiles ok, but there is no README file under /home/pi/RetroPie-Setup/tmp/build/lr-puae/ which breaks the install ('touch /home/pi/RetroPie-Setup/tmp/build/lr-puae/README' works around it - see here).

As for my post install scripts, I split my post install scripts to two parts, one that is specific to NUC and another that (I think, didn't test yet) will also work on Pi that sets up themes, overlays, emulator specific retroarch congifs, core overrides, sets up retroachievements.org account, etc.

Thanks again for all the work on the script. Very much appreciated!

johnodon

WIP... :)

I'm currently working out the code to separate pre-install and post-install package options.

(Radio buttons...only one selection allowed)

(Checklist...multiple selections allowed)

etheling

@etheling said in Retropie Installation on Ubuntu Server x64 18.04.1:

... but I had to recompile xpad afterwards to fix button mappings at the controller. E.g. select, start and others were not reporting at correct values when looking /dev/input/js* with jstest. I'll look into this maybe next weekend to see if it reproduces.

This reliably reproduced in different ways when my install order was: 1) pre-instal kernel , wpa_supplicant, start net, 2) main script, 3) post installs. I don't know what exactly is/was causing the problems but what seems to make things work smooth is to install custom kernel last (or after xpad has been installed, I suspect). So having modified my install order to 1) pre-instal wpa_supplicant & start net, 2) main script, 3) post installs (including install new kernel) installs/confiugres everything in a single smooth run. :)

etheling

@MisterB One thing I realized is that in function set_resolution_grub the script sets GRUB_GFXMODE=1920x1080x32,auto. It appears to me though that this (even when augmented with GRUB_GFXPAYLOAD_LINUX=keep) does not make kernel actually keep resolution that grub changes to (as per GRUB_GFXMODE), and at least with my NUC, kernel always switches back to 4k as it boots up.

I tried vga=ask, and vga=... kernel parameters as suggested by some not so recent docs, but alas, I think these are for some legacy use cases.

What worked for me though was video= kernel parameter. It's documentation leaves something to be desired, but ArchLinux had some useful documentation. Long story short: in my case with 8th Gen NUC, the kernel boot option for 5.8.x series kernel is just video=1920x1080which puts fb/console to that resolution during boot/before X is tarted (and especially after I exit X).

I think it might be useful to include video= to the script, but then again, I am not sure how well this works across different display graphics chipsets/hw and how portable that syntax is. Just wanted to mention this so it get's 'documented' here in case someone else wonders about this.

etheling

EDIT: See post 175 for most recent version of the script.

EDIT: See post 162 in this thread for updated version of the script. I updated this post to refer to updated script.

@MisterB , I did some changes to your script (see updated script here) that allows me to start ES and RetroArch in Linux KMS/DRM mode. This way everything (?) can be run without X (or Wayland). Not only this may be slightly more performant, it also seems to be a simpler approach with fewer moving parts (and is pretty close to how RetroPie runs on Pi).

I tested it to successfully boot to ES in KMS/DRM mode on 8th gen NUC, and in VM using 20.04 LTS default (5.4.0-42) kernel (e.g. I'm not thrilled about having to update to latest mainline with the NUC). With limited testing, retroarch cores and e.g. scummvm seem to work just as well as before.

There wasn't much information online that I could find about running RetroPie on PC/Linux this way. So maybe there is something I missed that makes this not such a great idea?

Anyways; here is a summary of changes I did in order of reading down the install script (in linked install script, search '# Eth:' to locate them).

Added fbset to RETROPIE_DEPENDS. This is used by /opt/retropie/supplementary/runcommand/runcommand.sh and was missing.
In install_retropie () patch RetroPie-Setup/scriptmodules/emulators/retroarch.sh (patch), RetroPie-Setup/scriptmodules/supplementary/sdl2.sh (patch), and RetroPie-Setup/scriptmodules/helpers.sh (patch) to enable KMS/DRM (on x11 platform). In particular I am curious about helpers .sh, as it appears that due to some conflict in Ubuntu 19.04, it wouldn't allow compiling (RetroPie copy) of SDL2 (recompiling is needed to enable KMS/DRM support). Also, is there a reason why kms/drm flags for retroarch and sdl2 have not been enabled for x11 platform?
In hide_boot_messages()set kernel command line to quiet vt.global_cursor_default=0 video=1920x1080. E.g. remove splash, add video. ES behaved poorly if I tried to start it with console at other (larger than 1080p) resolutions, even when using --resolution 1920 1080 parameter. See my earlier email about the video= parameter and that it maybe should use more complete syntax. But this seems to work, so... Also, I removed plymouth splash, as boot is now so fast that pacman pops up for a mere second or two (especially on NUC).
Add enable_autostart_emulationstation () to the script to create .bash_profile launch emulationstation from .bash_profile (instead of startx) and run it instead of enable_autostart_xwindows().
In fix_quirks() add $USER to the input group to allow /dev/input/* access (needed for keyboard to work in KMS/DRM mode)
In fix_quirks() allow fbset to be executed by non-root users. Fbset is used by /opt/retropie/supplementary/runcommand/runcommand.sh and it errors without this change.
In fix_quirks() add audio_driver = "alsa" to /opt/retropie/configs/all/retroarch.cfg to make audio work in KMS/DRM mode. I'm not sure about this one if it's absolutely required, but for me to get audio, I had to add that.
Comment out / don't run: enable_plymouth_theme "retropie-pacman", enable_autostart_xwindow, hide_openbox_windows, autostart_openbox_apps, set_resolution_xwindows "1920x1080" and add/run enable_autostart_emulationstation after enable_autologin_tty.

And that's it... I think. :)

Lastly some random notes:

Even if RetroPie now runs without X, I think it's a good practice to install X for a few reasons and if for nothing else than to allow running multiple Emacses, or more seriously, I want to have ability to run X to launch gog.com games via wine / .xsession (I'll put how I am doing that to another post at later time), and it enables using browser via ports that @johnodon mentioned earlier.
With vmware fusion 11.5.6 I had to use legacy bios to have 1920x1080 resolution available for grub / linux (e.g. /w UEFI videoinfo reported ~1024x768 etc. as a max)
With vmware fusion, for some reason 1080p wasn't available for retroarch cores, so I had to hit 'a' to change default resolution to something thats available.. I thought this was odd.

It would be great if someone can test this on real hw and report back. I am curious about how 'portable' this is. Especially to non-Intel graphics chipsets.

What are your thoughts about this approach over using X? Maybe keep X as a default, but if starting the script with --kms etc. parameter, set-up RetroPie using KMS.

praetorian55

@etheling

Awesome work! I didn't test the entire script since I had already installed on my machine, but was eager to test the KMS/DRM support. I'm using an atom z8350 (Intel CherryTrail) and your changes to get emulation station to run in KMS/DRM mode definitely worked.
I'm fighting resolutions settings on a 4k TV right now so I hope to be able to go through some of your other settings as well to see if I can get that taken care of. Unfortunately I'm buried at work and probably won't be able to spend to much time on it anytime soon - I was just really excited someone figured out how to get KMS/DRM working!

Edit: was able to use your grub/kernel modification to tame the resolution to a perfect 1920x1080. That's a setting I've been struggling to find for awhile - thanks!

etheling

@praetorian55 Thanks! Great to hear! :)

One fragile part here is SDL2 because it looks like RetroPie installs/compiles it's own version and then holds the package (via echo "libsdl2-dev hold" | dpkg --set-selections) to prevent OS replace it. Alas, installed version is not new enough for e.g. winehq which then refuses to install. It looks like RetroPie's 'sdl2. sh' would allow installing a newer version of SDL2, but with a quick test it completely broke the install (presumably I either fat-fingered something or didn't take time to understand how the script logic works...I'll try again later).

I didn't really thought this through (like at all), ~~but could RetroPie use local copy of SDL2, and statically link emulationstation and RetroArch against it when compiling~~? (whereas now it's creating and installing custom .deb package, that OS is trying to fight off). (edit: see post 162 in this thread, dynamic linker is way to go)

@MisterB - I'll experiment with SDL2 a bit and will post an updated script maybe over the weekend. I have couple of other improvements on my mind that I will add too (on which topic, we should add also GRUB_GFXPAYLOAD_LINUX="keep" to /etc/default/grub - it will avoid mode switch when Grub loads the kernel).