basic_install exits without error despite multiple errors

seriema

@mitu Thank you I'll run with all steps (not only build). I've tried to decipher the setup scripts and running with some trial and error for some day now. My main issue comes down to me having to stare at the output as error codes are often swallowed. It would help immensely if there was a sure-fire way to know that an installation failed, regardless of what package index/ID I use. I've tried:

Checking exit code. As explained by @BuZz it's not expected to be other than 0 for basic_install, so I assume the same applies to some other packages so it won't work.
Looking for logs as suggested by your older post but the folder $HOME/RetroPie-Setup/logs is often non-existent. I was able to get it once when building lr-atari800 manually and it failing, but haven't been able to reproduce it (sudo ./retropie_packages.sh lr-atari800 build fails, but no logs, so I might have done something different).
Checking if $HOME/RetroPie-Setup/tmp/build is empty, as it seems to be so on a successful install.

BuZz

You will need to make your own logs (the logs are part of the GUI setup. If you want to do things manually you will need to log manually also). Most packages will return error code calling them manually (with no parameters. Not calling the build / configure etc functions directly). You don't need to call each function.

mitu

@seriema said in basic_install exits without error despite multiple errors:

Looking for logs as suggested by your older post but the folder $HOME/RetroPie-Setup/logs is often non-existent. I was able to get it once when building lr-atari800 manually and it failing, but haven't been able to reproduce it (sudo ./retropie_packages.sh lr-atari800 build fails, but no logs, so I might have done something different).

The logs folder is only populated when you use the retropie_setup script (with the UI). If you run the scriptmodules functions manually, logs doesn't record anything - you'll have to make your own, as @BuZz mentioned.

seriema

Ok it's getting clearer. Thank you both.

Just to be sure, there's no way to know if sudo ./retropie_packages.sh setup basic_install failed?

My 3rd attempt above (looking in tmp/build) isn't guaranteed to work?

BuZz

After checking the code it looks like I did plan to have basic_install return errors, so I will check that. I've been doing work on some of the packaging so some command line functionality could change.

I'd still use the main module calls and not functions in the setup module though. They are not guaranteed to work outside of the setup gui.

seriema

@BuZz Glad to hear that. When I looked around the code I did see that there were correct return 1 in places where I expected, but somehow it was being swallowed up. I could contribute a fix but it'd be a significant investment of time, that could be worth it as I'll eventually need to learn your code anyway, so if you decide to change/fix the behavior let me know.

To give you some background on what I'm actually doing and why the correct exit code matters to me:

I'm working on a small project for RetroPie called Retro-Cloud, that stores all ROMs in the cloud (Azure File Share). It's similar to RetroPie-Setup in that it's a bunch of bash scripts that installs PowerShell, setups a cloud infrastructure (VM + File Share), mounts the File Share as a drive, sets environment variables, etc. The only valid test is to run on it on a clean RetroPie installation, but that takes too long so I created a Docker image (I know you're "not a fan"). It's been working great so far, but I've been improving the image this week and suddenly nothing was working anymore... I had been so careful, despite every image rebuild taking 2-3 hours...

Can you imagine my surprise when I notice that RetroPie-Setup had started failing (silently) around the time I did my changes but not related to my changes? 🤣 RetroPie has released a new version, 4.5.17 (3e827c1) which is failing all over the place. The last version that worked for me was 4.5.16 (3b6947c).

It could be my image, so now I'm trying to lock down the version, but every rebuild of my Docker image takes 2-3 hours and without a fool-proof way of knowing if the install failed that theory is still to be proven and it's incredibly time consuming. See my Docker build timeline for all my parallel attempts at

Catching a failed install
Locking down the RetroPie version

If they work I can finally move on to do an arm build and test out a first release of my project. 😀

mitu

@seriema said in basic_install exits without error despite multiple errors:

RetroPie has released a new version, 4.5.17 (3e827c1) which is failing all over the place.

Besides the lr-atari800 build failing, I think that's a gross exageration.

seriema

@mitu I tried to cut down my post as it was too long but some context might've gotten lost. The failures I referred to was my testing with the incomplete image, and not the RetroPie release itself. I apologize if that was misinterpreted.

BuZz

I debugged this. I'm not sure about fixing as changing it may break calls between modules that don't specifically set a return code. It's only an issue calling setup module functions. This behaviour hasn't changed since 4.5.16.

Get a list of packages you want and do it that way and you can trap the errors.

Having said that I will test with my fix and see if I notice any problems.

BuZz

I may have a workaround which shouldn't affect backward compatibility. I will test it.

seriema

@BuZz Amazing! Let me know how I can help.

4.5.17 likely wasn’t be the problem, as subsequent builds has not reproduced it consistently. I can’t tell why it went wrong, but the only emulator that was built was N64. Usually there’s 30 emulators (well, their ROMs folders at least). I think the failed atari800 stopped the script but didn’t bubble up the error. Maybe the log will tell you more: https://hub.docker.com/repository/registry-1.docker.io/seriema/retro-cloud/builds/38183daf-6bd1-432a-ada0-ada85af1ec02

Is there a base package I could/should start with? I’m thinking of a slimmer install for now to speed up development as I’ll be focusing on an ARM build.

BuZz

@seriema not sure what you mean by a base package. basic_install installs everything from core and main sections. But what you prefer install is up to you.

BuZz

You should get the right return code now if basic_install fails.

Just to note, the command line interface may change at any time. I try not to break backwards compatibility but it's not guaranteed to be the same. Especially calls outside of the standard module functions like build, install, configure etc.

seriema

@BuZz Maybe it's "core" that I mean? Basically a minimal installation. Something faster to run (compared to basic_install) but still scaffolds enough of the file system to be useful for my testing.

For testing what you did I queued a new build. The latest version is running here right now and should be done in 2-3 hours:
https://hub.docker.com/repository/registry-1.docker.io/seriema/retro-cloud/builds/8e568eb7-83d7-4405-ace5-cf54e68425c7

mitu

@seriema said in basic_install exits without error despite multiple errors:

https://hub.docker.com/repository/registry-1.docker.io/seriema/retro-cloud/builds/8e568eb7-83d7-4405-ace5-cf54e68425c7

This link (just like the previous one) is not public - seems like an account on docker's site is needed.

seriema

@mitu Oh. I assumed it'd be public but apparently that's a feature request.

Here's the log for the failed build I linked originally: https://gist.github.com/seriema/9867490750b8faae72ef5a851edb2671

I'll post the current one once it's done (~1h left).

seriema

@mitu @BuZz Here's the log for the latest RetroPie (558f25c) https://gist.github.com/seriema/05db8d60c11dc18d0285c6495b1b7dd7

The build succeeded, I think? I can't see an obvious error, and the roms folders are there as expected. I'm not sure what caused the failure last time, but it was likely a fluke. I'll see if I can provoke basic_install to fail and check the error code.

seriema

@BuZz I unintentionally got two failed installs so I'll share them as they had your exit code fix. They failed differently so at least it's something. :) The N64 emulator exited with 1 as expected, but Splashscreen exited with 0 (it didn't continue to install but I don't know how to tell when it's done done). Edit: It should end with Info: [message] on success?

1: basic_install exited with 1 when mupen64plus failed. Ran on Debian with amd64. See full log.

= = = = = = = = = = = = = = = = = = = = =
Building 'mupen64plus' : N64 emulator MUPEN64Plus
= = = = = = = = = = = = = = = = = = = = =
[snip]
make: Leaving directory '/home/pi/RetroPie-Setup/tmp/build/mupen64plus/mupen64plus-video-glide64mk2/projects/unix'
head: cannot open '/home/pi/RetroPie-Setup/tmp/build/mupen64plus/GLideN64/src/Revision.h' for reading: No such file or directory
current revision "50e83004"
last build revision
/home/pi/RetroPie-Setup/tmp/build/mupen64plus/GLideN64/projects/cmake /home/pi/RetroPie-Setup/tmp/build/mupen64plus /home/pi/RetroPie-Setup
CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
  CMake 3.9 or higher is required.  You are running version 3.7.2


-- Configuring incomplete, errors occurred!
make: *** No targets specified and no makefile found.  Stop.
/home/pi/RetroPie-Setup/tmp/build/mupen64plus /home/pi/RetroPie-Setup
Removing additional swap
/home/pi/RetroPie-Setup
Could not successfully build mupen64plus - N64 emulator MUPEN64Plus (GLideN64/projects/cmake/plugin/Release/mupen64plus-video-GLideN64.so not found).
Errors:
Could not successfully build mupen64plus - N64 emulator MUPEN64Plus (GLideN64/projects/cmake/plugin/Release/mupen64plus-video-GLideN64.so not found).
The command '/bin/sh -c if [ "$(uname -m)" = 'x86_64' ]; then sudo ./retropie_packages.sh setup basic_install; fi' returned a non-zero code: 1

2: basic_install exited with 0 when splashscreen failed. Ran on Debian with arm32v7. See full log.

= = = = = = = = = = = = = = = = = = = = =
Configuring 'splashscreen' : Configure Splashscreen
= = = = = = = = = = = = = = = = = = = = =

/home/pi/RetroPie-Setup/scriptmodules/supplementary/splashscreen.sh: line 91: systemctl: command not found
/home/pi/RetroPie-Setup
Removing intermediate container c40e0a51fa8f
 ---> 3e4f12bb8360
[snip]
Successfully built 63739c10746d
Successfully tagged rc:docker-multiarch