basic_install exits without error despite multiple errors

seriema

Hello,

When running sudo ./retropie_packages.sh setup basic_install I see errors such as:

E: Unable to locate package libraspberrypi-dev
Could not install package(s): libudev-dev libxkbcommon-dev libasound2-dev libusb-1.0-0-dev libraspberrypi-dev libgles2-mesa-dev libavcodec-dev libavformat-dev libavdevice-dev.

or

cc -c -oatari800/src/mzpokeysnd.o atari800/src/mzpokeysnd.c -march=native -O2 -pipe -DDEFAULT_CFG_NAME=\".lr-atari800.cfg\" -DGIT_VERSION=\"" 6998ad4"\" -O2 -DNDEBUG -fPIC -D__LIBRETRO__ -DINLINE="inline" -DHAVE_CONFIG_H -Wall -I. -I./atari800/src -I./libretro -I./libretro/libretro-common/include
atari800/src/mzpokeysnd.c: In function 'generate_sync':

atari800/src/mzpokeysnd.c:2438:9: error: 'volume' undeclared (first use in this function)
* (volume.s16 / 2 / MAX_SAMPLE / 4 * M_PI * 0.95)
^~~~~~

atari800/src/mzpokeysnd.c:2438:9: note: each undeclared identifier is reported only once for each function it appears in

Makefile:522: recipe for target 'atari800/src/mzpokeysnd.o' failed
make: *** [atari800/src/mzpokeysnd.o] Error 1

~/RetroPie-Setup
Could not successfully build lr-atari800 - Atari 8-bit/800/5200 emulator - Atari800 port for libretro (/home/pi/RetroPie-Setup/tmp/build/lr-atari800/atari800_libretro.so not found).

But the setup still exits without an error code (it returns 0). There's also no logs in ~/RetroPie-Setup/logs, but there can be files in ~/RetroPie-Setup/tmp/build. As I'm writing an automated install, the error code is the important part. My scripts need to be able to detect if the setup fails.

I think retropie_packages.sh is somehow swallowing the errors when installing multiple emulators, because if I run sudo ./retropie_packages.sh lr-atari800 build it exits with error code 1 as expected.

How can I detect if there's an error with basic install? I suspect it's a bug or known behavior.

BuZz

@seriema that function isn't really designed to be used like that. If you need return codes call each package manually - you probably are better building a list of own packages if automating.

seriema

Thank you for such a quick response!

Oh, ok. So is calling sudo ./retropie_packages.sh [emulator] build for every emulator the only way to go, or is there some other command or flag I could use for an automated install? I mainly need to know if it failed or not.

Perhaps sudo ./retropie_packages.sh setup basic_install does more than what I need at this stage of my project. What I really need, as a first step at least, is to scaffold the directories an installation would have.

Primarily all the roms folders: RetroPie/roms/[emulator]
Secondly the folders and symlinks such as
- .emulationstation -> /opt/retropie/configs/all/emulationstation
- .config/retroarch

Is there a smaller/faster installation that does that? That also returns the correct error code.

mitu

@seriema said in basic_install exits without error despite multiple errors:

Oh, ok. So is calling sudo ./retropie_packages.sh [emulator] build for every emulator the only way to go, or is there some other command or flag I could use for an automated install? I mainly need to know if it failed or not.

build only compiles the package, it doesn't install it. sudo ./retropie_packages.sh [package] will install from from source - dependency install/source install/build/install/configure.

The configure action is responsible for creating the necessary ROM folder and configure it with the right emulators.

Is there a smaller/faster installation that does that? That also returns the correct error code.

Take a look at the source and see which packages/package sections the basic_install function calls, then call the section/package install functions.

seriema

@mitu Thank you I'll run with all steps (not only build). I've tried to decipher the setup scripts and running with some trial and error for some day now. My main issue comes down to me having to stare at the output as error codes are often swallowed. It would help immensely if there was a sure-fire way to know that an installation failed, regardless of what package index/ID I use. I've tried:

Checking exit code. As explained by @BuZz it's not expected to be other than 0 for basic_install, so I assume the same applies to some other packages so it won't work.
Looking for logs as suggested by your older post but the folder $HOME/RetroPie-Setup/logs is often non-existent. I was able to get it once when building lr-atari800 manually and it failing, but haven't been able to reproduce it (sudo ./retropie_packages.sh lr-atari800 build fails, but no logs, so I might have done something different).
Checking if $HOME/RetroPie-Setup/tmp/build is empty, as it seems to be so on a successful install.

BuZz

You will need to make your own logs (the logs are part of the GUI setup. If you want to do things manually you will need to log manually also). Most packages will return error code calling them manually (with no parameters. Not calling the build / configure etc functions directly). You don't need to call each function.

mitu

@seriema said in basic_install exits without error despite multiple errors:

Looking for logs as suggested by your older post but the folder $HOME/RetroPie-Setup/logs is often non-existent. I was able to get it once when building lr-atari800 manually and it failing, but haven't been able to reproduce it (sudo ./retropie_packages.sh lr-atari800 build fails, but no logs, so I might have done something different).

The logs folder is only populated when you use the retropie_setup script (with the UI). If you run the scriptmodules functions manually, logs doesn't record anything - you'll have to make your own, as @BuZz mentioned.

seriema

Ok it's getting clearer. Thank you both.

Just to be sure, there's no way to know if sudo ./retropie_packages.sh setup basic_install failed?

My 3rd attempt above (looking in tmp/build) isn't guaranteed to work?

BuZz

After checking the code it looks like I did plan to have basic_install return errors, so I will check that. I've been doing work on some of the packaging so some command line functionality could change.

I'd still use the main module calls and not functions in the setup module though. They are not guaranteed to work outside of the setup gui.

seriema

@BuZz Glad to hear that. When I looked around the code I did see that there were correct return 1 in places where I expected, but somehow it was being swallowed up. I could contribute a fix but it'd be a significant investment of time, that could be worth it as I'll eventually need to learn your code anyway, so if you decide to change/fix the behavior let me know.

To give you some background on what I'm actually doing and why the correct exit code matters to me:

I'm working on a small project for RetroPie called Retro-Cloud, that stores all ROMs in the cloud (Azure File Share). It's similar to RetroPie-Setup in that it's a bunch of bash scripts that installs PowerShell, setups a cloud infrastructure (VM + File Share), mounts the File Share as a drive, sets environment variables, etc. The only valid test is to run on it on a clean RetroPie installation, but that takes too long so I created a Docker image (I know you're "not a fan"). It's been working great so far, but I've been improving the image this week and suddenly nothing was working anymore... I had been so careful, despite every image rebuild taking 2-3 hours...

Can you imagine my surprise when I notice that RetroPie-Setup had started failing (silently) around the time I did my changes but not related to my changes? 🤣 RetroPie has released a new version, 4.5.17 (3e827c1) which is failing all over the place. The last version that worked for me was 4.5.16 (3b6947c).

It could be my image, so now I'm trying to lock down the version, but every rebuild of my Docker image takes 2-3 hours and without a fool-proof way of knowing if the install failed that theory is still to be proven and it's incredibly time consuming. See my Docker build timeline for all my parallel attempts at

Catching a failed install
Locking down the RetroPie version

If they work I can finally move on to do an arm build and test out a first release of my project. 😀

mitu

@seriema said in basic_install exits without error despite multiple errors:

RetroPie has released a new version, 4.5.17 (3e827c1) which is failing all over the place.

Besides the lr-atari800 build failing, I think that's a gross exageration.

seriema

@mitu I tried to cut down my post as it was too long but some context might've gotten lost. The failures I referred to was my testing with the incomplete image, and not the RetroPie release itself. I apologize if that was misinterpreted.

BuZz

I debugged this. I'm not sure about fixing as changing it may break calls between modules that don't specifically set a return code. It's only an issue calling setup module functions. This behaviour hasn't changed since 4.5.16.

Get a list of packages you want and do it that way and you can trap the errors.

Having said that I will test with my fix and see if I notice any problems.

BuZz

I may have a workaround which shouldn't affect backward compatibility. I will test it.

seriema

@BuZz Amazing! Let me know how I can help.

4.5.17 likely wasn’t be the problem, as subsequent builds has not reproduced it consistently. I can’t tell why it went wrong, but the only emulator that was built was N64. Usually there’s 30 emulators (well, their ROMs folders at least). I think the failed atari800 stopped the script but didn’t bubble up the error. Maybe the log will tell you more: https://hub.docker.com/repository/registry-1.docker.io/seriema/retro-cloud/builds/38183daf-6bd1-432a-ada0-ada85af1ec02

Is there a base package I could/should start with? I’m thinking of a slimmer install for now to speed up development as I’ll be focusing on an ARM build.

BuZz

@seriema not sure what you mean by a base package. basic_install installs everything from core and main sections. But what you prefer install is up to you.

BuZz

You should get the right return code now if basic_install fails.

Just to note, the command line interface may change at any time. I try not to break backwards compatibility but it's not guaranteed to be the same. Especially calls outside of the standard module functions like build, install, configure etc.

seriema

@BuZz Maybe it's "core" that I mean? Basically a minimal installation. Something faster to run (compared to basic_install) but still scaffolds enough of the file system to be useful for my testing.

For testing what you did I queued a new build. The latest version is running here right now and should be done in 2-3 hours:
https://hub.docker.com/repository/registry-1.docker.io/seriema/retro-cloud/builds/8e568eb7-83d7-4405-ace5-cf54e68425c7

mitu

@seriema said in basic_install exits without error despite multiple errors:

https://hub.docker.com/repository/registry-1.docker.io/seriema/retro-cloud/builds/8e568eb7-83d7-4405-ace5-cf54e68425c7

This link (just like the previous one) is not public - seems like an account on docker's site is needed.

seriema

@mitu Oh. I assumed it'd be public but apparently that's a feature request.

Here's the log for the failed build I linked originally: https://gist.github.com/seriema/9867490750b8faae72ef5a851edb2671

I'll post the current one once it's done (~1h left).