N64 crashes after 10 min

BuZz

@quicksilver not yet - I will open an issue if we can track it down to a specific commit. We don't know that it is a GlideN64 bug (Could be the RPI GPU drivers or firmware for example).

BuZz

Currently testing with the intro for Legend of Zelda, The - Majora's Mask letting it run continously. Not seen the problem yet (Running for 18 mins or so). @psyke83 said he saw the issue after 20 mins. Does anyone have a non interactive way of reproducing the issue quicker - so I can leave it running in the background?

quicksilver

@buzz It has happened for me just loading a level and letting my character stand still for 10-20 mins. However, when the glitch occurs it seems to get worse when you are moving and will likely lead to a crash. Sometimes when it occurs I can stop moving and the glitch will pass and I can resume playing. It may be harder to to get it to occur if you are standing still. This is all from my own anecdotal experience. It has happened on just about every game I have tested for glide (I have never tried majoras mask). Diddy kong racing, battle tanx are two games it will happen for me everytime.

BuZz

@quicksilver I got the issue after 22 mins or so. This is going to be very slow progress to debug - I suspect it's some memory leak of sorts as it doesn't trigger for ages. I will test with the latest code on Jessie in case it's related to newer kernels/firmware also

quicksilver

@buzz I appreciate you looking into this buzz. Im sure you have your hands full with a lot of different things and waiting around for 22 mins for a bug to show up im sure isnt your idea of fun. In the meantime I have been using save states as a work around to avoid losing game progress.

BuZz

@quicksilver Thanks. I have RetroPie 4.4 ready to be released but this is holding it up. Always bugs :-) I'm aiming for a quick workaround (as quick as possible anyway), so if for example just rolling back the GLideN64 plugin fixes it for now, I will do that (and we can work out the cause and a fix later).

quicksilver

@buzz I have done a fair amount of researching the issue (here and on reddit) and I can find similar reports of this happening going all the way back to August 2017. Its hard to know for sure if its the same issue but some of the descriptions are uncannily similar. Im sure most people who encountered this issue just chalked it up crappy N64 emulation on the Pi. Im not a programer but if there is any leg work you need me to do to help you out, just let me know.

simonster

Just in-case it helps diagnose it:

I've just fired up some N64 games on fresh Stretch install on Pi3B+ (new 4.3.15 image and then updated this morning), when playing Bomberman 64 on the default settings (mupen64plus-auto) it uses mupen64plus-glide-hires and locked up completely after around 10-15 minutes. I rebooted and forced it to mupen64plus-glide. After 12-13 minutes, I got a visual glitch but it cleared itself within a few seconds and then continued to play fine for over 1 hour.

From a fresh boot, I played WaveRace64 on mupen64plus-auto and this didn't appear to be using glide as it didn't have the fps counter. This played without issue for over 40 minutes.

I did check for logs but it appears the machine crashed out before it could write anything back.

Bobberella

Be cool if someone would post what caused the glitch in the first place.

quicksilver

@bobberella Nobody knows otherwise it would most likely be solved already ;)

simonster

Bizarrely, I've just updated from source and loaded Bomberman 64 and played for over an hour on my Pi3B+ (using the stretch version) and there was no sign of the issue. Yet I had it the last 2 times I played it it crashed around the 10-15 minutes mark.

EndersEnigma

@BuZz @quicksilver

Sorry for my delayed response, I was helping my mate move this weekend and had very little time to mess with my PI or to check the forums.

What I did to replicate this process was to start a game like "aerofighters assault" where the game is in constant motion. This seems to work well with replicating the glitch. Downside, If you just let a normal game run, the graphics may start to "blur" without crashing if the game is sitting idle. I have found that the crash seems to be tied to forcing the game to process new graphical input while the game is in "blur" which happens around the 15 min mark very consistently. If I let the game sit the graphics will just blur but the game will recover because it is not processing anything new graphically. These are just my observations.

My process: I set a timer on my phone for 12 minuets and let the game sit idle until that timer goes off. Once 12 minutes pass, I pick up the controller and play for a couple of minuets until the "blur glitch" starts to happen. This is where I have noticed that if i let it sit, the game will blur and often recover, If I try to interact at all during the blur it will freeze and require a hardboot.

EndersEnigma

@buzz said in N64 crashes after 10 min:

@quicksilver Thanks. I have RetroPie 4.4 ready to be released but this is holding it up. Always bugs :-) I'm aiming for a quick workaround (as quick as possible anyway), so if for example just rolling back the GLideN64 plugin fixes it for now, I will do that (and we can work out the cause and a fix later).

I cannot speak outside of my 4.3 load that I have on my computer, but from a fresh install on a brand new PI motherboard, brand new SD card, I got the same issue to replicate with the stock N64 GLIDE core both before and after updating it from source. I do not have a base version from previous loads to test, but it seems to be an issue both before and after updating from source on 4.3.

quicksilver

@simonster I've had times where the glitch either didn't happen, or I suspect sometimes it comes and goes so quickly and doesn't always cause a crash. Part of the reason why I thought my case was causing the glitch previously.

simonster

@quicksilver oh well! Thought I was on to something for a moment. I am trying to run top in SSH at the same time to monitor memory/cpu usage and see if it is memory related.

simonster

I forced Super Mario 64 to run through Glide-HighRes and have been running top -d 0.1 on ssh, whilst playing the game (the fastest it seems to go is around 25-27 fps), the memory utilisation (according to top) seems to be stable at 12.2% and increased to 12.3% after 15 minutes on my Pi3B+ and CPU load averages out at 50% (I assume that it can't access more than 50% due to the way it is coded).

I did get the glitch after about 20 minutes whilst playing but the system recovered (despite me trying to crash it deliberately), no noticeable increase in cpu load or memory usage was seen in top. Also the memory usage didn't go down after the glitch.

I know it probably isn't the best tool diagnostically, but the system didn't think it was using more memory or cpu than usual.

dankcushions

@simonster said in N64 crashes after 10 min:

CPU load averages out at 50% (I assume that it can't access more than 50% due to the way it is coded).

no that's just the amount it needs :) try telling that to all the people who overclock their pi3 cpus, though...

BuZz

I can reproduce the freeze on Jessie and older code. It's probably a gpu memory leak issue. Unsure though - but it can't hold up the release sorry. Hopefully we can debug it via valgrind or something.

quicksilver

@buzz no worries. Jessie and older, does that imply its not on issue on stretch?

BuZz

@quicksilver I meant Jessie and older Gliden64. The bug is not limited to Stretch basically. Will need to debug more in the future.