Solved! Duke Nukem 3D, expansions game saves saving to parent game's directory

sleve_mcdichael

Solved! You can't change the behavior of saves and configs going to ~/.config/eduke32 but you can change what happens "behind the curtain" (that location was already just a symlink in the first place, all we have to do is change where it points.) All it takes is a few extra lines in the launch scripts (if your launch command uses the -addon {n} syntax go ahead and keep that, this is only my version I have to use with my GRP files for whatever reason). You can point the symlinks wherever you want, but just make sure the target exists (you have to create them):

(Edit: the log is just saved to . so instead of moving the config file after it's created we can just pushd into the config folder first, and then popd after. Also added test for symlink, just in case.)

#! /bin/bash

[[ -h "~/.config/eduke32" ]] && rm -f ~/.config/eduke32
ln -s /opt/retropie/configs/ports/duke3d/vacation ~/.config/eduke32

pushd ~/.config/eduke32; "/opt/retropie/supplementary/runcommand/runcommand.sh" 0 _PORT_ "duke3d" "-j /home/pi/RetroPie/roms/ports/duke3d -g vacation/vacation.grp -x vacation/game.con -j /home/pi/RetroPie/roms/ports/duke3d/vacation"; popd

[[ -h "~/.config/eduke32" ]] && rm -f ~/.config/eduke32
ln -s /opt/retropie/configs/ports/duke3d ~/.config/eduke32

Original Post:

Raspberry Pi model 4B. RetroPie version 4.7.12, from 4.7.1 image download. EDuke32 installed from binary. I followed the guide for manual installation of the three expansion packs. My command-line options are slightly different from the guide, because my .GRP and .CON files came from the installation CD-ROMs and not from Steam. These are the minimal commands I found for each expansion that would still boot and without just loading the Atomic Edition instead. They are all slightly different from each other, I presume because the different methods used to obtain the files from the discs or installed game directories:

/opt/retropie/configs/ports/duke3d-dc/emulators.cfg:
eduke32 = "/opt/retropie/ports/eduke32/eduke32.sh -j/home/pi/RetroPie/roms/ports/duke3d -gdc/dc.grp"
default = "eduke32"

/opt/retropie/configs/ports/duke3d/emulators.cfg:
eduke32 = "/opt/retropie/ports/eduke32/eduke32.sh -j/home/pi/RetroPie/roms/ports/duke3d"
default = "eduke32"

/opt/retropie/configs/ports/duke3d-nw/emulators.cfg:
eduke32 = "/opt/retropie/ports/eduke32/eduke32.sh -j/home/pi/RetroPie/roms/ports/duke3d -gnw/nwinter.grp -xnw/nwinter.con"
default = "eduke32"

/opt/retropie/configs/ports/duke3d-vacation/emulators.cfg:
eduke32 = "/opt/retropie/ports/eduke32/eduke32.sh -j/home/pi/RetroPie/roms/ports/duke3d -gvacation/vacation.grp -xvacation/game.con -j/home/pi/RetroPie/roms/ports/duke3d/vacation"
default = "eduke32"

/opt/retropie/configs/ports/duke3d-dc/eduke32.cfg | grep SelectedGRP:
SelectedGRP = "dc.grp"

/opt/retropie/configs/ports/duke3d/eduke32.cfg | grep SelectedGRP:
SelectedGRP = "duke3d.grp"

/opt/retropie/configs/ports/duke3d-nw/eduke32.cfg | grep SelectedGRP:
SelectedGRP = "nwinter.grp"

/opt/retropie/configs/ports/duke3d-vacation/eduke32.cfg | grep SelectedGRP:
SelectedGRP = "vacation.grp"

/home/pi/RetroPie/roms/ports/duke3d-dc.sh:
#!/bin/bash
"/opt/retropie/supplementary/runcommand/runcommand.sh" 0 _PORT_ "duke3d-dc" ""

/home/pi/RetroPie/roms/ports/duke3d-nw.sh:
#!/bin/bash
"/opt/retropie/supplementary/runcommand/runcommand.sh" 0 _PORT_ "duke3d-nw" ""

/home/pi/RetroPie/roms/ports/duke3d.sh:
#!/bin/bash
"/opt/retropie/supplementary/runcommand/runcommand.sh" 0 _PORT_ "duke3d" ""

/home/pi/RetroPie/roms/ports/duke3d-vacation.sh:
#!/bin/bash
"/opt/retropie/supplementary/runcommand/runcommand.sh" 0 _PORT_ "duke3d-vacation" ""

All of my saves, in all three expansions, are being saved to /opt/retropie/configs/ports/duke3d/*.esv instead of their own respective duke3d-XX folders. All of the saves are all visible in-game in all three expansions and the base game, but of course, the ones from wrong expansions all say "previous version, unable to load" or whatever.

Where is this save file path defined? I want each game to save to its own folder. I don't see anything that points to the configs/ports/duke3d folder directly, so I thought it must be set by which system was called by the runcommand, must use that same folder where emulators.cfg was found. But each launch script calls a separate _PORT_ "duke3d-XX" command and yet they're all saving to the duke3d folder. How fix?

mitu

I think the manual instructions may need some revision.
Why aren't you using the automated installation method ? It will add the necessary launch entries in the ports folder.

sleve_mcdichael

@mitu said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

Why aren't you using the automated installation method ? It will add the necessary launch entries in the ports folder.

Didn't work. The launch scripts were created, but the only one that worked (besides Atomic Edition) was nwinter. That's the only one that had a .GRP file on the disc, and the only one that matched the checksums from the EDuke32 FAQ. The other two, I had to get the .GRP files: for DC, created from installed files using KGROUP.EXE from the AE install disc. For Vacation, I had to (after getting a patched launcher that renames temp files with the $ symbol instead of ¥, which is not allowed in DOS-box), launch the game and then copy VACATION.GRP from the game folder while it was running, as the file didn't exist otherwise.

These two didn't match the checksums for the Steam versions, which I presumed was why the simple -addon # option didn't work to load them. They would boot, but they would just run Atomic Edition instead. Hence why I went the manual route. I still had some trouble, they still didn't work (at least some/one of them) until I got the .CON files, too. Maybe the automated way would work, too, with those files present? I don't think I tried that, since I didn't copy them over until I had already started doing the manual way.

mitu

The automated installation checks for the nw,vacation, dc sub-folders to enable/configure the add-ons. I think you'd need all the addon's files (the .grp/.con ) to be present in the sub-folder in order for the launchers to work. It doesn't check for any checksums, just adds the folder to the search path (via -j) and the -addon parameter.
You may be able to replicate this in your custom launchers, eduke32 may save the in the addon folder.

Also, please don't over-use the 'Edit post' feature if you want to add more info (spelling/typing errors are fine) or rewrite your post, it breaks the conversation. If you have relevant info that should be added and don't want to add another post (especially if someone else hasn't replied yet), just prefix the additions with an 'EDIT'.

sleve_mcdichael

@mitu said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

The automated installation checks for the nw,vacation, dc sub-folders to enable/configure the add-ons. I think you'd need all the addon's files (the .grp/.con ) to be present in the sub-folder in order for the launchers to work.

The guide only mentions .grp files for the automated method, so I thought that's all I'd need. And for NWinter, it was. Since the other two didn't work with my acquired .grp files, I began exploring the manual option.

It doesn't check for any checksums,

The install script doesn't, but EDuke32 itself might need exact versions of the files. The checksums show that what I've copied/built from the CD-ROMs (except for NWinter, which worked while the other two did not) are not the exact same as the ones listed on the EDuke32 wiki FAQ which are said to work.

The file they said works, works. The files they didn't say work, don't.

You may be able to replicate this in your custom launchers, eduke32 may save the in the addon folder.

Yeah I'll try this tomorrow with the .con files in the addon folders, too.

Still, though...what is it about my current configuration that's causing them to all share the configs/ports/duke3d/ folder, when all the scripts are calling individual _PORT_ "duke3d-xx" systems? Shouldn't the "working directory" be the different /opt/retropie/configs/ports/<system> folders for each one?

mitu

It looks like the save folder is always set to the same location, regardless if the command line is using -j<path_to_addon> -addon <X> or -g<path_to_grc> -x<path_to_con>.
As a workaround, use a pushd <path_to_addon> before starting the game in the emulators.cfg for the addon, it should set the save path to the addon folder.

sleve_mcdichael

@mitu said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

It looks like the save folder is always set to the same location,

But that's weird, right? It can't be hard-coded in EDuke32. Like if I'm on Ubuntu or Windows, it's not saving to /opt/retropie/ports..., is it? Something's gotta tell it where to save. What?

mitu

@sleve_mcdichael said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

But that's weird, right? It can't be hard-coded in EDuke32

It's not hard coded, it looks like it's using the CWD - as I mentioned in the workaround.

sleve_mcdichael

@mitu said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

@sleve_mcdichael said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

But that's weird, right? It can't be hard-coded in EDuke32

It's not hard coded, it looks like it's using the CWD - as I mentioned in the workaround.

What, probably very simple and obvious thing, am I missing here? CWD is current working directory, yes? What defines the CWD? Is it the "system" (or port) called by the runcommand?

What is the CWD when the runcommand says "/opt/retropie/supplementary/runcommand/runcommand.sh" 0 _PORT_ "<system>" ""? Is it /opt/retropie/configs/ports/<system>? I really thought that was it but it seems not to be the case.

Why is a workaround necessary? How does my CWD end up in duke3d/ when my runcommand, nor any of my options or config files, don't say "duke3d" but "duke3d-vacation" or similar? I did copy all the config folders from the original duke3d but I'm not seeing anything inside of them that points back to that folder.

Sorry for the noob questions. I'm learning as I go. This one's hard, I guess.

mitu

@sleve_mcdichael said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

What, probably very simple and obvious thing, am I missing here? CWD is current working directory, yes?

Yes, by CWD i mean the current working directory.

What defines the CWD? Is it the "system" (or port) called by the runcommand?

Not sure.

sleve_mcdichael

I gave the automated install scripts another shot. I tried copying the addon folders, complete with .con files this time (in addition to the .grp files I already had) to addons/, and running the install script again. Launch scripts were created, as expected, with the -addon parameter.

NWinter: works (always did)

Vacation: crashed

DC: launched, but when I started a game on the "DC" episode (didn't try Eps. 1 or 2), it kicked me back to the menu (the "new game/load game" menu, not back to EmulationStation.) Tried loading a previous "DC" save: worked, but only sort of. There was the White House, I could run around and shoot, but the graphics were all glitchy, like when you move a window in Windows and it "smears" across the screen, just drawing the new position on top without erasing the old one first. The background was all doing that.

(Remember: my .grp files are not from Steam, as assumed in the guide, but from the original installation CD-ROMs or installed game directories in DOS-box. I believe this is why they didn't work with the -addon parameter.)

Since the automated method didn't work (mostly), while the manual method does, and since it doesn't seem to affect the save file location anyway, I'm gonna stick with what is working. So, back to the issue...

@mitu said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

As a workaround, use a pushd <path_to_addon> before starting the game in the emulators.cfg for the addon, it should set the save path to the addon folder.

How would I do that? Just string it with a ;?

I tried this:

/opt/retropie/configs/ports/duke3d-vacation/emulators.cfg:
eduke32 = "pushd /opt/retropie/configs/ports/duke3d-vacation/; /opt/retropie/ports/eduke32/eduke32.sh -j/home/pi/RetroPie/roms/ports/duke3d -gvacation/vacation.grp -xvacation/game.con -j/home/pi/RetroPie/roms/ports/duke3d/vacation"
default = "eduke32"

Didn't work. Saves still saved to duke3d, not duke3d-vacation.
Also tried pushd /home/pi/RetroPie/roms/ports/duke3d/vacation/; (...); same result. Is there another way I'm supposed to do it?

I tried removing (renaming) the configs/ports/duke3d folder to see what would happen. It used my $HOME folder instead. It put duplicates there, of everything from the configs folder (eduke32.cfg, grpfiles.cache, et al.) It also screwed up the graphics, but in a different way. Less noticeable in-game, but the intro screens and animations were all lo-fi, with big splotches of solid colors like it was in a 16-color mode or something.

This got me thinking, maybe it has something to do with eduke32.cfg's location. So I added -cfg /opt/retropie:configs/ports/<addon >/eduke32.cfg to the options. (These .cfg files were all copied from the duke3d folder and only modified in the SelectedGRP = line.) More weirdness: first of all, it still did the weird color-splotch thing. Additionally, I found new files in the folder: eduke32 (no .cfg, but it was formatted like the default eduke32.cfg) and eduke32_settings.cfg which looked like a duplicate of the default settings.cfg.)

Even the main game did the glitchy-color thing when I used -cfg /opt/retropie/configs/ports/duke3d/eduke32.cfg. Removed that option, and it went back to normal.

For a minute. I decided to just be done with it and put everything back to the working way and just deal with the shared saves. But something I had done along the way, made the color glitch reappear.

Well [censored] me, right?

So I deleted the duke3d folder again, reran the install script. Went in to the new duke3d folder to fix the emulators.cfg command, and noticed it was empty. Just emulators.cfg and a bare-bones settings.cfg with only two lines. Everything else (eduke32.cfg, etc.) were generated at runtime.

Maybe something in one of those files, copied over from the original install, had the bit that told the saves where to save. So I gave the addons' config folders the same treatment: I deleted everything but emulators.cfg and copied over the bare-bones settings.cfg from ../duke3d/. After loading one of the addons, config and save files were generated...in the duke3d folder!

I'm dead. This has killed me. I am no longer alive; a corpse is typing this.

That last bit was hyperbole, but I am out of ideas. There is no reason I can see why it is using a CWD other than where emulators.cfg is found, and I know it's using the right emulators.cfg because that's the only place the different launch options are found.

I'm done. I guess there is just no way to keep the addons separate from one another. Since having a separate "system" configured for each one (/opt/retropie/configs/ports/<sys>/emulators.cfg) does not appear to actually be doing anything, I might just scrap that idea and move the command options back to the launch scripts, have them all call _PORT_ "duke3d" since that's what they're basically doing anyway. I don't like it, but if they won't do what I tell them to, I may as well just tell them to do whatever it is they're going to do so at least I can still feel like I'm in control. (Actually I do like having fewer systems configured, if it doesn't mean I lose functionality. And since it never had the functionality of separate saves, there's nothing to lose. What I don't like is just that it's not using the systems I configured, and that I can't figure out why.)

Other things that did not work either (I didn't think they would, since the files are saving in a folder named duke3d, not one named eduke32 as the emulator is called, but nothing else has worked so I gave it a shot anyway. But it was no surprise when it didn't suddenly "fix it."):

Changing the emulator name inside emulators.cfg, à la:

eduke32-vacation = "<command>"
default = "eduke32-vacation"

Duplicating the symlink:

$HOME/.config/eduke32 -> /opt/retropie/configs/ports/duke3d

for each of the addons, as in:

$HOME/.config/eduke32-vacation -> /opt/retropie/configs/ports/duke3d-vacation

Doing both of these things together.

Is eduke32.log meant to always be saved to my $HOME folder, or did I boff something else up along the way?

mitu

@sleve_mcdichael said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

How would I do that? Just string it with a ;?
I tried this:
/opt/retropie/configs/ports/duke3d-vacation/emulators.cfg:
eduke32 = "pushd /opt/retropie/configs/ports/duke3d-vacation/; /opt/retropie/ports/eduke32/eduke32.sh -j/home/pi/RetroPie/roms/ports/duke3d -gvacation/vacation.grp -xvacation/game.con -j/home/pi/RetroPie/roms/ports/duke3d/vacation"
default = "eduke32"

Didn't work. Saves still saved to duke3d, not duke3d-vacation.

Well, it was worth a try. I may have a closer look later on.

[..] Is eduke32.log meant to always be saved to my $HOME folder, or did I boff something else up along the way?

I noticed this too, I don't think it's something particular to your configuration.

sleve_mcdichael

I know I said I was done, but I couldn't let it go. I didn't get anywhere, but I found a new lead. I found this in eduke32.log:

Application parameters: -j /home/pi/RetroPie/roms/ports/duke3d -g vacation/vacation.grp -x vacation/game.con -j /home/pi/RetroPie/roms/ports/duke3d/vacation 
Using CON file "vacation/game.con".
Using /home/pi/ for game data
Using /home/pi/RetroPie/roms/ports/duke3d/ for game data
Using /home/pi/RetroPie/roms/ports/duke3d/vacation/ for game data
Using /home/pi/.config/eduke32/ for game data

In particular, the lines that say "Using <path> for game data":

The second and third ones are defined by the command options ("application parameters").

...the first one is CWD (if I run the launch script from command prompt, this path is replaced by whatever directory I was in at the time.)

...that last one, /home/pi/.config/eduke32 is where files are being saved (I presume just because it's the one most recently added to the list?). Where is this coming from?

This seems like it would likely be the culprit. .config/eduke32 is a symlink to /opt/retropie/configs/ports/duke3d; this is where the files are being saved. That would also explain the link between the emulator eduke32 defined in emulators.cfg and the files ending up in a duke3d folder.

...but that's still not it, at least not entirely. I re-created the duke3d-vacation "system" with an eduke32-vacation "emulator", then made a .config/eduke32-vacation symlink that points back to the duke3d-vacation system folder. This generated the logfile excerpted above, with the .config/eduke32 path still used instead of .config/eduke32-vacation as named in emulators.cfg.

Here are the files that generated this log:

Launch script:

pi@retropie:~ $ cat /home/pi/RetroPie/roms/ports/duke3d-vacation.sh 
#!/bin/bash
"/opt/retropie/supplementary/runcommand/runcommand.sh" 0 _PORT_ "duke3d-vacation" "-j /home/pi/RetroPie/roms/ports/duke3d -g vacation/vacation.grp -x vacation/game.con -j /home/pi/RetroPie/roms/ports/duke3d/vacation"

You can see it calls the system _PORT_ "duke3d-vacation". Here is that emulators.cfg:

pi@retropie:~ $ cat /opt/retropie/configs/ports/duke3d-vacation/emulators.cfg 
eduke32-vacation = "/opt/retropie/ports/eduke32/eduke32-vacation.sh %ROM%"
default = "eduke32-vacation"

I named the emulator within emulators.cfg as eduke32-vacation and then made a symlink with that name in .config, patterned after the eduke32 one that was already there:

pi@retropie:~ $ ls -l /home/pi/.config/ 
total 4
lrwxrwxrwx 1 pi pi   34 May 12 14:48 eduke32 -> /opt/retropie/configs/ports/duke3d
lrwxrwxrwx 1 pi pi   44 May 13 10:02 eduke32-vacation -> /opt/retropie/configs/ports/duke3d-vacation
drwx------ 2 pi pi 4096 Feb 23 23:34 procps
lrwxrwxrwx 1 pi pi   35 Jan 27 11:34 retroarch -> /opt/retropie/configs/all/retroarch

I even duplicated the eduke32.sh script that actually calls the executable:

pi@retropie:~ $ cat /opt/retropie/ports/eduke32/eduke32-vacation.sh 
#!/bin/bash
# HACK: force vsync for RPI Mesa driver for now
VC4_DEBUG=always_sync /opt/retropie/ports/eduke32/eduke32 $*

I didn't go as far as duplicating the executable itself.

Where else would you look?

When I removed the system folder (previous post -- symlink was intact but pointed nowhere; I hadn't found the log using it yet), it used my $HOME folder; when I removed the symlink to the system folder, just now, it created a new .config/eduke32 folder (not a symlink) and put the files in there.

re: eduke32.log saving to $HOME/ -- this one, in contrast, seems a rather easy fix, I think. This info is duplicated in /dev/shm/runcommand.log so it's safe to delete. So I just added that to the emulator run command:

pi@retropie:~ $ cat /opt/retropie/configs/ports/duke3d/emulators.cfg
eduke32 = "/opt/retropie/ports/eduke32/eduke32.sh %ROM% && rm eduke32.log"
default = "eduke32"

Will this work like I think it does? Initially, it seems to. The logfile is saved to CWD so deleting it from there seems to work, also.

If the eduke32 executable exits with an error, will this get passed to eduke32.sh and correctly interpret the &&, or will eduke32.sh just run, be done, and report an exit 0 regardless? In that case, I could move the hack into the emulators.cfg command:

eduke32 = "VC4_DEBUG=always_sync /opt/retropie/ports/eduke32/eduke32 %ROM% && rm eduke32.log"

...right? The idea being to only delete the logfile on clean exit, but leave it easily accessible if errors are detected.

Or I could just use a ; and let it delete the file regardless of exit status because, as I said, this info is also duplicated in runcommand.log.

mitu

@sleve_mcdichael said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

...that last one, /home/pi/.config/eduke32 is where files are being saved (I presume just because it's the one most recently added to the list?). Where is this coming from?

It's the default configuration folder for an application (i.e. $XDG_CONFIG_HOME/app).

Will this work like I think it does? Initially, it seems to. The logfile is saved to CWD so deleting it from there seems to work, also.

Yes, pretty much.

sleve_mcdichael

@mitu said in Duke Nukem 3D, expansions game saves saving to parent game's directory:

It's the default configuration folder for an application (i.e. $XDG_CONFIG_HOME/app).

Ah-hah! And it appears that this is something reported by the application, or at least not solely based on the filename; I duplicated everything all the way down to the executable, adding -vacation where appropriate ("duke3d-vacation" launch script calls a "duke3d-vacation" system with an "eduke32-vacation" emulator that calls an "eduke32-vacation" hack script that calls an "eduke32-vacation" executable, with an "eduke32-vacation" symlink in .config that points back to the "duke3d-vacation" system folder in /opt), and launched the game. eduke32.log (no vacation) was saved (I disabled the removing the log part for this test), and it still showed using .config/eduke32 (no vacation here, either), and game save was saved to the regular duke3d system folder in /opt.

...I don't know where to go from here. Guess I'll make my peace with them all sharing the save folder and let sleeping dogs lie.

sleve_mcdichael

Solved!

So the issue was, it was using ~/.config/eduke32 as the config directory where everything went. Even changing the executable's filename did not change this location used so it must be hard coded in the application somewhere (even if it's just "this application is called eduke32" and then it just used "whatever the application calls itself" as the config folder, that is what it is using.) So there really is no way to change the save location... [one more thing I don't think I tried: renaming the folder in which the executable exists -- but it doesn't matter because even if it did work (I have doubts), I think the solution I ended up with is more elegant than quadruplicating the executable folder four different times for four different games.]

...EXCEPT! That location is, already, just a symlink to the config folder in /opt/retropie/configs/ports/duke3d. So, inside of that folder, I made three new folders: dc, nw, and vacation. Then I edited my launch scripts to re-link the .config/eduke32 link. At first I got an error: "failed to create symbolic link: file exists." So I tried it with the -f(orce) option. Got a different error: "cannot overwrite directory." Finally, I decided to just remove the existing link before creating a new one (and then -f was no longer needed.) That worked! Now, before a game is launched, the link in ~/.config is reconfigured to point at the correct subfolder, and then saves and configs are stored separately per-game.

After the game exits, I set the link back to its original location and, for good measure, I move the log in $HOME to the config folder first.

Example duke3d-vacation.sh launch script:

#! /bin/bash

rm ~/.config/eduke32
  ln -s /opt/retropie/configs/ports/duke3d/vacation ~/.config/eduke32
    "/opt/retropie/supplementary/runcommand/runcommand.sh" 0 _PORT_ "duke3d" "-j /home/pi/RetroPie/roms/ports/duke3d -g vacation/vacation.grp -x vacation/game.con -j /home/pi/RetroPie/roms/ports/duke3d/vacation"

    mv ~/eduke32.log ~/.config/eduke32
  rm ~/.config/eduke32
ln -s /opt/retropie/configs/ports/duke3d ~/.config/eduke32

For the Original duke3d.sh script I still set the link to .../configs/ports/duke3d before launch, just in case anything prior left it in weird shape, but obviously I don't have to "fix" it after since that's the original default location anyway.