Using lr-puae for Amiga emulation

rkoster

I just spent a few evenings getting this to work. Here's a bit of what I learned. Not being an original Amiga user, a lot of this baffled me for a while.

lr-puae has been updated quite a bit since the docs that were on the RetroPie site. I've updated the docs with the latest from the github. The big things that have changed since the original docs were written are:

the core used to only support .uae files that pointed at .adf files. This is no longer true. It now loads all of these: .zip .uae .adf .dms .fdi .ipf .hdf .hdz .m3u
the core has WHDLoad support now. This requires games in .hdf or .hdz format. ~~It does not support the .lha files used by Amiberry.~~ (This is no longer true; the latest versions of lr-puae support .lha files.) See the next reply for a script that can convert .lha files for use in lr-puae.
there are special steps required for supporting .ipf disk images.
the core supports autoselection of machine configs by adjusting the ROM filename.

Why use lr-puae instead of Amiberry? It is definitely less plug n play. Mostly only if you want to use Retroarch features beyond the controls: shaders, overlays, etc. It also seems to support a few more file types, though I suspect that the docs are incorrect on Amiberry only supporting .lha (I'd be surprised if it didn't support .adf and .hdf).

Some of my learnings which may help others are below. I've merged much of this into the RetroPie docs already.

ROM formats

extension	file description	requirements
.uae	text config file pointing to .hdf, .adf, or .ipf	needs to be created per ROM
.adf	disk image	loads directly
.adz	zipped disk image	loads directly
.dms	disk image, seems rare	loads directly
.fdi	disk image, seems rare	loads directly
.ipf	disk image	requires capsimg.so in retroarch directory
.hdf	hard disk image	requires WHDLoad set up
.hdz	zipped hard disk image	requires WHDLoad set up
.m3u	text config file	needs to be created for multidisk games
.zip	must contain an .adf, .hdf, or .ipf	unzip is handled by Retroarch
.lha	zipped hard disk image in an Amiga-specific compression format	unzip is handled within the emulator

By far the commonest file formats you will find are .adf and .lha.

One thing that Amiga enthusiasts seem to point out repeatedly is that although you may be able to expand an .lha file on Windows, you often shouldn't; the Amiga operating system and Windows don't always agree on paths and special characters, with the result that you can corrupt the file when unzipping it.

How lr-puae loads things

The Retroarch config file sets the machine specs.
A game override may change the machine specs.
You select a rom.
If the file is a .m3u, the core then proceeds to parse it and see what the first rom in line is. Otherwise, it looks at the file directly.
If the file isn't a .uae file, the core automatically generates a file called puae_libretro.uae in your /roms/amiga directory with the config set to match the machine specs from Retroarch. Otherwise, your rom IS a .uae file with config specs.
The core loads the .uae file and boots the emulator to match. This sets machine specs which may override Retroarch!
If what the .uae file points at happens to be a hard disk image, the core loads ~/RetroPie/BIOS/WHDLoad.hdf, then attaches the second hard disk image, looks at it, decides what machine settings it needs (thereby possibly changing the machine specs again), and boots from the first hard disk image.
Hopefully the game runs.

Using .ADFs

.adf files can be placed directly in your /roms/amiga folder. They can be loaded directly, but there are many possible configurations for the emulated Amiga. If you follow the standard setup instructions, you will have three Kickstart BIOS roms in your ~/RetroPie/BIOS folder with the correct names. Retroarch will then support three presets, one each for Amiga 500, Amiga 600, and Amiga 1200. These will have several variables preconfigured selected for maximum compatibility. It does not mean that one of these three configs will run every game.

The other thing to note is that one of the other Retroarch options is floppy speed. It can run between 100% and 800% speed. Needless to say, this is a potentially nice speed-up in loading times, but some ROMs will not work unless they are at 100% speed because of copy-protection.

Try the .adf, and if it doesn't work, enter the Retroarch GUI and try out the next machine. You will have to exit Retroarch and relaunch the game to see if it works. If it does, you have two choices:

save a game override using the Retroarch Options menu
modify the ROM file name by adding a substring:
"(A500)" or "(OCS)" to use an Amiga 500 model.
"(A600)" or "(ECS)" to use an Amiga 600 model.
"(A1200)" or "(AGA)" to use an Amiga 1200 model.

Either way should result in generating puae_libretro.uae with the specific settings. Note that a game override is a bit more flexible since you can save not only machine type, but floppy speed and other settings per game. The substring method will only change the machine type and automatic settings associated with it.

Using .IPFs

There is additional setup required to use a .ipf disk image. I have copied over the instructions given on the github for the core to the RetroPie docs, and compiled the latest version, but have not succeeded in getting it to work on a Pi. According to the changelogs, this feature was only tested on Windows.

Using WHDLoad games

Take it as a given that you need to create ~/RetroPie/BIOS/WHDLoad.hdf per the instructions in the RetroPie docs.

Next, a lot of advice on the Net says to set your machine to be an Amiga 1200. The boot image on the WHDLoad.hdf apparently will adjust downwards if the game requires it. That said, an Amiga 1200 has twice the CPU speed of the earlier machines, making it that much more intensive to emulate; I was able to run Agony on an emulated A600 and buy a couple of frames.

The vast majority of WHDLoad format games out there are in .lha format for Amiberry, not in .hdf format. These will not load.

There are many that are in .zip format and just have the contents of an .hdf file, rather than being an .hdf.

On top of that, some .hdf files are not set up for WHDLoad. And others are, but won't load in lr-puae because of the directory structure. Many files out there have a directory structure that looks like this:

Agony (1992)(Psygnosis).hdf
  Agony.info
  Agony (folder)
    Agony.info
    Disk.1
    Disk.2
    Disk.3
    Agony.slave
    Manual
    Manual.info
    ReadMe
    ReadMe.info
    scores.sav

This will not work in lr-puae. To fix this by hand, you can use ADF Opus to create an HDF file that looks like this:

Agony (1992)(Psygnosis).hdf
    Agony.info
    Disk.1
    Disk.2
    Disk.3
    game.slave
    Manual
    Manual.info
    ReadMe
    ReadMe.info
    scores.sav

The two big changes:

There isn't a folder at the top level. If your .slave file isn't at the top level of the .hdf, you will get a DOS window popup stating "DOS Error #205 (object not found) on reading '.slave'".
Instead of <gamename>.slave, that file must be named game.slave. It doesn't matter (despite what you may see on the net) whether it's .Slave or .slave, based on my testing.

The next reply has a link to scripts that will fix HDFs, convert LHAs, and set up ZIPs so that they all work in lr-puae.

WHDLoad files have two advantages over .adf files:

They have generally been configured in advance to have the right machine settings. There's a large Amiga community building WHDLoad files with the right specs.
They load way faster.

One big difference from Amiberry is that lr-puae does not use an XML file and a network connection to validate or checksum WHDLoad files against a central game database.

Video and performance

To simplify things, I have set the Retroarch settings to PAL, overscan cropping false, and high resolution (most games are low resolution, but will automatically get line-doubled by the emu). I also run a lower res than I usually do, to reduce load.

If you get audio crackle, try setting your screen to a 50Hz refresh rate via the Runcommand menu, and make sure that it's also set that way in Retroarch.

The Retroarch "CPU compatible" option improved performance even when my machine was set to emulate an Amiga 1200. It's not supposed to; it's supposed to only work when emulating a 68000 chip, and the 1200 had a 68EC020. Some games will not work with "CPU Compatible" set to false, but I have it set this way as the default for now.

I'll keep exploring performance options; so far, I've doubled framerate in my test game (Agony, which is high-res and seems demanding, compared to my other test game, Archon).

I'll also document what the Retroarch options do, based on the configuration.txt file in the github; I think most people won't dig that deep to learn what options do.

~~I haven't tracked down where the .uaerc file is, I expect it must exist somewhere.~~ Turns out, it doesn't seem to!

rkoster

Retroarch configs

Setting	Choices (default in bold)	Notes
Model	A500 / A600 / A1200	This setting predefines a range of other settings including CPU, memory, and which Kickstart BIOS to use.
Video standard	PAL / NTSC	Most software is PAL.
High resolution	true / false	If off, high-res Amiga screens will have their horizontal resolution halved (only every other horizontal pixel will be drawn). Most classic games employ low-resolution screens.
Crop overscan	true / false	Crops the overscan, which may result in clipped images in some games. A combination of NTSC and overscan settings will let you achieve all the possible resolutions for the Amiga screen; see the table under "Rendering" in the docs.
Use analog	false / true
LEDs	standard / simplified / none	Shows drive activity and power LEDs at the bottom right.
CPU speed	real / max	'real' is recommended for games designed to run on an A500-class Amiga but which won't work on faster Amigas. This includes many classic Amiga games. If set to 'max', the CPU emulation will run at the maximum speed that the host CPU can achieve. lr-puae will spend as much time as it can emulating the 68000 CPU and will not wait at all per frame.
CPU compatible	false / true	If enabled, E-UAE will use a slower but more compatible version of the CPU false emulation. This may be necessary to run some some demos and games correctly.
Sound output	none / interrupts / normal / exact	* none - audio emulation is disabled. * interrupts - audio emulation is enabled but audio output is disabled. * normal - audio emulation is enabled and output enabled. * exact - audio emulation is enabled and exact output enabled.
Sound frequency	11025 / 22050 / 44100	The frequency of emulated audio output in Hertz. Typically, higher frequencies will require more work, but have better quality.
Sound channels	mono / stereo / mixed	The Amiga supports 4-voice stereo sound, with two channels output on the left channel and two on the right. mono - monophonic output; all Amiga voices are output on a single channel. stereo - stereo output; two Amiga voices are output on the left channel and two on the right. mixed - stereo output; the four Amiga voices are mixed and output on both left and right channels.
Sound interpolation	none / rh / crux / sinc	Interpolation is a technique which "smoothes out" the audio. The three other choices are different algorithms for it which may give varying results and performance hits.
Floppy speed	100 / 200 / 300 / 400 / 500 / 600 / 700 / 800	Valid values are 1x to 8x the speed of a standard Amiga floppy drive. Values other than 100 may affect compatibility with Amiga software, especially the floppy-based copy-protection systems included with some games.
Immediate blit	true / false	If enabled then blits performed by the Amiga chipset emulation will be reported as finishing immediately. This may improve performance at the price of compatibility.
NTSC chipset	true / false	Set this to match the video system
Vertical centering	none / simple / smart	Used for centering the screen vertically within the emulator display (which is then positioned by the Retroarch display)
Horizontal centering	none / simple / smart	See above.

rkoster

The largest Amiga game collections are in .lha format. The core should support .lha files now, so you may not need this script anymore.

Manually converting 'RetroPlay' style .lha files for use in lr-puae is a huge pain. Also, a ton of the files out there are actually .zip files with the contents of an .hdf file loose in the zip. Lastly, you also have some zips that have an .hdf inside them.

So I did this little bash script: https://github.com/raphkoster/Amiga-LHA-ZIP-to-HDF-converter/blob/master/README.md

Amiga-LHA-ZIP-to-HDF-converter

This bash script converts LHAs, .HDFs, or zipped directories to HDFs usable in lr-puae in RetroPie.

Installation

This requires Python, PiP, amitools, and lha.

sudo apt-get update
sudo apt-get install python-pip
sudo pip install amitools
sudo apt-get install -y lhasa
apt-get install zip unzip

Then download and install this package.

sudo chmod a+x amigatool*
sudo chown pi:pi amigatool*

Usage

Put the scripts in the directory with your Amiga roms.

To run on a single file

./amigatool.sh filename

If the file is a .lha, it will be unpacked into a temporary directory, then rebuilt into an .hdf and placed in a new /hdf directory with the main .slave file renamed to game.slave. This will then run directly in lr-puae just like an .adf.

If the file is a .zip with the Amiga files loose in it, it will be unzipped, then rebuilt into an .hdf compatible with lr-puae and copied to the /hdf directory.

If the file is a .zip with an .hdf in it already, the .hdf will be converted to be compatible, if necessary, and moved to the /hdf folder.

If the file is a .hdf, it will be converted if necessary, then moved to the /hdf folder.

Note that these are not bootable .hdf files, as they do not have Kickstarts in them. They presume that you have properly set up WHDLoad.hdf and placed it in ~/RetroPie/BIOS

The files will be output as zipped .hdf files which can then be scraped. Note that unzipped .hdf files will not work with some scrapers.

To convert an entire directory

./amigatool-batch.sh

This will go through every file in the folder in which it is run, and call the previous script on each one.

weirdocollector

Thank you for this!

rkoster

The script has been updated with

Batch support! Do a whole directory with one line.
Verification of lr-puae compatibility and conversion of non-compatible .hdf files.
...Including ones that were hidden inside of .lha files.
A much more elegant progress display.

It now gives output like this:

Batch processing 13 for conversion to lr-puae .hdf roms.
0/13 Agony (1992)(Psygnosis).hdf: extracting....unpacking...processing.......This file does not contain a WHDLoad game.
1/13 Agony_v1.3_0960.zip: extracting.....found game Agony...packing..........copying to /hdf...  Done!
2/13 Agony_v2.0_NTSC_2701.lha: extracting.....found game AgonyNTSC...packing..........copying to /hdf...  Done!
3/13 Alien Breed WHDLoad v2.3a.hdf: extracting....unpacking...copying to /hdf... Done!
4/13 Alien Breed WHDLoad v2.3a.zip: extracting....unpacking...processing.............packing...copying to /hdf...  Done!
5/13 Aliex_v1.1.lha: extracting.....found game Aliex...packing.....copying to /hdf...  Done!
6/13 amigatool-batch.sh isn't a .zip, .lha, or .hdf.
7/13 amigatool.sh isn't a .zip, .lha, or .hdf.
8/13 Archon2_v1.1_0905.hdf: extracting....unpacking...processing......packing...copying to /hdf...  Done!
9/13 Archon2_v1.1_0905.lha: extracting.....found game Archon2...packing.......copying to /hdf...  Done!
10/13 Archon_v1.2_NTSC_0914.lha: extracting.....found game ArchonNTSC...packing.......copying to /hdf...  Done!
11/13 Skipping directory blankhdfs.
12/13 Skipping directory hdf.

Instructions in the post above have been updated.

rkoster

~~I just batch processed a few thousand files overnight, and couple dozen edge cases popped up. I'll take a look at these and see about updating the script.~~

Scratch that -- I seem to have introduced a bug ~~and the files are all broken. So don't use this script until I upload a fix.~~

Edit two: Actually, it's just permissions. sudo chown pi:pi will fix the files that won't boot. ~~I'll update the script shortly, but in the meantime, it's not a big bug.~~

Script fixed. As noted in the new instructions, don't call these with "sudo" or you'll end up with roms owned by root that won't work.

rkoster

The script has been updated with

a report at the end of the batch process telling you how it went:

amigatool batch process complete.
Total processed:                 9
  Skipped directories:           1
  Skipped files:                 2
  Didn't contain a WHDload game: 4
  File was corrupt:              1
  Already compatible files:      0
  Converted files:               1
Converted .hdf files are located in the /hdf folder.

It now handles WHDLoad files with multiple .slave files in them.
It now gracefully detects and reports as corrupt files that couldn't be repacked into an hdf.
Better messaging in general.
Files are also leaner, and get zipped at the end so they can be scraped.

rkoster

One more update, trapping more ways in which lha files can be corrupt. I processed around 2800 files with this test, so I feel fairly good about it now. Let me know if anyone runs into issues with it.

One question I had is whether I should add [lr-puae] to the filenames, so that people can tell that these have been processed this way. But I don't know if that would affect scraping. Right now, they end up with exactly the same name as the RetroPlay WHDLoad files do.

BTW @muldjord, either Skyscraper or Screenscraper.fr doesn't recognize .hdf as a file extension for Amiga, not sure which.

muldjord

@rkoster It does now from 3.2.0, just released. :)

Headrush69

Thank you rkoster for this tool.

Right now I'm not using PUAE, but with the big changes that Retroarch just added to the core, I'm definitely going to be checking it out and your script will be so helpful.

https://www.libretro.com/index.php/core-news-puae-amiga-emulator-and-vice-commodore-64/

abunille

@rkoster

Hello and thank you for the Scripts.

I run into issues and don't know what i'm doing wrong.
All RetroPlay style .lha end with an Corrupt! message.

Example:

0/28 xxx_game.lha: extracting...../amigatool.sh: line 176: bc: command not found
'create size=M' IOError: can't determine geometry of HDF image file
.found game...building...Corrupt!
rm: cannot remove '../../hdf/xxx_game.hdf': No such file or directory

amigatool batch process complete.
Total processed:                 28
  Skipped directories:           1
  Skipped files:                 2
  Didn't contain a WHDload game: 0
  File was corrupt:              25
  Already compatible files:      0
  Converted files:               0
Converted .hdf files are located in the /hdf folder.

Maybe you have an idea what i'm doing wrong?

rkoster

@abunille It says

./amigatool.sh: line 176: bc: command not found

That sounds like you are missing the bc package. Try

sudo apt install bc

and try again?

abunille

@rkoster

looks much better now.
Thank you :)

rkoster

Great, glad to hear it.

Mick2K

I know it's not the right place to ask but i was never that deep into Amiga emulation but after reading this post i wanted to start.

I have a collection of a few thousand files in .lha format but many games are doubles or triples.

Is there a dat file or a batch script to remove the doubles and only keep one. I can't find anything.

jul16ares

Hello i get this errors i don't understand

I'm on archlinux

Cadaver_v2.2_0900.zip: unzipping....Traceback (most recent call last):
  File "/usr/bin/xdftool", line 11, in <module>
    load_entry_point('amitools==0.1.0', 'console_scripts', 'xdftool')()
  File "/usr/lib/python3.7/site-packages/amitools/tools/xdftool.py", line 827, in main
    code = queue.run()
  File "/usr/lib/python3.7/site-packages/amitools/tools/xdftool.py", line 100, in run
    exit_code = CommandQueue.run(self)
  File "/usr/lib/python3.7/site-packages/amitools/util/CommandQueue.py", line 42, in run
    exit_code = self.run_first(cmd_line, cmd)
  File "/usr/lib/python3.7/site-packages/amitools/tools/xdftool.py", line 163, in run_first
    exit_code = cmd.run(self.blkdev, self.volume)
  File "/usr/lib/python3.7/site-packages/amitools/tools/xdftool.py", line 58, in run
    self.blkdev = self.init_blkdev(self.args.image_file)
  File "/usr/lib/python3.7/site-packages/amitools/tools/xdftool.py", line 209, in init_blkdev
    return f.create(image_file, options=opts, force=self.args.force)
  File "/usr/lib/python3.7/site-packages/amitools/fs/blkdev/BlkDevFactory.py", line 161, in create
    if not geo.setup(options):
  File "/usr/lib/python3.7/site-packages/amitools/fs/blkdev/DiskGeometry.py", line 56, in setup
    (c, h, s) = self._parse_chs(options)
  File "/usr/lib/python3.7/site-packages/amitools/fs/blkdev/DiskGeometry.py", line 85, in _parse_chs
    if options.has_key('chs'):
AttributeError: 'dict' object has no attribute 'has_key'
.found game...building...Corrupt!

Twerkem

@Mick2K try https://lifehacker.com/the-best-duplicate-file-finder-for-windows-1696492476

Chuck_B

I'm trying to play lha games, but games don't work. Do I have to set something?

rkoster

@Chuck_B From the original post:

"It does not support the .lha files used by Amiberry. See the next reply for a script that can convert .lha files for use in lr-puae.'' "The vast majority of WHDLoad format games out there are in .lha format for Amiberry, not in .hdf format. These will not load."

This is because Amiberry-compatible files have been specifically set up to work with that emulator.

See https://retropie.org.uk/forum/post/195052 for a script that will convert these .lha files to something that lr-puae can load.

Alternatively, use Amiberry for them.

FetusSandwich

@rkoster said in Using lr-puae for Amiga emulation:

Then download and install this package.
sudo chmod a+x amigatool*
sudo chown pi:pi amigatool*

Hi there. I have a massive collection of Amiga games in .LHA format that I've done a lot of manual scraping on, so was happy to find my way to this post! Thanks for all of your hard work.

I've followed the above (I think) but am now receiving the following:

./amigatool-batch.sh: line 7: syntax error near unexpected token `newline'
./amigatool-batch.sh: line 7: `<!DOCTYPE html>'

Any ideas as to what is happening? I'm assuming it's user error along the way, but another pair of eyes (or opinions) would be great.