Using lr-puae for Amiga emulation

rkoster

The largest Amiga game collections are in .lha format. The core should support .lha files now, so you may not need this script anymore.

Manually converting 'RetroPlay' style .lha files for use in lr-puae is a huge pain. Also, a ton of the files out there are actually .zip files with the contents of an .hdf file loose in the zip. Lastly, you also have some zips that have an .hdf inside them.

So I did this little bash script: https://github.com/raphkoster/Amiga-LHA-ZIP-to-HDF-converter/blob/master/README.md

Amiga-LHA-ZIP-to-HDF-converter

This bash script converts LHAs, .HDFs, or zipped directories to HDFs usable in lr-puae in RetroPie.

Installation

This requires Python, PiP, amitools, and lha.

sudo apt-get update
sudo apt-get install python-pip
sudo pip install amitools
sudo apt-get install -y lhasa
apt-get install zip unzip

Then download and install this package.

sudo chmod a+x amigatool*
sudo chown pi:pi amigatool*

Usage

Put the scripts in the directory with your Amiga roms.

To run on a single file

./amigatool.sh filename

If the file is a .lha, it will be unpacked into a temporary directory, then rebuilt into an .hdf and placed in a new /hdf directory with the main .slave file renamed to game.slave. This will then run directly in lr-puae just like an .adf.

If the file is a .zip with the Amiga files loose in it, it will be unzipped, then rebuilt into an .hdf compatible with lr-puae and copied to the /hdf directory.

If the file is a .zip with an .hdf in it already, the .hdf will be converted to be compatible, if necessary, and moved to the /hdf folder.

If the file is a .hdf, it will be converted if necessary, then moved to the /hdf folder.

Note that these are not bootable .hdf files, as they do not have Kickstarts in them. They presume that you have properly set up WHDLoad.hdf and placed it in ~/RetroPie/BIOS

The files will be output as zipped .hdf files which can then be scraped. Note that unzipped .hdf files will not work with some scrapers.

To convert an entire directory

./amigatool-batch.sh

This will go through every file in the folder in which it is run, and call the previous script on each one.

weirdocollector

Thank you for this!

rkoster

The script has been updated with

Batch support! Do a whole directory with one line.
Verification of lr-puae compatibility and conversion of non-compatible .hdf files.
...Including ones that were hidden inside of .lha files.
A much more elegant progress display.

It now gives output like this:

Batch processing 13 for conversion to lr-puae .hdf roms.
0/13 Agony (1992)(Psygnosis).hdf: extracting....unpacking...processing.......This file does not contain a WHDLoad game.
1/13 Agony_v1.3_0960.zip: extracting.....found game Agony...packing..........copying to /hdf...  Done!
2/13 Agony_v2.0_NTSC_2701.lha: extracting.....found game AgonyNTSC...packing..........copying to /hdf...  Done!
3/13 Alien Breed WHDLoad v2.3a.hdf: extracting....unpacking...copying to /hdf... Done!
4/13 Alien Breed WHDLoad v2.3a.zip: extracting....unpacking...processing.............packing...copying to /hdf...  Done!
5/13 Aliex_v1.1.lha: extracting.....found game Aliex...packing.....copying to /hdf...  Done!
6/13 amigatool-batch.sh isn't a .zip, .lha, or .hdf.
7/13 amigatool.sh isn't a .zip, .lha, or .hdf.
8/13 Archon2_v1.1_0905.hdf: extracting....unpacking...processing......packing...copying to /hdf...  Done!
9/13 Archon2_v1.1_0905.lha: extracting.....found game Archon2...packing.......copying to /hdf...  Done!
10/13 Archon_v1.2_NTSC_0914.lha: extracting.....found game ArchonNTSC...packing.......copying to /hdf...  Done!
11/13 Skipping directory blankhdfs.
12/13 Skipping directory hdf.

Instructions in the post above have been updated.

rkoster

~~I just batch processed a few thousand files overnight, and couple dozen edge cases popped up. I'll take a look at these and see about updating the script.~~

Scratch that -- I seem to have introduced a bug ~~and the files are all broken. So don't use this script until I upload a fix.~~

Edit two: Actually, it's just permissions. sudo chown pi:pi will fix the files that won't boot. ~~I'll update the script shortly, but in the meantime, it's not a big bug.~~

Script fixed. As noted in the new instructions, don't call these with "sudo" or you'll end up with roms owned by root that won't work.

rkoster

The script has been updated with

a report at the end of the batch process telling you how it went:

amigatool batch process complete.
Total processed:                 9
  Skipped directories:           1
  Skipped files:                 2
  Didn't contain a WHDload game: 4
  File was corrupt:              1
  Already compatible files:      0
  Converted files:               1
Converted .hdf files are located in the /hdf folder.

It now handles WHDLoad files with multiple .slave files in them.
It now gracefully detects and reports as corrupt files that couldn't be repacked into an hdf.
Better messaging in general.
Files are also leaner, and get zipped at the end so they can be scraped.

rkoster

One more update, trapping more ways in which lha files can be corrupt. I processed around 2800 files with this test, so I feel fairly good about it now. Let me know if anyone runs into issues with it.

One question I had is whether I should add [lr-puae] to the filenames, so that people can tell that these have been processed this way. But I don't know if that would affect scraping. Right now, they end up with exactly the same name as the RetroPlay WHDLoad files do.

BTW @muldjord, either Skyscraper or Screenscraper.fr doesn't recognize .hdf as a file extension for Amiga, not sure which.

muldjord

@rkoster It does now from 3.2.0, just released. :)

Headrush69

Thank you rkoster for this tool.

Right now I'm not using PUAE, but with the big changes that Retroarch just added to the core, I'm definitely going to be checking it out and your script will be so helpful.

https://www.libretro.com/index.php/core-news-puae-amiga-emulator-and-vice-commodore-64/

abunille

@rkoster

Hello and thank you for the Scripts.

I run into issues and don't know what i'm doing wrong.
All RetroPlay style .lha end with an Corrupt! message.

Example:

0/28 xxx_game.lha: extracting...../amigatool.sh: line 176: bc: command not found
'create size=M' IOError: can't determine geometry of HDF image file
.found game...building...Corrupt!
rm: cannot remove '../../hdf/xxx_game.hdf': No such file or directory

amigatool batch process complete.
Total processed:                 28
  Skipped directories:           1
  Skipped files:                 2
  Didn't contain a WHDload game: 0
  File was corrupt:              25
  Already compatible files:      0
  Converted files:               0
Converted .hdf files are located in the /hdf folder.

Maybe you have an idea what i'm doing wrong?

rkoster

@abunille It says

./amigatool.sh: line 176: bc: command not found

That sounds like you are missing the bc package. Try

sudo apt install bc

and try again?

abunille

@rkoster

looks much better now.
Thank you :)

rkoster

Great, glad to hear it.

Mick2K

I know it's not the right place to ask but i was never that deep into Amiga emulation but after reading this post i wanted to start.

I have a collection of a few thousand files in .lha format but many games are doubles or triples.

Is there a dat file or a batch script to remove the doubles and only keep one. I can't find anything.

jul16ares

Hello i get this errors i don't understand

I'm on archlinux

Cadaver_v2.2_0900.zip: unzipping....Traceback (most recent call last):
  File "/usr/bin/xdftool", line 11, in <module>
    load_entry_point('amitools==0.1.0', 'console_scripts', 'xdftool')()
  File "/usr/lib/python3.7/site-packages/amitools/tools/xdftool.py", line 827, in main
    code = queue.run()
  File "/usr/lib/python3.7/site-packages/amitools/tools/xdftool.py", line 100, in run
    exit_code = CommandQueue.run(self)
  File "/usr/lib/python3.7/site-packages/amitools/util/CommandQueue.py", line 42, in run
    exit_code = self.run_first(cmd_line, cmd)
  File "/usr/lib/python3.7/site-packages/amitools/tools/xdftool.py", line 163, in run_first
    exit_code = cmd.run(self.blkdev, self.volume)
  File "/usr/lib/python3.7/site-packages/amitools/tools/xdftool.py", line 58, in run
    self.blkdev = self.init_blkdev(self.args.image_file)
  File "/usr/lib/python3.7/site-packages/amitools/tools/xdftool.py", line 209, in init_blkdev
    return f.create(image_file, options=opts, force=self.args.force)
  File "/usr/lib/python3.7/site-packages/amitools/fs/blkdev/BlkDevFactory.py", line 161, in create
    if not geo.setup(options):
  File "/usr/lib/python3.7/site-packages/amitools/fs/blkdev/DiskGeometry.py", line 56, in setup
    (c, h, s) = self._parse_chs(options)
  File "/usr/lib/python3.7/site-packages/amitools/fs/blkdev/DiskGeometry.py", line 85, in _parse_chs
    if options.has_key('chs'):
AttributeError: 'dict' object has no attribute 'has_key'
.found game...building...Corrupt!

Twerkem

@Mick2K try https://lifehacker.com/the-best-duplicate-file-finder-for-windows-1696492476

Chuck_B

I'm trying to play lha games, but games don't work. Do I have to set something?

rkoster

@Chuck_B From the original post:

"It does not support the .lha files used by Amiberry. See the next reply for a script that can convert .lha files for use in lr-puae.'' "The vast majority of WHDLoad format games out there are in .lha format for Amiberry, not in .hdf format. These will not load."

This is because Amiberry-compatible files have been specifically set up to work with that emulator.

See https://retropie.org.uk/forum/post/195052 for a script that will convert these .lha files to something that lr-puae can load.

Alternatively, use Amiberry for them.

FetusSandwich

@rkoster said in Using lr-puae for Amiga emulation:

Then download and install this package.
sudo chmod a+x amigatool*
sudo chown pi:pi amigatool*

Hi there. I have a massive collection of Amiga games in .LHA format that I've done a lot of manual scraping on, so was happy to find my way to this post! Thanks for all of your hard work.

I've followed the above (I think) but am now receiving the following:

./amigatool-batch.sh: line 7: syntax error near unexpected token `newline'
./amigatool-batch.sh: line 7: `<!DOCTYPE html>'

Any ideas as to what is happening? I'm assuming it's user error along the way, but another pair of eyes (or opinions) would be great.

mitu

@FetusSandwich Make sure the file has Unix line endings. If you copy/pasted this on a Windows machine, it might have the wrong line-endings.

FetusSandwich

@mitu Hi, mitu.

First off, longtime lurker, but thanks for all of your contributions to this community. I'm almost done with a yearlong build and I can't tell you how many times a solution to one of RetroPie's infinite little quarks rested somewhere on this forum with your name on it. You have my many thanks.

That's a great thought, but I verified them in notepad++. As a Windows user trying to force fit himself in this Linux world, let's take a further step back and see if it's maybe the user error issue.

As the user here, to run amigatool-batch.sh, I want:

A. To be at /usr/local/lib with the above packages/permissions installed and set.
B. To be at /usr/local/lib with the packages installed and then move and run the scripts from /home/pi/RetroPie/roms/amiga.
C. To be at /usr/local/lib with the above packages/permissions to stage the setup, then move the entirety of the above to /home/pi/RetroPie/roms/amiga and run the script.

Additionally, I received the following errors when attempting sudo pip install amitools:

Building wheels for collected packages: amitools
  Running setup.py bdist_wheel for amitools ... error
  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-FS0i4A/amitools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpv6p5VUpip-wheel- --python-tag cp27:
  use_cython: False
  /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type'
    warnings.warn(msg)
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

To get past this, I used sudo pip install amitools --no-cache-dir, which seemed to work, but here we are.

EDIT: Could it possibly be a Python 2.7 EOL thing?