mame 2003 gamelist.xml with data from the binary

grant2258

its my 3rd day on messing around with python. I will get there in the end thats another story though. I really would like to help with the the state of all things mame.

Now one of the hardest things to debug is when you dont have consistent game naming so i started my journey to rectify this. This game data is pretty handy debugging wise to see if a game is marked up wrong with its working status with this information.

My initial data was from mame 0.78. I then realized mame 2003 has updates and fixed some games so i had to dumpthe mame2003 data and used that.

The information it will give you is the right rom name and manufacturer ( only changes are ' ,<,>, are changed to space and & is changed to "and". I will fix this its just bad chars to use in xml it is easily fixed) . The year, number of player and emulation status is listed too.

The big picture idea of this is the catver.ini can be generated from mame data and only none working ones marked in mame will be marked and unplayable for catver.ini.

this gameslist.xml assumes you have all the roms in .mame2003 it not a list generator for the roms. It will tell you the status of them emulation and when you start a rom you will know if it wrong.

Fix it in the mame source and update the catver.ini from the mame data not the other way round

I was thinking of adding the original rom name in that information as well but i just want to get a feel of what people think of it. Im using the default theme carbon so i dont know how it will look on other skins. Any feed back is appreciated. If its a no brainer ill just stop now and only use it to list my roms and get the emulation status and input information. Make sure your game list view is detailed to see the information.

here is a link to the dat.
https://drive.google.com/open?id=1Qw3U6WOS3R5bacO9VU433L1jZgYeoeXm

grant2258

ok progress is going well. Ive got to grips with classes we now have something useful I have made you a new shiney catver.ini for mame 2003 as of todays mame source.
here is what i done

I got the mame info
checked mames driver status and checked against caver.ini Unplayabe as well
marked in all the unplayable and a reason for the unplayable
i then added the missing lightgun games that where missed as well

the code is nearly done i just want to tidy it up before releasing it, Just be gentle with me in only a few days into my python trek.

here is the link to the new carver.ini . If you want it added to git ill do it so its version
controlled
.

https://drive.google.com/file/d/1eUKh8WAqzQqZANmfwa6cPDjgpKIKw2sr/view?usp=sharing

just updated didnt set the config for the lightgun just done a print statement
all is good now. The catver apart is apart from 5 games that need verified as bad I know at least one is good

C:\Users\grant\Desktop\games>python run.py
found parsed xml loading
aquarush Unplayable / please verify and update flag in driver source
diehard Unplayable / please verify and update flag in driver source
dragngun adding lightgun
freekckb Unplayable / please verify and update flag in driver source
jpark adding lightgun
lockload adding lightgun
mazerbla adding lightgun
mjleague Unplayable / please verify and update flag in driver source
nstocker adding lightgun
rungun Unplayable / please verify and update flag in driver source
showdown adding lightgun
targeth adding lightgun
triplhnt adding lightgun
vsgshoe adding lightgun
zombraid adding lightgun

grant2258

might as well post on what one was wrong ps i use descriptive names for mysel because i fixed my gamelist.xml

Free Kick (bootleg) Unplayable / please verify and update flag in driver source

freekick should have been marked as unplayable not freekckb as thats playable

this should have been
C:\Users\grant\Desktop\games>python run.py
found parsed xml loading
Aqua Rush (AQ1/VER.A) Unplayable / please verify and update flag in driver source
Die Hard Arcade (US) Unplayable / please verify and update flag in driver source
Free Kick (bootleg) Unplayable / please verify and update flag in driver source
Major League Unplayable / please verify and update flag in driver source
Run and Gun (World ver. EAA) Unplayable / please verify and update flag in driver source

grant2258

I know most of this stuff im going on about is probably go people thinking what this all all about. Ill try explain It started off i just wanted the right rom names for mame. Then I seen there was catver.ini which some people/utilities use to make a roms collection. So after getting my rom names right i thought hey this catver.ini should be getting its information from mame and not the other way round.

so here is a new link for whats changed in the catver.ini im not sure if cocktail flip should be marked as unplayable ill test and assume yes for now.

here is a link of the changes that the mame data done in diff format

https://drive.google.com/file/d/17JaF28UittgMjg1hc_8ovsKxXZp-rKoc/view?usp=sharing

markwkidd

Cocktail flip issues can be resolved through retroarch settings, so I'll suggest that not be listed as impossible.

Where did you source your catver.ini? Does it come from the mame2003 GitHub repo?

grant2258

yes that the one i used but its parsed through mame data and if it doesnt match it prints a please verify. The original catver doesnt change a new one is just generated

grant2258

cocktail is removed the diff and catver have been updated same link

less source/catver.ini | grep -c "Unplayable"
391
less catver.ini | grep -c "Unplayable"
504

so its 113 new bad ones

markwkidd

Very cool. Let's spot check these.

If I recall correctly, the last time I updated catver.ini I went though the catver and the mame2003 source and I launched every game marked as unplayable via the original MAME 0.78 in Windows in order to avoid 'false positives' -- games incorrectly marked as unplayable.

My criteria for designating something playable would be 1) that it boots up with reasonably appropriate video and 2) that it accepts coins and other input.

Not a very comprehensive test but I think that's still good criteria. This turned up several games that were labelled unplayable in the source but that were in fact functional.

Your project seems kind of like the obvious second phase of what I did before to get rid of false positives.

Edit: Actually your project is less like the second phase and more like the proper way to do the thing I was doing manually :D I am definitely interested in using these results to improve catver.ini!

herb_fargus

When I was looking at this a ways back, perhaps this is a different issue, but I remember having a bit of confusion with all the filename changes between versions and it was a bit difficult for me to figure out which version I had. If I remember correctly there was a name change dat that exists that perhaps handle those cases

grant2258

yes that is exactly the it from mame to mame the rom realnames change so in one version of mame a.zip could be amazing game(jap)(unencryped) then a new dump arrives it becomes amazing game(world)

(now if these games get mixed up in the mame naming and you think your picking the right one and your not because its the wrong real name is there.

now that aside i was happy i got my naming right. Now i went the the catver.ini and just wanted to see how valid it was and if there was any missing games there wasnt btw. There was just one game done twice. Considering mark done this all manually the accuracy is damn good of what he marked up.

So now whats left if i want to help mame mark up games right and its wrong in the source i have all the information i need in the gamelist details section to know if its write or wrong

please try out the gamelist.xml and show the full details and let me know if you find this helpful for your testing.

here is the source please take it easy on with the coding in only a few days into python know c helped and and made some other things difficult.

ps use python 2.7

to run you need to use python2

type python run.py

new files will be in the build folder

i supplied the mame2003 data i pulled

ps ill post details on how to get the mame data from mame2003 itll be the same method to pull data for lr-mame2003-plus since there both mame2003

just replace the source/mame2003.xml with data you pull from mame plus

https://drive.google.com/open?id=1Dx4XYA96d-G921IB9Zdfz9rLrclMpkJv

ps you can run it on windows or unix it doesnt matter which

grant2258

well I just tried altered beast fancied a game all driver stats read good but the sprites are just lingering. I tried it on normal mame078 and it is fine this naming convention works great and its easy to check if the original mame had the same proplem or if it was introduced into the codebase

grant2258

on a unrelated subject ive fixed altered beast the 68k core needs an update to do it though. The makefile isint generating the 68k core properly either.

this needs added to the makefile anywhere at the top

CFLAGS = -std=gnu99 -Isrc -Isrc/includes -Isrc/$(CORE_DIR) -I$(CORE_DIR)/cpu/m68000 -Isrc/cpu/m68000

the make file should be compiling m68kmake.c and executing it to make the code

im not good with github but here is a bash command to update the core and compile it

cd /home/pi/RetroPie-Setup/
sudo ./retropie_packages.sh lr-mame2003 sources
git clone https://github.com/kstenerud/Musashi.git
cd Musashi/
#makefile in lrmame really needs to do this step it doesnt
gcc m68kmake.c -g -O2 -std=gnu99 -o m68kmake
#execute it and generate source files
./m68kmake
sudo cp *.c ../tmp/build/lr-mame2003/src/cpu/m68000/
sudo cp *.h ../tmp/build/lr-mame2003/src/cpu/m68000/

cd ..

sudo ./retropie_packages.sh lr-mame2003 build
sudo ./retropie_packages.sh lr-mame2003 install

grant2258

@markwkidd

https://github.com/grant2258/mame2003-libretro/tree/output_data

thats how you get the data dumped its a quick hack and could be done better but the main thing is it does what we want make sure you have write permission to where it writing or itll segfault

grant2258

one more update for catver.ini I just played goldnaxa its stats all read as good the game starts fine the collision detection is well off and its unplayable. I confirmed the same behavior in mame 0.78. This should be updated in catver.ini so someone knows to look at it to change the source flag. This helps with knowing whats good and bad so people will also know what needs worked on

markwkidd

I need to spend some time with all of your work on this in order for me to keep track of the different enhancements that need to be made. I would like to help as much as I can.

Very exciting!

I thought I would get started with a straightforward update: flagging goldnaxa as unplayble. I have submitted a PR: https://github.com/libretro/mame2003-libretro/pull/340

markwkidd

@grant2258 could you talk me through your Altered Beast fix?

Do I understand correctly that changing the Makefile alone does not address the problem, there is a bug in the mame2003 m68000 CPU which you are fixing by importing part the Musashi emulator?

grant2258

Mame does use the Musashi emulator in it core its curently 3.3 the latest version is 3.4. There is no problem with this update.
just grabs the source

git clone https://github.com/kstenerud/Musashi.git
history.txt
m68kconf.h
m68kcpu.c
m68kcpu.h
m68kdasm.c
m68k.h
m68k_in.c
m68kmake.c

thats all the files that are there.

Now you need to compile the source file that generates the m68k emulator source.

with
gcc m68kmake.c -g -O2 -std=gnu99 -o m68kmake

now execute this

pi@retropie:~/RetroPie-Setup/Musashi $ ./m68kmake

            Musashi v3.3 68000, 68010, 68EC020, 68020 emulator
            Copyright 1998-2000 Karl Stenerud (karl@mame.net)

Generated 1962 opcode handlers from 513 primitives

what does this do it spits out these files
m68kopac.c
m68kopdm.c
m68kopnz.c
m68kops.c
m68kops.h

sudo cp *.c ../tmp/build/lr-mame2003/src/cpu/m68000/
sudo cp *.h ../tmp/build/lr-mame2003/src/cpu/m68000/

just replaces the files that where there

I believe the subtle differences the way in the way gcc compiles the older version is the problem. If you used the original version of gcc to compile mame 078 it will probably work. But there no need to do that when the cores been updated

grant2258

make files uses this for the core

SOURCES_C += $(CORE_DIR)/cpu/m68000/m68kops.c $(CORE_DIR)/cpu/m68000/m68kopac.c
$(CORE_DIR)/cpu/m68000/m68kopdm.c $(CORE_DIR)/cpu/m68000/m68kopnz.c
$(CORE_DIR)/cpu/m68000/m68kcpu.c $(CORE_DIR)/cpu/m68000/m68kmame.c

https://github.com/grant2258/mame2003-libretro/commit/9955f1c75066cb4b9c2c2b511ffbb64c5d3442a7

if you want to see the diff of the files

dankcushions

probably the best way to solve this is to have the musashi git master as a git submodule in mame2003.

grant2258

yea that could work but the makefile will need updated to do the source generation as well. I need to get ready for a new job interview. If any other info is needed let me know.

ps really sorry for diverting this threads topic but its kinda related to it all in a way. It would be nice to have a option to dump the the xml from the retroach menu