NES RessurctionXtras 2.0 Project (thread previously titled: Does anybody work on Emulators to fix broken games here?)

Used2BeRX

Thanks for the reply. I thought about that after I posted and figured that I might have to look elsewhere with the compatibility issues and then try to get any fixes integrated here. I'm not at that point yet though. I would need to know more about what needs to be done first, and I won't know anything until I start doing tests on the Pi platform. Almost everything with my NES work has been limited to XBox at this point.

Though, I'm sure lots of us here would like to hear more about what you are working on. I'd personally like to see somebody make RadMobile work on Mame2003, haha.

(Brace yourself... It's going to be a long reply :))

What I'm working on currently is something that started about 12 or so years ago with a great team of people that were working together on XBox modding and emulation. It ended up going at a much faster rate than I expected with all of the volunteers we had helping out back then, and we ended up covering many console and handheld systems because of it.

For me though, the work was spread too thin and before it got anywhere near what I would consider complete I had to walk away from it. It kind of consumed my life, and in some ways it was literally driving me crazy.

After being away from the community for about 7 years I started getting back into it after my brother gave me a Pi Zero to try to work on because he wanted to make one for a few of his in-laws and friends. Fast forward about 6 months and I'm neck deep in an NES project that I never thought I'd be doing again. :)

The groundwork was laid out when this was all released back in 2010, but now I'm working alone at my own pace to tweak everything, one system at a time, at my own pace. There is also so much more available to the emulation community today as well in terms of high quality box and cart images and detailed information about even many obscure games that didn't even come up with a search engine hit back in the day.

I can't even begin to put a complete list of things I've been working on right now (although I intend to in the future), but here's some of the highlights.

As of today, there are 2020 games included in the collection I'm working on for the NES. This includes every licensed US and EU game. (Unless there is notable differences only one version is included and US is given preference). All Japanese licensed games that are playable by an English speaking audience are also included. (Again, US is given first preference, and if it was an EU/Japan release then EU was given preference).

Also included are somewhere close to 400 English translations for the NES and FDS. (Every current one available at romhacking.net as well as a few that aren't).

There are hundreds of Asian Pirate games, as well as Prototypes, original Homebrew games and even hundreds more rom hacks (most also from romhacking.net).

Every game has a "synopsis file" attached to it. Years ago for the XBox this could be a very large file that you were able to pull up from the rom-list or while in game to read info about it. Today, I've been re-editing most of it for use on the rom-list on the RetroPie. There's no room for novels on the gamelist screen, so I'm trying to keep these down under 1kb for each game (Some of them reached 50kb on the XBox).

Every synopsis also has a website at the bottom of it as a citation from where the info was obtained. Many are from places like mobygames, gamefaqs, consoleclassix and even wikipedia.

Every game has a box, cart, title screenshot and action screenshot attached to it. Because of the much improved artwork available today, this area was completely redone as well and now a majority of the images in the collection are of a HUGE size that isn't even playable on the Pi 3. When the time comes, these will all be re-sized 3 times. (Once for the XBox, once for the Pi Zero, and once for the Pi3).

We have what we called "Generic Boxart" for about 200 of the games. Originally this was created by a great French artist named Gilles for the XBox, but those images are now very small compared to the new boxart. Thanks to the talented @chelochelini2014 here, we have new and much larger images for these games. They are for the categories "Homebrew", "Pirate", "Unlicensed", "Pirate Fighting", and "Prototypes". Even today it's hard to find images for these categories, and sometimes they just never existed.

Although there are MUCH better cart images online today for many games, especially the official ones, I've personally made about 300-400 cart images for games missing them. This was usually done by taking the boxart for a game, making a "label" from it, and then putting it on a cart template image. (In the spreadsheet I will talk about below, these are always remarked so in the future it would be easy for somebody who wanted to try to update them with actual carts as they become available would know where to start).

@darknior who is also a truly talented graphic artist has been upgrading Japanese artwork for many different systems that had lousy scans online. Where available, these will also be included in the set. His team is only working on "best of" collections by system, so any games that were not touched up by him will be touched up by me instead. They are serviceable and look much better than anything else available online, but I'm nowhere near as talented as his team or chelochelini who made our new Generic Boxart.

Rom zipfile names adhere to a standard of 42 characters or less which was necessitated by the XBox file system.

On the XBox, all Madmab Edition emulators were able to display the full game title through the use of the top line of a game's synopsis. Thanks to @meleu, we now have a great script that will convert all information and the location of all related media files to a game into the gamelist.xml so everything will be usable so long as everyting is in the proper directories.

This script also takes the first line of these synopsis files to create the game title tag in the gamelist.xml that will be displayed in the rom list.

The spreadsheet...

I'm still not ready to give out the link, but here's some highlights of what kind of info will be included with it.

A. Zipfile Name
B. Full game title (Synopsis 1st line)
C. Name Inside of Zipfile (Nobody ever used this in any romset I've ever seen before. It's a great tool. I won't go into details now about why, but I will at a later date).
E. Matching GoodNES 3.23b CRC match file name (if available)
E. Matching No-Intro CRC match file name (if available)
F. The website cited on the synopsis for the game.
G. Compatibilty for both XBox emulators (in the future it will have compatibility for Pi emulators as well).
H. Associated preview video size and type. (We have this for around 1,600 games from the old set, but those are all 480 because XBox was just SDTV for most people back then. I plan on re-doing every single video myself to ensure that all preview videos have the same format. ie: volume level, playtime length, making sure it looks like somebody is actually playing instead of mindlessly dying or only showing boring parts of RPG style games instead of action parts, etc.
I. Is the game manual included?
J. Box art and Cart art maximum dimensions (before the resizing for each system mentioned above takes place).

Well.... that's long enough for now. There's so much more that I could go into about things that have been done. There have literally been over 1,200 hours on my part just this year working on this, and if you include all of the hours that all of the people put into it combined when the project was being worked on years ago it wouldn't surprise me if we're talking over 25-50k hours. :)

The spreadsheet will come soon. I know I keep promising it here and there, but I don't want to put it up until I feel it's ready.

Thanks for giving me an opportunity to talk about this now. This is all stuff that needed to be written out eventually. I'd expect to see a re-wording of a lot of this when I release the spreadsheet and start a thread about the project.

If you made it all the way to the end of this post, it's much appreciated. :)

EDITED TO ADD:

Whoops.... I forgot to mention a VERY important part.

I will never actually release any roms anywhere. I will be creating a datfile however that can be used to make acquiring the proper roms much easier for people on their own. This is where things in the spreadsheet like associated GoodNES and No-Intro names come in handy.

I'm hoping maybe @meleu or somebody else might help me create a script that could take the roms altered in romcenter or clrmame pro and move them into the proper folders for you as well so you wouldn't have to create many different folders on your own.

zerojay

Just a reminder that most of those Xbox emulators and some of the MAME engine changes they made for COINOPS never got their sources released as they were supposed to.

Used2BeRX

@zerojay Yeah. You're never going to see source from CoinOPS.

I wasn't even talking about porting over emus from the XBox though. I've already talked to people about porting old versions of mame that would work with the existing romsets I have from that era, but nobody is interested in doing that.

Most of those emulators are based off of old versions now anyhow. Because of physical limitations of the XBox they had to just make changes to them. What I'm talking about here is integrating any changes in the emulators used on the RetroPie setup where needed. Things like mapper updates.

Who knows.... Maybe most of them were fixed on emus used here already? I won't know until I begin testing. I was just wondering ahead of time what would need to be done to get these fixes in place.

I won't have any involvement in any arcade stuff other than just using what other people come up with. I'll be doing consoles and handhelds that generally already have non-changing verified dumps. Mame is too much of a hassle with all the constant changes and I can't be arsed with it anymore.

Used2BeRX

Does anybody think I should add a column to the spreadsheet for the matching CRC TOSEC game file name as well as the No-Intro and GoodNES 3.23b?

I'm not sure if people really use TOSEC anymore or not.

People told me not to bother with GoodNES 3.23b because they said it was massively out of date. It turns out that this wasn't really true at all. When you see the spreadsheet you'll find that the "Good" rom is a match to the most current No-Intro rom about 99% of the time. I actually took preference with Good over No-Intro on the NES collection because No-Intro completely removes the iNES header from the games since they weren't an actual part of the dump. This was a problem though because in most emulators even in 2017 the games won't play without this header.

arcadez2003

@capeman said in Does anybody work on Emulators to fix broken games here?:

I'd personally like to see somebody make RadMobile work on Mame2003, haha.

What's the problem with Rad Mobile exactly..??

Used2BeRX

So I've begun going through the synopsis files for the Official releases again in an attempt to make sure that all information fields are accounted for and filled in correctly as well as shrinking the body of them down to under 1kb a piece.

Something I've noticed is that a lot of the game descriptions out there are pretty boring and just informational, but if you look around at different gaming sites you can find some pretty good descriptions for the games... yanno... like a quick blurb that would be on the back of the box or a game advertisement to grab your attention.

I'm going to be doing my best to have a description like this for all of the games in the collection.

Example: Othello, NES, US Licensed

Launchbox description:

"Black and White. The battle of opposites... as old as time itself... comes to life in Othello. Its origins are part Western, part Eastern. Othello is fast-paced, as easy to learn as checkers, yet as fascinating as chess. Play against a friend. Or play against the Othello computer in four challenging skill levels. You play the black discs and your opponent plays the white. Trap his discs between two of your own, and his "flip" to your color. One move can change the entire game. Finding that move is the secret to Othello. That's why it takes a minute to learn... but a lifetime to master!"

Moby Games Description:

"Othello is a conversion of the board game. Each player tries to fill the game board with as many discs of his color as possible. On each turn the player places a disc on the board by selecting a location where the opponents discs will become trapped between two of his discs. The trapped discs are flipped over to the players color. The game ends when the board is filled or time runs out, and the player with the greatest number of his color discs on the board wins. Two players can play against each other, or one player against the computer."

One of these things is more exciting than the other. :)

But there are plenty of times when Moby Games has a better description too. Heck, even Gamefaqs wins that competition sometimes.

I'm not necessarily looking for the best one. I tend to look at Moby Games first, then launchbox, then gamefaqs, then wikipedia, then anywhere else. If the first choice or two have a great description, I don't even bother checking any more. I haven't found any in about 100 games so far where I couldn't find something pretty good like the one above.

Capeman

@gamez-fan said in Does anybody work on Emulators to fix broken games here?:

@capeman said in Does anybody work on Emulators to fix broken games here?:

I'd personally like to see somebody make RadMobile work on Mame2003, haha.

What's the problem with Rad Mobile exactly..??

As far as my testing goes, any game running on Sega's system32 is slooooooooowwww on a raspberry pi if it works at all.

arcadez2003

@capeman said in Does anybody work on Emulators to fix broken games here?:

As far as my testing goes, any game running on Sega's system32 is slooooooooowwww on a raspberry pi if it works at all.

Although my V60 will be different from the one in MAME78 aka 2003 with regards to clock devides this should work it's a trick i used to speed up
these games on the xbox when using the newer Segas32 video code which is very resource heavy, in MAME2003 you have the older faster video
code so hopefully underclocking the main CPU will get you a nice performance boost to make these games playable.

If you can compile your own source do the following and then make a new build, in drivers/system32.c find..............

static MACHINE_DRIVER_START( system32 )

/* basic machine hardware */
MDRV_CPU_ADD(V60, OSC_A/2/12) // Reality is 16.somethingMHz, use magic /12 factor to get approximate speed

And replace with

static MACHINE_DRIVER_START( system32 )

/* basic machine hardware */
MDRV_CPU_ADD(V60, 12000000 )// should be 16mhz but make it this to speed things up for the Xbox

If the above works we can likely do the same for the Sega Multi32 games eg Outrunners etc etc

Used2BeRX

Strangely enough, gamefaqs seems to have the "best" brief description for the games out of mobygames and launchbox and gamefaqs in a majority of the cases. I started out looking at gamefaqs last, but now that's the first place I check.

Somebody must have added some cool blurbs about most of the official licensed games on their site at some point over the years, because when I was using that site to do this years ago it was mostly generic descriptions just saying the name of the game, what type of game it was made, the developer and publisher and the date of release.

My original NES synopsis collection size was MUCH larger. Because of the nature of the gamelist.xml and how EmulationStation handles them, a smaller size is necessary. Even with this smaller size per-game, this body information might have to be left out of the entries on a Pi Zero, but hopefully with a Pi3 there won't be any problems with the massive amount of titles that will be included.

The amount of game titles and the combined size of the synopsis that was made 10 years ago: 1,769 files at 3.83MB

New files and size: 2,020 files at 2.29MB

I'm hoping to get this well under 2MB when it's done.

Used2BeRX

Synopsis Update:

US Licensed completely updated. Every single field of information is filled in for every single game, and are as follows for Official Licensed releases.

[Full Game Title] (Synopsis First Line on XBox, Romlist Name on RetroPie)
Platform:
Region:
Media:
Controller:
Genre:
Gametype:
Release Year:
Developer:
Publisher:
Players:

The only data that will be double checked and/or altered are Genre and Players. I will be verifying by hand the number of players for all Official games when I re-test them on the Pi. I will also homogenize the Genre field when I can see this displayed in the romlist on the Pi.

TOTAL US LICENSED SYNOPSIS FILES: 648
TOTAL US LICENSED SYNOPSIS SIZE: 529kb

TOTAL SYNOPSIS FILES: 2,020
TOTAL SYNOPSIS SIZE: 2.25mb

Every body text has been reduced in size to keep each individual games under 1kb (1024 characters or less). In many cases one of the 3 websites I cited before had a great description as-is. In quite a few other cases I edited these down by removing superfluous information to keep the file size per game down.

I still have around 300 Licensed games to go through, including Europe and Japan games that aren't clones of US games, as well as Zapper games and VS Arcade System games.

After that, I will be going through all of the unofficial games including Translations, Hacks, Unlicensed, Prototypes, Pirates and then it's on to Famicom Disk System releases.

Still working on the spreadsheet. I hope to release that soon. :)

Oh... and I've been reducing the original synopsis files made years ago. This is because we only need the small blurb about the game in the romlist and the "novels" we used to have would likely cause a lot of problems when the gamelist.xml files by system would be needlessly huge, even with a Pi3.

Original files and size: 1,769 files at 3.83MB
Current files and size: 2,020 files at 2.24MB

Hoping to get this well under 2.00MB by the time it's done. :)

Used2BeRX

I just finished testing NES US Licensed games (not zapper or power pad). 100% of the games work on a Pi Zero with proper CPU clock settings and smooth video set. The only one that has any problems at all that I could see is Goal! II. Quicknes won't even play it, fceumm plays it slow and Nestopia has some audio stutter. Plays best in Nestopia though, and is still entirely playable.

I still have to re-test any 4-Player games when I have more controllers to make sure that function works, but all of the 4-Player games work with up to 2 players for sure.

Rion

@used2berx I have to say im mighty impressed with your dedication to this project!

Have you taught about starting the same thread over at the libretro forums?

Used2BeRX

@rion I didn't even know there were libretro forums. :)

That would probably be the way to go when I conclude my NES testing and have a list of all of the non-working games though, huh?

I wasn't really expecting any problems with the Official games, especially the US/EU ones. I've got a feeling though that once I delve into the pirate/unlicensed/translations/hacks testing that there are going to be a lot of them.

Used2BeRX

Finished testing US unlicensed games (Tengen, Codemasters, Wisdom Tree, American Video Entertainment, etc.)

100% working again, although unfortunately the 1 game that seems to suffer a bit with audio stutter is Alien Syndrome that wont' run at all in Quicknes, suffers gameplay slowdown in fceumm and has some light audio stutter in Nestopia on the Pi Zero.

I don't believe that any of these games would have any problems running on a Pi 3. The Unlicensed games overall were not a huge fan of the Quicknes emulator. About half or more of them would not play at all in Quicknes. Most of these played fine in either Nestopia or fceumm though.

Used2BeRX

Does anybody know if there is light gun support for NES games on the RetroPie, and if there is, how to set it up?

I'm going to skip testing these games until somebody lets me know. Onto Europe Licensed games for now.

EDIT: I'm actually referring to the ability to use the D-Pad with a cursor on screen. Both NES emulators on the XBox allow you to do this.

I'd also be curious to know if this could be done with some sort of USB Zapper gun, and if you would need a CRT television or not to do it.

THANKS!!!

Used2BeRX

Finished testing Europe Licensed games.

Only problem game was Devil World. It plays fine, but for some reason I am unable to select 2 player mode in any of the 3 emulators. Strange.

Used2BeRX

Finished testing Japan Licensed games.

All work except for 2 controller issue games.

Hyper Sports (Requires either Hyper Shot or Zapper support - Works on XBox).
Exciting Boxing (Requires special "Pack'In" Inflatable boxer control. Doesn't work on XBox either. Saw a video on Youtube that shows it supported in an emulator, but unsure what they're using.)

Used2BeRX

Nintendo VS Arcade machine testing complete. 5 non-working games.

VS Duck Hunt, VS Freedom Force, VS Gumshoe, VS Hogan's Alley don't work because I don't know how to use the Zapper on RetroPie. Additionally, Gumeshoe doesn't seem to work at all because none of the three NES emulators will even load the game at all.

VS Ninja Jajamaru-Kun "works" very limited. Fceumm is the only emulator that will load the game, but The only way you can play the game at all is with a 2 Player game. 1st player is always non-responsive. You have to wait for 1st player to die and then you can play the 2nd player.

All of these games work on the XBox.

On the Pi, none of them work with Quicknes... most of them just kick back to romlist, some a black screen on load and some play with very bad colors. You can't seem to insert coin with Quicknes anyhow.

Nestopia plays nearly all but 3 or 4 of them. You can insert coins using the trigger buttons in either Nestopia or Quicknes.

Used2BeRX

It will be assumed unless somebody tells me otherwise that there is no support for Zapper games (via actual zapper/CRT TV, or on the gamepad). Even if there is support for Power Pad (via joystick), it's nearly impossible to map any of those games in a way that makes sense, even on the XBox, so they will be considered unsupported here as well unless somebody lets me know otherwise.

Total Official NES games tested on the Pi Zero by category:

US Licensed: 648 Total, 647 play great, 1 plays mediocre.

US Unlicensed: 79 Total, 78 play great, 1 plays mediocre.

Zapper Games (Non VS machine): 22 Total, 0 play at all (unknown if zapper supported).

Europe Licensed: 29 Total, 29 play great (although Devil World won't let you play 2 players on any emulator).

Japan Licensed: 116 Total, 114 play great. (Exciting Boxing: Unsupported Controller, Hyper Sports: Needs Hyper Shot or Zapper support).

VS Arcade Machines: 29 Total, 24 play great. (4 light gun games not supported, Gumeshoe & Ninja Jajamaru-Kun won't play right in any of the 3 emulators.

Unsuppored Controller (Mainly Power Pad): 10 Total, 0 supported.

TOTAL UNIQUE LICENSED (AND US UNLICENSED) NES GAMES: 933

TOTAL SUPPORTED/WORKING IN RETROPIE: 892

Used2BeRX

Good news so far on the Translations. 100% of the games play so far out of the first 65 of 346 translated games. This includes a few of the Chinese pirates through the letter D as well. The only one with slight problems on the Pi Zero is the pretty sweet Platformer "Armadillo" with some slight audio crackle in Nestopia, but it shouldn't be a problem for anybody with a PI 3.

It will probably be a while before I get back to testing games, but I'll try to push on whenever I get the chance.

EDIT: Sweet.... Out of curiosity, I zipped way forward to "The Legend of Zelda: Triforce of the Gods" and it works. This was a Chinese Pirate back-port of "Legend of Zelda: A Link to the Past" on the SNES for the Famicom that was recently translated. It didn't play in Quicknes or Nestopia, but it works in Fceumm. :)

EDIT 2: Double sweet... "Bio Hazard", in the Chinese Pirate section works too. It was a Chinese Pirate back-port of the Playstation's first Resident Evil game for the Famicom. Unfortunately, this game isn't translated yet, but one can only hope.

Don't want to jinx anything yet, but the compatibility list for the more obscure games might be better than I was thinking on the Pi.