UI Ideas from NES Classic - Save States

CourierSS

I think most have seen the new NES Classic/Famicom Classic Edition from Nintendo, but for those who haven't:

NES Classic Edition:
http://www.nintendo.com/nes-classic

Famicom Classic Edition:
https://www.nintendo.co.jp/clv/

Basically, it's a miniaturized version of the NES/Famicom that is pre-installed with 30 games. It's offline only, doesn't play cartridges, and no additional games can be added. It's essentially a simple system that caters to nostalgia and has positioned itself to sell during the holiday season.

What interests me are some of the UI choices they did, in particular save states, which I think can be helpful in improving the UI of Retropie. So I'll start off by describing how the NES Classic does them and then talk about how they can be useful on the Retropie. Then I'll describe possible implementation of them. Here's an imgur album of the UI of the NES Classic that I'm interested in with descriptions http://imgur.com/a/DeHns (also can be seen in the Youtube links above, except for locking save state which is in other youtube videos), which I'll also list below.

If you're already aware of how NES Classic does it or just want to skip how NES Classic implemented it, scroll down to the bold letters below.

Save State Indicator

The NES Classic allows four suspend points, or save states, per game. Below each title are four circles that indicate whether or not the game has save states, hollow being not used and filled in to indicate in use.

The Retropie has much more than just 4 save state slots, so while the 4 circles look nice they aren't as practical for the Retropie where people can have many save states. Nevertheless, having an indicator as well as knowing which slots are occupied is useful particularly for those who use many save states. For example, maybe a person had played an RPG and made many save states but hasn't played it for a while. When the person returns, he might not know or remember which save states correspond to his last save. He might not even remember how many he has made. He might even skip slots while making saves. Currently, the person would either have to load the game and manually increment and load a state and then figure out which is the correct state he last left off in or use ssh/Samba into the folder and check which save state was last modified.

The proposed way would be for there to be save icon (floppy disk/usb stick/hard drive) with a number next to it corresponding to the number of save states made for that game. This indicator can be placed next to the title of the game or in the detailed description. This way a person can tell at a glance how many save states there are. It would also be nice if the save states can be seen while in the menu along with a corresponding time stamp, which I'll describe in more detail in a bit.

Creating a Save State
To create a save state in-game a person must hit the reset button on the NES Classic. Reason for this is most likely because the controller doesn't have extra buttons in order to preserve the original form factor and having hotkeys might not work out for the target demograph which includes those who aren't as comfortable with UI elements that can't be seen. Hitting the reset button sends the user back to the menu whereupon an icon with a picture of the moment when the game was exited indicates that a save state can be saved. Pressing the appropriate button then allows a person to choose which slot they wish to save to.

Making a save state like this isn't as useful on the RetroPie. In particular, due to the shortness of the controller wire, a person using the NES Classic will pretty much be within arm or leg's reach of the device at all times whereas a person using the RetroPie may have wireless controllers and can use hotkeys for saving.

What I do like about this though is that it creates a save state and saves a picture on exit. This makes it so that a person who accidentally exits the games, such as a friend, family member, and stranger who aren't as familiar with the Retropie, can easily return to the game and glance at the image and note where they left off. Of course, this doesn't help much in the case of rage quiting and creating an unoptimal save, but there's not much that can be done in that case other then hoping the person had previous save states beforehand. Still creating a save state upon exit is a nice idea.

Locking a Save State

When highlighting a save state, a person can choose to lock it to prevent it from being overwritten. This is indicated by the lock symbol.

This one is fairly straight-forward. Sometimes people might make saves to return to certain moments in-game, like fighting a cool boss fight again or running some obstacles. This guarantees that those save states can't be overwritten on accident while in-game.

Display Mode

When choosing between CRT filter, 4:3, and Pixel Perfect displays, there are pictures that indicate how each will look.

This one I also like since it indicates a general idea of what these options do. However, I figure this would be difficult to implement, considering that everyone has their own tastes on filters and how different filters may be better for different systems or games. Some people might like filters on certain games but not on others, so a universal toggle might not work well. I guess the proposed way would be to have standard filters decided upon for each system. For those who like to choose their own filters or no filters, they can override the default (via Retroarch?). This way, when the display modes are toggled, all games of that system will then be affected by that option except for those that have custom filters/settings applied.

Now how I think a save states UI can be implemented in RetroPie.

Anytime a save state is created in-game, the screenshot feature should be triggered and the time should be recorded. Time might not be accurate but the idea is that a person can look at it and determine which states are saved earlier and later. Maybe playtime instead.
From the list of games in each system, a person should be able to see a save state indicator with a number (next to the title of the game, in the detailed description section, or somewhere) to indicate the number of save states. Pressing a button, such as the X or Y (saving one button for favoriting and can't use arrows due to the potential Grid view), would then bring up a detailed view of the game. In the future, this detailed view could have information like a video of the game, screenshots, manual, etc., and save states. For now the first four aren't likely to happen soon, so I'll just focus on save states.
Upon pressing the button, a menu would pop-up or change to another screen with all of the save states of that game. The latter might be easier to implement since it would be changing the screen just like going from the carousel to the system, but I'm not knowledgeable at all about how Emulation Station works. Additionally, it makes it so that Emulation Station doesn't have to immediately have all save states available to be seen upon RetroPie being turned on. This would be more useful if a person has many save states. But again, I'm just taking a guess on how Emulation Station might be working.
Each save state would have a corresponding image of when the state was saved, a time stamp or playtime (to tell which save states are earlier or later), the save state number, and maybe a possible short title that a person can edit (using select button. Ideally RetroPie would have an on-screen keyboard). For example, "Jenova Boss Fight" for Final Fantasy VII.
The first save slot would be reserved for the save state created upon exiting the game which would trigger both the save state and screenshot features.
For each of the save state, except for the first, a person can lock the save state to prevent it from being over written (X or Y button or Select). This would then place a lock symbol next to the save state.
By clicking a save state in the menu, the emulator would load the game and then load the save state.
Ideally while in-game a person could invoke a similar pop-up menu with save state numbers, picture, and time stamp, but I'm not sure if that's possible.

I think the above would be very nice from a UI perspective in making Retropie more user-friendly. I'm not sure how feasible it actually is in implementing what I mentioned, but I figure they're worth hearing and considering.

backstander

@CourierSS I like your concepts for overhauling the Save States UI and here our my thoughts on improvements.

Anytime a save state is created in-game, the screenshot feature should be triggered and the time should be recorded. Time might not be accurate but the idea is that a person can look at it and determine which states are saved earlier and later. Maybe playtime instead.

I think the playtime would be best for 2 reasons:

1st, the Raspberry Pi doesn't have a RTC so it looses the time on every reboot but gets the time from the internet. If you're not connected to the internet then you don't have accurate time.
2nd, sometime I will go back and play an older save, like if I messed up or wondered what would happen if I made an alternate choice but then I want to go to that further save again. I don't know if that makes any sense but it would make this easy if it had playtime.

For those who like to choose their own filters or no filters, they can override the default (via Retroarch?)

I kind of wonder if this would be better accomplished in the Runcommand and from there you could set global settings for that game system or per game settings.

Ideally RetroPie would have an on-screen keyboard

The Save State UI would probably be best to be part of Retroarch because you would want to be able to do it while playing a game but I also think an on-screen keyboard would be nice in EmulationStation to help with scraping games. The original author of EmulationStation has decided to not work on it anymore so I have a feeling that if someone else doesn't fork it, than RetroPie will need to use another GUI (like for example Attract-Mode).

Of course, this doesn't help much in the case of rage quiting and creating an unoptimal save, but there's not much that can be done in that case other then hoping the person had previous save states beforehand. Still creating a save state upon exit is a nice idea.

I think it would be nice to have an option to be asked if you want to save a state while exiting a game and also have an option to automatically save state on exit then automatically load that same save state on load of game, or an option to have this off (for those that don't like it). Maybe have these options globally per system or just per game.

Some of these ideas might not be feasible with the amount of memory and process power that's included with the Raspberry Pi. Might be more fitting for an x86 box running Linux and RetroPie. Then again I kind of like RetroPie because it's lightweight and is very customize (if you know where to look) and adding too many features would loose this lightweightness. Anyways I do enjoy your vision of Retro Gaming Future!

CourierSS

@backstander

I agree with your ideas. I would like to contribute to the code, but I only have basic programming skills and lack familiarity with RetroPie so even if I wanted to implement the above I'm not quite sure where to start (actually not even sure how to compile).

1st, the Raspberry Pi doesn't have a RTC so it looses the time on every reboot but gets the time from the internet. If you're not connected to the internet then you don't have accurate time.

For the playtime, I know that SDL has SDL_GetTicks which returns the amount of time elapsed since it's been initialized in milliseconds. Unfortunately, again my lack of familiarity with RetroPie means that I have no idea how SDL is being used (so someone please correct me if I'm wrong), but assuming that SDL is continuously running, SDL_GetTicks can be used to grab the time when starting a game and saving a game. Taking the difference would give the current session's playtime and then adding it to previous total playtime (which should be recorded into the gamelist.xml files every time a game is exited) would then give total playtime. The downside is that the command works for about 49 days before it loops back to 0 milliseconds, which is problematic to those who continuously run their RetroPie without restarting, but I think it can be worked out. So this function can maybe be used to record playtime for games in general as well as playtime for save states.

This assumes that SDL doesn't crash (maybe some other function that grabs the time from the OS would be better), but maybe this would be the simplest to begin with first? I think many people would be interested in how long they've played a game even without implementing State State UI.

Actually, a bit off topic but I was thinking about starting with something simple to familiarize myself with the RetroPie code.

The idea was that if a person accidentally presses Configure Input in the Emulation Station start menu, he should be able to get out of it without needing to press Esc on a keyboard or inputting all of the buttons again. I think this can be done by saying to hold down a directional key + any button for a couple of seconds.

It can't just be any random two buttons because on first setup if a person accidentally holds down two buttons, they'll be kicked out of the Configure Input menu with a controller that's not configured.
If we look at it from the perspective of those who aren't unfamiliar with RetroPie (friends, family, strangers), it also might not work to say press the A + B or X + Y buttons because people might be using arcade sticks (which probably aren't labeled) or the buttons that EmulationStation says to press might not correspond to the name of the buttons on a controller (SNES vs Xbox vs PlayStation controller layouts).
Additionally, it prevents someone with an improperly configured controller from easily exiting the Configure Input menu.

Essentially, a properly configured controller should be able to get out of Configure Input while a controller that isn't should not be able to.

So a while ago I looked at the seeing how to implement this but became confused on what EmulationStation was using for handling inputs as SDL provides Joystick functions and Game Controller functions, but the code made it look like it used the Joystick one even though Game Controller appears to supersede it. And then I wanted to determine the number of buttons held down, which appears to be done by SDL_GetKeyboardState but I'm not sure if it works with joysticks/game controller. However, it seems like another thread, Analog Trigger Buttons in Retropie, mentions how EmulationStation is handling SDL so I guess I can take a look at it again.

I guess this became a bit of a ramble.

Rookervik

Yeah, RetroPie is a collection of software, not a central application. A program called EmulationStation is the UI that shows you all of your games and allows you to run them. Then a large portion of systems are launched with Retroarch, which is a totally different application. Many other emulators are needed to run the remaining systems.

Save states are managed by the individual emulators. "RetroPie" really has nothing to do with save states at all.

We also can't currently make an EmulationStation theme similar to the NES Classic. We don't have scrolling images right now, and probably won't. There are some people working on Grid-view. Which would be similar to the NES Classic, but more up-and-down and not a sliding view like WiiFlow.

But do remember, RetroPie is the gathering of all of these softwares, and the scripts used to update them and write controller configurations. A lot of the things mentioned here are specifically EmulationStation or specifically RetroArch, which RetroPie doesn't really have any say in what direction they go.

Zigurana

Hi @CourierSS, thanks for taking the time to sketch out your use-cases, because, well, if it isn't really feasible at this point, it at least gives a clear end-goal for what a consolidated Retro-gaming solution might look like. I think all development should start with such "dreams" because it creates a momentum (also for other devs) much greater than 'try to build tiny incremental enhancement X' could ever achieve.

Said that, the picture you paint is (at this point in time) only remotely doable for the RetroArch - based systems. As @Rookervik indicated above, Retropie really is a collection of programs/scripts, and RetroArch is just one of the many emulator-engines that we use.
For DOS-based games for example, we often use dosbox, which then emulates a dos environment, and savestates would be handled completely different for that system. Similar for Amiga, ScummVM, etc, etc.
The main issue is that EmulationStation is basically agnostic to all these systems, it just runs the runcommand scripts on start and upon exiting a game. Once the 'on-end' script gains focus, it is too late to create a screenshot and a save point / snapshot.

Regarding your other idea on the controller setup, why don't you start out with a 5 or 10 sec time-out to exit the setup-wizard automatically? It will give you enough headaches in terms of getting to know ES, and will bring 90% of the wanted functionality. (I think the button - combo thing will be a pain to communicate clearly to the user).

CourierSS

After reading both yours and Rookervik's explanation, I think I understand a bit more of the relationships between the various programs/scripts and that I should fiddle around more with RetroPie to see how everything works together. Seems like implementing a save state UI is more complex than I imagined, not that it wasn't already complicating enough. Just testing a moment ago, I realized that saving to the same slot with lr-picodrive and lr-genesis-plus-gx cores for the Sega Master System would overwrite the same file because they save to the same folder by default which also brings its own problems.

When I was thinking about an alternative way of exiting the controller setup, I was recalling how I would usually configure inputs in PC games and some emulators and the immediacy of being able to back out if desired which was why I suggested the button combo. However, having an exit based on a timer does sound simpler to understand than the button combo. This feels like both would need to be implemented and tested to see which could be better. I think the relevant file would be GuiDetectDevice.cpp so I'll look into that when I have the time.

I guess if it had to be a specific button combo, it would be any button plus the directional key Up and hope that people would associate that with going up a folder in a file explorer or going back up to the menu haha