NEW RetroPie Web Console

[MileSacul]

Hello All,

I am excited to tell you that the RetroPie Web Console Alpha release is now available.

This is a web-based tool that hooks into the RetroPie-Setup scripts and allows you to install packages.

The best part is you can select everything you want to install, walk away from the computer and periodically see what it is currently being executed, failed, or successful. It also utilizes a queue with priority settings, so you can choose what goes in first and what goes in last.

Keep in mind, it currently does not support interfaces that use a GUI or manual input like ps3controller pair.

Supports now:

• Automatic pull of RetroPie-Setup if not existing
• Basic Install
• Manage Packages (Including search by type, name, description, and installed)
• Update RetroPie-Setup Script
• Reboot computer

Plans for the future:

• Execution History
• Queue update (Change priority order)
• Kill job (GUI executions hang)
• Batch update settings
  • Run updates/installs when you are not playing games.
• Upload/Download ROMS, saves, and states
• Export configs
• GUI hooks
• Uninstall RetroPie
• Install multiple things at once
• Sync between other RetroPie Web Consoles (RetroPie Super Web Console?)
• Development hooks
  • Develop Node.js scripts that I can hook into with the Web Console
  • Compile code.
  • Push to GitHub.
• VNC Viewer?
• Performance Metrics?
• Cook bacon?

[caver01]

@milesacul Is there a video demo of this in action?

[meleu]

@milesacul screenshots, please! 😉

[MileSacul]

I will work on getting a video in.

[MileSacul]

[MileSacul]

I also added these images to the GitHub Wiki

[Ganja]

This looks Awesome!

grabbing this tonight to test.

[herb_fargus]

This is probably the 5th retropie web console of sorts? Always good to have variety I guess.

I think it's a tad funny where the readme says " The purpose of this utility is to simplify the setup of RetroPie, which requires command line knowledge." Yet there is more command line work to install the tool than there is to install retropie. If this is meant to be simple, it would probably be beneficial to code an installation module that can be run through the setup script.

Just some constructive criticism, I'm looking forward to seeing the development.

[MileSacul]

Regarding, 'more command line work' I totally agree with you. While writing the README, I was telling myself that I need to simplify that part too (Hard part is adding to crontab). Once at Beta stage, I will have that figured out.

As for being another RetroPie Web Console, I only was aware of RetroPie-Manager, which it didn't support installing and configuring RetroPie.

I will look into the other web consoles to see if there are more offerings I could potentially add.

[cyperghost]

@milesacul Just for interest. Why Node.JS? How big is the install of the whole package? Can we compare with a php plattfrom like lighttpd or nginx?

[MileSacul]

@cyperghost There are several reasons why I went with NodeJS.

1. I was interested in learning NodeJS in general. The thought of building a web application with javascript sounded amazing.
2. NodeJS allows npm, which really helps me manage what external tools I am using.
3. Configuring REST APIs with NodeJS is easier than PHP (Disclaimer, I haven't used PHP in years, so it might have changed)
4. NodeJS works well with MonogDB, which is a database program that handles JSON objects.

I ran a check on all the apts my tool uses. On individual installs I see this:
nodejs
After this operation, 50.9 MB of additional disk space will be used.
mongodb
After this operation, 208 MB of additional disk space will be used.
libssl-dev
After this operation, 6,480 kB of additional disk space will be used.
libcurl4-openssl-dev
After this operation, 961 kB of additional disk space will be used.
vim
After this operation, 28.2 MB of additional disk space will be used.

As to the full size of my application and all of its dependencies, it totals ~1.3 GB. I verified this by running du-ch before and after running install.sh.

[darknior]

WOAW i love your project <3
It's always ccol to have new easy interface to manage our PI :)

I have think a moment to take Retropie-Manager and update it with JQuery, because i love it and it is powerful.
But Node.JS is better ... you can do many more things with it :)
Now like write @herb_fargus there are some manager already existing ... we can't use them all ...
Have you plan to include Retropie-Manager functions to your manager ?
To let us with only one manager, more powerful, and with the install script to add it from Retropie-Setup :D
And maybe the last Recalbox manager can also give you some good idées ;)

Thanks a lot

[herb_fargus]

Recalbox web manager:

https://retropie.org.uk/forum/topic/2303/retropie-manager-web-app-recalbox-manager-fork-mod/

https://github.com/botolo78/RetroPie-Manager

Retropie web gui 1

https://retropie.org.uk/forum/topic/4627/retropie-web-gui/

https://github.com/fechy/retropie-web-gui

Retropie web gui 2

https://github.com/gwhitcher/RetroPie-Web-GUI

Rom visualiser

https://retropie.org.uk/forum/topic/8020/rom-visualizer/

Webtropie

https://retropie.org.uk/forum/topic/10164/webtropie-wip-was-web-app-wip-please-give-it-a-name/

https://github.com/gazpan/WebtroPie

YARman

https://retropie.org.uk/forum/topic/8266/yarman-web-beta

https://github.com/daeks/yarman

Retropie metadata editor

https://retropie.org.uk/forum/topic/4152/retropie-metadata-editor

https://rme.codeplex.com/releases

And now yours :) though there are probably a few others floating around too

[caver01]

@herb_fargus But you gotta love "Webtropie" for that name alone!

[darknior]

WOAW idon't know this exist all these Web interface, so fantastic <3
I will try some of them ...
Webtropie and YARman are so excellent :p

Wish will merge all these fantastic WEB UI lol

[MileSacul]

@darknior As mentioned, "I will look into the other web consoles to see if there are more offerings I could potentially add."

I agree that having 8 different web consoles for managing one application is excessive. The hard part is making this work on Mobile devices.

My biggest goal is the RetroPie Super Web Console. This will essentially be a manager where you can push configurations/ROMS/etc from a single computer to all of your Pis. I already kind of have special hooks for this by utilizing REST APIs. For example, if you go to http://{IPADDRESS}:3000/apps you will see JSON objects of all available packages on your Pi.

[cyperghost]

@herb_fargus

• What are the reasons you think, that all these projects get stuck in development?
• What frontend got the best development? (I knew only Recalbox web manager)
• I think WebtroPie and YARMAN have both also good development ;)

@MileSacul
About filesize... I compared nginx and lighttpd install via aptitude. They need just round about 1MB and the server setup is usually very fast. Therefore I asked for NODE.JS and it's size. Also the database setting a SQLite* base would be a more common used one (up to now!).

So it's of course your decision what you want to do in your sparetime but if you really want to develop a frontend then fork the farthest developed frontend for now. I think it's the Recalbox web manager as it was one of the first. Maybe herb or other members can get annother idea here what seems the most developed RetroPie-WebUI.

Really no offence just my two cents! As there were so much frontends being in development stages but noone gets finished and it seems for me that all available programming languages were used :)
Thank you for your efforts ;) ... and I hope that one webgui gets finished

EDIT: Read more about webservices of RetroPie here

---

*I deeply hope that the gameslists and the metadata will be converted in one database like SQLite for portability reasons.

[herb_fargus]

@cyperghost idk. Everyone has their own idea of doing things and most weren't planning on long term dev or maintenance so much as just doing it cause they could. Some are bloaty, some are lightweight. We all do this in our free time so people pursue things they are interested in.

Anyways I'm interested to see how OP handles it, I welcome any new ideas, regardless of how long they last

[cyperghost]

@herb_fargus said in NEW RetroPie Web Console:

Anyways I'm interested to see how OP handles it, I welcome any new ideas, regardless of how long they last

Agreed ;)

@MileSacul Give us a shot of coding magic ;)

[MileSacul]

@cyperghost
While I like the idea of making this thing lightweight, I also have to look at application response times, developer adoption, performance footprint, the lifespan of used technologies.

I used these sites to collect initial thoughts:
HammerPrinciple SQLITE VS MONGODB
StackShare's MongoDB vs SQLLite
Web Technologies Survey
Node vs Apache vs Lighttpd vs Nginx
Apache Vs Nginx Vs Node.js And What It Means About The Performance Of WordPress Vs Ghost
Top 8 Node.js Best Practices For Your Startup

Keep in mind, the only sources I really consider reliable are StackShare and the Web Technologies Survey, but to be honest, finding valid performance reports on these technologies were a failure. I probably have to build them myself if I ever plan to (probably not).

Response times
A lot of analyses showed that Nginx pretty much beats NodeJS as a web server. While Lighthttpd is faster in smaller input/output, it gets slower as you give it more and more requests. Overal, NodeJS gives slower response times as a web server even when clustering.

As for MogoDB vs SQLLight, there were only thoughts on adoption and that both perform equally as well.

Developer Adoption
I was blown away by this statement from "Top 8 Node.js Best Practices For Your Startup:" To summarize, JavaScript is the #1 most-used language on GitHub and it looks like this trend will continue.

Although, the Web Technologies Survey showed that Nginx was only second to Apache and claimed over a third of the market while Node.js only took .3%.

As mentioned before, the fact that I can use Javascript both for my web application and my web server it means I don't have to switch my brain from going from one technology to the other. Especially when going from PHP (which Nginx uses) to JavaScript. Partnered with AngularJS, I can get help from anybody that knows JavaScript. Using one language to rule them all kind of strategy :P.

As for MongoDB vs SQLite, you are right about more people using SQLite. After all, it follows Oracle, DB2, and MySQL in terms of query syntax; I had quite a learning curve figuring out MongoDB because I am used to production databases. Also, only SQLite supports foreign keys, which is something I really like. There are a few pros for MongoDB though:

1. It stores JSON objects, which means no custom hooks to convert my queries
2. It allows JSON objects for query search, which yet again means no conversion.
3. It requires no real database schema. While Mongoose does use a schema, updating it is as simple as changing it in my JavaScript code. In terms of development, that means I don't have to include SQL update scripts with each database schema change.
4. It allows storing of arrays in a column. All of the functions that you see on the manage page is stored as a single array in a column.

** Performance Footprint **
Even though you can run very intensive operations (opt/exp lr-mame builds from source) with my app, I think keeping the performance footprint low is a good idea. Keep in mind, I designed the web console to always be running. Even when users are playing their video games.

The big thing I see from most of the nginx and NodeJS application is how many threads they run on. NodeJS is designed to run a single thread. This means that it is less likely to cause performance issues, with exception of calling all the install commands because I use NodeJS child_process to essentially run outside of NodeJS on its own.

Ningx creates a new thread for each connection to it. This is troublesome because it means it can be a resource hog when a ton of people are trying to run things. Although I don't expect many people connecting to the Web Console at once, that could change when designing the RetroPie Super Web Console.

As for resource cost of SQLite vs MongoDB, I didn't see any metrics available online, so I will probably have to prototype both of them and do comparisons. Since a lot of the lr-mame builds generate MB worth of log results (that I store in the DB), I can just execute a full install of everything and monitor performance during both operations. Although, I doubt either of them are CPU intensive since they are really based of I/O speeds.

Lifespan of used Technologies
This one is probably the hardest of all. After all, when in high school I used Yahoo as my primary search engine and developed my first web application with NotePad and Netscape browser.

A lot of the reports show that NodeJS is more of a small fry and Nginx is 2nd only to Apache. While I think NodeJS can be considered a solid investment based solely on the fact that it is built off of the Google JavaScript engine, there is a chance that Google can end up like Yahoo. PHP has stood the testament of time.

I will probably have to use GitHub as my primary source for my decision. The trends are pointing to JavaScript, so I should try to at least move in that direction even if NodeJS might be replaced.

As for the databases, SQLite seems like it will stand the testament of time more, solely because it utilizes a lot of the well known database CRUD operations. Not to mention it is used by all Andriod phones. For now, I will continue to use MongoDB and after I get my Schema strategy really down, I will probably move over to SQLite or figure out some sort of hybrid between the two.

Overal Summary
All of them look like they could work as the backbone of my application, but I've actually already spent two months (in my free time) learning NodeJS and MongoDB, then applying them in an application. Maybe once I truly know what I am doing, I will dabble with moving around, but for now, I will stay on the same path.

On a side note, I found some very interesting ideas like using a hybrid Nginx and NodeJS application, since Nginx transfers static files like CSS and images faster.