WebtroPie

darknior

System is empty

Toggle Collapsed | use cfdump format: 
Query:
 - JSONQuery
{
-app: {
LoadAllSystems: false
ScanAtStartup: false
ShowEmptySystems: false
ShowEmptyDirectories: false
ShowAddFields: true
ShowGameCounts: true
ShowThemeSelect: true
ShowViewSelect: true
LogSystemTotals: false
ThemeSet: "carbon"
Language: "en"
DateFormat: "DD/MM/YYYY"
ViewTransitions: "Slide"
ViewStyle: ""
}
-env: {
has_gd: 1
has_launch: 1
read_only: 0
}
-lang: {
-md_labels: {
name: "Name"
rating: "Rating"
releasedate: "Released"
developer: "Developer"
publisher: "Publisher"
genre: "Genre"
players: "Players"
lastplayed: "Last Played"
playcount: "Times Played"
human_size: "Size"
image: "Image"
marquee: "Marquee"
video: "Video"
year: "Year"
hidden: "Hidden"
kidgame: "Kid"
favorite: "Favourite"
stars: "Stars"
description: "Description"
}
-buttons: {
launch: "Launch"
scrape: "Scrape"
save: "Save"
cancel: "Cancel"
reset: "Reset"
delete: "Delete"
move: "Move"
options: "Options"
menu: "Menu"
search: "Search"
back: "Back"
system: "system"
choose: "choose"
edit: "Edit"
open: "Open"
select: "Select"
}
-languages: {
en: "English (UK)"
en-us: "English (US)"
}
}
-es: {
BackgroundJoystickInput: false
CaptionsCompatibility: true
DrawFramerate: false
EnableSounds: true
MoveCarousel: true
ParseGamelistOnly: false
QuickSystemSelect: true
SaveGamelistsOnExit: false
ScrapeRatings: true
ScreenSaverControls: true
ScreenSaverOmxPlayer: true
ShowHelpPrompts: true
ShowHiddenFiles: false
SlideshowScreenSaverCustomImageSource: false
SlideshowScreenSaverRecurse: false
SlideshowScreenSaverStretch: false
SortAllSystems: false
StretchVideoOnScreenSaver: false
UseCustomCollectionsSystem: true
VideoAudio: true
VideoOmxPlayer: true
hideQuitMenuOnKidUI: false
MaxVRAM: "100"
ScraperResizeHeight: "0"
ScraperResizeWidth: "400"
ScreenSaverSwapImageTimeout: "5000"
ScreenSaverSwapVideoTimeout: "35000"
ScreenSaverTime: "300000"
AudioDevice: "PCM"
CollectionSystemsAuto: "all,favorites,recent"
CollectionSystemsCustom: ""
GamelistViewStyle: "automatic"
Local: "fr_FR.UTF-8"
OMXAudioDev: "both"
PowerSaverMode: "default"
Scraper: "TheGamesDB"
ScreenSaverBehavior: "random video"
ScreenSaverGameInfo: "always"
SlideshowScreenSaverBackgroundAudioFile: "/home/pi/.emulationstation/slideshow/audio/slideshow_bg.wav"
SlideshowScreenSaverImageDir: "/home/pi/.emulationstation/slideshow/image"
SlideshowScreenSaverImageFilter: ".png,.jpg"
StartupSystem: ""
ThemeSet: "dark-steel"
TransitionStyle: "fade"
UIMode: "Full"
UIMode_passkey: "uuddlrlrba"
}
-themes: {
stretch-carbon: true
stretch-clean-look: true
stretch-crt: true
stretch-cygnus: true
stretch-material: true
stretch-tronkyfran: true
}
systems: [ ]
-themes_list: [
-{
name: "famicom-mini"
}
-{
name: "carbon"
}
-{
name: "pixel"
}
-{
name: "ComicBook"
}
-{
name: "workbench"
}
-{
name: "snes-mini"
}
-{
name: "art-book"
}
-{
name: "dark-steel"
}
-{
name: "ComicBook_SE-Wheelart"
}
-{
name: "nes-mini"
}
-{
name: "material"
}
-{
name: "mini-psx"
}
]
}

kixut

@darknior thanks, I think it's going to turn out to be a permissions problem, does your web server run as a user that can read the gamelists files ?

darknior

@kixut It's work before ... i don't know. I supose Apache use root user, how can i verify this ? i use you script copy from here.

And i install your script i think at good place in : /opt/retropie/supplementary/WebtroPie/ with root user

darknior

if ($_GET['get'] & SYSTEMS)
{
	echo($_GET['get']);
	echo(SYSTEMS);

result :

25532
... systems":[]," ....

It is strange because your application is installed by part as ROOT and some other files PI
I always have execute it with root user for installation, maybe it's bad ?
But /opt/retropie/supplementary/ is a user root directory.

kixut

@darknior it should work from anywhere, there is no hard coded path to where it's installed.
I would just open a tab to http://192.168.1.31/app/svr/config.php?get=255 and edit the config.php and temporarily add some echo commands (see below), then refresh your tab, the list of systems should be every system from your es_systems.cfg, and the ones that have a true file_exists are the ones that go back to the client, if the path look right and the ones that you know exist say that they don't exist then it would be permissions, but maybe it's not getting the list from es_systems, or maybe there something wrong with the paths

    echo "<pre>\n";
    print_r($systems['system']);
    echo "</pre>";

    for($i=0; $i<count($systems['system']);$i++)
    {

        $system_name = $systems['system'][$i]['name'];

        echo "<pre>\n";
        echo $system_name."\n";
        echo "[".$systems['system'][$i]['path']."/gamelist.xml"."]";
        echo file_exists($systems['system'][$i]['path']."/gamelist.xml")."\n";
        echo "[".ROMSPATH.'/'.$system_name."/gamelist.xml"."]";
        echo file_exists(ROMSPATH.'/'.$system_name."/gamelist.xml")."\n";
        echo "[".HOME_ES."/gamelists/".$system_name."/gamelist.xml"."]";
        echo file_exists(HOME_ES."/gamelists/".$system_name."/gamelist.xml")."\n";
        echo "[".ES_PATH."/gamelists/".$system_name."/gamelist.xml"."]";
        echo file_exists(ES_PATH."/gamelists/".$system_name."/gamelist.xml")."\n";
        echo "</pre>";

        if (file_exists($systems['system'][$i]['path']."/gamelist.xml") ||
            file_exists(ROMSPATH.'/'.$system_name."/gamelist.xml") ||
            file_exists(HOME_ES."/gamelists/".$system_name."/gamelist.xml") ||
            file_exists(ES_PATH."/gamelists/".$system_name."/gamelist.xml")
            )
        {
            $systems['system'][$i]['has_gamelist'] = true;
            if (filesize($systems['system'][$i]['path']."/gamelist.xml") > 40 ||
                filesize(ROMSPATH.'/'.$system_name."/gamelist.xml") > 40 ||
                filesize(HOME_ES."/gamelists/".$system_name."/gamelist.xml") > 40 ||
                filesize(ES_PATH."/gamelists/".$system_name."/gamelist.xml") > 40
                )
                $systems['system'][$i]['has_games'] = true;
        }
       $config['systems'][$system_name] = $systems['system'][$i];
    }
exit; // this is all we're interested in at the moment

... never mind ... lets add another last one "/opt/retropie/configs/all/emulationstation/"

Edit: Ive added /opt/retropie/configs/all/emulationstation and pushed, I feel dumb now

darknior

If i understand fine :

if ($_GET['get'] & SYSTEMS)
{
    $array_types = array('system'=>true);
    if (file_exists(HOME_ES."/es_systems.cfg"))
        $systems = load_file_xml_as_array(HOME_ES."/es_systems.cfg");
    else
        $systems = load_file_xml_as_array(ES_PATH."/es_systems.cfg");
    $config['systems'] = array();

    echo "<pre>\n";
    print_r($systems['system']);
    echo "</pre>";

    for($i=0; $i<count($systems['system']);$i++)
    {

        $system_name = $systems['system'][$i]['name'];

        echo "<pre>\n";
        echo $system_name."\n";
        echo "[".$systems['system'][$i]['path']."/gamelist.xml"."]";
        echo file_exists($systems['system'][$i]['path']."/gamelist.xml")."\n";
        echo "[".ROMSPATH.'/'.$system_name."/gamelist.xml"."]";
        echo file_exists(ROMSPATH.'/'.$system_name."/gamelist.xml")."\n";
        echo "[".HOME_ES."/gamelists/".$system_name."/gamelist.xml"."]";
        echo file_exists(HOME_ES."/gamelists/".$system_name."/gamelist.xml")."\n";
        echo "[".ES_PATH."/gamelists/".$system_name."/gamelist.xml"."]";
        echo file_exists(ES_PATH."/gamelists/".$system_name."/gamelist.xml")."\n";
        echo "</pre>";

        if (file_exists($systems['system'][$i]['path']."/gamelist.xml") ||
            file_exists(ROMSPATH.'/'.$system_name."/gamelist.xml") ||
            file_exists(HOME_ES."/gamelists/".$system_name."/gamelist.xml") ||
            file_exists(ES_PATH."/gamelists/".$system_name."/gamelist.xml")
            )
        {
            $systems['system'][$i]['has_gamelist'] = true;
            if (filesize($systems['system'][$i]['path']."/gamelist.xml") > 40 ||
                filesize(ROMSPATH.'/'.$system_name."/gamelist.xml") > 40 ||
                filesize(HOME_ES."/gamelists/".$system_name."/gamelist.xml") > 40 ||
                filesize(ES_PATH."/gamelists/".$system_name."/gamelist.xml") > 40
                )
                $systems['system'][$i]['has_games'] = true;
        }
       $config['systems'][$system_name] = $systems['system'][$i];
    }
exit; // this is all we're interested in at the moment
}

I have an empty page :(

<pre>
</pre>

I don't understand fine for /opt/retropie/configs/all/emulationstation ?

darknior

/opt/retropie/configs/all/emulationstation is PI user
/opt/retropie/configs/all/emulationstation/gamelists PI
Each system and gamelist.xml PI user too.

darknior

No right problem, i have finally found :)
It's my fault, but i think you must upgrade a little your XML PARSER for guy"s that write code it don't like.
Now you can remove the code you add yesterday for me, and put again the game counter to show duplicate (PSX) games in console.log.

Explanations :

I have add error_reporting(E_ERROR | E_WARNING); to your svr/config.php file.
And i think you should add it too, for the moment we are not in production mode but most dev mode :p
I read my Apache error.log and then found :

[Tue Nov 14 13:51:48.686359 2017] [:error] [pid 26599] [client 192.168.1.11:58586] PHP Warning:  SimpleXMLElement::__construct(): Entity: line 406: parser error : xmlParseEntityRef: no name in /opt/retropie/supplementary/WebtroPie/app/svr/xml_util.php on line 154
[Tue Nov 14 13:51:48.686442 2017] [:error] [pid 26599] [client 192.168.1.11:58586] PHP Warning:  SimpleXMLElement::__construct():     &lt;command&gt;cd /opt/retropie/ports/openbor/ &amp;&amp; ./OpenBOR %ROM%&lt;/command&gt;  in /opt/retropie/supplementary/WebtroPie/app/svr/xml_util.php on line 154
[Tue Nov 14 13:51:48.686519 2017] [:error] [pid 26599] [client 192.168.1.11:58586] PHP Warning:  SimpleXMLElement::__construct():                                                ^ in /opt/retropie/supplementary/WebtroPie/app/svr/xml_util.php on line 154

Then i remember in my es_systems.cfg i have add a command line for OpenBOR, i give it to you, with :

<command>cd /opt/retropie/ports/openbor/ && ./OpenBOR %ROM% > /dev/null 2>&1</command>

I know it is not fine to not escape in XML, but ES is cool and understand it without crashing.
I have replace now by :

<command>cd /opt/retropie/ports/openbor/ && ./OpenBOR %ROM% < /dev/null 2<&1</command>

And all is working fine :)

darknior

Game Show / Counter :

Lynx : 44 same
Jaguar : 42 same
Atari800 : Not showed !
Atari7800 : 40 you show 35
Atari 5200 : Not showed !
Atari 2600 : 130 you show 100
Arcade : 436 you show 500 (i know it is because i have move my CPS games)

If i understand fine, it's because ES also count games on HDD not on gamelist.xml and add them.
For the not showed systems, it's because there is no gamelist, only file in directory to test games.

It's really cool to see this difference, to fix it if we forgot a game.
But it is also a problem to add a new game, we must first add it on PI, and restart ES to save the change.

Do you think it is possible for you to show files not in XML ? It is complicate because you must scan the directory with PHP, and it will take time to compare with XML games, but maybe you can do it only on demand. With a SCAN button in the bottom menu when we are on a system we want to work on.
And on the game list view, add a TAG we can show with filter, that show us if a game in the XML is on the HDD or not.

User interface

I have multiple bugs ... not same from the different skins ... i know it will be hard/long/impossible for you to fix them all ...
And for some i know you can't do nothing because a theme use % placement, it's cool, but it is made for a resolution, 1080p, 720p, 4/3, 16/9 ... and on PC we have other resolutions ... I assume skinner are not real coder and don't take the good origin point to place elements.
But the most important bugs are not many more :

The first is more a question than a bug, how do you choose the z-index of an element ?
My friend made it :

		<image name="logo2" extra="true">
			<path>./art/controller.svg</path>
            <maxSize>0.270 0.427</maxSize>
			<pos>0.335 0.100</pos>
			<origin>0 0</origin>
			<color>bbbbbb</color>
		</image>

And you put the element with z-index 200 in front of the video.
ES if i understand fine, show elements in order it read them, we declare them, but we can also add a z-index now.
My friend for sure has make the error to not use s-index to prevent bugs.

0_1510670904260_Sans titre-1.jpg

The BOXART image, change the z-Index to show it under the video.
You can see it on PIXEL theme, in front of the video.

0_1510670929891_Sans titre-2.jpg

The LOGO it is in front of the VIDEO too, and it is normal when you use VLC. But maybe you can add a switch to show it behind when the guy using OMXPlayer and you can see it in the ES settings from config.php

VideoOmxPlayer: true

Always the Last player at bad place, but i see no other skin showing it to compare...

MAYBE SKIN CODING PROBLEM :

In the NES MINI it use a caroussel_bar.png that have a z-index 100 in front of all other. We can't see the text and we can't use the menu. When i remove the ZINDEX all is fine.

0_1510674317724_Sans titre-3.jpg

Same with SNES MINI. On home screen we can't see system images because he can't found them, on game list same than NES, an effect is in front of the menu and we can't use it.

0_1510675166875_Sans titre-4.jpg
0_1510675174144_Sans titre-5.jpg

I don't read the skin code for the moment, but i think it is not your fault. They must better code the skins. Or it is also because you don't use the same ES element default oder.

IMPROVEMENT :

Not a bug but only an improvement, is it possible to PAUSE the video preview when we EDIT a game, because with time the music will take us silly lol Especially on Coleco Vision where the sound is crappy :p
Or/And is it possible to have a MUTE button on the bottom menu ?

Editing interface :

When we already have the image, you don't show the button/dragndrop to change/update it. Maybe you can let us clic the image to update it ?
When we don't have it i see the buttons, but the DragnDrop is not working for me on Chrome, and when i browse it don't upload the image to pi.
When i modify a text like a description, it don't save on pi too.
When do you save on gamelist.xml ?
Where do you upload images/video ? Are you using the default or my MEDIA path ? Because you can't know it on upload if we can't set it somewhere.

kixut

@darknior I'm so glad that you found the problem, when I tested I was using the earlier es_systems.cfg file that you posted and unless I've got them mixed up it didn't have the invalid code.

I think a priority must be for me to make that area more robust, it needs the es_systems config for the correct paths, rom extensions and system theme so it's important that it is loaded.

If scanning is switched on it will search for roms that aren't in the game list file, also the logging console mod that I made can be switched on in the settings.cfg (it's not in the app menu).

Edit: You can switch on scanning for all systems or as you say you can scan an individual directory, these show in the list with "New: " at the start of the name, you are right, it would be useful to be able to filter on that.

Edit: Regarding test system not in es_systems file, the old way I used to do it was to scan rom folder for folders with a game list, I could have it work that way but if you scan that directory it will not know what extension to look for but it's better maybe to show it and scan fail than not to show it at all ..

I know I have a problem with z-index, I need to understand properly how ES does it, the docs say that when extra is true that the elements are rendered in order that they appear in the file, and this what I try to do - unless z-index has been explicitly set, however, a theme can include other files, views may be combined, different elements can have different attributes from many files so when parts of themes can be in different included multiple files it gets hard to know what the element order really is. I know I should just put time aside to study the source code, but if someone could explain the algorithm in plain english I would be very happy :)

... I will have a read through the rest of the issues, and am happy to tackle them,

darknior

Ok cool, i will stand you read me :)

I think a priority must be for me to make that area more robust, it needs the es_systems config for the correct paths, rom extensions and system theme so it's important that it is loaded.

You are right because i have 2 problems after that. My 3DO and PCEngine CD are not loaded. 0 games.
And when i search in the XML i found two "&" in description. I replace them and it's rock again. Now i know, but some other can not understand why that don't load.

If scanning is switched on it will search for roms that aren't in the game list file, also the logging console mod that I made can be switched on in the settings.cfg (it's not in the app menu).

I have try without success this option. When i enter AMIGA i have 10 roms and it show me only 3 title. And on the 3 titles 2 roms are removed and it not hide them.
<bool name="ScanAtStartup" value="true"/>
It works perfectly for LOG, i love this option <3 I will remove my duplicate entry lol
<bool name="LogSystemTotals" value="true"/>

Edit: Regarding test system not in es_systems file

I don't have test system not in the es_system file, You not understand me but what where, i don't know what about you speak ? Maybe you speak about my SCAN option with a game TAG. It is es_system directories, i speak to scan real rom file to show or hide them from gamelist.xml . Like this the count will be same than on ES, and we can work on new games to add them to our gamelist.xml

I know I have a problem with z-index

Ok it's what's i understand to, ES read a CFG and show the element, the last element read is on the top.
I think with time more skinner will use z-index, i wish, because it is a problem and can make some bugs in ES too.

I stand you answer le last questions about Editing inferfase lol

kixut

@darknior

I will change the editor so that you can drag and drop to the image to upload a new image
The upload tries to write the image to either 'images', 'marquees' or 'videos' directory within the rom directory and should put the uploaded path in the relevant editor field.
I think it's failing if the directory doesn't exist, so it should try to create it which I will fix.
I know you would prefer it write to your external medias directory but it won't know to put it in for example ~/medias/snes/box art front directory... maybe I could add a setting similar to variables
<string name="uploadImageDirectory" value="~/medias/${system_name}/box art front"/>
I will test / fix saving issues
When you make a change it doesn't save immediately, you can undo, when you click save it should save

darknior

The upload tries to write the image to either 'images', 'marquees' or 'videos' directory within the rom directory and should put the uploaded path in the relevant editor field. I think it's failing if the directory doesn't exist, so it should try to create it which I will fix.

Yes i think same, i don't have these directories. I can make the trick.

I know you would prefer it write to your external medias directory but it won't know to put it in for example ~/medias/snes/box art front directory... maybe I could add a setting similar to variables
<string name="uploadImageDirectory" value="~/medias/${system_name}/box art front"/>

If you can do it, first time only in the settings.cfg to try, it will be perfect :)

When you make a change it doesn't save immediately, you can undo, when you click save it should save

Yes i have make the change, change game, come back to it, and i can see my change, but when i refresh the page my change disappear. Only saved in cache i think and not on the PI.

kixut

@darknior thanks, I hope to have a look at some of these changes over the weekend, I've fixed the little saving bug, sorry about that
Edit: I've also improved z-index calculation

darknior

Your Webtropie in action :)

Really so excellent, i stand for the 3 edition bugs update to try it more.

Description saving.
Images saving and upload.
Possibility to scan a system and show news games to add gamelist.xml

kixut

@darknior nice video although I will have a look at hiding that scrollbar, I think it only happens on Windows, was that with Chrome or IE ?, I have a few more uncommited changes to do with the NES MINI theme to get the carousel feature working properly with it and to better position its fonts. I think it's turned into a bit of an acid test for WebtroPie :)

darknior

@kixut said in WebtroPie WIP:

@darknior nice video although I will have a look at hiding that scrollbar, I think it only happens on Windows, was that with Chrome or IE ?, I have a few more uncommited changes to do with the NES MINI theme to get the carousel feature working properly with it and to better position its fonts. I think it's turned into a bit of an acid test for WebtroPie :)

Yes it's on Windows with Chrome, and i forgot to telle you to hide the scrollbar. On PI there isn't :p
If you can make NES mini working fine on Webtropie it will be excellent. Bugs are little different with SNES Mini. PSX Mini is almost working fine, it has the best code, but it is less interesting for your code challenge lol

For my part i stand more for editing features, because it is the real purpose of this application.
I have many stuff to manage, it will do a great test for your Webtropie ... and on 4, 5 different computers.

darknior

News updates yesterday ...
WOAW, NES and SNES Mini are really working better now :)

darknior

Hi, any news about your project please ? :)
Can you first correct the description saving?
I know you don't have so many free time, but if you can, it will save me many time on game editing for many gameslist.xml i must work on :p
I understand it will be take more time to fix image upload :(

madmodder123

@kixut THIS IS AWESOME