*nix based VM (VMWare) as RetroPie play and test environment

Ashpool

prelude

In past years I've used, from time to time, a Virtual Machine (VMWare Player) with Ubuntu as a guest system to try out some retropie stuff/settings/configs/artwork [Memory Lane: I think the first time i've done this was with Ubuntu 18.04]. Recently I tried it again to test some new version of the module-script generator for lr-mess, lr-mame and mame standalone. As @Folly pointed out Ubuntu may not be the best choice for this and there are various issues encountered and most propably more to find.

So, to offload such discussions from that thread (where it would be offtopic), I am opening this one to collect issues/workarounds (if possible) and discuss/mindmap/brainstorm the use of retropie within a *nix based VM using VMWare (Player/Workstation/Fusion) in general. As this is a somewhat fuzzy concept/idea at the moment the following posts will be reserved/left blank for now (and their content/designation may change in/over time) and we may see how this one will turn out/where it may lead us to :]

Some of the basic Questions include

What *nix Distro(s) are ideal to be used here (least problematic), what desktop environment to use.
what settings to be used when setting up the vm (minimum requirements, what could be considered overkill).

retropie in a virtual machine
There are some reasons why one may consider/decide to run retropie from within a virtual machine. One use case obviously is to test and/or develop stuff on it and even a "to play" setup may be worth a thought or two. Some examples given:

on real ("productivity") systems one may not want to junk the system with too many installs and simply (in regards to retropie) creating a new vm, or reverting to a backup of one (backing up a vm is simply copy'n'paste) or to a snapshot (vmware workstation) may be simpler than to deal with real partitions/disk/card-images.
it may sound overkill and wasted space, but having a setup for the systems of your choice that aren't clustering your HOST system may be convenient for casual playing, consider such a setup alike to a portable app [may involve settings in .vmx configuration for the VM, more on that later] (app-virtualizers may be an alternative, but are most of the time bound to the host systems os).
if you are testing/developing backdrops/overlays/bezels for your raspi retropie system and your computers display is large enough to accomodate the resolution of that targeted system, you may simply set the vm's display dimensions to that.

VMWare - requirements:
Given that your system holds up to the requirements to use the vmware-virtualizer of your choice, there may be some basic requirements to the guest as a system to be usable as a retropie-platform:
As installing cores via precompiled binaries is only availabe on raspberry pies, the neeeded amount of allocated memory to the VM depends mainly on the memory needed to compile the cores you want to use on that machine. If your Host has more than 8 GB available, this would be a recommended minimum and in addition to that it is not advisable to go beyond vmwares recommended maximum [1]. The recommended size for a Host with 8 GB of memory is 5632 MB (5.5 GB), but to compile e.g. lr-mess/lr-mame on that one, it is needed to increase the swap space added from their module-scripts to 8192 see this post.

1: during the setup of a virtual machine, whence configuring its hardware - vmware(-player) will show 3 recommendations for the memory setting, the Guest OS minimum, a recommended size (which most propably is to low for our needs) and a maximum recommended size (assignments beyond that value will lead to memory swapping on the host and most propably will stall not only the vm, but also the host).

P.S.: A better testenvironment could most possible be achieved via QEMU, as with that one we could use the precompiled binarys and aren't wasting the time and also aren't in need of the extra memory it takes to compile from source. Sadly I am totally unfamiliar with QEMU (and afaik for now it ain't supporting the raspi4 model(s)), but i would welcome any (nowadays) guide/instruction to set it up and like to invite/encourage anyone who know how to do this to post it here on the retropie forum!

Ashpool

vmware - general tips/issues

you may consider to add uuid.action="keep" to the vms.vmx config file, otherwise VMWare will always ask you whether you have copied or moved the vm if its path has changed (the action="keep" setting would be identical to answering 'moved' here, answering copy would result in new uuid and MAC values for that VM. Shouldn't matter for *nix installs, but (offtopic) for a activated windows guest, this one may be the hardware change that makes its current activation invalid). Don't do this if you intent to use the copied vm(/cloned system) as another system that runs at the same time on the same network as its parent/source (use case could be to test netplay).

open-vm-tools - issues

There is a known and unresolved issue with using shared folders on *nix guest systems (at least on windows-hosts, untested on *nix based ones) that the share is only working right after enabling it, but fails on further reboots/restarts. A workaround for this is to add vmhgfs-fuse /mnt/hgfs fuse defaults,allow_other 0 0 to the guests '/etc/fstab'.

Ashpool

reserved

Ashpool

Guest System: Debian

As VMWare Workstation (Player) ain't suporting Debian 12 as a Guest System so far (20230720) [Link is requesting current suport for DEB12, not frozen state for nowadays comment!], so for now Debian 11 (old stable) is the option of choice (and remain so for some time).

Among the various install options, there is mostly to be considered the much smaller CD variant, which relies on an internet connection during the install, and the DVD one, which can be used to install debian without an internet connection. In our case the first is the better option, as no internet shouldn't be a concern (after all: retropie and its content cannot be installed without an internet connection) and most propably all of the extra packages on the DVD ain't needed for just a simple/lightweight install. Also whence installing from the DVD, the DVD will be added as a source-package and certain apt actions may later ask for it, this (imho PITA) can be avoided by removing it from /etc/apt/sources.list.

Installing Debian inside a VM

Unless opting for "I will install the operatin system later", open-vm-tools, and if opting for a Desktop-Environment during the install also open-vm-tools-desktop (otherwise it has to be installed after the DE), will be installed unto the guest system.
After setting up the VMs (Virtual Disk Size and Hardware) specifications and once the setup is started from the install media, below the VMs Guest-Screen an "Information-Bar" is telling you to "install the OS as you would do on a physical computer" after that's done and the installed guest OS is booted, simply confirm the end of install process by clicking "I finished Installing".

During the Install Debian is asking for a domain name, this can (afaik) simply be left blank (FYI: correct domain name for a home desktop linux machine, especially this answer).
It also will ask for a root password. If one is entered, the user won't have sudo privileges (those can/have to be added manually after the install), by simply leaving it blank the created user will be a sudoer.
some steps further ahead, the installer will ask for what basic packages should be installed:
- Desktop Environment - opting for one (of personal choice) here may be a convinient option, as it avoids to add one and needed applications (on a per demand basis) later, but it also has the drawback of installing (what exactly may vary between the available Desktop Environments) lots of extras (Office/Browser/WhatsNot) that are not needed within a dedicated "RetroPie-VM". A Core DE could well be installed later on (see below) and for a pure "RetroPie-VM" a pure plain/simple/minimalistic/lightweight OS install is all that is needed (and why wasting storage space (especially whence considering backups)?).
- web server - most propably unneeded.
- SSH - may well be installed/enabled via RetroPie later on.
- standard system utilities - Well, if you feel adventurous or are a real *nix geek knowing and wanting the implications, you may well opt them out, but for most of us it is safer to install them (FYI: whats the consequences if I don't...).

Installing the Desktop Environment (skip if already done during the install)

For those DEs missing a "Display Manager" by their "Core Install", lightdm is choosen here just because it is used in all other install-scenarious/cases, feel free to replace it by another one of your choice.

XFCE
```
sudo apt install xfce4
sudo apt install open-vm-tools-desktop
sudo apt install xfce4-terminal (optional)
```
- the basic terminals are imho unusable due to their default sizing, installing the xfce#-terminal is recommended just out of convinience reasons here.
- unless installing a web-browser of choice later on, the preinstalled startmenu-entry and quicklaunch-bar icon for the browser are useless and can/should be removed.
Cinnamon
```
sudo apt install cinnamon-core
sudo apt install open-vm-tools-desktop
```
With Cinnamon, the Network.Manager supplied is somewhat troublesome, as it only manages connection not setup within /etc/network/interfaces. Best option for now is most propably to comment out the interface within that file.

Mate

sudo apt install mate-desktop-environment-core
sudo apt install lightdm
sudo apt install open-vm-tools-desktop
sudo apt install mate-media

LXDE

sudo apt install lxde-core
sudo apt install open-vm-tools-desktop

LXQt

sudo apt install lxqt-core
sudo apt install open-vm-tools-desktop
sudo apt install lightdm

configuring auto-login with lightdm

to enable auto login into the guest os (whence using lightdm as a display manager), it is needed to edit /etc/lightdm/lightdm.conf.

Either you may uncomment and fill in the information for autologin-user=<username> under [Seat:*], or (easier to remember and be done) just add [SeatDefaults] at the EndOfFile, followed by: autologin-user=<username>

Ashpool

reserved

Ashpool

reserved

Ashpool

@Folly So here we go.... I am not really sure how to summarize/condensate in the reserved posts (vmware general/ubu-based/debian(pure)-based/???

The issue with rpie/ES not beeing fullscreen in ubu would be a ubu/wayland topic... the other ones (you encountered)?
For vmware in general it would be recommend 8GB minimum (at least for compiling (current) mame(/mess?)). Copy'n'paste with open-vm-tools bugged would be general vm (and i haven't found a workaround solution), same for the shared folders problem I encountered, but there the solution presented here in the vm-ware forum worked for me (at least in debian 11.07, will try ubu next).

It may be dissapointing for fellow forum users finding no real material here for now/current-day-cycle, but I won't be adding to this thread/posts any further for now ( besides that I want to try out some things in a new debian vm install it is also time to get some early sleep once in a while - Sorry Folks! ;| )
[Edit: apparently, at least in debian 11.7, entering a blank admin pw during the install voids the need to add the user to the sudoers group]

Edit: Next thing I want to test/try out is the lubuntu distro for ubu 22.04, as maybe some of the problems for ubu are wayland/gnome related and a more "leightweight" desktop-environment could be better suited.

Folly

@Ashpool

Great !
You have build up the basics very nicely ;-)
I think it will be very useful, indeed.

I will reserve 5 posts too, just in case.

Folly

reserved

Folly

reserved

Folly

reserved

Folly

reserved

Folly

reserved

Folly

@Ashpool

Are you open into changing the topic name a bit ?
I would like to change it into :
*nix based VM (VMWare) as RetroPie play and test environment

I think the play part will definitely attract more people to use VM's.
I certainly do play games on my VM's when it comes to it ;-)

Using (VMWare) only in the name makes us a bit more flexibel in the discussion about what programs we use.
For example, I use VMware fusion on OSX. ( Can you add "Fusion" in your first post too ? )

Do you agree for these changes ?

Folly

@Ashpool said in *nix based VM (VMWare Player) as retropie test environment:

@Folly So here we go.... I am not really sure how to summarize/condensate in the reserved posts (vmware general/ubu-based/debian(pure)-based/???

I think it would be nice to summarize in the reserved posts on how to install a specific OS and how to overcome the issues that are faced.
Basically that would make these posts very interesting as a tutorial for each specific OS.

Let's see what will go on in the discussion.

For vmware in general it would be recommend 8GB minimum (at least for compiling (current) mame(/mess?)).

I did several tests on different VM's and compiling will work with 8GB depending a bit on how much cpu cores you use.
4 cpu cores is probably the maximum with 8GB minimum if computer allows these specs.

If not :
Mame standalone will be a bit more forgiving as it will add 8192 MB swap when compiling on a 64 bit OS.
However lr-mess/lr-mame will add only a swap of 4096 MB when compiling on a 64 bit OS
So when using 5632 MB and 3 cpu cores compiling lr-mess/lr-mame froze.
Compiling mame standalone compiles fine with using 5632 MB and 3 cpu cores.

So if you have a computer with 8 GB it's recommended to use a minimum 5632 MB and 3 cpu cores.
For lr-mess/lr-mame to compile you need to increase the swap space in the module-scripts to 8192 in :

~/RetroPie-Setup/scriptmodules/libretrocores/lr-mame
~/RetroPie-Setup/scriptmodules/libretrocores/lr-mess

( requirements is probably something to add in the fist post )

Edit :
Changed 5192 into 5632

Ashpool

@Folly said in *nix based VM (VMWare Player) as retropie test environment:

I think the play part will definitely attract more people to use VM's.
Using (VMWare) only in the name makes us a bit more flexibel in the discussion about what programs we use.

agreed on both

@Folly said in *nix based VM (VMWare Player) as retropie test environment:

So if you have a computer with 8 GB it's recommended to use a minimum 5192 MB and 3 cpu cores.
...
( requirements is probably something to add in the fist post )

Assuming the Host has that many cores to start with ;] My main work-pc is still an Dualcore AMDX2 (3.00GHz) with 12GB (VMWare 16.2.5 installed), whereas my main gaming machine is an Octacore Ryzen7 @3.2GHz with 16GB (VMWare 17.0.2 *).
Yeah, requirements for vmware in the 1st post sounds good (will do that later), though I am inclined to use the 1st reserved post for vmware/(open-)vm-ware-tools topics.

Btw. so far I haven't thought about it and used the default settings for "display", so no accelerated 3D - If you're using it, would you recommend it to be used and if yes, what should the shared memory setting be?

Edit: @Folly as wednesday is our established pub meeting day since '91, i leave it for that so far ;>

Ashpool

@Folly said in *nix based VM (VMWare) as RetroPie play and test environment:

Changed 5192 into 5632

Btw. is that one vmwares recommended max. on 8 GB Host machines, or why 5.5 GB (instead of 5 or 6 - so far I've always prefered multiples of 1024 (out of habbit and thinking in terms of memory modules)?

Edit: And I am also unsure of the difference needed to run emulator-cores vs. compiling them. None of my (host-)systems available for testing this has less than 8 GB available (ok just to add: Some of those systems aren't able to run VMWare Player Version > 14, and both my *nix machines are only on 8 GB and are using an onboard GPU's (so, 8 GB - shared memory with GPU)).
Maybe it would be possible/use-case-scenario-to-be-considered (play case) to setup/compile cores on a machine that is capable to do it and later on to play that VM on another host with edited/downgraded ~memory&|core limits for the VM (.?.).

Besides, as you are using Fusion on a Mac, and I am just on Windows/*buntu Systems (though I have yet to see if I can run any recent VMWare Player after 14 on any of my *nix System), maybe Mac-specific experiences/tips/issues would better be placed in one of your reserved posts, though I don't have any idea how to exactly split guest-system related topics in regards to Host OS.

'nother thought I had and not that I am aware of any: Does anyone know if there are emulators utilizing AMD-V/Intel VT-X (maybe not gaming consoles, more likely computer systems ?) and would therefore benefit from enabling that for the VM?

Folly

@Ashpool said in *nix based VM (VMWare) as RetroPie play and test environment:

@Folly said in *nix based VM (VMWare) as RetroPie play and test environment:

Changed 5192 into 5632

Btw. is that one vmwares recommended max. on 8 GB Host machines, or why 5.5 GB (instead of 5 or 6 - so far I've always prefered multiples of 1024 (out of habbit and thinking in terms of memory modules)?

I am using it on my Apple mac mini m1 which only has 8 GB of memory.
5632 is the maximum what VMWare recommends to use.

Edit: And I am also unsure of the difference needed to run emulator-cores vs. compiling them.

The difference is that almost all emulators just use 1 core to emulate stuff.
Only thing is that if you have a multicore cpu then other tasks can go to a secondary core, etc so the load is less on the core that is used for emulation.

Only a few emulators are trying to use multiple cores.
For what I read it's very difficult to use multiple cores with emulation because everything has to be in sync to get a fluid emulation experience.
So syncing is the problem here.

Whereas compiling can be done over multiple cores more easily because syncing isn't important.
Basically, if a program needs compiling and it has, for example, 100 files then the workload can be slit up over the cores.
So with 4 cores every core can compile 25 files which can be done simultaneously so it is eventually 4 times quicker.
The disadvantage is that compiling needs memory so if you have 4 cores compiling the you will get the picture.
4 cores have to use the same memory so basically 4 cores need 4 times the memory than with 1 core.

So if you don't have enough memory in you Host or VM than you can reduce the cores in the VM configuration so less memory is consumed.

None of my (host-)systems available for testing this has less than 8 GB available (ok just to add: Some of those systems aren't able to run VMWare Player Version > 14, and both my *nix machines are only on 8 GB and are using an onboard GPU's (so, 8 GB - shared memory with GPU)).
Maybe it would be possible/use-case-scenario-to-be-considered (play case) to setup/compile cores on a machine that is capable to do it and later on to play that VM on another host with edited/downgraded ~memory&|core limits for the VM (.?.).

Indeed, basically it would be possible.
However when compiling stuff on one specific cpu will not always work on an other specific cpu.
Most programs, when compiling, detect the features of the cpu and use the best available.
For example the Intel I7 is better than an I5 or I3.
Compiling a program on an I7 transferring it to a I5 or I3 and it will not work because it misses specific specs.
The other way around can sometimes be possible because then you compile your program with less specs running it on a better cpu with backwards-compatible specs.

A solution can be cross-compiling a program on a CPU with better specs compiling on that CPU for a CPU with lesser specs.
However cross-compiling is very difficult and you have to have quite some knowledge to accomplish.
I have done this once in a hackish way, compiling mame for an older 32 bit x86 cpu pc.
Compiling it in a x86-32bit VM with a changed module-script.
With this solution I was able to compile mame in 1:30 hours whereas it would take more that 24 hours on that older x86 32 bit PC.

Besides, as you are using Fusion on a Mac, and I am just on Windows/*buntu Systems (though I have yet to see if I can run any recent VMWare Player after 14 on any of my *nix System), maybe Mac-specific experiences/tips/issues would better be placed in one of your reserved posts, though I don't have any idea how to exactly split guest-system related topics in regards to Host OS.

Good idea,
I can use my preserved ones for Fusion specific stuff.
If needed, you can just link to these preserved posts in one of you posts.

'nother thought I had and not that I am aware of any: Does anyone know if there are emulators utilizing AMD-V/Intel VT-X (maybe not gaming consoles, more likely computer systems ?) and would therefore benefit from enabling that for the VM?

Processors with AMD-V/Intel VT-X can be used to improve the experience of running virtual machines.
If available and enabled in VMWare then the Virtual Machine uses hardware of the Host more directly and so the speed will drastically improve.
I don't think it can be enabled and used again within a Virtual Machine.

Hope everything makes sense in this post ;-)

Ashpool

@Folly said in *nix based VM (VMWare) as RetroPie play and test environment:

Processors with AMD-V/Intel VT-X can be used to improve the experience of running virtual machines.
If available and enabled in VMWare then the Virtual Machine uses hardware of the Host more directly and so the speed will drastically improve.
I don't think it can be enabled and used again within a Virtual Machine.

Ok, I allways thought, that as hardware-assisted-virtualization was a requirement for running 64bit guest systems vmware will always utilize it if available and enabling Virtualize Intel VT-x/EPT or AMD-V/RVI actually meant it to be virtualized within the guest (for nested solutions).
Just to be sure of that/verify it: I've just installed VMWare-Player 15 within a Windows7Pro(x64)-Guest (virtualize hardware-assisted-virtualization enabled) and then an XPx64 Guest within it, which went fine and the VM within the VM was usable. Then I've deactivated the virtualization for Intel VT-X/AMD-V for the Win7 VM ("Host"-"Guest") and when trying to start the XPx64 VM ("Guest"-"Guest") again, it failed with "Error while powering on: This host does not support AMD-V …".

Hope everything makes sense in this post ;-)

It does! And I feel lucky now, that for my past usage of windows guest systems, that system specific compilation wasn't a problem when moving a VM between my various Intel/AMD Based Host Systems over all those years (Some of my VMs I kept age back to VMWare Player 12 times (older ones simply became obsolete or where migrated to VMP12 by mounting and cloning the old "HDD".vmdk to a new VM.vmdk (don't remember what they where based upon, propably +-~ 6 or 7 ones [1]))) and "installed" programms always utilize what they find and are cabable of utilizing.

1: (P.S.:) This ain't in violation of my answer/comment in the "module-script-thread", it was just that trying out whether virtual-box may had been an alternative to vmware (and haven't used VMs for some while in between):

I've tried virtualbox in pre-Win7 times, but since vmware(-player) offered native support for xpmode vms, i've switched to that player and well, stayed with it so far (though for legacy systems as win9x, i would prefer QEMU/PCem/DOSBox-x))

Folly

@Ashpool said in *nix based VM (VMWare) as RetroPie play and test environment:

and then an XPx64 Guest within it, which went fine and the VM within the VM was usable. Then I've deactivated the virtualization for Intel VT-X/AMD-V for the Win7 VM ("Host"-"Guest") and when trying to start the XPx64 VM ("Guest"-"Guest") again, it failed with "Error while powering on: This host does not support AMD-V …".

Interesting, I never tried it.
So you proved you can still utilize Intel VT-x/EPT or AMD-V/RVI running a "VM" within a "VM".

that system specific compilation wasn't a problem when moving a VM between my various Intel/AMD Based Host Systems

Aha, indeed you are lucky.
But than again it could be rationally explained by the fact that compilation took place on older cpu's.
Running the software on cpu's from the same period or on newer cpu's can work as the cpu's used can have the backwards compatible specs.

Virtual box

It has also been a very long time since I used it.
Not sure how it is right now.