*nix based VM (VMWare) as RetroPie play and test environment

Ashpool

@Folly said in *nix based VM (VMWare) as RetroPie play and test environment:

Processors with AMD-V/Intel VT-X can be used to improve the experience of running virtual machines.
If available and enabled in VMWare then the Virtual Machine uses hardware of the Host more directly and so the speed will drastically improve.
I don't think it can be enabled and used again within a Virtual Machine.

Ok, I allways thought, that as hardware-assisted-virtualization was a requirement for running 64bit guest systems vmware will always utilize it if available and enabling Virtualize Intel VT-x/EPT or AMD-V/RVI actually meant it to be virtualized within the guest (for nested solutions).
Just to be sure of that/verify it: I've just installed VMWare-Player 15 within a Windows7Pro(x64)-Guest (virtualize hardware-assisted-virtualization enabled) and then an XPx64 Guest within it, which went fine and the VM within the VM was usable. Then I've deactivated the virtualization for Intel VT-X/AMD-V for the Win7 VM ("Host"-"Guest") and when trying to start the XPx64 VM ("Guest"-"Guest") again, it failed with "Error while powering on: This host does not support AMD-V …".

Hope everything makes sense in this post ;-)

It does! And I feel lucky now, that for my past usage of windows guest systems, that system specific compilation wasn't a problem when moving a VM between my various Intel/AMD Based Host Systems over all those years (Some of my VMs I kept age back to VMWare Player 12 times (older ones simply became obsolete or where migrated to VMP12 by mounting and cloning the old "HDD".vmdk to a new VM.vmdk (don't remember what they where based upon, propably +-~ 6 or 7 ones [1]))) and "installed" programms always utilize what they find and are cabable of utilizing.

1: (P.S.:) This ain't in violation of my answer/comment in the "module-script-thread", it was just that trying out whether virtual-box may had been an alternative to vmware (and haven't used VMs for some while in between):

I've tried virtualbox in pre-Win7 times, but since vmware(-player) offered native support for xpmode vms, i've switched to that player and well, stayed with it so far (though for legacy systems as win9x, i would prefer QEMU/PCem/DOSBox-x))

Folly

@Ashpool said in *nix based VM (VMWare) as RetroPie play and test environment:

and then an XPx64 Guest within it, which went fine and the VM within the VM was usable. Then I've deactivated the virtualization for Intel VT-X/AMD-V for the Win7 VM ("Host"-"Guest") and when trying to start the XPx64 VM ("Guest"-"Guest") again, it failed with "Error while powering on: This host does not support AMD-V …".

Interesting, I never tried it.
So you proved you can still utilize Intel VT-x/EPT or AMD-V/RVI running a "VM" within a "VM".

that system specific compilation wasn't a problem when moving a VM between my various Intel/AMD Based Host Systems

Aha, indeed you are lucky.
But than again it could be rationally explained by the fact that compilation took place on older cpu's.
Running the software on cpu's from the same period or on newer cpu's can work as the cpu's used can have the backwards compatible specs.

Virtual box

It has also been a very long time since I used it.
Not sure how it is right now.

Ashpool

Ok, I've tried to gather some data:
I installed Debian 11 with just one Desktop Environment of choice & Ubuntu flavoured DE Installs with the basic setup of 30GB virtual HDD (non-split)/8GB/4 Cores/3d Acceleration with max. 4GB shared and after (manually installing open-vm-tools(-desktop) where needed) doing a sudo apt update && sudo apt upgrade (really did nothing on the deb installs, just on the *ubu ones), setting the screen to 1280x960 and after restarting the VM, calling 'top' from within a terminal and then having sampled after ~5 min idle time the memory usage and also, after shutting down the vm, the size of its .VMDK (HDD) file.

System             | Idle Mem | .VMDK size
-------------------+----------+------------
DEB11-XFCE         |  ~435 MB |    4.62 GB
DEB11-KDE (PSB)    |  ~640 MB |    6.7  GB
DEB11-Cinnamon     |  ~770 MB |    6.82 GB
DEB11-LXQT         |  ~440 MB |    5.23 GB
DEB11-Mate         |  ~430 MB |    5.15 GB
-------------------+----------+------------
Raspian OS (32bit) |  ~320 MB |   11.1  GB
-------------------+----------+------------
DEB12-KDE (PSB)    |  ~820 MB |   10.5  GB
-------------------+----------+------------
Kubu LTS (PSB)     | ~1000 MB |   15.7  GB
Xubu LTS           |  ~890 MB |   12.66 GB

KDE really proved problematic (ok, for this 'venture it ain't the DE of choice, but it is my preference for real installs), for the Version(s) of Plasma used in (around) Debian 11 , there is a known issue (which won't be resolved for those OS, and the workaround available ain't worth to be considered in our context).
...
And Kubuntu LTS was another mess too, its version of plasma don't have the will not resize-bug, but with VMs Display: 3D Hardware acceleration enabled set it will blank whence opening settings and even without it it has some glitches that are voiding the use of KDE as a DE in a VM Guest OS.

Just recently Debian 12 "Bookworm" was released, but as VMWare aren't aware of it by now, it ain't an option for now (open-vm-tools are unavailable, it has to be installed as deb11x64 and there may be more hazards to be encountered by doing so - experimental, nothing to discuss atm).

For some Ubuntu Installs (e.g. not Lubuntu) VMWare offers "easy install" - and I am not sure if that could be recommended, it will auto install open-vm-tools (what it will also do on recognized OS as Debian11 where it ain't using easy install)... but from my experience so far, in cases where it will use "easy install" it is IMHO better to opt for "I will install the OS later" during setting up the VM and later install open-vm-tools manually (and I still don't understand why they are still offering their vmware-tools.iso disc image to install the tools on *nix guests, if vmware itself is advising the use of open-vm-tools(-desktop) :insane:.

P.S.: @Folly I haven't forgotten/abandoned this endeavour so far ;) But as I had bought the DoW master collection at GoGs discounted 10€ launch sale... well, I distracted (and still am distracting) myself once again in serving the God-Emperor ;] But I think that next Midweek+ I may have some skeletal guide entries within my reserved posts (fingers crossed). And from above Data, I am inclined to go primary with pure Debian XFCE Installs as a base (though I haven't tried installing RetroPie on any of those for now).

@mitu If I am refering to external websites, I guess linking to a wayback-machine archive is frowned upon in this forum (because of being identical/so close to....) .?.

Folly

@Ashpool said in *nix based VM (VMWare) as RetroPie play and test environment:

sudo apt update && sudo apt upgrade (really did nothing on the deb installs, just on the *ubu ones)

sudo in Debian OSes only works if you add your user name (pi for example) to the /etc/sudoers file.

# User privilege specification
root    ALL=(ALL:ALL) ALL
pi    ALL=(ALL:ALL) ALL

Looking at the /etc/sudoers file on raspberry pi OS it also could be like this :

# User privilege specification
root    ALL=(ALL:ALL) ALL

# Allow members of group sudo to execute any command
%sudo   ALL=(ALL:ALL) ALL

You can edit the file like this :

pi@raspberrypi:~ $ su
Password:  (enter your password)
root@raspberrypi:/home/pi# nano /etc/sudoers

Now you can use 'sudo apt .......`.

Will have a look at the other stuff later, when I can.

Ashpool

@Folly said in *nix based VM (VMWare) as RetroPie play and test environment:

sudo in Debian OSes only works if you add your user name (pi for example) to the /etc/sudoers file.

Sorry, wasn't clear on that (will end up in the 1st. setup guide) -> it worked, but the system was up-2-date after the install (which it ain't for the various ubuntu ones I tried (they had lots of stuff upgradeable)). And as long as you are not providing an admin password during debians setup dialogue, the created user will be a sudoer ;) Pretty convenient and I don't think that we are in need of any hardened security settings within a VM of this context. But thanks for the reminder, I guess I would have forgotten to add that info (which of course should be in here) otherwise!

Edit: @add atm I trying to figure out what went wrong on my try with installing just system-tools without a DE and afterwards doing that from the console (for a manual xfce install compared to it being installed via the debian setup, that brought the initial vmdk size down from around 5 to just 3 GB, so it may be worth it. But whence trying to do it again with the lxde-core package, the vm had no sound and I had to set sound.VirtualDevice to hdaudio, but that really shouldn't be needed as debian has drivers for the creative AudioPCI which seem to be the default VMWare emulates for the other installs (and only adds hdaudio in vmx files for windows7 or later) (though I hadn't tried to add sound.VirtualDevice = "es1371" to that lxde vmx (and the VM where I installed lxde with the regular setup worked well without the extra vmx line and used the creative driver).

Folly

@Ashpool

Sorry that I a not around that much.
Tried to make images and sort these on how to install Debian in VMware.
Want to make an animated gif of it.
However, everything seems to take an awful lot of time (which I don't have).
Hope I will manage in the near future, just to let you know ;-)
Otherwise you know why.

Ashpool

@Folly said in *nix based VM (VMWare) as RetroPie play and test environment:

Hope I will manage in the near future, just to let you know ;-)
Otherwise you know why.

No Problem at all! Main reason I haven't posted anything in recent times was: at first I had once again underestimated the quirks and hazards encountered in wording/drafting/layouting and then (last week+) spend some time away from home again, and whilst I had gathered all data I needed to work on on an external HDD and thought to prepare some text, the bloody murphy handicap (I had to live with) was: I had left my AC Adapter for my thinkpad at home :( (And since some years it has no working battery pack attached anymore) ... I hope that I may come up with some, at least rudimentary, text in regards to a minimalistic debian install [Edit: but this week without access to the data/mindmap/thoughts has now left me pretty much departed from it and I may need some time again to dig into the stuff again (as you hinted at: it is a PITA to setup/query/verify stuff by setups that each in itself takes hours/whole evenings) :( ].

Ashpool

@Folly BTW.: As I was just haggling again (for maybe 1-2 hours - every time I try again) with the Debian Website to find the download area for the old-stable branche, i have it finally bookmarked for now... but times they are a-changing, and so far I haven't found a suitable/recommandable/mentionable path that could be followed on the Debian Pages to reach a specific version/iso ... IMHO they are just a PITA, but sadly the more I experiment with Debian and (#)ubuntu installs, the more I am inclined to opt for a minimal DEB install without a DE (just system tools) and manual DE install afterwards (not only for VMs, but also for my regular *nix based Desktops) and so the more I am saddened that their website is such a mess to navigate (and even if DEB 12 may be installable in VMWare *, as long as they are not officially supporting it as a guest, 11 will remain my target for now). ... Sorry, just a short rant I had to make now - navigating the debian pages ain't really no fun at all (unless there is some small-print/link I always overlook)!
Edit: In fact, I only came up with the link above via my browsers history and today I really wasn't able to reach it by a regular access of/from debians main website :(

mitu

@Ashpool said in *nix based VM (VMWare) as RetroPie play and test environment:

so far I haven't found a suitable/recommandable/mentionable path that could be followed on the Debian Pages to reach a specific version/iso ...

https://cdimage.debian.org/cdimage/archive/, linked from https://www.debian.org/CD/faq/#old.

Folly

@Ashpool said in *nix based VM (VMWare) as RetroPie play and test environment:

Sorry, just a short rant I had to make now - navigating the debian pages ain't really no fun at all (unless there is some small-print/link I always overlook

Indeed, have that same problem once in a while.

Good to hear that you find the minimal install the best way.
Seems indeed obvious if we want to use it mostly for RetroPie.

Ashpool

Ok_ I know I am focusing on diversions from the main topic, but I am also notorious known for my focus on data as such :/ So, no (what I had hoped for), guide to a minimal debian (bullseye) install within a vmware vm. but just those notes below...

So for the state of the previous day (230705), this is what I spend my time upon to gather some data for the available Desktop Environments (may sound worse then it actually is: for the debian installs, the base VM without installing a DE was a good copy&paste base towards later DE installs).

Nevertheless... here we go with current data samples, if there are discrepancies towards my previous table , consider it to be due to updates made available since that time!

Data gathered on: 20230705
VM-Settings:
30 GB HDD
8192 GB RAM
4 Cores, Virtualize Vt-x/* enabled
Network: NAT
GFX: Accelerate 3D, 4 GB shared

Just to be sure: right after each install sudo apt update and sudo apt upgrade (if update mentioned the need) where performed.

If not done by the install, open-vm-tools(-desktop) are installed via sudo apt install afterwards.
Debian: on xfce4 minimal installs xfce4-terminal was also apt installed next to the DE.

Install Type: Minimal Install means:
UBU-based -> minimal option during install
DEB-based -> just standard system utilities, no desktop environment during initial setup
             Desktop Environment (and open-vm-tools-desktop) installed via CLI

Install Medias used:
(d1) debian-11.7.0-amd64-DVD-1iso      (CRC32: 0EE03455)
(d2) debian-11.7.0-amd64-netinst.iso   (CRC32: EB19C571)
(u1) ubuntu-22.04.2-desktop-amd64.iso  (CRC32: 1A29639F)
(u2) xubuntu-22.04.2-desktop-amd64.iso (CRC32: D8ABF8D5)
(u3) lubuntu-22.04.2-desktop-amd64     (CRC32: B6382DFC)

Idle Mem   -> memory usage read from top after +-~ 5min IDLE time (rounded up to multiple of 5)
Disk usage -> OS partition reported via File-Manager (max. space omitted, depends on the installers default partitioning scheme) [converted from GB to GiB by hand where needed]
.vmdk size -> VMWares .vmdk HDD image size (reported from Windows Explorer)

[keep in mind that this ain't a scientific report, nor a valid statistical sampling! So don't read those numbers by their value, but by their approximation & range they may represent]


System/Install    | Idle Mem | Disk usage |   .vmdk  | Install | Note
------------------+----------+------------+----------+---------+------
Ubuntu       (u1) |  900 MiB |  17.97 GiB | 12.5  GB | Minimal |
xUbuntu      (u2) |  530 MiB |  17.76 GiB | 12.4  GB | Minimal |
xUbuntu      (u2) |  610 MiB |  17.06 GiB | 12.8  GB | Full    |
lUbuntu      (u3) |  510 MiB |  19.4  GiB | 12.8  GB | Full    | *1, *2
DEB xfce     (d1) |  380 MiB |  25.0  GiB |  2.88 GB | Minimal |
DEB xfce     (d2) |  370 MiB |  24.86 GiB |  3.06 GB | Minimal |
DEB xfce     (d2) |  445 MiB |  23.36 GiB |  5.33 GB | Full    |
DEB Cinnamon (d2) |  670 MiB |  24.16 GiB |  3.95 GB | Minimal |
DEB LXDE     (d2) |  285 MiB |  24.3  GiB |  2.99 GB | Minimal |
DEB LXQt     (d2) |  390 MiB |  24.3  GiB |  3.66 GB | Minimal | *3
DEB Mate     (d2) |  365 MiB |  24.36 GiB |  3.74 GB | Minimal | *3

----
1: LUbuntu doesn't offer a minimal install option during its setup
2: LUbuntu ain't recognized by VMWare as an Ubuntu based Install (XUbuntu is!)
3: The DE install doesn't included a DisplayManager, lightdm was installed via cli.

Ashpool

@Ashpool (memo to self & @Folly FYI) some notes/details I've encountered during the manual setup of the Desktop Environments (Debian 11).

some are (pre-)populating their panels/menus with icons to applications which aren't installed by the minimal DE setup itself (e.g. webbrowser in xfce) [note: removing orphaned start menu entries].
even if the network itself works fine, cinnamon integrates the network-manager into the desktop but that one is only managing interfaces not setup in /etc/network/interfaces and therefore always showing not connected/no connection [which may irritate most users] (?options?: any way to remove the network-interface from cinnamon? comment out the interface in above config and let the network-manager handle the connection?)
some Desktop Environments aren't preinstalling a display manager (lxqt, mate). Stick to lightdm (as all other installs are using it), or is lxdm for lxqt a reasonable alternative?
At least configuring an autologin for lightdm is pretty easy and sticking to it would eventually avoid the need for different (2be documented) approaches towards that option.
weird issues with the soundcard/sound-device: on most(/all manual installed) DEs it is muted on 1st startup and with the exception of mate simply unmuting it and raising the volume solve the no sound issue. For mate, the install of the package mate-media was neccessary [edit: and raising pcm value slider in alsamixer afterwards]: Further research needed why installing alsa-tools wasn't sufficient.
Any benefits by using VMWares "shared folders" capability? IMHO SSH and RetroPies Samba shares are far more convinient ways of transfering files between host&guest. (VMWare Tools copy&pase somewhat the worst option as it is leaving most of the time its junk within the hosts temp folder(s).