USB File Transfer, NTFS.

[Efriim]

RetroPie 4.4.14
RaspberryPi3B+
64gb PNY USB 3.0 Flash Drive

I have my drive formatted to NTFS allocation size 4096, and it mounts and I have a fstab and it is shared.
NTFS handles large writes where with exfat/fat32 I could not transfer a file over 500mb it would just drop it or timeout everytime.

Now with NTFS, it is transferring 11mb/s high, dropping to 6mb/s. I can transfer large files, but with smaller non-contiguous files, it will eventually just stop on one of them and it will give me an unexpected network error and ask if I want to try again.
If I try again it is only 10mb/s for a half-minute then it will usually slow and stop again.

I'm thinking it is something like the cache flush, I'm running out of this virtual memory that it uses as a buffer to write possible. I have swap disabled, the usb is mounted async. I don't know, Ideas please?

[mitu]

If it's NTFS, why not connect it to your PC and transfer the files directly ?

[Efriim]

@mitu
I'm actually doing that right now.

I'm noticing that the transfer slows down a lot on smaller files that are many. It will go 0-10KB/s. I just can't remember the times or if that is truly normal. I managed to get everything on there once before, through samba.

[Efriim]

It seems really off balance, like its got to write the table and flush then cycle twice for every single file.

Actually, I may have transferred directly. It was a brand new upgrade from a 16gb, and I just remember I got it going and it was done.

I do think I remember it getting 20MB/s, though now on a different computer it is only 11MB/s tops.

Would it matter if it was formatted quickly or not? It's been abused, from this virus and dosbox crashing and some transfers crashing.

I guess I'll try to transfer on the computer I used before after this current transfer is over. I think it seems slow because I'm staring at it, and because windows 10 looks different.

I think I remember it getting over 30MB/s on my laptop. I guess I will find out soon enough.

[backstander]

Maybe set your Removal policy for external storage devices to "Better performance" in Windows 10:
https://www.zdnet.com/article/microsoft-changes-how-windows-10-disconnects-usb-storage-devices/

If you are going back and forth from Windows to Linux, you'll probably want to format your flash drive to exFAT. It has a maximum file size limit of 16 EB (exabyte).

[Efriim]

@backstander
Thanks, I forget about this stuff with passing generations of windows, and always use the safely remove hardware anyways.

I did a full length format and direct transfer speeds were a lot faster for smaller files, it no longer takes 2 additional seconds per file. Though I guess now I still don't know if that was exclusive to my other computer.

I will use exfat, as you suggest. NTFS has been problematic, though I did not try re-enabling the swap.

And exFat has been working well comparatively, luckily most dos file objects aren't over 500mb (for network transfer).

For samba in my New windows 10 installation to work, I had to enable insecure connections via
https://www.raspberrypi.org/forums/viewtopic.php?t=200482
Set the following registry value to 1
HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters\AllowInsecureGuestAuth

Now I just need to figure out how to activate the thing, I guess they are no longer free upgrades, and Windows 10 Education edition doesn't seem to activate on its own.

[Parabolaralus]

@Efriim way off topic:

You might have to call that in if you have a real key for it. I also think education keys expire which causes the hassle most likely because you won't be a student forever...

Normally when you apply for non-profit and I'm assuming discounts on student/education editions of windows as well as office you are set up with a MSVL service center account using your email. The status of your keys including activation limits are listed when logging in which you may have hit though you may not have been given this info since it's normally sent to admins which then give you or IT a key and say good day!

Source: I'm the IT guy for a number of organizations that has setup and managed this for a few years now, but not for a university - yet!

[Efriim]

@Parabolaralus
Yeah I'll either figure it out or install linux (what is a good distro for dell laptops?). This pc I found at a garbage dump, the hard drive was failing, the battery shot, and it was a little beaten up from being discarded. I restored it with a ssd, and a new battery.
I think it came with windows 7, but there was no sticker on the back.

I downloaded the USB Windows 10 upgrade from microsoft, these were free upgrades some time ago. I had assumed Education postulating that it would activate for free, but I have no qualifications for those terms.

Back on topic:
ExFat has been working great with allocation unit size 8192b.
NTFS-3g may have been a problem in particular with dosbox because of the write syncs.