The power switch on the ControlBlock lead to a small voltage-drop at the switching component (as does every electronically component). From other projects I know that especially additional USB devices can be problematic. I guess that when you disconnect, one or both USB devices (SSD, converter) the under voltage disappears. Alternatively, you could get an active USB hub that powers the additional USB devices and keep the connection between the RPi and the USB devices.