The PS1 had somewhate "simplified" 3d capabilities compared to the Nintendo 64. For instance, PSX didn't have any sort of camera perspective correction, so moving the camera would stretch/warp shapes. Of course you probably wouldn't notice this since SONY did textures better than Nintendo. The biggest downfall of the N64 (well graphics-wise at least) was the 4K texture size. I resulted on blurry, ugly, stretch out textures. Combine that with Hardware AA, and the result was pretty much always blurry.

It was weird playing N64 on Retropie The first time with smoothing off, because it looked so much better than the actual N64 hardware.

In the end, SONY probably made the smarter choice to go with less advance hardware and No AA. It didn't hurt that they had great marketing and went after an older audience that was probably a better fit for the system.