Uh dang, I need to make some retractions here.

I went back to re-examine the joystick circuitry and stuff and it looks like the thumbstick works fine at any voltage. (I felt my logic was odd when lying in bed the other day). The "lopsided-ness" I was talking about is actually caused by setting the range incorrectly. Ignore what I said for #1 in the previous post. The thumbstick appears to behave the same with 1.8V as with 5V (from a design standpoint, you'll probably save a little power by driving it at 1.8V).

Most modern joysticks expect something in the range of -32767 to +32767 with the center/neutral point (when the joystick is not being touched) being 0. Why? Because this is the min and max values you can represent with a signed 16 bit integer. To be extremely pedantic, you can actually go to -32768, but we pretend it's -32767 to make it symmetric.

When I did the mapping function, I take an input range of values going from ~70 to ~325 to 0 <-> 16384. I think this is where the "lopsided-ness" comes in. I'm using a built in function from the Arduino library called map() so I don't exactly know what is going on, but it places the center point somewhere around 4000 instead of ~8150 as you would expect. This isn't a problem if the output range is -32767 to 32767 for some reason.