I tested the trakballs in the test mode. 3 of them work fine. The 4th one only moved on one axis.
I then swapped the bad one with one of the others and it worked fine.
So, I determined that it's the encoder pcb, not the main logic board or wiring.
There are different revisions of the encoder board, but no difference in the x and y axis.