They actually just wear, believe it or not. It seems counter intuitive, but when you look closely at the way the trakball actually works, there is almost always friction between the ball and one of the three rollers, depending on which direction you're moving it in.
Over time the ball just wears crescents into the rollers. The only real solution is to replace them (and make sure the bearings are running free and lubricated), if you want the best performance out of the trakball.
There are numerous threads about refurbishing bearings, as it's possible to restore ones that are frozen up or dried out, if you don't want to replace them. I've actually been doing some experimenting lately, as I've wanted to look into this further for some time.
If the original bearings are pitted, you can get them to spin again (by soaking them in vinegar, and spinning them up in a drill, flushing them out, and lubing them with light oil). They will actually spin pretty well, though they tend to be noisier than new bearings.
I've also been experimenting with different types of new bearings, and have yet to find any that are as silent as the original ones that are in my Millipede's trakball, which I believe are the original Atari factory ones, and were never left to dry out. I've even tried ceramic bearings, and they still aren't as smooth, though none of the sets I've tried would be considered 'high end' bearings, and were all from China. But it gives me a starting point to hopefully improve on. I hope to post more about it, if I find ones that I think are significantly better, as in my experience, they all can technically work, but do each give a different feel, especially if you compare them side by side.