Goo Gone is ideal, yes. Luckily you're just doing some cocktail overlays so the project may not be too maddening to deal with that way. Only bad part is I'd imagine there is probably chipped powder coat or rust around the trackball hole.
On the panels I've stripped without completely repainting, I've found that a little satin black Rustoleum can be used for minor touch ups with decent results. Just gotta mask properly before you use it so you don't end up with little flecks of overspray that mess with adhesion.