I thought the same when I first started to play it. I have found that if I put the tama down on a table and just concentrate on pressing buttons without holding it, I do so much better.
It also helps to always move your character back to the centre position when waiting for the next ball so you only need to move 1 place either way.
