robot design

Give your schematic or something else to tell us about your bot.
A robot might not move in a straight line due to a hundred reasons..

Well, specifically, the motor driver circuit might be faulty. Or, the problem might be in your remote, as you've said it's a manual bot. I'd say, most probably problem is with the remote. Check if the motor works fine separately, ie, without the robot. If it works fine, then remote is malfunctioning.

It may be that, when you press forward button, some other signal like left or right may be transmitted by the remote. That mihgt be the reason for the awkward movement. Check it up.