I did it - so far it has worked out great.
I use XBees to talk between the handheld and the controller board and have had very few communications issues (occasionally on one of the handhelds the XB goes offline but the controller correctly detects it and shuts down the driver. Power the handheld off and back on and it all comes back. This has not been a problem on the other 3 boards)
I use an Adafruit 9DOF in the handheld to determine the tilt of the controller. horizontal is neutral, down is brake, up is go fast. I have both a power switch and an “arm” button on the handheld. My biggest mistake was putting these right next to each-other. it confuses people. Next time I’ll make the power switch a slide switch and nowhere near the arm button.
I use Teensy 3.2’s on both the controller and the handheld. This works great but they are a bit power hungry. I haven’t played around with decreasing clock speed. I did have one Teensy die from (best guess) static, so now I am in the process of covering all control boards with a metal shield.
I also have a display but it’s mounted to the board since the act of looking at the display on the handheld would cause you to change the throttle position. I get my displays from Buydisplay.com.
There’s a lot of space for innovative controllers - good luck!