I've done the Live CD/USB, and that worked fine, but for the lack of being able to save data. Persistence is an option, but what I'm really looking for is a full fledged bootable install on a USB ...

The Intel Compute Stick is a tiny computer that lets you run desktop apps on a TV when you plug the stick into an HDMI port. Intel offers Windows and Ubuntu Linux versions of the Compute Stick: but ...