That was pretty cool, I wonder if you could use ioctl's to use the gpu and bypass the gpu driver completely, and whether that would be at all usable :P
No, ioctl resolves to a particular function in the driver registered to the file handle. So the driver stays in full control of communication with the GPU. It's basically a way to get around the everything-is-a-file mentality.
int (*ioctl) (struct inode *, struct file *, unsigned int, unsigned long);
The ioctl system call offers a way to issue device-specific commands (like formatting a track of a floppy disk, which is neither reading nor writing). Additionally, a few ioctl commands are recognized by the kernel without referring to the fops table. If the device doesn't offer an ioctl entry point, the system call returns an error for any request that isn't predefined (-ENOTTY, "No such ioctl for device"). If the device method returns a non-negative value, the same value is passed back to the calling program to indicate successful completion.
and some extra info in http://www.xml.com/ldd/chapter/book/ch05.html
if the driver doesn't understand the command you send it (the unsigned int) it just gives an error.
The only way to get direct communication with the gpu is to write your own kernel module for it to enable it from userspace.