It's still possible to insert usb/cd/floppy in modern CPU and directly boot from it and run only your code, not OS code.
The biggest problem with this approach will be how to use GPU hardware. For pure software solution there is no problem. Just set up your video mode and write pixels to it. But to use GPU hardware will be huge pain. First of all - there is so much different hardware - AMD, NVIDIA, Intel. Each very different and even different models of same vendor are very different architecture/interface. So for developer to support all GPU's is huge amount of work. Like really really huge. Take a look at Linux - there a lot of good developers working on open-source drivers, but still after how many years the open-source drivers are simply not as good/complete as closed source drivers directly from vendor (on Windows).
And second issue - not all documentation for GPU hardware is available. So you won't be able to do anything with GPU hardware without support from vendor. Intel has published complete documentation of their GPU's, for example here's docs for Skylake: https://01.org/linuxgraphics/docu...processors-based-skylake-platform
There are some docs available for AMD, but I don't think they keep it up to date: http://developer.amd.com/resources/developer-guides-manuals/
I don't think there are any reasnonable docs available for NVIDIA. But people are reverse engineering it.
Additionally to GPU there are some other issues with hardware - like network/WiFi which can also require proprietary drivers without docs.
That's why on PC games don't do this.