I have been working on HMH videos (I'm 40 hours in now), but I did not come here with the goal of doing video game programming. I came here to learn how to do graphics programming without having to write to the html+css api or another api like QT. At this point in the HMH series I see the topics starting to veer off in the direction of simulation of physical/mechanical systems which fall squarely into the domain of video games.

I would appreciate any thoughts about programming the kinds of "entities" that are used in 2D GUI applications. Like buttons, layout containers, images, text etc. I think I could have much more control over a web application if I coded it in webassembley. And the way HMH allocates everything in the start and then goes to a game layer seems like a good fit to how the webassembley spec is defined.

I do not know if this is a good idea or not. What I do know that in my experience as a web developer, I have always worked at too high a level of abstraction. Not only am I forced to recreate every piece of graphics that a designer makes in figma or sketch using html + css again, but after I'm done with it, its usually in a state where the designer making even a small change means I have to rewrite a lot of stuff. This stuff which invariably is html/css/jsx invariably turns out to be brittle. In fact I think in the entire frontend web works, the amount of work that we are forced to duplicate every time is at a factor of 2 times. Once the designer makes it, then the frontend developer makes it again.

Additionally, the amount of tuning that I can do to make a program faster on the web is very limited. The v8 engine blog shows so many different kinds of ways a piece of code can be optimized or deoptimized on the fly that a developer has little control over. And the way the chrome profiler that shows memory usage or cpu usage does not make it very easy for me to think about my programs in the same way that Casey Muratori shows how he optimizes his programs (for example the grass layout algorithm video).

Moreover, I cannot even use the DOM model to its full efficiency because literally every project uses either react or vuejs. Unfortunately the kind of code that computed properties/watchers etc. defined by these frameworks promote is neither thoughtful of ram/cpu usage in itself nor do they make it easy for me to take actions to do such optimizations. Its like I'm running on a virtual machine (vue or react runtime) that is running on another virtual machine (v8). And the first one is written in javascript.

I guess my question here is, "Am I on a fool's errand in trying to write an application using only the base api of the canvas element that allows me to put pixels on the screen and webassembly?"

I am scared because it implies that I have to code a lot of stuff on my own that even HMH will not be going through (graphics wise) , like how to layout text or how to render boxes of ui elements on the screen. Has anyone attempted anything similar aside from ImGUI for implementing actual business applications? ImGUI gives me a lot of heart because it shows plainly that this is definitely possible.

Eventually I would like to make something like figma or sketch that allows me to have some sort of a "level editor" for building business applications. In my imagination, this allows artists to update or revamp the ui without me having to do a lot of work, it also allows me to program in the C model so I can work on optimizing stuff. Ideally, I would like to program on the web like an engine developer will program in a game studio.

So, am I talking nonsense here or can it go somewhere?

visa-viz was made for last Handmade Network "Wheel Reinvention" Jam.

Forum topic on it: https://handmade.network/forums/jam/t/8137-jam_submission_-_twitter_thread_graph_explorer

Website to try it: https://visa-viz.github.io/

And video on it: https://youtu.be/1RjU5XJqysc?t=7519

If I remember correctly, it is built with emscripten, all the layout, UI & rendering happens from wasm, but JS canvas is used to capture text to use for rendering input.

I don't know about webassembly but programming simple UI controls isn't that hard. Casey does a little bit of UI programming (e.g. simple buttons) in handmade hero and he has a old video about IMGui. There are also a few articles on his blog.

Textbox and textfield are a bit harder to create but doable.

Text layout is just keeping track of where you are in a line, and keeping track of the last valid wrapping position, so that when the text is too large for the desired width or height, you can wrap and continue.

In my layout code (which I don't claim to be good) I have a few function to change the state (font, color, wrapping parameter...) and few functions to add items: advance position, placeholder (space for image, ui controls...), string, metadata (storing a pointer, to user data, for example the url of a link). Everything is kept in a list (like render commands).

You are definitely NOT talking non-sense. I dream about developing the kind of UI code you describe, writing directly to pixel buffer and not relying on framework of any kind. Before MS Windows and all the web frameworks came about, this is how UIs were built - e.g. most MS-DOS apps. However the web aspect complicates things a little bit, because of all the necessary layers between your code and the pixel buffer. Do you really want to develop a web app, or a native desktop UI will work? imGui can do this for sure, but it is a framework you rely on then.

imgui also has a web demo based on wasm that work inside a canvas using emscripten. But I would rather use the new clang compiler option instead of emscripten to compile a GUI app to the web and native with something like a platform layer that is different for the web and windows.

I saw after posting that these episodes exist. Ill keep going till I reach them.

This is a nice exploration, but its not really at the level that I want to reach. For example, I would like to select the text. Also, from a cursory examination of the code, it doesn't look like this codebase is actually tackling rendering anything from scratch which I'm after and which ImGUI for example does.

This project seems to be using the canvas just for the graph basically and all the drawing is still done using the html5 apis.