separating platform code in non-game programs

I've followed Handmade Hero through the end of the prototyping of the Windows platform layer, and I think I have a handle on how the multi-platform support works - for each platform, there are files containing only code specific to those platforms, and then there are files of shared code that each platform layer calls into. In particular, the relationship goes only one way, with the shared code never calling back out into platform code. I can see right away one way this is desirable, in that it appears very easy to call C code from the Objective-C or Swift that macOS requires, but not the other way around.

Is it always possible and practical to enforce this one-way relationship? I'm working on a couple programs, so far conceived only for Windows, that make much more use of winapi than at least the prototype of the Handmade Hero Windows layer does - dialog boxes and standard controls and such - and it's not obvious to me how to extract all the winapi calls from the control flow. For example, I have a chess program with a function that executes a move. Aside from altering the Piece and Board objects as necessary, which is platform-agnostic, it calls InvalidateRect on the squares that need to be redrawn. Usually, that means the starting and ending square, but if the move was castling or an en passant capture, other squares are involved as well. Also, if the move was a pawn promotion, it launches a dialog where the player can choose what kind of piece to promote to. Is the correct approach to break this up into a bunch of smaller functions whose return values prompt the caller in the platform layer to make each of the winapi calls themselves, and leave the rather complicated branching control flow of which of these functions to call when up to the caller as well? Or perhaps a two-way relationship between the layers is less of a problem than I imagine?

Edited by AlexKindel on Reason: Initial post
This is really discussion about GUI frameworks you are using. If you are using IMGUI style UI, then there is no issues with similar way how HH structures platform code.

In your case you are not doing IMGUI style UI. That will require some kind of wrappers as you already have realized.

Qt & GTK does this with events. They capture platform specific events and call necessary callbacks in users code to perform action.

Alternative is message passing - something what SDL2 is doing. But that will require splitting complex actions - like your showing dialog choice, into multiple pieces where each of them will process small piece of logic in your code.
I think it's accurate to say that I'm not using a framework, yeah. I might try one someday, but I'm just using the API's provided by the OS vendors for these projects.

From the sounds of it, both of the approaches I mused about are possible. It would be helpful to see an example of what code callable from C that can draw a rectangle in a window on either Windows or macOS depending on the presence of a pound define would look like.
I've finished separating the back end code into a static library. Do I understand correctly that in order to use it with a macOS front end, I have to build it on a Mac? Which, since it uses a Boost library, will also involve building Boost on the Mac?
Yes. Officially you can build macOS binaries (and iOS) only on macOS. Unofficially, if you get the right toolchain files (includes, libraries and compiler), it can work from any OS.