I saw Casey mention this on stream is there a good alternative debugger on windows.
I found windbg.exe but have no clue setting it up.
Does any Windows debugging experts know on how to setup Windbg in a handmade hero fashion.

There are no good windows debugger alternatives on Windows. There are some debuggers, but they don't feel very good as MSVS debugger. Windbg probably is most functional one, but debugging in it is more similar to debugging in gdb than in MSVS. It also includes cdb.exe which is pure-console debugger instead of GUI.

What do you mean by setting up windbg in a handmade hero fashion?

Thanks for the information.
And what I meant by handmade fashion is like how casey did in handmade hero instead of default vs solution file setup.Then I guess msvc is the only reliable option.

What do you mean by that? He heavily customized building process from "default" MSVS IDE one. The way how he uses debugger is pretty standard for MSVS, there's nothing very different from "default" debugger setup there.