There was a lot of thinking to figure out why doing the elimination converted the Identity to the Inverse. It seems pretty simple though (although I make a bunch of mistakes with math all the time so tell me if Im wrong lol)

lets say the gaussian eliminations represented by T1, T2 and T3 convert A to I, then

T1 . T2 . T3 . A = I

or

T2 . T3 . A = T1' . I

or 

T3 . A = T2' . T1' . I

or 

A = T3' . T2' . T1' . I

or 

A = T3' . (T1 . T2)' . I

or 

A = (T1 . T2 . T3)' . I

or

A = (T1 . T2 . T3)'

or 

A' = (T1 . T2 . T3)

And its with the equations like Casey Muratori wanted.

That looks a lot longer evaluation than this:

T1 . T2 . T3 . A = I

T1 . T2 . T3 . A . A' = I . A'  // multiply both sides with A'

T1 . T2 . T3 . I = A'           // A . A' is I

T1 . T2 . T3 = A'

This ones better

Hey so I also found an algebraic argument for the way the normals transform. Do you have something for this too?

Lets say we have a vector P and its normal N. Then,

And lets say we apply a transformation T to the vector P. Then, we want to find the normal such that:

So suppose there is a transformation T1 that transforms N to Nprime. Then:

So if we want to keep the dot product 0 then its sufficient that

Therefore we can say that if we transform a vector by the matrix T then the normal should be transformed by T1 to keep the dot product 0 and hence the transformed normal perpendicular to the transformed vector. (Incidentally, I think this means we can tranform vectors at any angle while preserving the angle like this, not just normals).

Transformation is a rotation

Now, if the transformation T is a rotation matrix (R) then:

And for rotation matrices we know that:

So,

Which means the normal vector n will tranform by the same rotation matrix R as the vector p.

Transformation is scaling

For scaling we need to multiply with a diagonal matrix with the diagonal elements representing the scaling factors. So,

Now scaling matrices will be diagonal matrices like

and their inverse will be

and transpose of the diagonal matrix leaves it unchanged so that we have

General transformation

So for a general transformation involving both rotation and scaling given by the product of transformation matrices:

The normal must be transformed as

or

So we just invert all the diagonal elements of the scaling matrices and use the rotation matrices directly for transforming normals while keeping them normal.

To keep the dot product at zero and the converted normal perpendicular to the transformed vector, we can state that if we transform a vector by the matrix T, we must also change the normal by the matrix T1. (Incidentally, I believe this means that, in addition to normals, we may transform vectors at any angle while maintaining the angle in this manner.)

Yeah I thought about this some more. Keeping the magnitude of the inner product constant is useful to ensure normal vectors since the magnitude is 0 which is only achievable if the vectors are normal (with the assumption that your scaling matrix is not scaling either one or both of them to 0). But if the angle is not 90 degrees then the same magnitude of inner product can be obtained at different lengths of the vectors at different angles. So maintaining the same angle across transforms when the angle is not 90 degrees is much more tricky. I didn't try to derive a way to do this since it wasn't really useful to the material at hand.