Hey so I also found an algebraic argument for the way the normals transform. Do you have something for this too?
Lets say we have a vector P and its normal N. Then,
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/390c1a3b-cb09-441b-933a-b10db8687909/image.png)
And lets say we apply a transformation T to the vector P. Then, we want to find the normal such that:
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/c5eed59b-24ba-4d5e-8474-3f14c4e9e28b/image.png)
So suppose there is a transformation T1 that transforms N to Nprime. Then:
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/12e62e2f-02d2-4ddb-af46-61cc01df81e0/image.png)
So if we want to keep the dot product 0 then its sufficient that
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/58a6494a-3b58-44a7-9d5a-d8e873f61e6c/image.png)
Therefore we can say that if we transform a vector by the matrix T then the normal should be transformed by T1 to keep the dot product 0 and hence the transformed normal perpendicular to the transformed vector. (Incidentally, I think this means we can tranform vectors at any angle while preserving the angle like this, not just normals).
Transformation is a rotation
Now, if the transformation T is a rotation matrix (R) then:
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/0d4e29bf-bf02-4876-92b2-be178213a4e9/image.png)
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/f42df891-570a-46a4-9c96-9a71e557dfb8/image.png)
And for rotation matrices we know that:
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/7d24f4ae-d8cf-49e7-be7b-e1224f1494b3/image.png)
So,
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/13757065-a9f9-4421-a22c-77e6638a6de1/image.png)
Which means the normal vector n will tranform by the same rotation matrix R as the vector p.
Transformation is scaling
For scaling we need to multiply with a diagonal matrix with the diagonal elements representing the scaling factors. So,
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/a3a69e7a-96d3-4e22-abc3-bae3da1a3abb/image.png)
Now scaling matrices will be diagonal matrices like
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/3ca73a5e-f04a-462c-ad5c-e0b7e5eaa1ec/image.png)
and their inverse will be
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/48e16116-7cf1-4cd1-a592-95f1210ecf73/image.png)
and transpose of the diagonal matrix leaves it unchanged so that we have
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/ff215656-7c55-4954-a381-cef22450e65e/image.png)
General transformation
So for a general transformation involving both rotation and scaling given by the product of transformation matrices:
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/4448ed3f-0c8e-42c3-9677-630262cb739d/image.png)
The normal must be transformed as
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/76f5c352-599a-42e8-a9b2-54edc8bfa69a/image.png)
or
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/16101fa0-f0e6-4ab2-9cc4-715879ab7b52/image.png)
![image.png](https://hmn-assets-2.ams3.cdn.digitaloceanspaces.com/227d5ef6-bc41-4586-8f82-cd42c5e85ec0/image.png)
So we just invert all the diagonal elements of the scaling matrices and use the rotation matrices directly for transforming normals while keeping them normal.