u/LinearAlgebraWorld

This is a follow-up to our earlier post deriving the orthogonal projection formula

P = U(UᵀU)⁻¹Uᵀ

from the geometric definition of projection:

https://www.reddit.com/r/LinearAlgebra/comments/1u4cjog/derive_the_projection_formula_from_the_definition/

Here we look at the same formula as a serial transformation, following what happens when the factors are applied from right to left:

Uᵀ → (UᵀU)⁻¹ → U.

The diagram tracks both the standard basis vectors and the two directions spanning col(U). It also shows why

U⁺ = (UᵀU)⁻¹Uᵀ

acts as a left inverse of U, and how applying U afterward gives the orthogonal projection onto col(U).

If the columns of U were orthonormal, then UᵀU = I and the middle correction would disappear.

This is a follow-up to our previous post on how the choice of free variable changes the real and imaginary parts of a complex eigenvector:

https://www.reddit.com/r/LinearAlgebra/comments/1vl5s28/complex_eigenvectors_of_a_22_matrix_how_the_free/

For a general 2×2 real matrix with complex eigenvalues, changing the phase of the free variable rotates and scales the pair Re(x̃₁), Im(x̃₁) through a matrix-transformation ellipse.

Here we look at the special case where the matrix itself is a rotation-scaling matrix.

In this case, for every nonzero choice of the free variable, Re(x̃₁) and Im(x̃₁) remain perpendicular and equal in length. The numerical examples show the same phase experiment as in the previous post, but now the ellipse becomes a circle.

Thanks to u/StanleyDodds for pointing out an unnecessary detour in the previous version. We have corrected and simplified it here.

This is the next page of the chapter on complex eigenvectors and the rotation-scaling theorem. More to follow.

This is a follow-up to our previous post on how the choice of free variable changes the real and imaginary parts of a complex eigenvector:

https://www.reddit.com/r/LinearAlgebra/comments/1vl5s28/complex_eigenvectors_of_a_22_matrix_how_the_free/

For a general 2×2 real matrix with complex eigenvalues, changing the phase of the free variable rotates and scales the pair Re(x̃₁), Im(x̃₁) through a matrix-transformation ellipse.

Here we look at the special case where the original matrix itself is a rotation-scaling matrix.

In this case, for every nonzero choice of the free variable, Re(x̃₁) and Im(x̃₁) remain perpendicular and equal in length. The numerical examples at the bottom show the same phase experiment as in the previous post, but now the ellipse becomes a circle.

This is the next page of the chapter on complex eigenvectors and the rotation-scaling theorem. More to follow.

These are the first two pages of our chapter on complex eigenvectors and the rotation-scaling theorem. This part explains how the choice of free variable changes the real and imaginary parts of an eigenvector. More to follow.

Following our earlier posts on QR factorization by direct rotations, we have added a visual comparison with the corresponding Householder factorization for the same 3×3 matrix.

The image compares:

• the Q and R factors produced in each case
• the geometric action of R on the basis vectors
• the subsequent action of Q
• the relationship between the two factorizations

We have also added a formal derivation of the relationship between the direct-rotation step and the corresponding Householder step:

https://www.graphmath.com/la/qr/qr-direct-rotations.html

It appears in the first section, “An Unpaved Road to Main Street: from direct rotation to a Householder-mirroring QR algorithm.”

We would welcome comments on the comparison, the derivation and the presentation.

We posted an earlier visual explanation of powers of a 2D real matrix with complex eigenvalues.

Since then, we substantially updated one of the images and added a new companion image.

The updated image compares several examples of

A = X S X⁻¹

where S is a rotation-scaling matrix. It shows how the behavior changes for |λ| < 1, |λ| = 1 and |λ| > 1, and how the corresponding orbits look in the original coordinates versus the rotation-scaling coordinates.

The new image looks at the same powers from another angle: which directions expand or contract after the next step. The colored sectors come from the quadratic condition

vᵀ(AᵀA − I)v = 0,

which separates directions where the radius increases, decreases or stays unchanged.

We thought the two views complement each other: one emphasizes the rotation-scaling factorization, while the other emphasizes directional expansion and contraction in the original coordinates.

As always, we welcome feedback on clarity and presentation.

We have been developing a direct-rotation construction for QR factorization and would be very interested to know whether anyone has seen an equivalent method.

The idea is to eliminate all subdiagonal entries of each active column with one rotation:

• construct the plane spanned by the active column and its target coordinate direction
• rotate inside that plane until the column is aligned with the coordinate axis
• leave every vector perpendicular to that plane unchanged
• deflate the matrix and repeat on the remaining active submatrix

The GIF shows the construction for a 3×3 example. The following images explain how the rotation is built, extended to the full space and applied recursively.

We developed this construction independently. Rotation-based QR methods certainly exist, with Givens rotations being the standard example, but this is a different construction. A Givens rotation acts in a coordinate plane and eliminates one selected entry at a time. Here, the rotation plane is spanned by the active column and its target coordinate direction, so one direct rotation eliminates the entire subdiagonal part of that column.

Thus, for an m×n matrix, the construction uses one rotation per active column rather than one Givens rotation per subdiagonal entry.

Our main questions are: have you encountered this particular construction, or something mathematically equivalent to it? Does this construction suggest any practical advantage or application beyond its geometric interpretation?

We are also working on the computational implementation and have made substantial progress. In operation count and structure, it currently appears to lie between Givens and Householder, closer to Householder, and there is still room for improvement.

Householder remains the standard general-purpose method, so we are not claiming a practical advantage. Possible value may lie in the geometric interpretation, in applying one direct rotation to an active column or perhaps in implementations where this rotation structure can be exploited.

We would especially welcome references, thoughts about possible applications and criticism of the construction or computational approach.

Another addition to our linear algebra project. This page compares several common classes of diagonalizable 2×2 matrices, showing their geometry, eigenvectors, and corresponding change-of-basis factorization side by side. Feedback is welcome.

We made a side-by-side visual summary of matrix size, rank, the four fundamental subspaces and pseudoinverses.

The table compares six common matrix types:

square full-rank
square rank-deficient
tall full-column-rank
tall rank-deficient
wide full-row-rank
wide rank-deficient

For each case, it shows the dimensions of row(M), null(M), col(M) and left-null(M), whether the map is one-to-one or onto, what happens to Mx⃗ = 0 and Mx⃗ = b⃗, and how the pseudoinverse behaves.

Hope you don’t mind the size and density — we wanted to keep all cases together so they could be compared directly. Opening the image at full size is recommended.

As always, we welcome feedback on clarity and presentation.

These images reconstruct a matrix from the identity by applying the inverse elementary row operations from right to left.

The geometric effects are visible at every step: scaling a row multiplies signed area or volume by the scale factor, adding a multiple of one row to another produces a shear and leaves it unchanged, and swapping two rows reverses orientation and changes the sign.

The 2D example starts with the unit square and ends with signed area -3. The 3D example follows the same idea with a unit cube and also ends with signed volume -3. Multiple viewpoints are shown in 3D because no single projection makes every transformation equally clear.

We made a visual explanation of powers of a real 2×2 matrix with complex eigenvalues.

For such a matrix, we can write

A = X S X⁻¹

where S is a rotation-scaling matrix. Then powers are computed as

Aᵗ = X Sᵗ X⁻¹.

The idea is that Sᵗ is easy to understand geometrically: it rotates by tθ and scales by |λ|ᵗ. The change of basis by X and X⁻¹ turns this circular rotation-scaling picture into the ellipse-like spirals seen in the original coordinates.

The first image follows one example through the factorization. The second shows more numerical examples with |λ| < 1, |λ| = 1 and |λ| > 1.

As always, we welcome feedback on clarity and presentation.

We previously posted separate 3D animations of QR decomposition by Gram–Schmidt, Givens rotations and Householder reflections.

Here is a 2D comparison of the same three geometric routes.

The goal is to show that all three methods reach the same kind of result, A = QR, but by very different geometric actions:

1. Gram–Schmidt: subtract projections and normalize
2. Givens rotations: rotate selected components to zero
3. Householder reflections: reflect vectors across chosen lines or planes

In 2D, the Givens case is almost trivial: only one rotation is needed to zero the lower-left entry. In higher dimensions, Givens QR proceeds by many such rotations, one entry at a time.

One detail worth noticing: Gram–Schmidt, in its standard form, produces positive diagonal entries in R. Givens and Householder versions may produce different signs depending on rotation/reflection sign choices. This is normal: QR is unique only after an extra sign convention is imposed, such as requiring positive diagonal entries in R.

More explanation and the 3D versions are here:
https://www.graphmath.com/la/visuals/qr/qr-three-geometric-routes.html

We will also add these 2D animations to that page.

We welcome feedback on clarity and presentation.

We made three animated comparisons showing how eigenvectors and eigenvalues behave across different families of 2D linear transformations.

1. Non-symmetric matrices with real eigenvalues, where the eigenvector directions need not be perpendicular
2. Symmetric matrices with real eigenvalues, where the eigenvector directions are orthogonal
3. Real matrices with complex eigenvalues, where no nonzero real direction remains on the same line

In each animation, the transformation develops continuously from the identity matrix to the displayed matrix. The goal is to make the difference between these three cases visible rather than only algebraic.

Full-size animations and explanations:
https://www.graphmath.com/la/visuals/eigenvectors-eigenvalues-2d-transformations.html

As always, we welcome feedback on clarity and presentation.

A visual definition of eigenvectors, followed by examples across several common 2D transformations.

The table compares uniform and non-uniform scaling, shear, triangular and symmetric matrices, projection, reflection, rotation and related cases. It shows which directions remain on the same line after transformation, their corresponding eigenvalues, and when no real eigenvectors exist.

Hope you don’t mind the size and density — we wanted to keep all the examples together so they could be compared directly. Opening the image at full size is recommended.

The goal is to make the definition A x⃗ = λx⃗ visible across many different matrix types.

We welcome feedback on clarity and presentation.

UPDATE: we have added a few more examples, you can see the full updated version on our web-site:
https://www.graphmath.com/la/visuals/eigenvectors-definition-and-examples.html

We made a visual derivation of least squares from the original overdetermined system.

The first image shows how projecting the data vector b onto the column space of A leads to the normal equations

AᵀAβ = Aᵀb

and gives the best-fit line y = β₁ + β₂x.

The second keeps the same data but changes the model to y = β₁ + β₂x². The second column of A changes from x to x², so the prediction plane changes, but the same projection method applies.

The point is to show least squares as geometry rather than a formula to memorize.

As always, we welcome feedback on clarity and presentation.

We made a paired visual derivation of orthogonal projection, starting from its defining condition:

the projection lies in the target subspace, and the leftover residual is orthogonal to that subspace.

For projection onto a single vector, this immediately gives the familiar scalar projection formula. Replacing the single vector by the columns of a matrix gives the same derivation for

P = U(UᵀU)⁻¹Uᵀ.

The point is not just to know the formulas, but to be able to reconstruct them instead of memorizing them.

As always, we welcome feedback on clarity and presentation.

Made this 2D version after apt and kind suggestion of u/strange-the-quark and watching relevant 3Blue1Brown video.

We made a visual derivation of Cramer’s rule using signed volumes.

The image shows the 3D case: the denominator is the signed volume of the parallelepiped spanned by the columns of A, while the numerator replaces one column by b. By projecting both shapes onto the same normal direction, the ratio of volumes becomes the corresponding coordinate xᵢ​.

A related page with the full visual explanation, including both the 3D derivation and a compact ℝⁿ version, is here:
https://www.graphmath.com/la/visuals/cramers-rule-geometric-derivation.html

We welcome feedback on clarity and presentation.

The determinant permutation list can be built recursively:

choose the first-row entry, then permute what remains.

The “what remains” part is exactly the permutation list of the corresponding submatrix.

So the same recursive object is called Sₙ​ in the full determinant, and Sₙ₋₁ inside each cofactor submatrix.

4×4 version:
www.graphmath.com/la/figures/determinant/permutations4x4.png