Linear Algebra Done Right

1. Vector Spaces: The Foundation of Linear Algebra

Linear algebra is the study of linear maps on finite-dimensional vector spaces.

Defining Vector Spaces. Vector spaces are the fundamental structures in linear algebra, generalizing the familiar concepts of R2 (the plane) and R3 (ordinary space). A vector space consists of a set V, equipped with addition and scalar multiplication operations that satisfy specific axioms, such as commutativity, associativity, existence of an additive identity and inverse, and distributive properties. These axioms ensure that vector spaces behave in a predictable and consistent manner, allowing for powerful algebraic manipulations.

Examples of Vector Spaces. While F^n (lists of scalars) are the most common examples, vector spaces can be more abstract. The set of all polynomials with coefficients in F, denoted P(F), forms a vector space under standard polynomial addition and scalar multiplication. Similarly, F^∞, the set of all sequences of elements from F, is a vector space. These examples demonstrate that vector spaces can encompass a wide range of mathematical objects beyond simple lists of numbers.

Subspaces: Vector Spaces within Vector Spaces. A subspace is a subset of a vector space that is itself a vector space, inheriting the addition and scalar multiplication operations from the parent space. To verify that a subset U of V is a subspace, it suffices to check that U contains the zero vector, is closed under addition (u, v ∈ U implies u + v ∈ U), and is closed under scalar multiplication (a ∈ F, u ∈ U implies au ∈ U). Subspaces are essential for understanding the structure of vector spaces and the behavior of linear maps.

2. Finite-Dimensional Vector Spaces: Span, Independence, Basis, and Dimension

In a finite-dimensional vector space, the length of every linearly independent list of vectors is less than or equal to the length of every spanning list of vectors.

Span and Linear Combinations. A linear combination of vectors is a sum of scalar multiples of those vectors. The span of a list of vectors is the set of all possible linear combinations that can be formed from them. If the span of a list equals the entire vector space, the list is said to span the space.

Linear Independence and Dependence. A list of vectors is linearly independent if the only way to obtain the zero vector as a linear combination of them is by setting all the scalars to zero. Otherwise, the list is linearly dependent. Linear independence ensures that each vector in the list contributes uniquely to the span.

Basis and Dimension. A basis of a vector space is a list of vectors that is both linearly independent and spans the space. All bases of a finite-dimensional vector space have the same length, which is defined as the dimension of the space. The dimension is a fundamental property of a vector space, characterizing its "size." For example:

• The standard basis of F^n is ((1, 0, ..., 0), (0, 1, 0, ..., 0), ..., (0, ..., 0, 1)), and dim(F^n) = n.
• The basis of Pm(F) is (1, z, ..., z^m), and dim(Pm(F)) = m + 1.

3. Linear Maps: Transforming Vector Spaces

The key result here is that for a linear map T, the dimension of the null space of T plus the dimension of the range of T equals the dimension of the domain of T.

Defining Linear Maps. A linear map (or linear transformation) is a function T: V → W between two vector spaces that preserves the linear structure. This means that T(u + v) = T(u) + T(v) for all u, v ∈ V (additivity) and T(av) = aT(v) for all a ∈ F and v ∈ V (homogeneity). Linear maps are the central objects of study in linear algebra, as they describe how vector spaces can be transformed while preserving their essential properties.

Null Space and Range. The null space (or kernel) of a linear map T is the set of all vectors in V that are mapped to the zero vector in W. The range (or image) of T is the set of all vectors in W that can be obtained as the output of T for some input vector in V. The null space and range are subspaces of V and W, respectively, and they provide important information about the behavior of T.

Dimension Theorem. A fundamental result in linear algebra is the dimension theorem, which states that for a linear map T: V → W, dim(V) = dim(null T) + dim(range T). This theorem connects the dimensions of the domain, null space, and range, providing a powerful tool for analyzing linear maps. It implies that if the domain is "larger" than the target space, the linear map cannot be injective, and if the domain is "smaller" than the target space, the linear map cannot be surjective.

4. Polynomials: Algebraic Tools for Linear Algebra

Even if your students have already seen some of the material in the first few chapters, they may be unaccustomed to working exercises of the type presented here, most of which require an understanding of proofs.

Polynomials as Functions. Polynomials are functions of the form p(z) = a_0 + a_1z + a_2z^2 + ... + a_mz^m, where a_i are scalars from F and z is a variable. The degree of a polynomial is the highest power of z with a non-zero coefficient. Polynomials are essential tools in linear algebra, particularly when studying operators.

Roots of Polynomials. A root of a polynomial p is a scalar λ such that p(λ) = 0. Roots play a crucial role in factoring polynomials and understanding their behavior. A key result is that λ is a root of p if and only if p(z) = (z - λ)q(z) for some polynomial q.

Fundamental Theorem of Algebra. A cornerstone of complex analysis, the Fundamental Theorem of Algebra states that every non-constant polynomial with complex coefficients has at least one complex root. This theorem has profound implications for the structure of polynomials and their factorizations. It allows us to express any complex polynomial as a product of linear factors.

5. Eigenvalues and Eigenvectors: Unveiling Operator Structure

Once determinants have been banished to the end of the book, a new route opens to the main goal of linear algebra—understanding the structure of linear operators.

Invariant Subspaces. A subspace U of V is invariant under an operator T ∈ L(V) if T(u) ∈ U for all u ∈ U. Invariant subspaces are crucial for understanding the structure of operators, as they allow us to decompose the operator into simpler pieces. The simplest non-trivial invariant subspaces are one-dimensional.

Eigenvalues and Eigenvectors. A scalar λ ∈ F is an eigenvalue of T if there exists a non-zero vector v ∈ V such that T(v) = λv. The vector v is called an eigenvector of T corresponding to λ. Eigenvalues and eigenvectors reveal fundamental properties of linear operators, indicating directions in which the operator simply scales the vectors.

Linear Independence of Eigenvectors. Eigenvectors corresponding to distinct eigenvalues are linearly independent. This theorem is a cornerstone for diagonalizing operators. If an operator has enough linearly independent eigenvectors to form a basis, then the operator can be represented by a diagonal matrix with respect to that basis.

6. Inner-Product Spaces: Adding Geometry to Vector Spaces

Even in a book as short as this one, you cannot expect to cover everything.

Inner Products: Generalizing Dot Products. An inner product on a vector space V is a function that takes two vectors and returns a scalar, satisfying positivity, definiteness, additivity, homogeneity, and conjugate symmetry. Inner products generalize the dot product in R^n, allowing us to define notions of length, angle, and orthogonality in abstract vector spaces.

Norms and Orthogonality. The norm of a vector is defined as the square root of its inner product with itself, providing a measure of its length. Two vectors are orthogonal if their inner product is zero, generalizing the concept of perpendicularity. The Pythagorean theorem holds in inner-product spaces: if u and v are orthogonal, then ||u + v||^2 = ||u||^2 + ||v||^2.

Orthonormal Bases and Gram-Schmidt. An orthonormal basis is a basis consisting of vectors that are pairwise orthogonal and have norm 1. Orthonormal bases simplify many calculations in inner-product spaces. The Gram-Schmidt procedure provides an algorithm for constructing an orthonormal basis from any linearly independent list of vectors.

7. Operators on Inner-Product Spaces: Self-Adjoint and Normal Transformations

The spectral theorem, which characterizes the linear operators for which there exists an orthonormal basis consisting of eigenvectors, is the highlight of Chapter 7.

Self-Adjoint Operators. An operator T is self-adjoint if T = T*, where T* is the adjoint of T. Self-adjoint operators play a role analogous to real numbers in the context of complex numbers. Every eigenvalue of a self-adjoint operator is real.

Normal Operators. An operator T is normal if it commutes with its adjoint, i.e., TT* = T*T. Normal operators generalize self-adjoint operators and have important spectral properties. On complex inner-product spaces, the Spectral Theorem states that an operator is normal if and only if there exists an orthonormal basis consisting of eigenvectors of the operator.

Spectral Theorem. The Spectral Theorem is a central result in the theory of operators on inner-product spaces. It states that a linear operator T on a finite-dimensional complex inner-product space V has an orthonormal basis consisting of eigenvectors if and only if T is normal. This theorem provides a powerful tool for analyzing the structure of normal operators.

8. Complex Vector Spaces: Generalized Eigenvectors and Jordan Form

The minimal polynomial, characteristic polynomial, and generalized eigenvectors are introduced in Chapter 8.

Generalized Eigenvectors. A generalized eigenvector of an operator T corresponding to an eigenvalue λ is a vector v such that (T - λI)^j(v) = 0 for some positive integer j. Generalized eigenvectors extend the concept of eigenvectors and are crucial for understanding the structure of operators that are not diagonalizable.

Nilpotent Operators. An operator N is nilpotent if N^j = 0 for some positive integer j. Nilpotent operators play a key role in the decomposition of operators on complex vector spaces. Every operator can be decomposed into a diagonalizable part and a nilpotent part.

Jordan Form. The Jordan Form Theorem states that for any linear operator T on a finite-dimensional complex vector space V, there exists a basis of V such that the matrix of T with respect to this basis is in Jordan form. This form consists of blocks along the diagonal, where each block is an upper-triangular matrix with the eigenvalue on the diagonal and 1s directly above the diagonal.

9. Real Vector Spaces: Invariant Subspaces and Block Triangular Forms

Linear operators on real vector spaces occupy center stage in Chapter 9.

Two-Dimensional Invariant Subspaces. Every operator on a finite-dimensional, non-zero, real vector space has an invariant subspace of dimension 1 or 2. This result is crucial because real vector spaces may not have eigenvalues, and thus may not have one-dimensional invariant subspaces.

Block Upper-Triangular Matrices. Every operator on a real vector space has a block upper-triangular matrix with respect to some basis, where each block is a 1-by-1 matrix or a 2-by-2 matrix with no eigenvalues. This result is analogous to the result that every operator on a complex vector space has an upper-triangular matrix with respect to some basis.

Real Spectral Theorem. The Real Spectral Theorem states that a linear operator T on a finite-dimensional real inner product space V has an orthonormal basis consisting of eigenvectors if and only if T is self-adjoint. This theorem is analogous to the Complex Spectral Theorem, but it applies only to self-adjoint operators on real vector spaces.

10. Trace and Determinant: Numerical Summaries of Linear Transformations

Once determinants have been banished to the end of the book, a new route opens to the main goal of linear algebra—understanding the structure of linear operators.

Trace of an Operator. The trace of an operator T is defined as the sum of the diagonal entries of its matrix with respect to any basis. The trace is independent of the choice of basis and provides a numerical summary of the operator. The trace of an operator equals the sum of its eigenvalues, counting multiplicity.

Determinant of an Operator. The determinant of an operator T is defined as (-1)^dim(V) times the constant term in the characteristic polynomial of T. The determinant is independent of the choice of basis and provides another numerical summary of the operator. The determinant of an operator equals the product of its eigenvalues, counting multiplicity.

Properties of Trace and Determinant. The trace and determinant satisfy several important properties. For example, trace(ST) = trace(TS) and det(ST) = det(S)det(T). An operator is invertible if and only if its determinant is non-zero. These properties make the trace and determinant powerful tools for analyzing linear operators.

Last updated: February 27, 2025