## Unit 1

### Section 1 (Vectors and Matrices I)

• What is an n-vector, how to add and subtract vectors, and multiply them with scalars (real numbers).
• What is the dot product of vectors.
• How is the dot product related to angle, orthogonality, Euclidean norm (length) and distance of vectors.
• What is an m × n (“m by n”) matrix and how to correctly determine matrix dimensions.
• How to add and subtract matrices, and multiply them with scalars.
• How to use indices when accessing individual entries in vectors and matrices.
• How to calculate matrix-vector and matrix-matrix products, and when they are defined and undefined.

### Section 2 (Vectors and Matrices II)

• More about vectors and matrices, including zero vectors, zero matrices, identity matrices, and diagonal matrices.
• About commutative, associative, and distributive properties of vector and matrix operations.
• That some important vector and matrix operations are not commutative.
• How to count arithmetic operations involved in various vector and matrix operations.
• Which vector and matrix operations are most computationally demanding.
• About matrix transposition, and how to work with transposed and symmetric matrices.

### Section 3 (Linear Systems)

• Basic facts about linear systems and their solutions.
• Basic terminology including equivalent, consistent and inconsistent systems.
• About the existence and uniqueness of solutions to linear systems.
• How to transform a linear system to a matrix form.

### Section 4 (Elementary Row Operations)

• How to perform three types of elementary row operations with matrices.
• That elementary row operations are reversible, and how to reverse them.
• How to create the augmented matrix (A|b) of a linear system Ax = b.
• How applying elementary row operations to an augmented matrix changes the underlying linear system.
• That applying elementary row operations to an augmented matrix does not change the solution to the underlying linear system.
• How to solve linear systems using Gauss elimination.
• That the Gauss elimination has two phases – forward elimination and back substitution.
• That the Gauss elimination is reversible.
• How to do the Gauss elimination efficiently.

### Section 5 (Echelon Forms)

• What is an echelon form of a matrix and how to obtain it.
• How to use the echelon form to infer existence and uniqueness of solution.
• What is an under- and over-determined system.
• How to determine pivot positions and pivot columns.
• What are basic and free variables.
• What is the reduced echelon form and how to obtain it.
• About the uniqueness of the reduced echelon form.
• How to express solution sets to linear systems in parametric vector form.

## Unit 2

### Section 6 (Linear Combinations)

• What is a linear combination of vectors.
• How to write a linear system as a vector equation and vice versa.
• How to write a vector equation as a matrix equation Ax = b and vice versa.
• What is a linear span, and how to check if a vector is in the linear span of other vectors.
• The Solvability Theorem for linear systems.
• That the linear system Ax = b has a solution for every right-hand side if and only if the matrix A has a pivot in every row.

### Section 7 (Linear Independence)

• What it means for vectors to be linearly independent or dependent.
• How to determine linear independence or dependence of vectors.
• How is linear independence related to linear span.
• How is linear independence related to the matrix equation Ax = 0.
• About singular and nonsingular matrices.
• About homogeneous and nonhomogeneous linear systems.

### Section 8 (Linear Transformations I)

• About linear vector transformations of the form T(x) = Ax.
• How to figure out the matrix of a linear vector transformation.
• About rotations, scaling transformations, and shear transformations.
• That surjectivity of the transformation T(x) = Ax is equivalent to the existence of a solution to the linear system Ax = b for any right-hand side.
• How to easily check if the transformation T(x) = Ax is onto (surjective).
• That injectivity of the transformation T(x) = Ax is equivalent to the uniqueness of solution to the linear system Ax = b.
• How to easily check if the transformation T(x) = Ax is one-to-one (injective).
• That the solution to the linear system Ax = b is unique if and only if the matrix A has a pivot in every column.
• That bijectivity of the transformation T(x) = Ax is equivalent to the existence and uniqueness of solution to the linear system Ax = b for any right-hand side.

### Section 9 (Linear Transformations II)

• How to find a spanning set for the range of a linear transformation.
• What is the kernel of a linear transformation.
• How to find a spanning set for the kernel.
• That the kernel determines injectivity of linear transformations.
• About compositions of linear transformations.
• How to obtain the matrix of a composite transformation.
• About inverse matrices and inverse transformations.
• How to obtain the matrix of an inverse transformation.
• What can one do with inverse matrices.

### Section 10 (Special Matrices and Decompositions)

• What are block matrices and how to work with them.
• How to count arithmetic operations in the Gauss elimination.
• How to work with block diagonal, tridiagonal and diagonal matrices.
• What are upper and lower triangular matrices.
• What are sparse and dense matrices.
• What are elementary matrices.
• Gauss elimination in terms of elementary matrices.
• Matrix inversion in terms of elementary matrices.
• About LU decomposition of nonsingular matrices.
• What is the definiteness of symmetric matrices.
• How to check if a symmetric matrix is positive-definite, positive-semidefinite, negative-semidefinite, negative-definite or indefinite.
• About Cholesky decomposition of symmetric positive-definite (SPD) matrices.

## Unit 3

### Section 11 (Determinants I)

• What is a permutation and how to calculate its sign.
• What is a determinant, the Leibniz formula.
• Simplified formulas for 2×2 and 3×3 determinants.
• How to calculate determinants using Laplace (cofactor) expansion.
• How to calculate determinants using elementary row operations.
• How to calculate determinants of diagonal, block diagonal and triangular matrices.

### Section 12 (Determinants II)

• About the determinants of elementary matrices and their inverses.
• That matrix A is invertible if and only if det(A) ≠ 0, and det(A-1) = 1/det(A).
• That for any two n×n matrices A and B it holds det(AB) = det(A)det(B) .
• That  det(A+B) ≠ det(A) + det(B) .
• That  det(A) = det(AT) and what it means for column operations.
• How to use the so-called Cramer’s rule to solve the linear system Ax=b.
• About the relations between det(A), linear dependence of columns and rows in the matrix A, properties of the linear transformation T(x)=Ax, and the existence and uniqueness of solution to the linear system Ax=b .
• How to use determinants to calculate area and volume.

### Section 13 (Linear Spaces)

• What is a linear space and how it differs from a standard set.
• How to check whether or not a set is a linear space.
• That linear spaces may contain other types of elements besides vectors.
• That a linear span always is a linear space.
• How to check if a set of vectors generates a given linear space.
• How to work with linear spaces of matrices.
• How to work with polynomial spaces.

### Section 14 (Basis, Coordinates, Subspaces)

• What is a basis, and that a linear space can have many bases.
• How to find the coordinates of an element relative to a given basis.
• How to determine the dimension of a linear space.
• What is a subspace of a linear space, and how to recognize one.
• About the union and intersection of subspaces.
• How the change of basis in a linear space affects the coordinates of its elements.

### Section 15 (Null Space, Column Space, Rank)

• What is the null space, how to check if a vector belongs to the null space.
• How to determine the dimension of the null space and find its basis.
• How is the null space related to the uniqueness of solution to the linear system Ax=b
• What is the column space, how to check if a vector belongs to the column space.
• How to determine the dimension of the column space and find its basis.
• How is the column space related to the existence of solution to the linear system Ax=b
• How is the null space related to the kernel of the linear transformation T(x)=Ax
• How is the column space related to the range of the linear transformation T(x)=Ax
• What is the rank of a matrix, and its implications for the existence and uniqueness of solution to the linear system Ax = b.
• What does it mean for a matrix to have full rank or be rank-deficient.
• The Rank Theorem.

## Unit 4

### Section 16 (Eigenproblems I)

• What is an eigenproblem.
• About important applications of eigenproblems in various areas of science and engineering.
• How to verify if a given vector is an eigenvector, and if a given value is an eigenvalue.
• How to calculate eigenvalues and eigenvectors.
• About the characteristic polynomial and characteristic equation.
• How to handle matrices with repeated eigenvalues.
• About algebraic and geometric multiplicity of eigenvalues.
• How to determine algebraic and geometric multiplicity of eigenvalues.
• How to find a basis of an eigenspace.

### Section 17 (Eigenproblems II)

• The geometrical meaning of eigenvalues and eigenvectors in R2 and R3.
• That a matrix is singular if and only if it has a zero eigenvalue.
• That the null space of a matrix is the eigenspace corresponding to the zero eigenvalue.
• That eigenvectors corresponding to different eigenvalues are linearly independent.
• The Cayley–Hamilton theorem (CHT).
• How to use the CHT to efficiently calculate matrix inverse.
• How to use the CHT to efficiently calculate matrix powers and matrix exponential.
• About similar matrices, diagonalizable matrices, and eigenvector basis.
• How to use the eigenvector basis to efficiently calculate arbitrary matrix functions including matrix powers, the inverse matrix, matrix exponential, the square root of a matrix, etc.

### Section 18 (Complex Linear Systems)

• About complex numbers and basic complex arithmetic.
• How to solve complex linear equations with real and complex coefficients.
• How to solve complex linear systems with real and complex matrices.
• How to perform complex elementary row operations.
• How to determine the rank of a complex matrix.
• How to check if a complex matrix is nonsingular.
• How to use the Cramer’s rule for complex matrices.
• How to check if complex vectors are linearly independent.
• How to invert complex matrices.
• How to transform complex linear systems into real ones.
• About the structure of complex roots of real-valued polynomials.
• About complex eigenvalues and eigenvectors of real matrices.
• How to diagonalize matrices with complex eigenvalues.

### Section 19 (Normed Spaces, Inner Product Spaces)

• Important properties of the Euclidean norm and dot product in Rn
• The Cauchy-Schwarz inequality and triangle inequality.
• How are norm and inner product defined and used in general linear spaces.
• About important norms and inner products in spaces of matrices and polynomials.
• How to calculate norms, distances and angles of matrices and polynomials.
• That every inner product induces a norm, but not every norm induces an inner product.
• About the norm and inner product of complex vectors in Cn.
• About the norm and inner product in general complex linear spaces.

### Section 20 (Orthogonality and Best Approximation)

• About orthogonal complements and orthogonal subspaces.
• How to find a basis in an orthogonal complement.
• That Row(A) and Nul(A) of a m×n matrix A are orthogonal complements in Rn.
• About orthogonal sets and orthogonal bases.
• About orthogonal decompositions and orthogonal projections on subspaces.
• That orthogonal projection operators are idempotent.
• How to calculate orthogonal decomposition with and without orthogonal basis.
• How to calculate the best approximation of an element in a subspace.
• How to calculate the distance of an element from a subspace.
• The Gram-Schmidt process.
• How to orthogonalize vectors, matrices, and polynomials.
• About the Fourier series expansion of periodic functions.
• How to obtain Legendre polynomials.
• How to use Legendre polynomials for best polynomial approximation of functions.

## Unit 5

### Section 21 (Spectral Theorem)

• About the basis and dimension of the complex vector space Cn
• What are conjugate-transpose matrices and how to work with them.
• About Hermitian, orthogonal, and unitary matrices.
• That the eigenvalues of real symmetric and Hermitian matrices are real.
• That the eigenvectors of real symmetric and Hermitian matrices can be used to create an orthogonal basis in Rn and Cn.
• About orthogonal diagonalization of real symmetric matrices.
• About unitary diagonalization of Hermitian matrices.
• The Spectral Theorem for real symmetric and Hermitian matrices.
• About the outer product of vectors.
• How to perform spectral decomposition of real symmetric and Hermitian matrices.
• How eigenvalues are related to definiteness of real symmetric and Hermitian matrices.
• How to calculate eigenvalues of large matrices using Numpy.

### Section 22 (QR Factorization and Least-Squares Problems)

• How to perform the QR factorization of rectangular and square matrices.
• What are Least-Squares problems and how to solve them.
• Properties of the normal equation, existence and uniqueness of solution.
• How to fit lines, polynomials, and bivariate polynomials to data.
• How to fit implicitly given curves to data.
• How to solve Least-Square problems via QR factorization.

### Section 23 (Singular Value Decomposition)

• The basic idea of SVD.
• About the shared eigenvalues of the matrices \(A^TA\) and \(AA^T.\)
• What are singular values and how to calculate them efficiently.
• That the number of nonzero singular values equals the rank of the matrix.
• How to calculate right and left singular vectors, and perform SVD.
• The difference between full and compact SVD.
• About the Moore-Penrose pseudoinverse and its basic properties.
• What types of problems can be solved using the M-P pseudoinverse.
• How to calculate the M-P pseudoinverse using SVD.
• How to calculate the SVD of complex matrices.
• How are singular values related to the Frobenius norm.
• What is the spectral (operator) norm of a matrix.
• How to calculate the error caused by truncating the SVD.
• How to use SVD for rank estimation in large data sets.
• How SVD is used in image processing.

### Section 24 (Large Linear Systems)

• The condition number and how to calculate it using singular values.
• The condition number of real symmetric and Hermitian matrices.
• The role the condition number plays in the solution of linear systems.
• Mantissa, exponent, and the representation (round-off) error.
• Machine epsilon and finite computer arithmetic.
• Instability of the Gauss elimination.
• Memory issues related to storing large matrices.
• COO, CSR and CSC representation of sparse matrices.
• Sparsity-preserving and sparsity-breaking matrix operations.

### Section 25 (Linear Algebra with Python)

• Import Numpy and Scipy.
• Define (large) vectors and matrices.
• Perform standard vector and matrix operations.
• Edit matrices and vectors using the Python for-loop.
• Extract row and column vectors from matrices.
• Use direct and iterative matrix solvers.
• Determine the rank of a matrix.
• Perform LU and Cholesky factorizations.
• Calculate determinants, eigenvalues and eigenvectors.
• Diagonalize matrices, calculate functions of matrices.
• Perform QR factorization and orthogonalize sets of vectors.
• Solve Least-Squares problems.
• Perform spectral decomposition and SVD.