## Unit 1

### Section 1 (Vectors and Matrices I)

- What is an n-vector, how to add and subtract vectors, and multiply them with scalars (real numbers).
- What is the dot product of vectors.
- How is the dot product related to angle, orthogonality, Euclidean norm (length) and distance of vectors.
- What is an m × n (“m by n”) matrix and how to correctly determine matrix dimensions.
- How to add and subtract matrices, and multiply them with scalars.
- How to use indices when accessing individual entries in vectors and matrices.
- How to calculate matrix-vector and matrix-matrix products, and when they are defined and undefined.

### Section 2 (Vectors and Matrices II)

- More about vectors and matrices, including zero vectors, zero matrices, identity matrices, and diagonal matrices.
- About commutative, associative, and distributive properties of vector and matrix operations.
- That some important vector and matrix operations are not commutative.
- How to count arithmetic operations involved in various vector and matrix operations.
- Which vector and matrix operations are most computationally demanding.
- About matrix transposition, and how to work with transposed and symmetric matrices.

### Section 3 (Linear Systems)

- Basic facts about linear systems and their solutions.
- Basic terminology including equivalent, consistent and inconsistent systems.
- About the existence and uniqueness of solutions to linear systems.
- How to transform a linear system to a matrix form.

### Section 4 (Elementary Row Operations)

- How to perform three types of elementary row operations with matrices.
- That elementary row operations are reversible, and how to reverse them.
- How to create the augmented matrix (A|b) of a linear system Ax = b.
- How applying elementary row operations to an augmented matrix changes the underlying linear system.
- That applying elementary row operations to an augmented matrix does not change the solution to the underlying linear system.
- How to solve linear systems using Gauss elimination.
- That the Gauss elimination has two phases – forward elimination and back substitution.
- That the Gauss elimination is reversible.
- How to do the Gauss elimination efficiently.

### Section 5 (Echelon Forms)

- What is an echelon form of a matrix and how to obtain it.
- How to use the echelon form to infer existence and uniqueness of solution.
- What is an under- and over-determined system.
- How to determine pivot positions and pivot columns.
- What are basic and free variables.
- What is the reduced echelon form and how to obtain it.
- About the uniqueness of the reduced echelon form.
- How to express solution sets to linear systems in parametric vector form.

## Unit 2

### Section 6 (Linear Combinations)

- What is a linear combination of vectors.
- How to write a linear system as a vector equation and vice versa.
- How to write a vector equation as a matrix equation Ax = b and vice versa.
- What is a linear span, and how to check if a vector is in the linear span of other vectors.
- The Solvability Theorem for linear systems.
- That the linear system Ax = b has a solution for every right-hand side if and only if the matrix A has a pivot in every row.

### Section 7 (Linear Independence)

- What it means for vectors to be linearly independent or dependent.
- How to determine linear independence or dependence of vectors.
- How is linear independence related to linear span.
- How is linear independence related to the matrix equation Ax = 0.
- About singular and nonsingular matrices.
- About homogeneous and nonhomogeneous linear systems.

### Section 8 (Linear Transformations I)

- About linear vector transformations of the form T(x) = Ax.
- How to figure out the matrix of a linear vector transformation.
- About rotations, scaling transformations, and shear transformations.
- That surjectivity of the transformation T(x) = Ax is equivalent to the existence of a solution to the linear system Ax = b for any right-hand side.
- How to easily check if the transformation T(x) = Ax is onto (surjective).
- That injectivity of the transformation T(x) = Ax is equivalent to the uniqueness of solution to the linear system Ax = b.
- How to easily check if the transformation T(x) = Ax is one-to-one (injective).
- That the solution to the linear system Ax = b is unique if and only if the matrix A has a pivot in every column.
- That bijectivity of the transformation T(x) = Ax is equivalent to the existence and uniqueness of solution to the linear system Ax = b for any right-hand side.

### Section 9 (Linear Transformations II)

- How to find a spanning set for the range of a linear transformation.
- What is the kernel of a linear transformation.
- How to find a spanning set for the kernel.
- That the kernel determines injectivity of linear transformations.
- About compositions of linear transformations.
- How to obtain the matrix of a composite transformation.
- About inverse matrices and inverse transformations.
- How to obtain the matrix of an inverse transformation.
- What can one do with inverse matrices.

### Section 10 (Special Matrices and Decompositions)

- What are block matrices and how to work with them.
- How to count arithmetic operations in the Gauss elimination.
- How to work with block diagonal, tridiagonal and diagonal matrices.
- What are upper and lower triangular matrices.
- What are sparse and dense matrices.
- What are elementary matrices.
- Gauss elimination in terms of elementary matrices.
- Matrix inversion in terms of elementary matrices.
- About LU decomposition of nonsingular matrices.
- What is the definiteness of symmetric matrices.
- How to check if a symmetric matrix is positive-definite, positive-semidefinite, negative-semidefinite, negative-definite or indefinite.
- About Cholesky decomposition of symmetric positive-definite (SPD) matrices.

## Unit 3

### Section 11 (Determinants I)

- What is a permutation and how to calculate its sign.
- What is a determinant, the Leibniz formula.
- Simplified formulas for 2×2 and 3×3 determinants.
- How to calculate determinants using Laplace (cofactor) expansion.
- How to calculate determinants using elementary row operations.
- How to calculate determinants of diagonal, block diagonal and triangular matrices.

### Section 12 (Determinants II)

- About the determinants of elementary matrices and their inverses.
- That matrix A is invertible if and only if det(A) ≠ 0, and det(A-1) = 1/det(A).
- That for any two n×n matrices A and B it holds det(AB) = det(A)det(B) .
- That det(A+B) ≠ det(A) + det(B) .
- That det(A) = det(AT) and what it means for column operations.
- How to use the so-called Cramer’s rule to solve the linear system Ax=b.
- About the adjugate (adjunct) matrix and how to use it to calculate matrix inverse.
- About the relations between det(A), linear dependence of columns and rows in the matrix A, properties of the linear transformation T(x)=Ax, and the existence and uniqueness of solution to the linear system Ax=b .
- How to use determinants to calculate area and volume.

### Section 13 (Linear Spaces)

- What is a linear space and how it differs from a standard set.
- How to check whether or not a set is a linear space.
- That linear spaces may contain other types of elements besides vectors.
- That a linear span always is a linear space.
- How to check if a set of vectors generates a given linear space.
- How to work with linear spaces of matrices.
- How to work with polynomial spaces.

### Section 14 (Basis, Coordinates, Subspaces)

- What is a basis, and that a linear space can have many bases.
- How to find the coordinates of an element relative to a given basis.
- How to determine the dimension of a linear space.
- What is a subspace of a linear space, and how to recognize one.
- About the union and intersection of subspaces.
- How the change of basis in a linear space affects the coordinates of its elements.

### Section 15 (Null Space, Column Space, Rank)

- What is the null space, how to check if a vector belongs to the null space.
- How to determine the dimension of the null space and find its basis.
- How is the null space related to the uniqueness of solution to the linear system Ax=b
- What is the column space, how to check if a vector belongs to the column space.
- How to determine the dimension of the column space and find its basis.
- How is the column space related to the existence of solution to the linear system Ax=b
- How is the null space related to the kernel of the linear transformation T(x)=Ax
- How is the column space related to the range of the linear transformation T(x)=Ax
- What is the rank of a matrix, and its implications for the existence and uniqueness of solution to the linear system Ax = b.
- What does it mean for a matrix to have full rank or be rank-deficient.
- The Rank Theorem.

## Unit 4

### Section 16 (Eigenproblems I)

- What is an eigenproblem.
- About important applications of eigenproblems in various areas of science and engineering.
- How to verify if a given vector is an eigenvector, and if a given value is an eigenvalue.
- How to calculate eigenvalues and eigenvectors.
- About the characteristic polynomial and characteristic equation.
- How to handle matrices with repeated eigenvalues.
- About algebraic and geometric multiplicity of eigenvalues.
- How to determine algebraic and geometric multiplicity of eigenvalues.
- How to find a basis of an eigenspace.

### Section 17 (Eigenproblems II)

- The geometrical meaning of eigenvalues and eigenvectors in R2 and R3.
- That a matrix is singular if and only if it has a zero eigenvalue.
- That the null space of a matrix is the eigenspace corresponding to the zero eigenvalue.
- That eigenvectors corresponding to different eigenvalues are linearly independent.
- The Cayley–Hamilton theorem (CHT).
- How to use the CHT to efficiently calculate matrix inverse.
- How to use the CHT to efficiently calculate matrix powers and matrix exponential.
- About similar matrices, diagonalizable matrices, and eigenvector basis.
- How to use the eigenvector basis to efficiently calculate arbitrary matrix functions including matrix powers, the inverse matrix, matrix exponential, the square root of a matrix, etc.

### Section 18 (Complex Linear Systems)

- About complex numbers and basic complex arithmetic.
- How to solve complex linear equations with real and complex coefficients.
- How to solve complex linear systems with real and complex matrices.
- How to perform complex elementary row operations.
- How to determine the rank of a complex matrix.
- How to check if a complex matrix is nonsingular.
- How to use the Cramer’s rule for complex matrices.
- How to check if complex vectors are linearly independent.
- How to invert complex matrices.
- How to transform complex linear systems into real ones.
- About the structure of complex roots of real-valued polynomials.
- About complex eigenvalues and eigenvectors of real matrices.
- How to diagonalize matrices with complex eigenvalues.

### Section 19 (Normed Spaces, Inner Product Spaces)

- Important properties of the Euclidean norm and dot product in Rn.
- The Cauchy-Schwarz inequality and triangle inequality.
- How are norm and inner product defined and used in general linear spaces.
- About important norms and inner products in spaces of matrices and polynomials.
- How to calculate norms, distances and angles of matrices and polynomials.
- That every inner product induces a norm, but not every norm induces an inner product.
- About the parallelogram law.
- About the norm and inner product of complex vectors in Cn.
- About the norm and inner product in general complex linear spaces.

### Section 20 (Orthogonality and Best Approximation)

- About orthogonal complements and orthogonal subspaces.
- How to find a basis in an orthogonal complement.
- That Row(A) and Nul(A) of a m×n matrix A are orthogonal complements in Rn.
- About orthogonal sets and orthogonal bases.
- About orthogonal decompositions and orthogonal projections on subspaces.
- That orthogonal projection operators are idempotent.
- How to calculate orthogonal decomposition with and without orthogonal basis.
- How to calculate the best approximation of an element in a subspace.
- How to calculate the distance of an element from a subspace.
- The Gram-Schmidt process.
- How to orthogonalize vectors, matrices, and polynomials.
- About the Fourier series expansion of periodic functions.
- How to obtain Legendre polynomials.
- How to use Legendre polynomials for best polynomial approximation of functions.

## Unit 5

### Section 21 (Spectral Theorem)

- About the basis and dimension of the complex vector space Cn.
- What are conjugate-transpose matrices and how to work with them.
- About Hermitian, orthogonal, and unitary matrices.
- That the eigenvalues of real symmetric and Hermitian matrices are real.
- That the eigenvectors of real symmetric and Hermitian matrices can be used to create an orthogonal basis in Rn and Cn.
- About orthogonal diagonalization of real symmetric matrices.
- About unitary diagonalization of Hermitian matrices.
- The Spectral Theorem for real symmetric and Hermitian matrices.
- About the outer product of vectors.
- How to perform spectral decomposition of real symmetric and Hermitian matrices.
- How eigenvalues are related to definiteness of real symmetric and Hermitian matrices.
- How to calculate eigenvalues of large matrices using Numpy.

### Section 22 (QR Factorization and Least-Squares Problems)

- How to perform the QR factorization of rectangular and square matrices.
- What are Least-Squares problems and how to solve them.
- Properties of the normal equation, existence and uniqueness of solution.
- How to fit lines, polynomials, and bivariate polynomials to data.
- How to fit implicitly given curves to data.
- How to solve Least-Square problems via QR factorization.

### Section 23 (Singular Value Decomposition)

- The basic idea of SVD.
- About the shared eigenvalues of the matrices \(A^TA\) and \(AA^T.\)
- What are singular values and how to calculate them efficiently.
- That the number of nonzero singular values equals the rank of the matrix.
- How to calculate right and left singular vectors, and perform SVD.
- The difference between full and compact SVD.
- About the Moore-Penrose pseudoinverse and its basic properties.
- What types of problems can be solved using the M-P pseudoinverse.
- How to calculate the M-P pseudoinverse using SVD.
- How to calculate the SVD of complex matrices.
- How are singular values related to the Frobenius norm.
- What is the spectral (operator) norm of a matrix.
- How to calculate the error caused by truncating the SVD.
- How to use SVD for rank estimation in large data sets.
- How SVD is used in image processing.

### Section 24 (Large Linear Systems)

- The condition number and how to calculate it using singular values.
- The condition number of real symmetric and Hermitian matrices.
- The role the condition number plays in the solution of linear systems.
- Mantissa, exponent, and the representation (round-off) error.
- Machine epsilon and finite computer arithmetic.
- Instability of the Gauss elimination.
- Memory issues related to storing large matrices.
- COO, CSR and CSC representation of sparse matrices.
- Sparsity-preserving and sparsity-breaking matrix operations.

### Section 25 (Linear Algebra with Python)

- Import Numpy and Scipy.
- Define (large) vectors and matrices.
- Perform standard vector and matrix operations.
- Edit matrices and vectors using the Python for-loop.
- Extract row and column vectors from matrices.
- Use direct and iterative matrix solvers.
- Determine the rank of a matrix.
- Perform LU and Cholesky factorizations.
- Calculate determinants, eigenvalues and eigenvectors.
- Diagonalize matrices, calculate functions of matrices.
- Perform QR factorization and orthogonalize sets of vectors.
- Solve Least-Squares problems.
- Perform spectral decomposition and SVD.