System of linear equations

tags
Applied maths

Such a system with \(m\) equations and \(n\) unknowns is often denoted \(Ax = b\) where \(A\) is a matrix \(m\times n\) and \(b\) is a vector of size \(m\).

There are multiple methods to solve such a systems with different sets of hypotheses.

System types

Square matrix with full rank

In the most simple case: a square matrix with full rank, the solution exists and is unique: \(x = A^{-1} b\)

Underdetermined systems

In general, such systems have a family of solutions that can be parametrized by some constraint on their components.

They can be solved with the following equation: \[ x = A^+ b + (I - A^+ A)w \], where \(w\) is a vector and \(A^+\) is the Moore-Penrose inverse of the matrix \(A\). The solutions form a vector space.

Overdetermined systems

In overdetermined systems, there is no actual solution. However, one can look for the best solution according to some criterion. If that criterion is the sum of squared residuals, this leads to ordinary least squares.

Links to this note


← Back to Notes