Equation 6.1:
\begin{equation} \begin{cases} \begin{array}{rrrl} 2 & \!\!\!\!\!\! x_1 + 4 &\!\!\!\!\!\!\!x_2 - 2 &\!\!\!\!\!\!x_3 = \hid{-}16, \\ - & \!\!\!\!\!\! x_1 - 7 &\!\!\!\!\!\!x_2 + 2 &\!\!\!\!\!\!x_3 = -27, \\ & 3 &\!\!\!\!\!\!x_2 - 6 &\!\!\!\!\!\!x_3 = -21. \\ \end{array} \end{cases} \end{equation}

Equation 6.5:
\begin{align} \mx{A}=& \left( \begin{array}{rrr} 2 & 4 & -2 \\ -1 & -7 & 2 \\ 0 & 3 & -6 \end{array} \right) = \left( \begin{array}{c} -\,\,\, \vc{a}_{1,}^\T - \\ -\,\,\, \vc{a}_{2,}^\T - \\ -\,\,\, \vc{a}_{3,}^\T - \end{array} \right), \\ &\text{where } \vc{a}_{1,} = \left( \begin{array}{rrr} 2 \\ 4\\ -2 \end{array} \right), \ \ \vc{a}_{2,} = \left( \begin{array}{rrr} -1 \\ -7\\ 2 \end{array} \right), \ \ \text{and } \vc{a}_{3,} = \left( \begin{array}{rrr} 0 \\ 3\\ -6 \end{array} \right). \end{align}

Equation 6.4:
\begin{align} \mx{A}=& \left( \begin{array}{rrr} 2 & 4 & -2 \\ -1 & -7 & 2 \\ 0 & 3 & -6 \end{array} \right) = \left( \begin{array}{ccc} \vert & \vert & \vert \\ \vc{a}_{,1} & \vc{a}_{,2} & \vc{a}_{,3} \\ \vert & \vert & \vert \end{array} \right), \\ &\\ &\\ &\text{where } \vc{a}_{,1} = \left( \begin{array}{rrr} 2 \\ -1\\ 0 \end{array} \right), \ \ \vc{a}_{,2} = \left( \begin{array}{rrr} 4 \\ -7\\ 3 \end{array} \right), \ \ \text{and } \vc{a}_{,3} = \left( \begin{array}{rrr} -2 \\ 2\\ -6 \end{array} \right). \end{align}

Equation 6.6:
\begin{equation} \mx{A} = \bigl(\vc{a}_{,1} \,\,\, \vc{a}_{,2} \,\,\,\dots\,\,\, \vc{a}_{,c}\bigr) = \left( \begin{array}{c} \vc{a}_{1,}^\T\\ \vc{a}_{2,}^\T\\ \vdots \\ \vc{a}_{r,}^\T\\ \end{array} \right). \end{equation}

Definition 6.6: Symmetric Matrix
A square matrix is called symmetric if $\mx{A}=\mx{A}^\T$.

Definition 6.7: Matrix Multiplication by a Scalar
A matrix $\mx{A}$ can be multiplied by a scalar $k$ to form a new matrix $\mx{S} = k \mx{A}$, which is of the same size as $\mx{A}$. \begin{equation} \mx{S}= \left( \begin{array}{cccc} s_{11} & s_{12} & \dots & s_{1c} \\ s_{21} & s_{22} & \dots & s_{2c} \\ \vdots & \vdots & \ddots & \vdots \\ s_{r1} & s_{r2} & \dots & s_{rc} \end{array} \right) = \left( \begin{array}{cccc} k a_{11} & k a_{12} & \dots & k a_{1c} \\ k a_{21} & k a_{22} & \dots & k a_{2c} \\ \vdots & \vdots & \ddots & \vdots \\ k a_{r1} & k a_{r2} & \dots & k a_{rc} \end{array} \right) \end{equation} This is more compactly expressed as: $[ s_{ij} ] = k[ a_{ij} ] = [ k a_{ij} ]$.

Definition 3.4: Dot Product Calculation in Orthonormal Basis
In any orthonormal basis, the dot product between two $n$-dimensional vectors, $\vc{u}$ and $\vc{v}$, can be calculated as \begin{equation} \vc{u}\cdot\vc{v} = \sum_{i=1}^{n} u_i v_i, \end{equation} which is a sum of component-wise multiplications. The two- and three-dimensional dot products are calculated as \begin{align} \mathrm{two\ dimensions\ } &:\ \ \vc{u}\cdot\vc{v} = u_xv_x + u_yv_y, \\ \mathrm{three\ dimensions\ } &:\ \ \vc{u}\cdot\vc{v} = u_xv_x + u_yv_y +u_zv_z. \\ \end{align}

Definition 6.2: Row and Column Vectors from a Matrix
The $i$:th row vector of an $r \times c$ matrix, $\mx{A}$, is denoted by $\vc{a}_{i,}^\T$, and it has $c$ scalar elements in it. The $i$:th column vector of $\mx{A}$ is denoted by $\vc{a}_{,i}$, which has $r$ scalar elements in it. Using vectors, a matrix can thus be written in the following two ways, \begin{equation} \mx{A} = \bigl(\vc{a}_{,1} \,\,\, \vc{a}_{,2} \,\,\,\dots\,\,\, \vc{a}_{,c}\bigr) = \left( \begin{array}{c} \vc{a}_{1,}^\T\\ \vc{a}_{2,}^\T\\ \vdots \\ \vc{a}_{r,}^\T\\ \end{array} \right). \end{equation}

Equation 6.22:
\begin{equation} [p_{ij}] = \Biggl[\sum_{k=1}^s a_{ik} b_{kj}\Biggr] = \bigl[ \vc{a}_{i,} \cdot \vc{b}_{,j} \bigr], \end{equation}

Equation 6.20:
\begin{equation} (r \times \bcancel{s})\, (\bcancel{s} \times t) \longrightarrow (r \times t), \end{equation}

Definition 6.9: Matrix-Matrix Multiplication
If $\mx{A}$ is an $r \times s$ matrix and $\mx{B}$ is an $s\times t$ matrix, then the product matrix $\mx{P}=\mx{A}\mx{B}$, which is an $r \times t$ matrix, is defined as \begin{align} \mx{P} =& \mx{A}\mx{B} = \left( \begin{array}{ccc} a_{11} & \dots & a_{1s} \\ \vdots & \ddots & \vdots \\ a_{r1} & \dots & a_{rs} \end{array} \right) \left( \begin{array}{ccc} b_{11} & \dots & b_{1t} \\ \vdots & \ddots & \vdots \\ b_{s1} & \dots & b_{st} \end{array} \right)\\ &\\ =& \left( \begin{array}{ccc} \sum_{k=1}^s a_{1k} b_{k1} & \dots & \sum_{k=1}^s a_{1k} b_{kt} \\ \vdots & \ddots & \vdots \\ \sum_{k=1}^s a_{rk} b_{k1} & \dots & \sum_{k=1}^s a_{sk} b_{kt} \end{array} \right) = \left( \begin{array}{ccc} p_{11} & \dots & p_{1t} \\ \vdots & \ddots & \vdots \\ p_{r1} & \dots & p_{rt} \end{array} \right). \end{align} Note that there must be as many columns in $\mx{A}$ as there are rows in $\mx{B}$, otherwise the matrix-matrix multiplication is not defined. The matrix-matrix multiplication is more compactly expressed as $\bigl[p_{ij}\bigr] = \Bigl[\sum_{k=1}^s a_{ik} b_{kj}\Bigr]$.

Equation 6.8:
\begin{equation} \underbrace{ \mx{A} }_{3\times 1} = \left( \begin{array}{c} 3 \\ 2 \\ 6 \end{array} \right) = \underbrace{\vc{a}}_{\begin{array}{c} \text{column} \\ \text{vector} \end{array} } ,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, \underbrace{ \mx{B} }_{1\times 3} = \bigl(5\,\,\, 1 \,\,\, 4\bigr) = \underbrace{ \vc{b}^\T }_{\begin{array}{c} \text{transposed} \\ \text{column} \\ \text{vector} \end{array} } \end{equation}

Equation 6.27:
\begin{gather} \begin{cases} \begin{array}{r} z_1 = a_{11} y_1 + a_{12} y_2 \\ z_2 = a_{21} y_1 + a_{22} y_2 \end{array} \end{cases} \,\,\,\,\,\,\,\,\text{and}\,\,\,\,\,\,\,\, \begin{cases} \begin{array}{r} y_1 = b_{11} x_1 + b_{12} x_2 \\ y_2 = b_{21} x_1 + b_{22} x_2 \end{array} \end{cases} . \end{gather}

Equation 6.28:
\begin{gather} \begin{cases} \begin{array}{r} z_1 = a_{11} (b_{11} x_1 + b_{12} x_2) + a_{12} (b_{21} x_1 + b_{22} x_2) \\ z_2 = a_{21} (b_{11} x_1 + b_{12} x_2) + a_{22} (b_{21} x_1 + b_{22} x_2) \end{array} \end{cases} \\  \Longleftrightarrow \\ \begin{cases} \begin{array}{r} z_1 = (a_{11} b_{11} + a_{12} b_{21}) x_1 + (a_{11} b_{12} + a_{12} b_{22}) x_2) \\ z_2 = (a_{21} b_{11} + a_{22} b_{21}) x_1 + (a_{21} b_{12} + a_{22} b_{22}) x_2) \end{array} \end{cases} . \end{gather}

Equation 6.21:
\begin{align} \mx{P} = \mx{A}\mx{B} &= \left( \begin{array}{c} -\,\,\, \vc{a}_{1,}^\T - \\ \textcolor{#cc0000}{-}\,\,\, \textcolor{#cc0000}{\vc{a}_{2,}^\T} \textcolor{#cc0000}{-} \\ \vdots \\ -\,\,\, \vc{a}_{r,}^\T - \end{array} \right) \left( \begin{array}{ccccc} \vert & \vert & \textcolor{#cc0000}{\vert} & & \vert \\ \vc{b}_{,1} & \vc{b}_{,2} & \textcolor{#cc0000}{\vc{b}_{,3}} & \dots & \vc{b}_{,t} \\ \vert & \vert & \textcolor{#cc0000}{\vert} & & \vert \end{array} \right) \\ &\\ &= \left( \begin{array}{ccccc} \vc{a}_{1,}\cdot \vc{b}_{,1} & \vc{a}_{1,}\cdot \vc{b}_{,2} & \vc{a}_{1,}\cdot \vc{b}_{,3} & \dots & \vc{a}_{1,}\cdot \vc{b}_{,t} \\ \vc{a}_{2,}\cdot \vc{b}_{,1} & \vc{a}_{2,}\cdot \vc{b}_{,2} & \textcolor{#cc0000}{\vc{a}_{2,}\cdot \vc{b}_{,3}} & \dots & \vc{a}_{2,}\cdot \vc{b}_{,t} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \vc{a}_{r,}\cdot \vc{b}_{,1} & \vc{a}_{r,}\cdot \vc{b}_{,2} & \vc{a}_{r,}\cdot \vc{b}_{,3} & \dots & \vc{a}_{r,}\cdot \vc{b}_{,t} \\ \end{array} \right). \end{align}

Theorem 5.2: Gaussian Elimination Rules
The solutions to a linear system of equations do not change if we

1. swap the order of two equations,
2. multiply an equation with a constant $\neq 0$, or
3. add a multiple of another equation to an equation.

Definition 6.10: Two-Dimensional Rotation Matrix
A $2\times 2$ rotation matrix is defined by \begin{align} \mx{R}(\phi) = & \left(\begin{array}{rr} \cos \phi & -\sin \phi \\ \sin \phi & \cos \phi \end{array} \right), \end{align} where $\phi$ is the number of radians that the matrix rotates by (counter-clockwise).

Definition 6.12: Two-Dimensional Shear Matrix
A two-dimensional shear matrix is defined by either of \begin{align} \mx{H}_{xy}(s) = \begin{pmatrix} 1 & s \\ 0 & 1 \end{pmatrix} \ \ \ \mathrm{or} \ \ \ \mx{H}_{yx}(s) = \begin{pmatrix} 1 & 0 \\ s & 1 \end{pmatrix} . \end{align} Note that the first subscript of $\mx{H}$ refers to which coordinate is changed, and the second subscript refers to the coordinate that is used to scale by $s$ and add to that first coordinate.

Theorem 3.1: Dot Product Rules
The following is a set of useful rules when using dot products. \begin{align} \begin{array}{llr} (i) & \vc{u} \cdot \vc{v} = \vc{v} \cdot \vc{u} & \spc\text{(commutativity)} \\ (ii) & k(\vc{u} \cdot \vc{v}) = (k\vc{u}) \cdot \vc{v} & \spc\text{(associativity)} \\ (iii) & \vc{v} \cdot (\vc{u} +\vc{w}) = \vc{v} \cdot \vc{u} + \vc{v} \cdot \vc{w} & \spc\text{(distributivity)} \\ (iv) & \vc{v} \cdot \vc{v} = \ln{\vc{v}}^2 \geq 0, \mathrm{with\ equality\ only\ when\ } \vc{v}=\vc{0}. & \spc\text{(squared length)} \\ \end{array} \end{align}

Equation 6.23:
\begin{equation} [p_{ij}] = \Biggl[ \sum_{k=1}^s a_{ik} b_{kj} \Biggr] = \biggl[ \vc{a}_{i,}^\T \vc{b}_{,j} \biggr] = \Biggl[ \Bigl(a_1\spc a_2 \dots a_s \Bigr) \left( \begin{array}{c} b_1\\ b_2\\ \vdots\\ b_s \end{array} \right) \Biggr]. \end{equation}

Definition 6.4: Zero Matrix
A zero matrix, $\mx{O}$, has all its matrix elements equal to zero.

Equation 6.50:
\begin{gather} \mx{A}\mx{B} = \mx{A}\mx{C} \\ \Longleftrightarrow \\ \mx{A}\mx{B} - \mx{A}\mx{C} = \mx{O} \\ \Longleftrightarrow \\ \mx{A}(\mx{B} -\mx{C}) = \mx{O}, \end{gather}

Equation 6.49:
\begin{align} \mx{A}= \left( \begin{array}{rr} 2 & 1 \\ 6 & 3 \end{array} \right) \spc\spc\text{and}\spc\spc \mx{B}= \left( \begin{array}{rr} -2 & 3 \\ 4 & -6 \end{array} \right) \\ \mx{A}\mx{B}= \left( \begin{array}{rr} 2\cdot(-2) + 1\cdot 4 & 2\cdot 3 + 1\cdot (-6) \\ 6\cdot(-2) + 3\cdot 4 & 6\cdot 3 + 3\cdot (-6) \end{array} \right) = \left( \begin{array}{rr} 0 & 0 \\ 0 & 0 \end{array} \right) =\mx{O}. \end{align}

Theorem 6.1: Matrix Arithmetic Properties
In the following, we assume that the sizes of the matrices are such that the operations are defined. \begin{equation} \begin{array}{llr} (i) & k(l\mx{A}) = (kl)\mx{A} & \spc\text{(associativity)} \\ (ii) & (k+l)\mx{A} = k\mx{A} +l\mx{A} & \spc\text{(distributivity)} \\ (iii) & k(\mx{A}+\mx{B}) = k\mx{A} +k\mx{B} & \spc\text{(distributivity)} \\ (iv) & \mx{A} + \mx{B} = \mx{B} + \mx{A} & \spc\text{(commutativity)} \\ (v) & \mx{A}+(\mx{B}+\mx{C})=(\mx{A}+\mx{B})+\mx{C} & \spc\text{(associativity)} \\ (vi) & \mx{A}+ (-1)\mx{A} = \mx{O} & \spc\text{(additive inverse)} \\ (vii) & \mx{A}(\mx{B}+\mx{C})=\mx{A}\mx{B}+\mx{A}\mx{C} & \spc\text{(distributivity)} \\ (viii) & (\mx{A}+\mx{B})\mx{C}=\mx{A}\mx{C}+\mx{B}\mx{C} & \spc\text{(distributivity)} \\ (ix) & (\mx{A}\mx{B})\mx{C}=\mx{A}(\mx{B}\mx{C}) & \spc\text{(associativity)} \\ (x) & \mx{I}\mx{A}=\mx{A}\mx{I}=\mx{A} & \spc\text{(multiplicative one)} \\ (xi) & (k\mx{A})^\T=k\mx{A}^\T & \spc\text{(transpose rule 1)} \\ (xii) & (\mx{A}+\mx{B})^\T=\mx{A}^\T+\mx{B}^\T & \spc\text{(transpose rule 2)} \\ (xiii) & (\mx{A}^\T)^\T=\mx{A} & \spc\text{(transpose rule 3)} \\ (xiv) & (\mx{A}\mx{B})^\T=\mx{B}^\T\mx{A}^\T & \spc\text{(transpose rule 4)} \\ \end{array} \end{equation} In addition, we have the following trivial set of rules: $1\mx{A}=\mx{A}$, $0\mx{A}=\mx{O}$, $k\mx{O}=\mx{O}$, and $\mx{A}+\mx{O}=\mx{A}$.

Equation 6.55:
\begin{equation} ax = b. \end{equation}

Equation 6.54:
\begin{equation} \mx{A}\mx{X} = \mx{B}. \end{equation}

Equation 6.58:
\begin{equation} \mx{A}\mx{A}^{-1} = \mx{A}^{-1}\mx{A} = \mx{I}. \end{equation}

Definition 6.17: Matrix Inverse
The square matrix $\mx{A}$ is said to be invertible if there exists a matrix $\mx{A}^{-1}$, which is called the inverse of $\mx{A}$, such that \begin{equation} \mx{A}\mx{A}^{-1} = \mx{A}^{-1}\mx{A} = \mx{I}. \end{equation} For $\mx{A}\mx{A}^{-1} = \mx{I}$, $\mx{A}^{-1}$ is called a right-side inverse, while $\mx{A}^{-1}$ is called a left-side inverse if $\mx{A}^{-1}\mx{A} = \mx{I}$.

Theorem 6.3: Two-Dimensional Matrix Inverse
For a $2\times 2$ matrix, $\mx{A}$, the inverse is \begin{equation} \mx{A}^{-1} = \begin{pmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{pmatrix} ^{-1} = \frac{1}{a_{11}a_{22} - a_{12}a_{21}} \begin{pmatrix} \hid{-}a_{22} & -a_{12} \\ -a_{21} & \hid{-}a_{11} \end{pmatrix} , \end{equation} if $a_{11}a_{22} - a_{12}a_{21} \neq 0$, otherwise, the inverse does not exist.

Equation 6.64:
\begin{gather} \mx{A}\vc{x} = \vc{y} \\ \Longleftrightarrow \\ \mx{A}\vc{x} = \mx{I}\vc{y}, \end{gather}

Equation 6.26:
\begin{gather} \begin{cases} \begin{array}{rrrl} 2 & \!\!\!\!\!\! x_1 + 4 &\!\!\!\!\!\!\!x_2 - 2 &\!\!\!\!\!\!x_3 = \hid{-}16 \\ - & \!\!\!\!\!\! x_1 - 7 &\!\!\!\!\!\!x_2 + 2 &\!\!\!\!\!\!x_3 = -27 \\ & 3 &\!\!\!\!\!\!x_2 - 6 &\!\!\!\!\!\!x_3 = -21 \\ \end{array} \end{cases} \\ \Longleftrightarrow \\ \underbrace{ \left( \begin{array}{rrr} -2 & 4 & -2 \\ -1 & -7 & 2 \\ 0 & 3 & -6 \end{array} \right) }_{\mx{A}} \, \underbrace{ \left( \begin{array}{c} x_1\\ x_2\\ x_3 \end{array} \right) }_{\vc{x}} = \underbrace{ \left( \begin{array}{r} 16\\ -27\\ -21 \end{array} \right) }_{\vc{b}} \\ \Longleftrightarrow \\ \mx{A} \vc{x} = \vc{b}. \end{gather}

Theorem 6.4: Matrix Inverse Properties
In the following, if the matrices $\mx{A}$ and $\mx{B}$ are invertible, then $\mx{A}^{\T}$, $\mx{A}^{-1}$, $\mx{B}^{-1}$, and $\mx{A}\mx{B}$ are also invertible, and \begin{equation} \begin{array}{llr} (i) & (\mx{A}^{-1})^{-1} = \mx{A} & \spc\text{(inverse inverse)}, \\ (ii) & (\mx{A}\mx{B})^{-1} = \mx{B}^{-1}\mx{A}^{-1} & \spc\text{(product inverse)}, \\ (iii) & (\mx{A}^{-1})^{\T} = (\mx{A}^{\T})^{-1} & \spc\text{(inverse transpose)}. \\ \end{array} \end{equation} Note the order of $(ii)$, that is, that the order of $\mx{A}$ and $\mx{B}$ are reversed on the right hand side, compared to the left side.

Definition 5.2: Linear Independence and Dependence
The set of vectors, $\vc{v}_1,\dots,\vc{v}_n$, is said to be linearly independent, if the equation \begin{equation} k_1\vc{v}_1 + k_2 \vc{v}_2 + \dots + k_n \vc{v}_n = \vc{0}, \end{equation} only has a single solution which is \begin{equation} k_1 = k_2 = \dots = k_n =0. \end{equation} If there is at least one other solution, then the set of vectors is linearly dependent.

Definition 5.3: Span
The set of vectors $\{\vc{v}_1,\dots,\vc{v}_q\}$ in $\R^n$ is said to span $\R^n$, if the equation \begin{equation} k_1\vc{v}_1 + k_2 \vc{v}_2 + \dots + k_q \vc{v}_q = \vc{u}, \end{equation} has at least one solution, for every vector $\vc{u}$.

Theorem 6.2: Matrix Inverse Existence
Let us call the left-side inverse, $\mx{A}_l^{-1}$ and the right-side inverse, $\mx{A}_r^{-1}$, i.e., $\mx{A}_l^{-1} \mx{A} = \mx{I}$ and $\mx{A}\mx{A}_r^{-1} = \mx{I}$. Then the following holds $(i)$ If $\mx{A}_l^{-1} \mx{A} = \mx{A}\mx{A}_r^{-1} = \mx{I}$ then $\mx{A}_l^{-1} =\mx{A}_r^{-1}$. $(ii)$ There is only one matrix inverse, $\mx{A}^{-1}$, to a matrix $\mx{A}$, i.e., $\mx{A}^{-1}= \mx{A}_l^{-1} =\mx{A}_r^{-1}$.

Theorem 5.5: The Basis Theorem

1. Two vectors $\vc{u}$ and $\vc{v}$ in the plane constitute a basis if and only if they are linearly independent.
2. Three vectors $\vc{u}$, $\vc{v}$, and $\vc{w}$ in 3D space constitute a basis if and only if they are linearly independent.
3. Three or more vectors in the plane are always linearly dependent.
4. Four or more vectors in 3D space are always linearly dependent.

Theorem 6.8: Matrices and Linear Independence
If the columns of $\mx{A}$ span $\R^p$ then there exists a right-inverse $\mx{A}_r^{-1}$ to the matrix $\mx{A}$.

Theorem 6.6: Matrices and Linear Independence
If there exists a left-inverse $\mx{A}_l^{-1}$ to the matrix $\mx{A}$, then the columns of $\mx{A}$ are linearly independent.

Equation 5.102:
\begin{gather} \hat{\vc{e}}_1 = x_{11} \vc{e}_1 + x_{21} \vc{e}_2,\\ \hat{\vc{e}}_2 = x_{12} \vc{e}_1 + x_{22} \vc{e}_2. \end{gather}

Equation 6.83:
\begin{align} \hat{\vc{e}}_j = \sum_{i=1}^n b_{ij}\vc{e}_i, \end{align}

Equation 6.82:
\begin{align} \vc{v} &= \sum_{i=1}^n x_i\vc{e}_i \\ &= \sum_{i=1}^n \hat{x}_i\hat{\vc{e}}_i, \end{align}

Theorem 2.5: Coordinates in Three Dimensions
Let $\vc{e}_1$, $\vc{e}_2$, and $\vc{e}_3$ be three non-zero basis vectors, and that there is no plane that is parallel with all three vectors. For every vector, $\vc{v}$, in the three-dimensional space, there is a single coordinate triplet, $(x,y,z)$, such that \begin{equation} \vc{v} = x\vc{e}_1 + y\vc{e}_2 + z\vc{e}_3. \end{equation}

Equation 6.81:
\begin{align} \vc{v} = \mx{B}\hat{\vc{v}} = \begin{pmatrix} b_{11} & b_{12} & \dots & b_{1,n} \\ b_{21} & b_{22} & \dots & b_{2,n} \\ \vdots & \vdots & \ddots & \vdots \\ b_{n,1} & b_{n,2} & \dots & b_{n,n} \end{pmatrix} \vc{\hat{v}}, \end{align}

Definition 3.3: Orthonormal Basis
For an $n$-dimensional orthonormal basis, consisting of the set of basis vectors, $\{\vc{e}_1, \dots, \vc{e}_n\}$, the following holds \begin{equation} \vc{e}_i \cdot \vc{e}_j = \begin{cases} 1 & \text{if } i=j, \\ 0 & \text{if } i\neq j. \\ \end{cases} \end{equation} This simply means that the basis vectors are of unit length, i.e., they are normalized, and that they are pairwise orthogonal.

Definition 6.18: Orthogonal Matrix
An orthogonal matrix $\mx{B}$ is a square matrix where the column vectors constitute an orthonormal basis.

Theorem 6.12: Orthogonality and Length Preservation
If $\mx{B}$ is an orthogonal matrix, then $\ln{\mx{B}\vc{v}} = \ln{\vc{v}}$, and vice versa, i.e., the transform preserves length.

Theorem 6.14:
If $\mx{A}$ and $\mx{B}$ are orthogonal matrices, then $\mx{A}\mx{B}$ is orthogonal as well.

Theorem 6.10: Change of Base
Given the following relationship between the two $\R^n$ bases $\{\vc{e}_1,\vc{e}_2,\dots,\vc{e}_n\}$ and $\{\hat{\vc{e}}_1,\hat{\vc{e}}_2,\dots,\hat{\vc{e}}_n\}$, \begin{align} \hat{\vc{e}}_1 &= b_{11} \vc{e}_1 + b_{21} \vc{e}_2 + \dots + b_{n,1} \vc{e}_n, \\ \hat{\vc{e}}_2 &= b_{12} \vc{e}_1 + b_{22} \vc{e}_2 + \dots + b_{n,2} \vc{e}_n, \\ &\dots \\ \hat{\vc{e}}_n &= b_{1,n} \vc{e}_1 + b_{2,n} \vc{e}_2 + \dots + b_{n,n} \vc{e}_n, \end{align} where a particular vector $\vc{v}$ has the following two representations \begin{align} \vc{v} &= v_1\vc{e}_1 + v_2\vc{e}_2 + v_3\vc{e}_3+\dots+v_n\vc{e}_n = \\ &= \hat{v}_1\hat{\vc{e}}_1 + \hat{v}_2\hat{\vc{e}}_2 + \hat{v}_3\hat{\vc{e}}_3+\dots+\hat{v}_n\hat{\vc{e}}_n, \end{align} and let $\mx{B}$ be the matrix with $\hat{\vc{e}}_i$ as column vectors, then it holds that \begin{align} \vc{v} = \mx{B}\hat{\vc{v}} = \begin{pmatrix} b_{11} & b_{12} & \dots & b_{1,n} \\ b_{21} & b_{22} & \dots & b_{2,n} \\ \vdots & \vdots & \ddots & \vdots \\ b_{n,1} & b_{n,2} & \dots & b_{n,n} \end{pmatrix} \vc{\hat{v}}, \end{align} where $\vc{v}=(v_1,v_2,v_3,\dots,v_n)$ and $\hat{\vc{v}}=(\hat{v}_1,\hat{v}_2,\hat{v}_3,\dots,\hat{v}_n)$.

弹出帮助:
基(复数为bases)是一组线性无关的(基)向量,使得空间中的每个向量 都可以描述为基向量的线性组合。 如果基为 $\{\vc{e}_1,\vc{e}_2,\vc{e}_3\}$,则空间中的任意向量 $\vc{v}$ 都可以由 三个数 $(x,y,z)$ 描述,使得 $\vc{v} = x\vc{e}_1 +y\vc{e}_2 +z\vc{e}_3$。通常基是隐含的, 在这种情况下我们写 $\vc{v} = (x,y,z)$,或者为了区分从一个向量 $\vc{v}$ 到另一个 $\vc{u}$ 的向量元素, 我们可以使用 $\vc{v}=(v_x,v_y,v_z)$ 和 $\vc{u}=(u_x,u_y,u_z)$ 的记号。

弹出帮助:
$n$ 维列向量 $\vc{v}$ 相对于一个基表示,由一列 $n$ 个标量值组成。 The vector elements are sometimes denoted $v_1$, $v_2$,..., $v_n$. For two- and three-dimensional vectors, we sometimes also use $v_x$, $v_y$, and $v_z$. The notation is

\begin{equation} \underbrace{ \vc{u} = \begin{pmatrix} u_x \end{pmatrix} = \begin{pmatrix} u_1 \end{pmatrix}}_{\text{1D vector}}, \spc\spc \underbrace{ \vc{v} = \begin{pmatrix} v_x \\ v_y \end{pmatrix} = \begin{pmatrix} v_1 \\ v_2 \end{pmatrix}}_{\text{2D vector}}, \spc\spc \underbrace{ \vc{w} = \begin{pmatrix} w_x \\ w_y \\ w_z \end{pmatrix} = \begin{pmatrix} w_1 \\ w_2 \\ w_3 \end{pmatrix}}_{\text{3D vector}}, \end{equation}

where $\vc{u} = u_x \vc{e}_1$, $\vc{v} = v_x \vc{e}_1 + v_y \vc{e}_2$, and $\vc{w} = w_x \vc{e}_1 + w_y \vc{e}_2 + w_z \vc{e}_3$. 注意 $\vc{e}_i$ 是基向量。
在我们的文本中,我们也使用简写记号 $\vc{w} = \bigl(w_1,w_2,w_3\bigr)$,其意义与上面相同 (注意向量元素之间的逗号)。 但行向量之间没有逗号。

弹出帮助:
交换性是指改变运算的两个操作数的顺序不改变结果。 例如,向量加法是交换的,因为 $\vc{u}+\vc{v} = \vc{v}+\vc{u}$。 同样,点积也是交换的,因为 $\vc{u}\cdot\vc{v} =\vc{v}\cdot\vc{u}$。 然而,矩阵乘法一般不是交换的,即 $\mx{A}\mx{B} \neq \mx{B}\mx{A}$, 但矩阵加法是交换的: $\mx{A}+\mx{B} = \mx{B} + \mx{A}$。

弹出帮助:
分配性是指一个操作数与多个操作数相乘,并且这可以 被一个和替代,其中每个项是第一个操作数与其他每个操作数相乘。
例如,向量缩放是分配的,因为 $k(\vc{u}+\vc{v}) = k\vc{u}+k\vc{v}$ 且 $(k+l)\vc{u} = k\vc{u}+l\vc{u}$。
对于点积,我们有 $\vc{v}\cdot(\vc{u}+\vc{w}) = \vc{v}\cdot\vc{u} + \vc{v}\cdot\vc{w}$。
对于向量积,有 $\vc{u} \times (\vc{v} + \vc{w}) = \vc{u} \times \vc{v} + \vc{u} \times \vc{w}$ 且 $(\vc{u} + \vc{v}) \times \vc{w} = \vc{u} \times \vc{w} + \vc{v} \times \vc{w}$。
对于矩阵,我们有以下规则: $(k+l)\mx{A} = k\mx{A} +l\mx{A}$、 $k(\mx{A}+\mx{B}) = k\mx{A} +k\mx{B}$、 $\mx{A}(\mx{B}+\mx{C})=\mx{A}\mx{B}+\mx{A}\mx{C}$ 以及 $(\mx{A}+\mx{B})\mx{C}=\mx{A}\mx{C}+\mx{B}\mx{C}$。

弹出帮助:
点积,也称为标量积,是一个标量值,记为 $\vc{u}\cdot \vc{v}$:

\begin{equation} \vc{u}\cdot \vc{v} = \left\{ \begin{array}{ll} \ln{\vc{u}}\ \ln{\vc{v}} \cos[\vc{u},\vc{v}], & \text{if } \vc{u}\neq \vc{0} \text{ and } \vc{v}\neq \vc{0}\\ 0, & \text{if } \vc{u}=\vc{0} \text{ or } \vc{v}=\vc{0}, \end{array} \right. \end{equation}

其中 $[\vc{u},\vc{v}]$ 是 $\vc{u}$ 和 $\vc{v}$ 之间的最小角。 如果 $\vc{u} \cdot \vc{v} = 0$,则向量是正交(垂直)的,反之亦然。 我们还有 $\vc{u} \cdot \vc{v}>0 \Leftrightarrow 0 < [\vc{u},\vc{v}] < \pi/2$ 以及 $\vc{u} \cdot \vc{v}<0 \Leftrightarrow \pi/2 < [\vc{u},\vc{v}] \leq \pi$。
在正交基中,我们有 $\vc{u}\cdot\vc{v} = \sum_{i=1}^{n} u_i v_i$,其中 $n$ 是维数。 例如,对于三维,我们有 $\vc{u}\cdot\vc{v} = u_xv_x + u_yv_y +u_zv_z$。

弹出帮助:
高斯消元法是将线性方程组每一行的自由变量数目逐个减少, 直到最底行为 $kx_n=l$,即 $x_n=l/k$ 的过程。之后,其余的 自由变量可以通过替换来恢复。

弹出帮助:
在参数形式下,直线由一个起点 $S$、一个直线方向 $\vc{d}$ 和一个标量参数 $t$ 定义, 使得直线上的任意点都可以从直线方程得到,即 $P(t) = S + t\vc{d}$。这种表示适用于任意维数。
在隐式形式下,直线由一个起点 $S$ 和一个法向量 $\vc{n}$ 定义。对于直线上的任意点 $P$, 有 $\vc{n}\cdot (P-S) = 0$。这种表示仅适用于二维。

弹出帮助:
如果方程组只包含零次和一次的多项式项, 即常数或一次项,如 $9x$、$-2y$ 和 $0.5z$,则称其为线性的。

弹出帮助:
向量集 $\vc{v}_1,\dots,\vc{v}_n$ 称为线性无关的,如果方程

\begin{equation} k_1\vc{v}_1 + k_2 \vc{v}_2 + \dots + k_n \vc{v}_n = \vc{0}, \end{equation}

仅有唯一解 $k_1 = k_2 = \dots = k_n =0$。 如果至少存在一个其他解,则该向量集是线性相关的。

弹出帮助:
在二维中,正定向与逆时针顺序相同,而负定向是顺时针顺序。 平面中的两个向量 $\vc{u}$ 和 $\vc{v}$ 是正定向的,如果 $\vc{u}$ 可以以正定向旋转, 使得最小角 $[\vc{u},\vc{v}]$ 变为零。
对于三维向量,假设 $\vc{u}$、$\vc{v}$ 和 $\vc{w}$ 都从公共原点开始。 如果想象你坐在 $\vc{w}$ 的顶端,朝向原点看,并且从这个位置看 $\vc{u}$ 和 $\vc{v}$ 是正定向的,则称向量 $\vc{u}$、$\vc{v}$ 和 $\vc{w}$ 为正定向的。对于负定向向量集也是如此, 只是从这个位置看 $\vc{u}$ 和 $\vc{v}$ 必须是负定向的。

弹出帮助:
正交性是指,例如,两条直线成直角。我们说这些直线是正交的(彼此)。 当两个向量成 $90^\circ$ 或 $\pi/2$ 弧度的角度时,即 $[\vc{u},\vc{v}]=\pi/2$,它们是正交的。 这也记为 $\vc{u} \perp \vc{v}$,这表明向量是正交的, 也就是垂直的。 还要注意,如果两个向量是正交的,则点积为零,即 $\vc{u}\cdot\vc{v}=0$。 正交矩阵 $\mx{B}$ 是一个方阵,其列向量构成一个标准正交基。

弹出帮助:
$n$ 维标准正交基由 $n$ 个基向量 $\vc{e}_i$ 组成,使得 当 $i\neq j$ 时 $\vc{e}_i\cdot \vc{e}_j$ 为 0,当 $i=j$ 时为 1。标准基是标准正交的。 在二维中,我们可以看到这是正确的, 因为 $\vc{e}_1\cdot\vc{e}_2=$$(1,0)\cdot(0,1)=0$ 且 $\vc{e}_i\cdot\vc{e}_i=1$。

弹出帮助:
在参数形式(也称显式形式)下,平面方程由 $P(t_1, t_2) = S + t_1 \vc{d}_1 + t_2\vc{d}_2$ 给出, 其中 $S$ 是平面上的一点,$\vc{d}_1$ 和 $\vc{d}_2$ 是平面上的两个非平行非零向量。 使用参数 $t_1$ 和 $t_2$,可以达到任何点 $P(t_1,t_2)$。
在隐式形式下,平面方程由 $\vc{n}\cdot(P-S)=0$ 给出,其中 $\vc{n}$ 是平面的法向量, $S$ 是平面上的一点。只有对于位于平面中的点 $P$, 该公式才等于 $0$。

弹出帮助:
正定向的向量集也称为右手系。

弹出帮助:
行向量是已转置的列向量,使得列变成了行。 在本书中,列向量是默认向量,因此行向量可以通过 对列向量 $\vc{v}$ 进行转置来产生,即 $\vc{v}^\T$。 因此行向量为 $\vc{v}^\T = \bigl(v_1\ \ v_2 \ \ v_3\bigr)$。注意向量元素之间的空格。 这与列向量 $\vc{w} = \begin{pmatrix} w_1 \\ w_2 \\ w_3 \end{pmatrix}$ 成对比。

弹出帮助:
(线性)方程组只是若干个(线性)方程,人们希望一起求解,使得 解对每个方程都成立。

弹出帮助:
将两个向量 $\vc{u}$ 和 $\vc{v}$, 相加形成一个新向量,记为 $\vc{u}+\vc{v}$. 这通过平移 $\vc{u}$ 使其尾点位于 $\vc{v}$ 的顶点来完成, 或反之亦然,因为向量加法是交换的,即 $\vc{u}+\vc{v} = \vc{v}+\vc{u}$。

弹出帮助:
向量 $\vc{v}$ 的长度记为 $\ln{\vc{v}}$。在标准正交基中, 长度的平方为 $\ln{\vc{v}}^2 = \vc{v}\cdot\vc{v}$,即 $\ln{\vc{v}} = \sqrt{\vc{v}\cdot\vc{v}}$。 例如,对于三维向量,这简化为 $\ln{\vc{v}} = \sqrt{v_x^2 + v_y^2 + v_z^2}$。

第5章:高斯消元法(上一章)

第7章:行列式(下一章)