To err is human, to blame it on someone else is even more human, Jacob’s Law
Irony is wasted on the stupid, Oscar Wilde
Definition. A vector $\vec{AB}$ is a geometric object or quantity that has both magnitude (or length) and direction. Vectors in an n-dimensional Euclidean space can be represented as coordinates vectors in a Cartesian coordinate system.
Definition. The magnitude of the vector $\vec{A}$, also known as length or norm, is given by the square root of the sum of its components squared, $|\vec{A}|~ or~ ||\vec{A}|| = \sqrt{a_1^2+a_2^2+a_3^2}$, e.g., $\vert{}\vert{}\langle 3, 2, 1 \rangle\vert{}\vert{} = \sqrt{3^2+2^2+1^2} = \sqrt{14}$, $\vert{}\vert{}\langle 3, -4, 5 \rangle\vert{}\vert{} = \sqrt{3^2+(-4)^2+5^2} = \sqrt{50} = 5\sqrt{2}$, or $\vert{}\vert{}\langle 1, 0, 0 \rangle\vert{}\vert{} = \sqrt{1^2+0^2+0^2} = \sqrt{1} = 1$.
Vector Addition: The sum of two vectors is the sum of their components, $\vec{A}+ \vec{B} = (a_1+b_1)\vec{i}+(a_2+b_2)\vec{j}+(a_3+b_3)\vec{k} = \langle a_1+b_1, a_2+b_2, a_3+b_3 \rangle$.
Vector Subtraction: The subtraction of two vectors is similar to addition and is also done component-wise, it is given by simply subtracting their corresponding components (x, y, and z in 3D), $\vec{A} - \vec{B} = (a_1-b_1)\vec{i}+(a_2-b_2)\vec{j}+(a_3-b_3)\vec{k} = \langle a_1-b_1, a_2-b_2, a_3-b_3 \rangle$.
Scalar multiplication is the multiplication of a vector by a scalar, a real number, changing its magnitude without altering its direction (unless the scalar is negative, which reverses the direction). It is effectively multiplying each component of the vector by the scalar value, $c\vec{A} = (ca_1)\vec{i}+(ca_2)\vec{j}+(ca_3)\vec{k} = \langle ca_1, ca_2, ca_3 \rangle$.
The dot or scalar product is a fundamental operation between two vectors. It produces a scalar quantity that represents the projection of one vector onto another. The dot product is the sum of the products of their corresponding components: $\vec{A}·\vec{B} = \sum a_ib_i = a_1b_1 + a_2b_2 + a_3b_3.$, e.g. $\vec{A}·\vec{B} = \sum a_ib_i = \langle 2, 2, -1 \rangle \cdot \langle 5, -3, 2 \rangle = (2 \cdot 5) + (2 \cdot -3) + (-1 \cdot 2) = 10 - 6 - 2 = 2.$
Geometrically, it is the product of their magnitudes multiplied by the cosine of the angle $\theta$ between them, $\boxed{\vec{A} \cdot \vec{B} = \vert{}\vert{}\vec{A}\vert{}\vert{} \cdot \vert{}\vert{}\vec{B}\vert{}\vert{} \cos(\theta)}$
The projection of $\vec{a}$ onto $\vec{b}$ is $proj_{\vec{b}}\vec{a} = \frac{\vec{a}·\vec{b}}{||\vec{b}||^2}\vec{b}$.
The cross product, denoted by $\vec{A} \times \vec{B}$, is a binary operation on two vectors in three-dimensional space. It yields a new vector that is perpendicular (orthogonal, at a right angle) to both original vectors and has a magnitude equal to the area of the parallelogram formed by the two input vectors.
The cross product $\vec{A}x\vec{B}$ can be computed using the determinant of a $3 \times 3$ matrix formed by the standard basis vectors ($\hat{\mathbf{i}}, \hat{\mathbf{j}}, \hat{\mathbf{k}}$) and the components of the two vectors: $\vec{A} \times \vec{B} = det(\begin{smallmatrix} \hat{\mathbf{i}} & \hat{\mathbf{j}} & \hat{\mathbf{k}} \\\ a_1 & a_2 & a_3\\\ b_1 & b_2 & b_3\end{smallmatrix}) =|\begin{smallmatrix}a_2 & a_3\\\ b_2 & b_3\end{smallmatrix}|\vec{i}-|\begin{smallmatrix}a_1 & a_3\\\ b_1 & b_3\end{smallmatrix}|\vec{j}+|\begin{smallmatrix}a_1 & a_2\\\ b_1 & b_2\end{smallmatrix}|\vec{k}$.
Example: If $\vec{A} = \langle 1, 2, 3 \rangle$ and $\vec{B} = \langle 4, 5, 6 \rangle$: $\vec{A} \times \vec{B} = \begin{vmatrix} \hat{\mathbf{i}} & \hat{\mathbf{j}} & \hat{\mathbf{k}} \\ 1 & 2 & 3 \\ 4 & 5 & 6 \end{vmatrix} = (2\cdot 6 - 3\cdot 5)\hat{\mathbf{i}} - (1\cdot 6 - 3\cdot 4)\hat{\mathbf{j}} + (1\cdot 5 - 2\cdot 4)\hat{\mathbf{k}} = -3\hat{\mathbf{i}} + 6\hat{\mathbf{j}} - 3\hat{\mathbf{k}} = \langle -3, 6, -3 \rangle$
In various applications, the use of different coordinate systems becomes essential. For instance, disciplines like physics, engineering, and computer graphics often involve dealing with transformations between various coordinates systems, such as Cartesian, polar, cylindrical, and spherical coordinates. These transformations are typically defined by sets of equations, which are commonly linear in nature.
Consider the following system of linear equations: $\begin{cases} u_1 = 2x_1 + 3x_2 + 3x_3 \\ u_2 = 2x_1 + 4x_2 + 5x_3 \\ u_3 = x_1 + x_2 + 2x_3 \end{cases}$
Matrices offers an efficient way to solve systems of linear equations. By utilizing matrices, we can represent these equations more concisely and work with them computationally.
For the system above, we can express it in matrix form as: $\begin{pmatrix} 2 & 3 & 3 \\ 2 & 4 & 5 \\ 1 & 1 & 2 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \begin{pmatrix} u_1 \\ u_2 \\ u_3 \end{pmatrix}$
This notation, A · X = U is more convenient and concise where:
By performing dot products between the rows of the coefficient matrix $A$ and the column vector $X$, we arrive at the resulting vector $U$.
The concept of matrices is so powerful, that in many cases, we make our lives simpler by viewing or treating a vector $\vec{v} = (v_1, v_2, ···, v_n)$ as a special type of matrix. When we view $\vec{v} = (v_1, v_2, ···, v_n)$ as a matrix, we write it as an n x 1 column matrix: $\vec{v} = \biggl(\begin{smallmatrix}v_1\\\ v_2\\\ .\\\ .\\\ .\\\ v_n\end{smallmatrix}\biggr)$.
When dealing with two matrices $A$ and $B$, where $A$ has dimensions $m \times n$ and $B$ has dimensions $n \times p$, the resultant matrix $C = AB$ will have dimensions $m \times p$.
⚠️ Important Note: For matrix multiplication to be valid, the number of columns in the first matrix must exactly equal the number of rows in the second matrix.
The elements of matrix $C$ are computed by taking the dot products of the rows of $A$ and the columns of $B$. Specifically, the entry in the $i$-th row and $j$-th column of $C$ is computed by taking the dot product of the $i$-th row of $A$ with the $j$-th column of $B$: $\boxed{c_{ij} = \sum_{k=1}^n a_{ik}b_{kj}}$
To compute the product AB, we will proceed as follows: $AB = \begin{pmatrix} (1\cdot1 + 2\cdot3 + 3\cdot5) & (1\cdot2 + 2\cdot4 + 3\cdot6) \\ (4\cdot1 + 5\cdot3 + 6\cdot5) & (4\cdot2 + 5\cdot4 + 6\cdot6) \\ (7\cdot1 + 8\cdot3 + 9\cdot5) & (7\cdot2 + 8\cdot4 + 9\cdot6) \end{pmatrix}$
Performing the multiplication and addition gives us a $3 \times 2$ matrix: $AB = \begin{pmatrix} 1 + 6 + 15 & 2 + 8 + 18 \\ 4 + 15 + 30 & 8 + 20 + 36 \\ 7 + 24 + 45 & 14 + 32 + 54 \end{pmatrix} = \begin{pmatrix} 22 & 28 \\ 49 & 64 \\ 76 & 100 \end{pmatrix}$
AB = $\Bigl(\begin{smallmatrix}3 & 4 & 4\\\ 1 & 0 & 2\\\ 1 & 2 & 1\end{smallmatrix}\Bigr)\Bigl(\begin{smallmatrix}1 & 2\\\ 1 & 9\\\ 1 & 0\end{smallmatrix}\Bigr) = \Bigl(\begin{smallmatrix}3⋅1+4⋅1+4⋅1 & 3⋅2+4⋅9+4⋅0 \\\ 1⋅1+0⋅1+2⋅1 & 1⋅2+0⋅9+2⋅0 \\\ 1⋅1+2⋅1+1⋅1 & 1⋅2+2⋅9+1⋅0\end{smallmatrix}\Bigr)$ =[Performing the multiplications and additions:] $\Bigl(\begin{smallmatrix}3+4+4 & 6+36+0 \\\ 1+0+2 & 2+0+0 \\\ 1+2+1 & 2+18+0\end{smallmatrix}\Bigr) =\Bigl(\begin{smallmatrix}11 & 42\\\ 3 & 2\\\ 4 & 20\end{smallmatrix}\Bigr)$
AB = $\Bigl(\begin{smallmatrix}3 & 4 & 2\\\ 1 & 0 & 2\\\ 1 & 2 & 1\end{smallmatrix}\Bigr)\Bigl(\begin{smallmatrix}1 & 2 & 3\\\ 1 & 2 & 0\\\ 0 & 1 & 1\end{smallmatrix}\Bigr) = \Bigl(\begin{smallmatrix}7 & 16 & 11\\\ 1 & 4 & 5\\\ 3 & 7 & 4\end{smallmatrix}\Bigr)$
⚠️ For matrix multiplication to be valid, the number of columns in the first matrix must be equal to the number of rows in the second matrix. The resulting matrix, known as the matrix product, will have the number of rows of the first matrix and the number of columns of the second matrix.
For example, if matrix A is of size $m \times n$ and matrix B is of size $n \times p$, the product AB will be of size $m \times p$.
Matrices and their properties form the backbone of Linear Algebra and Calculus. They follow specific rules that sometimes mirror regular algebra, but occasionally differ in crucial ways (like the lack of commutativity in multiplication).
Any image or vector in a plane can be altered using different operations or transformations. There are three main types of transformations, which are reflections, rotations, and translations.
Rotation matrices are essential tools in two-dimensional Euclidean space to perform transformations that involve rotating points around the origin.
When we rotate a point in the plane, we are changing its position according to a specific angle around the origin (0, 0). The rotation can be counterclockwise or clockwise. By standard convention, counterclockwise rotations are positive angles, and clockwise rotations are negative.
Consider a point $Q$ with Cartesian coordinates $(x, y)$. In polar coordinates, this point is described by its distance from the origin $r$ and the initial angle $\gamma$ it makes with the positive x-axis: $x = r\cos(\gamma), y = r\sin(\gamma)$ (illustration i).

If we want to rotate this point counterclockwise by an angle $\theta$, the new coordinates $(x', y')$ can be found using the trigonometric angle addition identities: $\cos(A+B) = \cos(A)\cos(B) - \sin(A)\sin(B)$ and $\sin(A+B) = \sin(A)\cos(B) + \cos(A)\sin(B)$
Our new coordinates are based on the combined angle $(\theta + \gamma)$: $\begin{cases} x' = r\cos(\theta + \gamma) \\ y' = r\sin(\theta + \gamma) \end{cases}$
Applying these identities to our rotation problem: $x' = r(\cos(\theta)\cos(\gamma) - \sin(\theta)\sin(\gamma))$, $y' = r(\sin(\theta)\cos(\gamma) + \cos(\theta)\sin(\gamma))$
Since we know that $x = r\cos(\gamma)$ and $y = r\sin(\gamma)$, we can substitute $x$ and $y$ back into the equations:
$\begin{cases} x' = x\cos(\theta) - y\sin(\theta) \\ y' = x\sin(\theta) + y\cos(\theta) \end{cases}$
This beautifully translates into a standard matrix multiplication format:
$\begin{pmatrix} x' \\ y' \end{pmatrix} = \begin{pmatrix} \cos(\theta) & -\sin(\theta) \\ \sin(\theta) & \cos(\theta) \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}$ where the $2 \times 2$ matrix containing the trigonometric functions is the Rotation Matrix.
Consider rotating a point $(x, y)$ counterclockwise by 90° ($\theta = 90^\circ$) around the origin. We plug 90° into our standard rotation matrix: $R = \begin{pmatrix} \cos(90^\circ) & -\sin(90^\circ) \\ \sin(90^\circ) & \cos(90^\circ) \end{pmatrix} = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$
Applying this to a general point $(x, y)$: $\begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} -y \\ x \end{pmatrix}$
What does this look like in practice?
Let’s rotate the unit vector on the x-axis, $\vec{i} = (1, 0)$: $\begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 0 \\ 1 \end{pmatrix}$
This illustrates that the rotation matrix $R$ transforms the horizontal vector $\vec{i} = (1, 0)$ directly into the vertical vector $\vec{j} = (0, 1)$.
Analogously, rotating $\vec{j}$: $R\vec{j} = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 0 \\ 1 \end{pmatrix} = \begin{pmatrix} -1 \\ 0 \end{pmatrix} = -\vec{i}$
For example, rotating the point (1, 0): $(\begin{smallmatrix}0 & -1\\\ 1 & 0\end{smallmatrix})(\begin{smallmatrix}1\\\ 0\end{smallmatrix}) = (\begin{smallmatrix}0\\\ 1\end{smallmatrix})$. This illustrates that the rotation matrix R transform the vector $\vec{i} = (1, 0)$ into $\vec{j} = (0, 1)$.
Combining Rotations: When we multiply two rotation matrices, we get a matrix representing the combined rotation.
When we multiply two rotation matrices, we get a matrix representing the combined rotation. For instance, rotating by 90 degrees twice means squaring the matrix: $R^2 = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} = \begin{pmatrix} -1 & 0 \\ 0 & -1 \end{pmatrix} = -I_{2\times2}$
This is equivalent to a 180-degree rotation (flipping the plane), represented perfectly by the negative identity matrix.
To navigate linear algebra, you need to recognize several special types of matrices and their components.
Main Diagonal: In an $n \times n$ square matrix, this consists of the entries $a_{ii}$ where i ranges from 1 to n (the row number equals the column number). These elements lie on the imaginary line running from the top left corner to the bottom right corner, e.g., in the following $3 \times 3$ matrix, $\begin{pmatrix} \mathbf{a} & b & c \\ d & \mathbf{e} & f \\ g & h & \mathbf{i} \end{pmatrix}$, the main diagonal elements are $a, e,$ and $i$.
Trace: The trace of a square matrix, denoted as $\text{trace}(A)$, is simply the sum of the elements on its main diagonal. The trace is a linear transformation from square matrices to the real numbers $\mathbf{R}$, e.g., $trace\bigl(\begin{smallmatrix}a & b & c\\\ d & e & f\\\ g & h & i\end{smallmatrix}\bigr) = a + e + i.$
Properties:
Cyclic property: $\text{trace}(ABC) = \text{trace}(BCA) = \text{trace}(CAB)$..
Transpose property, $\text{trace}(A^T) = \text{trace}(A)$, that is, the trace of a matrix is equal to that of its transpose.
A diagonal matrix is a square matrix in which the entries outside the main diagonal are all zero, e.g., * A diagonal matrix is a square matrix in which the entries outside the main diagonal are all zero, e.g., $\bigl(\begin{smallmatrix}d_1 & 0 & 0\\\ 0 & d_2 & 0\\\ 0 & 0 & d_3\end{smallmatrix}\bigr)$.
A lower triangular matrix is a square matrix in which all the entries above the main diagonal are zero, e.g., $\bigl(\begin{smallmatrix}l_{11} & 0 & 0 \\ l_{21} & l_{22} & 0 \\ l_{31} & l_{32} & l_{33}\end{smallmatrix}\bigr)$.
Similarly, an upper triangular matrix is a square matrix in which all the entries below the main diagonal are zero, e.g., $\bigl(\begin{smallmatrix}u_{11} & u_{12} & u_{13} \\ 0 & u_{22} & u_{23} \\ 0 & 0 & u_{33}\end{smallmatrix}\bigr)$.
A symmetric matrix is a square matrix that is equal to its transpose. In other words,$A^T = A$.
It mirrors itself across the main diagonal, e.g., $\bigl(\begin{smallmatrix}s_{11} & s_{12} & s_{13} \\ s_{12} & s_{22} & s_{23} \\\ s_{13} & s_{23} & s_{33}\end{smallmatrix}\bigr)$.
The concept of the inverse matrix is crucial for solving linear systems algebraically. When we talk about an inverse matrix, we are referring to a special kind of matrix that, when multiplied by the original matrix, yields the identity matrix.
Given a square matrix A (a matrix with an equal number of rows and columns), the inverse of $A$, denoted as $A^{-1}$, is a special matrix that satisfies this equation: $A \times A^{-1} = A^{-1} \times A = I$ where I is the identity matrix, which is a matrix with 1’s on the diagonal and 0’s elsewhere.
Essentially, multiplying a matrix by its inverse reverses (“undoes”) the effect of the original matrix.
⚠️ Existence Rule: The inverse $A^{-1}$ exists if and only if $A$ is non-singular. A matrix is non-singular if its determinant is non-zero ($\det(A) \neq 0$).
Let’s find the inverse of matrix $A$: $A = \begin{pmatrix} 2 & 3 & 3 \\ 2 & 4 & 5 \\ 1 & 1 & 2 \end{pmatrix}$
Calculate the Cofactor Matrix $C$.
Each element in $C$ (i-th row and j-th column of C) is found using the formula $C_{ij} = (-1)^{i+j} \times \text{minor}(A_{ij})$.
The minor is the determinant of the $2\times2$ submatrix obtained by deleting or removing the i-th row and j-th column from A.
The signs $(-1)^{i+j}$ in the cofactor matrix follow a checkerboard pattern: $\begin{pmatrix} + & - & + \\ - & + & - \\ + & - & + \end{pmatrix}$.
Let’s calculate two example minors:
$\text{minor}(A_{11})$ (top left): Cross out row 1, col 1. $\det\begin{pmatrix} 4 & 5 \\ 1 & 2 \end{pmatrix} = (4\cdot2) - (5\cdot1) = 3$. Because position $(1,1)$ is “+”, $C_{11} = +3$.
$\text{minor}(A_{12})$ (top middle): Cross out row 1, col 2. $\det\begin{pmatrix} 2 & 5 \\ 1 & 2 \end{pmatrix} = (2\cdot2) - (5\cdot1) = -1$. Because position $(1,2)$ is “-”, $C_{12} = -(-1) = 1$.
Calculating all 9 gives us the cofactor matrix: $C = \begin{pmatrix} 3 & 1 & -2 \\ -3 & 1 & 1 \\ 3 & -4 & 2 \end{pmatrix}$
Transpose the Cofactor matrix C to obtain the Adjugate matrix, $\text{adj}(A).$ Informally, to transpose a matrix is to create a new matrix by swapping its rows and columns (flip the original matrix about its main diagonal).
$\text{adj}(A) = C^T = \begin{pmatrix} 3 & -3 & 3 \\ 1 & 1 & -4 \\ -2 & 1 & 2 \end{pmatrix}$
Calculate the Determinant det(A).
In linear algebra, the determinant is a crucial concept that provides valuable insights into the properties of a square matrix. One of the key properties of determinants is that the determinant of the product of two matrices is equal to the product of their determinants. Mathematically, this is expressed as: $\vert A \cdot B \vert = \vert A \vert \cdot \vert B \vert$, where $\vert A \vert = det(A)$.
For a 3 x 3 matrix A, the determinant can be calculated using the formula: $det(\begin{pmatrix}a & b & c\\\ d & e & f\\\ g & h & i\end{pmatrix}) = a(ei - fh) - b(di - fg) + c(dh - eg).$
Expanding along the top row of our original matrix A: $\text{det}(A) = 2(4 \times 2 - 5 \times 1) - 3(2 \times 2 - 5 \times 1) + 3(2 \times 1 - 4 \times 1) = 2(8 - 5) - 3(4 - 5) + 3(2 - 4) = 2(3) - 3(-1) + 3(-2) = 6 + 3 - 6 = 3.$
$A^{-1} = \frac{1}{\text{det}(A)} \times \text{adj}(A) = \frac{1}{3}(\begin{smallmatrix}3 & -3 & 3\\\ 1 & 1 & -4\\\ -2 & 1 & 2\end{smallmatrix})$
The calculation is quite easy because A is not singular (det(A) = 0), so the inverse $A^{-1}$ does not exist.
$det(\begin{smallmatrix}1 & 2 & 3\\\ 1 & 2 & 3\\\ 0 & 2 & 1\end{smallmatrix})$ =[$det(\begin{smallmatrix}a & b & c\\\ d & e & f\\\ g & h & i\end{smallmatrix})$ = a(ei - fh) - b(di - fg) + c(dh - eg).] 1·(2·1-3·2) -2(1·1-3·0) +3(1·2-0·2) = -4-2+6 = 0.