To err is human, to blame it on someone else is even more human, Jacob’s Law
Irony is wasted on the stupid, Oscar Wilde
Definition. A vector $\vec{AB}$ is a geometric object or quantity that has both magnitude (or length) and direction. Vectors in an n-dimensional Euclidean space can be represented as coordinates vectors in a Cartesian coordinate system.
Definition. The magnitude of the vector $\vec{A}$, also known as length or norm, is given by the square root of the sum of its components squared, $|\vec{A}|~ or~ ||\vec{A}|| = \sqrt{a_1^2+a_2^2+a_3^2}$, e.g., $||< 3, 2, 1 >|| = \sqrt{3^2+2^2+1^2}=\sqrt{14}$, $||< 3, -4, 5 >|| = \sqrt{3^2+(-4)^2+5^2}=\sqrt{50}=5\sqrt{2}$, or $||< 1, 0, 0 >|| = \sqrt{1^2+0^2+0^2}=\sqrt{1}=1$.
The sum of two vectors is the sum of their components, $\vec{A}+ \vec{B} = (a_1+b_1)\vec{i}+(a_2+b_2)\vec{j}+(a_3+b_3)\vec{k}$ = < (a1+b1), (a2+b2), (a3+b3) >.
The subtraction of two vectors is similar to addition and is also done component-wise, it is given by simply subtracting their corresponding components (x, y, and z in 3D), $\vec{A} - \vec{B} = (a_1-b_1)\vec{i}+(a_2-b_2)\vec{j}+(a_3-b_3)\vec{k}$ = < (a1-b1), (a2-b2), (a3-b3) >.
Scalar multiplication is the multiplication of a vector by a scalar, a real number, changing its magnitude without altering its direction. It is effectively multiplying each component of the vector by the scalar value, $c\vec{A} = (ca_1)\vec{i}+(ca_2)\vec{j}+(ca_3)\vec{k} = < ca_1, ca_2, ca_3>$.
The dot or scalar product is a fundamental operation between two vectors. It produces a scalar quantity that represents the projection of one vector onto another. The dot product is the sum of the products of their corresponding components: $\vec{A}·\vec{B} = \sum a_ib_i = a_1b_1 + a_2b_2 + a_3b_3.$, e.g. $\vec{A}·\vec{B} = \sum a_ib_i = ⟨2, 2, -1⟩·⟨5, -3, 2⟩ = a_1b_1 + a_2b_2 + a_3b_3 = 2·5+2·(-3)+(-1)·2 = 10-6-2 = 2.$
It is the product of their magnitudes multiplied by the cosine of the angle between them, $\vec{A}·\vec{B}=||\vec{A}||·||\vec{B}||·cos(θ).$
The cross product, denoted by $\vec{A}x\vec{B}$, is a binary operation on two vectors in three-dimensional space. It results in a vector that is perpendicular to both of the input vectors, has a magnitude equal to the area of the parallelogram formed by the two input vectors, and has a direction determined by the right-hand rule.
The cross product $\vec{A}x\vec{B}$ can be computed using the following formula, $\vec{A}x\vec{B} = det(\begin{smallmatrix}i & j & k\\ a_1 & a_2 & a_3\\ b_1 & b_2 & b_3\end{smallmatrix}) =|\begin{smallmatrix}a_2 & a_3\\ b_2 & b_3\end{smallmatrix}|\vec{i}-|\begin{smallmatrix}a_1 & a_3\\ b_1 & b_3\end{smallmatrix}|\vec{j}+|\begin{smallmatrix}a_1 & a_2\\ b_1 & b_2\end{smallmatrix}|\vec{k}$
In various applications, the use of different coordinate systems becomes essential. For instance, disciplines like physics, engineering, and computer graphics often involve dealing with transformations between various coordinates systems, such as Cartesian, polar, cylindrical, and spherical coordinates. These transformations are typically defined by sets of equations, commonly linear in nature.
One example of such equations can be represented as:
$\begin{cases} u_1 = 2x_1 + 3x_2 +3x_3 \\ u_2 = 2x_1 + 4x_2 +5x_3 \\ u_3 = x_1 +x_2 + 2x_3 \end{cases}$
Matrices offers an efficient way for solving systems of linear equations. Utilizing matrices, we can represent these equations more concisely and conveniently.
For the system above, we can express it in matrix form as:
$\Bigl (\begin{smallmatrix}2 & 3 & 3\\ 2 & 4 & 4\\ 1 & 1 & 2\end{smallmatrix} \Bigr) \Bigl(\begin{smallmatrix}x_1\\ x_2\\x_3\end{smallmatrix} \Bigr) = \Bigl (\begin{smallmatrix}u_1\\ u_2\\u_3\end{smallmatrix} \Bigr )$
This notation, A · X = U is more convenient and concise where A = $\Bigl(\begin{smallmatrix}2 & 3 & 3\\ 2 & 4 & 4\\ 1 & 1 & 2\end{smallmatrix}\Bigr)$ is a 3 x 3 matrix representing the coefficients, X = $\Bigl(\begin{smallmatrix}x_1\\ x_2\\x_3\end{smallmatrix}\Bigr)$ is a 3 x 1 column vector representing the variables, and U = $\Bigl (\begin{smallmatrix}u_1\\ u_2\\u_3\end{smallmatrix} \Bigr )$ is a 3 x 1 column vector representing the results. By performing dot products between the rows of the coefficient matrix A and the column vector X, we can arrive at the resulting vector U.
The concept of matrices is so powerful, that in many cases, we make our lives simpler by viewing a vector $\vec{v} = (v_1, v_2, ···, v_n)$ as a special type of matrix. When we view vectors as matrices, we actually view them as a rotated version of the standard form. When we view $\vec{v} = (v_1, v_2, ···, v_n)$ as a matrix, we write it as an n x 1 column matrix: $\vec{v} = \biggl(\begin{smallmatrix}v_1\\ v_2\\ .\\ .\\ .\\ v_n\end{smallmatrix}\biggr)$.
When dealing with two matrices A and B, where A has dimensions m × n and B has dimensions n × p, the resultant matrix C = A⋅B will have dimensions m × p. The elements of matrix C are computed by taking dot products of the rows of A and the columns of B. Specifically, the entry in the ith row and jth column of C is computed by taking the dot product of the ith row of A with the jth column of B, i.e., $c_{ij} = \sum_{k=1}^n a_{ik}b_{kj}$
To compute the product AB, we will proceed as follows:
AB = $\Bigl(\begin{smallmatrix}1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9\end{smallmatrix}\Bigr)\Bigl(\begin{smallmatrix}1 & 2 \\ 3 & 4 \\ 5 & 6\end{smallmatrix}\Bigr) =\Bigl(\begin{smallmatrix}1\cdot1 + 2\cdot3 + 3\cdot5 & 1\cdot2 + 2\cdot4 + 3\cdot6 \\ 4\cdot1 + 5\cdot3 + 6\cdot5 & 4\cdot2 + 5\cdot4 + 6\cdot6 \\ 7\cdot1 + 8\cdot3 + 9\cdot5 & 7\cdot2 + 8\cdot4 + 9\cdot6\end{smallmatrix}\Bigr)$ =[Performing the multiplications and additions:] $\Bigl(\begin{smallmatrix}1 + 6 + 15 & 2 + 8 + 18 \\ 4 + 15 + 30 & 8 + 20 + 36 \\ 7 + 24 + 45 & 14 + 32 + 54\end{smallmatrix}\Bigr) = \Bigl(\begin{smallmatrix}22 & 28 \\ 49 & 64 \\ 76 & 100 \end{smallmatrix}\Bigr)$
AB = $\Bigl(\begin{smallmatrix}3 & 4 & 4\\ 1 & 0 & 2\\ 1 & 2 & 1\end{smallmatrix}\Bigr)\Bigl(\begin{smallmatrix}1 & 2\\ 1 & 9\\ 1 & 0\end{smallmatrix}\Bigr) = \Bigl(\begin{smallmatrix}3⋅1+4⋅1+4⋅1 & 3⋅2+4⋅9+4⋅0 \\ 1⋅1+0⋅1+2⋅1 & 1⋅2+0⋅9+2⋅0 \\ 1⋅1+2⋅1+1⋅1 & 1⋅2+2⋅9+1⋅0\end{smallmatrix}\Bigr)$ =[Performing the multiplications and additions:] $\Bigl(\begin{smallmatrix}3+4+4 & 6+36+0 \\ 1+0+2 & 2+0+0 \\ 1+2+1 & 2+18+0\end{smallmatrix}\Bigr) =\Bigl(\begin{smallmatrix}11 & 42\\ 3 & 2\\ 4 & 20\end{smallmatrix}\Bigr)$
AB = $\Bigl(\begin{smallmatrix}3 & 4 & 4\\ 1 & 0 & 2\\ 1 & 2 & 1\end{smallmatrix}\Bigr)\Bigl(\begin{smallmatrix}1 & 2\\ 1 & 9\\ 1 & 0\end{smallmatrix}\Bigr) = \Bigl(\begin{smallmatrix}11 & 42\\ 3 & 2\\ 4 & 20\end{smallmatrix}\Bigr)$
AB = $\Bigl(\begin{smallmatrix}3 & 4 & 2\\ 1 & 0 & 2\\ 1 & 2 & 1\end{smallmatrix}\Bigr)\Bigl(\begin{smallmatrix}1 & 2 & 3\\ 1 & 2 & 0\\0 & 1 & 1\end{smallmatrix}\Bigr) = \Bigl(\begin{smallmatrix}7 & 16 & 11\\ 1 & 4 & 5\\ 3 & 7 & 4\end{smallmatrix}\Bigr)$
⚠️ For matrix multiplication to be valid, the number of columns in the first matrix must be equal to the number of rows in the second matrix. The resulting matrix, known as the matrix product, will have the number of rows of the first matrix and the number of columns of the second matrix.
For example, if matrix A is of size m×n and matrix B is of size n×p, the product AB will be of size m×p.
One important property of matrix multiplication is that it is associative. This means that for matrices A, B, and C (of compatible sizes): (AB)X = A(BX).
This is the transformation of a vector $\vec{X}$ by first applying matrix or transformation B, and then matrix or transformation A.
This means that you can first apply matrix B to vector $\vec{X}$, and then apply matrix A to the result, yielding the same final outcome as applying the combined transformation AB directly to $\vec{X}$. ⃗
Matrices and their properties form the backbone of Linear Algebra and Calculus.
Matrices have several important properties that are fundamental in linear algebra:
Any image in a plane could be altered by using different operations or transformations. There are three main types of transformations, which are reflections, rotations, and translations.
Rotation matrices are essential tools in two-dimensional Euclidean space to perform transformations that involve rotating points around the origin.
When we rotate a point in the plane, we are changing its position according to a specific angle around the origin (0, 0). The rotation can be counterclockwise or clockwise, but we will focus on counterclockwise rotations, which are positive angles in the standard convention.
Consider a point Q with coordinates (x, y). In polar coordinates, this point can be described by its distance from the origin r and the angle γ it makes with the positive x-axis: x = r·cos(γ), y = r·sin(γ) (illustration i).
If we want to rotate this point counterclockwise by an angle θ, the new coordinates (x′, y′) can be found using trigonometric identities, namely the sum and difference identities for cosine and sine, cos(A+B) = cos(A)cos(B) - sin(A)sin(B) (i), sin(A+B) = sin(A) cos(B) + cos(A)sin(B) (ii).
$\begin{cases} x’ = rcos(θ + γ) \\ y’= rsin(θ + γ) \end{cases}$
Applying these identities to our rotation problem:
x′ = rcos(θ+γ) =[(i)] r(cos(θ)cos(γ) − sin(θ)sin(γ)), y′=[(ii)] rsin(θ+γ)=r(sin(θ)cos(γ) + cos(θ)sin(γ))
Since x = r·cos(γ), y = r·sin(γ), we can substitute:
$\begin{cases} x’ = xcos(θ) -ysin(θ) \\ y’= xsin(θ) +ycos(θ) \end{cases}$
This can be written or expressed in matrix form as:
$(\begin{smallmatrix}x’\\ y’\end{smallmatrix})=(\begin{smallmatrix}cos(θ) & -sin(θ)\\ sin(θ) & cos(θ)\end{smallmatrix})(\begin{smallmatrix}x\\ y\end{smallmatrix})$
Consider rotating a point (x, y) counterclockwise by 90° (θ = 90°) around the origin. The rotation matrix for 90 degrees becomes:
$(\begin{smallmatrix}cos(90) & -sin(90)\\ sin(90) & cos(90)\end{smallmatrix})= (\begin{smallmatrix}0 & -1\\ 1 & 0\end{smallmatrix})$
Applying this to a point (x, y):
$(\begin{smallmatrix}0 & -1\\ 1 & 0\end{smallmatrix})(\begin{smallmatrix}x\\ y\end{smallmatrix}) = (\begin{smallmatrix}-y\\ x\end{smallmatrix})$.
For example, rotating the point (1, 0):
$(\begin{smallmatrix}0 & -1\\ 1 & 0\end{smallmatrix})(\begin{smallmatrix}1\\ 0\end{smallmatrix}) = (\begin{smallmatrix}0\\ 1\end{smallmatrix})$. This illustrates that the rotation matrix R transform the vector $\vec{i} = (1, 0)$ into $\vec{j} = (0, 1)$.
Analogously, $R\vec{j} = (\begin{smallmatrix}0 & -1\\ 1 & 0\end{smallmatrix})(\begin{smallmatrix}0\\ 1\end{smallmatrix}) = (\begin{smallmatrix}-1\\ 0\end{smallmatrix}) = -\vec{i}$
When we multiply two rotation matrices, we get a matrix representing the combined rotation. For instance, rotating by 90 degrees twice:
$R^2=(\begin{smallmatrix}0 & -1\\ 1 & 0\end{smallmatrix})(\begin{smallmatrix}0 & -1\\ 1 & 0\end{smallmatrix})=(\begin{smallmatrix}-1 & 0\\ 0 & -1\end{smallmatrix})=-I_{2x2}$. This is equivalent to a 180-degree rotation, or flipping the plane, represented by $-I_{2x2}$ where $I_{2x2}$ is the identity matrix.
The main diagonal of a square matrix consists of the entries aii where i ranges from 1 to n (for an n x n square matrix).These elements lie on the imaginary line running from the top left corner to the bottom right corner of the matrix, e.g. in the following 3 x 3 matrix, $\bigl(\begin{smallmatrix}a & b & c\\ d & e & f\\ g & h & i\end{smallmatrix}\bigr)$, the main diagonal elements are a, e, and i.
The trace of a square matrix, denoted as trace(a), is the sum of the elements on its main diagonal. The trace is a linear transformation from square matrices to the real, e.g., $trace\bigl(\begin{smallmatrix}a & b & c\\ d & e & f\\ g & h & i\end{smallmatrix}\bigr) = a + e + i.$ Properties:
A diagonal matrix is a square matrix in which the entries outside the main diagonal are all zero, e.g., $\bigl(\begin{smallmatrix}d_1 & 0 & 0\\ 0 & d_2 & 0\\ 0 & 0 & d_3\end{smallmatrix}\bigr)$.
A lower triangular matrix is a square matrix in which all the entries above the main diagonal are zero, e.g., $\bigl(\begin{smallmatrix}l_{11} & 0 & 0 \\l_{21} & l_{22} & 0 \\l_{31} & l_{32} & l_{33}\end{smallmatrix}\bigr)$.
Similarly, an upper triangular matrix is a square matrix in which all the entries below the main diagonal are zero, e.g., $\bigl(\begin{smallmatrix}u_{11} & u_{12} & u_{13} \\0 & u_{22} & u_{23} \\0 & 0 & u_{33}\end{smallmatrix}\bigr)$.
A symmetric matrix is a square matrix that is equal to its transpose. In other words,AT = A, e.g., $\bigl(\begin{smallmatrix}s_{11} & s_{12} & s_{13} \\s_{12} & s_{22} & s_{23} \\s_{13} & s_{23} & s_{33}\end{smallmatrix}\bigr)$.
The concept of the inverse matrix is crucial in linear algebra. When we talk about an inverse matrix, we are referring to a special kind of matrix that, when multiplied by the original matrix, yields the identity matrix.
Given a square matrix A (a matrix with an equal number of rows and columns), the inverse of A, denoted as A-1, exists if and only if A is non-singular. A matrix is non-singular if its determinant is non-zero (det(A) ≠ 0).
The property of an inverse matrix is: $A \times A^{-1} = A^{-1} \times A = I$ where I is the identity matrix, which is a matrix with 1’s on the diagonal and 0’s elsewhere. Essentially, multiplying a matrix by its inverse reverses (“undoes”) the effect of the original matrix.
Consider the matrix A: $A = (\begin{smallmatrix}2 & 3 & 3\\ 2 & 4 & 5\\ 1 & 1 & 2\end{smallmatrix}).$
To calculate the inverse, follow these steps:
$\text{minors}=(\begin{smallmatrix}3 & -1 & -2\\ 3 & 1 & -1\\ 3 & 4 & 2\end{smallmatrix}),$ e.g., $\text{minor}(A_{00}) = det(\begin{smallmatrix}4 & 5\\1 & 2\end{smallmatrix}) = 8 - 5 = 3.~ \text{minor}(A_{01}) = det(\begin{smallmatrix}2 & 5\\1 & 2\end{smallmatrix}) = -1$
So, the cofactor matrix C is:
$C = (\begin{smallmatrix}3 & 1 & -2\\ -3 & 1 & 1\\ 3 & -4 & 2\end{smallmatrix})$.
The signs in the cofactor matrix follow a “checkerboard pattern". Namely, (-1)i+j is pictured in this matrix:
$(\begin{smallmatrix}+ & - & +\\ - & + & -\\ + & - & +\end{smallmatrix})$
$\text{adj}(A) = (\begin{smallmatrix}3 & -3 & 3\\ 1 & 1 & -4\\ -2 & 1 & 2\end{smallmatrix})$
In linear algebra, the determinant is a crucial concept that provides valuable insights into the properties of a square matrix. One of the key properties of determinants is that the determinant of the product of two matrices is equal to the product of their determinants. Mathematically, this is expressed as: |A·B| = |A| · |B|, where |A| = det(A).
For a 3 x 3 matrix A, the determinant can be calculated using the formula: $det(\begin{smallmatrix}a & b & c\\ d & e & f\\ g & h & i\end{smallmatrix}) = a(ei - fh) - b(di - fg) + c(dh - eg).$
$\text{det}(A) = 2(4 \times 2 - 5 \times 1) - 3(2 \times 2 - 5 \times 1) + 3(2 \times 1 - 4 \times 1) = 2(8 - 5) - 3(4 - 5) + 3(2 - 4) = 2(3) - 3(-1) + 3(-2) = 6 + 3 - 6 = 3.$
$A^{-1} = \frac{1}{\text{det}(A)} \times \text{adj}(A) = \frac{1}{3}(\begin{smallmatrix}3 & -3 & 3\\ 1 & 1 & -4\\ -2 & 1 & 2\end{smallmatrix})$
The calculation is quite easy because A is not singular (det(A)=0), so the inverse A-1 does not exist.
$det(\begin{smallmatrix}1 & 2 & 3\\ 1 & 2 & 3\\ 0 & 2 & 1\end{smallmatrix})$ =[$det(\begin{smallmatrix}a & b & c\\ d & e & f\\ g & h & i\end{smallmatrix})$ = a(ei - fh) - b(di - fg) + c(dh - eg).] 1·(2·1-3·2) -2(1·1-3·0) +3(1·2-0·2) = -4-2+6 = 0.
Calculate the cofactor matrix C of A. The element in C are determined by the cofactor formula, where each element (the ith row and jth column of C) is the product $C_{ij} = (-1)^{i+j} \times \text{minor}(A_{ij})$.
C = $(\begin{smallmatrix}0 & 3 & -2\\ 0 & -2 &2\\2 & -1 & 0\end{smallmatrix})$
Transpose the cofactor matrix C to obtain the adjugate matrix, adj(A) = CT = $(\begin{smallmatrix}0 & 0 & 2\\ 3 & -2 & -1\\ -2 & 2 & 0\end{smallmatrix})$
Calculate the Determinant, det(A) = a(ei - fh) - b(di - fg) + c(dh - eg) = 1(2⋅0−3⋅0) −2(1⋅0−3⋅1) +2(1⋅0−2⋅1) = 1⋅0 −2⋅(−3) +2⋅(−2) = 0 +6 −4 = 2.
Finally, calculate the inverse matrix $A^{-1}$ using the formula: $A^{-1} = \frac{1}{\text{det}(A)} \times \text{adj}(A)$ where $A^{-1} = \frac{1}{\text{det}(A)} \times \text{adj}(A) = (\begin{smallmatrix}0 & 0 & 1\\ \frac{3}{2} & -1 & \frac{-1}{2}\\ -1 & 1 & 0\end{smallmatrix})$