Matrix Exponentials

Recall

The most terrifying words in the English language are: I’m from the government and I’m here to help, Ronald Reagan

Forget the politicians. The politicians are put there to give you the idea you have freedom of choice. You don’t. You have no choice. You have owners. They own you. They own everything. They own all the important land, they own and control the corporations that’ve long since bought and paid for, the senate, the congress, the state houses, the city halls, they got the judges in their back pocket, and they own all the big media companies so they control just about all of the news and the information you get to hear. They got you by the balls. They spend billions of dollars every year lobbying to get what they want. Well, we know what they want. They want more for themselves and less for everybody else. But I’ll tell you what they don’t want. They don’t want a population of citizens capable of critical thinking. They don’t want well informed, well educated people capable of critical thinking. They’re not interested in that. That doesn’t help them, George Carlin

Differential equations

Definition. A differential equation is an equation that involves one or more dependent variables, their derivatives with respect to one or more independent variables, and the independent variables themselves, e.g., $\frac{dy}{dx} = 3x +5y, y’ + y = 4xcos(2x), \frac{dy}{dx} = x^2y+y, etc.$

It involves (e.g., $\frac{dy}{dx} = 3x +5y$):

• Dependent variables: Variables that depend on one or more other variables (y).
• Independent variables: Variables upon which the dependent variables depend (x).
• Derivatives: Rates at which the dependent variables change with respect to the independent variables, $\frac{dy}{dx}$

First-Order Linear Ordinary Differential Equations (ODEs)

Definition. A first-order linear ordinary differential equation is an ordinary differential equation (ODE) involving an unknown function y(x), its first derivative y′, and functions of the independent variable x, which can be written in the general form:: a(x)y' + b(x)y = c(x) where:

• y = y(x) is the unknown function of the independent variable x.
• y′ = $\frac{dy}{dx}$ is the derivative of y with respect to x.
• a(x), b(x), and c(x) are known functions of x.

These equations are termed “linear” because the unknown function y and its derivative y’ appear to the first power and are not multiplied together or composed in any nonlinear way.

If the function c(x)=0 for all x in the interval of interest, the equation simplifies to: a(x)y’ + b(x)y = 0. Such an equation is called a homogeneous linear differential equation.

The Existence and Uniqueness Theorem provides crucial insight into the behavior of solutions to first-order differential equations ODEs. It states that if:

• The function f(x, y) (the right-hand side of the ODE) in the equation y’ = f(x, y) is continuous in some neighborhood around a point (x₀, y₀) and
• Its partial derivative with respect to y, denoted as $\frac{∂f}{∂y}$, is also continuous near (x₀, y₀).

Then, the differential equation y' = f(x, y) has a unique solution to the initial value problem through the point (x₀, y₀), meaning that it satisfies the initial condition y(x₀) = y₀.

This theorem ensures that under these conditions, the solution exists and is unique near x = x₀.

Matrix Exponentials; Application to Solving Systems.

In advanced calculus and differential equations, solving systems of linear differential equations is a fundamental topic. Specifically, we focus on systems of the form: $\vec{x’}(t) = A\vec{x}(t)$ where:

• A is a constant 2 x 2 matrix.
• $\vec{x}(t)$ is a vector of unknown functions, $\vec{x}(t) = \frac{x(t)}{y(t)}$.
• $\vec{x’}(t)$ denotes the derivaive of $\vec{x}(t)$ with respect to t.

Our goal is to find the general solution to this system using matrix exponentials. This method provides a powerful and efficient way to solve linear systems of differential equations.

Fundamental Matrix Solution

A fundamental matrix, X(t), for the system $\vec{x'} = A\vec{x}$, is a matrix whose columns are linearly independent solutions to the system. In other words, if $\vec{x_1}$ and $\vec{x_2}$ are two linearly independent solutions, then we can construct the fundamental matrix as:

X(t) = $(\begin{smallmatrix}\vec{x_1}(t) : \vec{x_2}(t)\end{smallmatrix}) = (\begin{smallmatrix}x_1(t) & x_2(t)\\ y_1(t) & y_2(t)\end{smallmatrix})$ where the first colum is the vector $\vec{x_1}(t)$ and the second column is the vector $\vec{x_2}(t)$.

There is not a unique fundamental matrix because its columns can be formed by any two linearly independent solutions, and there can be many such pairs.

Properties of the Fundamental Matrix

1. Non-zero determinant. A key property of a fundamental matrix X(t) is that its determinant is non-zero for all t: det(X(t)) ≠ 0. This ensures that the columns of X(t) are linearly independent.
2. The matrix X(t) satisfies the original differential equation: X’(t) = AX(t) where X’(t) is the derivative of X(t) with respect to t. This is because each column of X(t) is a solution to $\vec{x’} = A\vec{x}$.

General Solution

Once the fundamental matrix X(t) is found, we can express the general solution to the system $\vec{x’} = A\vec{x}$ as:

$\vec{x}(t) = c_1\vec{x_1}(t)+ c_2\vec{x_2}(t) ↭ \vec{x} = X(t)\vec{c} = [\vec{x_1}\vec{x_2}](\begin{smallmatrix}c_1\\ c_2\end{smallmatrix})$ where $\vec{c} = (\begin{smallmatrix}c_1\\ c_2\end{smallmatrix})$ is a vector of constants determined by initial conditions.

The general solution can also be written in matrix form: $\vec{x}(t) = X(t)\vec{c}$ where C = $(\begin{smallmatrix}\vec{c_1}\\ \vec{c_2}\end{smallmatrix})$

This expression emphasizes that any solution $\vec{x}(t)$ is a linear combination of the fundamental solutions.

There isn’t a unique fundamental matrix; any matrix formed by two independent solutions can serve as a fundamental matrix.

Futhermore, if C is any invertible 2 x 2 matrix (det(C) ≠ 0), then multiplying it by the fundamental matrix X(t): $\vec{x}(t) = X(t)C$ also represents a fundamental matrix solution. This shows that the general solution can be expressed with different constants or scaling factors, but the essential structure remains the same.

Matrix Exponential

The matrix exponential is a powerful tool for solving systems of linear differential equations. For a system of the form: $\vec{x’} = A\vec{x}$ the solution can be expressed using the matrix exponential e^At.

To understand how the matrix exponential works, let’s first consider the simpler case of a scalar differential equation: x’ = ax where a is a constant. The solution is x(t) = ce^at.

The exponential function e^at can be expressed as its power series expansion: $e^{at} = 1 + at + a^2\frac{t^2}{2!}+ a^3\frac{t^3}{3!}+ ···$

This series converges, and differentiating it (term by term) gives the derivative:

$\frac{de^{at}}{dt} = 0 + 1·a + a^2t + a^3\frac{t^2}{2} + ··· =[\text{Factoring a}] a(1 +at + a^2\frac{t^2}{2} + ···) = ae^{at}$. Thus, e^at solve the equation x’ = ax.

For a system of the form: $\vec{x’} = A\vec{x}$, the solution can be expressed using the matrix exponential of A. The matrix exponential, denoted e^At, is defined as: $e^{At} := I_2 + At + A^2\frac{t^2}{2!} + A^3\frac{t^3}{3!} + ···$ where I₂ is the identity matrix, and Aⁿ represents the matrix A to the n-th power.

The matrix exponential e^At provides the fundamental matrix for the system $\vec{x’} = A\vec{x}$. The general solution to the system is given by $\vec{x}(t)=e^{At}\vec{c}$ where $\vec{c}$ is a constant vector determined by the initial conditions of the system.

For a system $\vec{x’} = A\vec{x}$, the matrix exponential e^At plays the same role as the scalar exponential e^at for the scalar case. Just as e^at solve x’ = ax, the matrix exponential solves: $\frac{d}{dt}e^{At}=Ae^{At}$

First, note that the derivative of e^At can be computed by differentiating its series term-by-term:

$\frac{d}{dt}e^{At} =[e^{At} \text{ is defined as }] \frac{d}{dt}(I_2 + At + A^2\frac{t^2}{2!} + A^3\frac{t^3}{3!} + A^4\frac{t^4}{4!} + ···) = A + A^2t + \frac{A^3t^2}{2!} + \frac{A^4t^4}{3!} ··· =[\text{Factoring A}] A(I + At + \frac{A^2t^2}{2!} + \frac{A^3t^3}{3!}+···) = Ae^{At}$

Thus, e^At satisfies X' = AX, proving that it is a fundamental matrix for the system. The general solution to the system $\vec{x'} = A\vec{x}$ can be expressed as $\vec{x}(t) = e^{At}\vec{c}$ where $\vec{c}$ is a vector of constants determined by initial conditions.

Initial condition. At t = 0, the matrix exponential reduces to the identity matrix: e^A·0 = I. Thus, det(e^At) = 1 ≠ 0, ensuring that e^At is non-singular (invertible) and can serve as a valid fundamental matrix.

Example: Solving the System using Matrix Exponentials

Consider the system of differential equations: $\begin{cases} x’ = y \\ y’ = x \end{cases}$

We can rewrite this system in matrix form as: $\vec{x’} = A\vec{x} = (\begin{smallmatrix}0 & 1\\ 1 & 0\end{smallmatrix})(\begin{smallmatrix}x\\ y\end{smallmatrix})$ where $A = (\begin{smallmatrix}0 & 1\\ 1 & 0\end{smallmatrix})$ and $\vec{x} = (\begin{smallmatrix}x\\ y\end{smallmatrix})$

To solve the system, we will compute the matrix exponential e^At, which is defined by the power series: $e^{At} = I_2 + At + A^2\frac{t^2}{2!} + A^3\frac{t^3}{3!} + ··· $ where I₂ is the 2×2 identity matrix.

Computing Powers of A: $A^2 = (\begin{smallmatrix}0 & 1\\ 1 & 0\end{smallmatrix})(\begin{smallmatrix}0 & 1\\ 1 & 0\end{smallmatrix}) = (\begin{smallmatrix}1 & 0\\ 0 & 1\end{smallmatrix}) = I_2, A^3 = A^2A = I_2A = A, A^4 = A^3A = AA = I_2$

Substituting the computed powers into the series:

$e^{At} = I_2 + At + A^2\frac{t^2}{2!} + A^3\frac{t^3}{3!} + ··· = (\begin{smallmatrix}1 & 0\\ 0 & 1\end{smallmatrix}) + (\begin{smallmatrix}0 & 1\\ 1 & 0\end{smallmatrix})t + (\begin{smallmatrix}1 & 0\\ 0 & 1\end{smallmatrix})\frac{t^2}{2!} + (\begin{smallmatrix}0 & 1\\ 1 & 0\end{smallmatrix})\frac{t^3}{3!}+··· = (\begin{smallmatrix}1+\frac{t^2}{2!}+\frac{t^4}{4!}+··· & t+\frac{t^3}{3!}+···\\ t+\frac{t^3}{3!}+··· & 1+\frac{t^2}{2!}+\frac{t^4}{4!}+···\end{smallmatrix}) = (\begin{smallmatrix}cosh(t) & sinh(t)\\ sinh(t) & cosh(t)\end{smallmatrix})$ where we have recognized the series expansions: $cosh(t) = 1+\frac{t^2}{2!}+\frac{t^4}{4!}+···, sinh(t) = t+\frac{t^3}{3!}+···$

The matrix $(\begin{smallmatrix}cosh(t) & sinh(t)\\ sinh(t) & cosh(t)\end{smallmatrix})$ can also be expressed in terms of exponentials: $\frac{1}{2}(\begin{smallmatrix}e^t+e^{-t} & e^t-e^{-t}\\ e^t-e^{-t} & e^t+e^{-t}\end{smallmatrix})$

$cosh(t) = \frac{e^t+e^{-t}}{2}, sinh(t) = \frac{e^t-e^{-t}}{2}$

If we are dealing with an initial value problem $\vec{x’} = A\vec{x}$ with initial conditions $\vec{x}(0) = \vec{x_0}$

The general solution to the system is $\vec{x}(t) = e^{At}\vec{C}$ where $\vec{C}$ is a constant vector determined by the initial conditions.

If we are given an initial condition $\vec{x}(0) = \vec{x_0}$, we have $\vec{x_0} = e^{A·0}\vec{C} = I_2\vec{C} = \vec{C} ⇒ \vec{C} = \vec{x}(0)$

Therefore, the general solution is $\vec{x} = e^{At}\vec{x}(0)$

Properties of the Matrix Exponential.

1. Solves the differential equation. It satisfies $\frac{d}{dt}e^{At} = Ae^{At}$
2. Initial condition at t = 0, the matrix exponential reduces to the identity matrix, and the general solution is $\vec{x} = e^{At}\vec{x}(0)$.

Exponential Law for Matrices

In scalar algebra, we have the exponential law: e^a+b = e^ae^b

However, in matrix algebra, the exponential law e^A+B = e^Ae^B does not generally hold for matrices unless matrices A and B commute, meaning AB = BA.

Some special cases where the Exponential law holds include:

1. Scalar Matrices: A = cI, where c is a scalar and I is the identity matrix, then A commutes with any matrix B because: AB = cIB = cB = Bc = BcI = BA. Example: $A = (\begin{smallmatrix}c & 0\\ 0 & c\end{smallmatrix})$.
2. Opposite Matrices: B = -A, then A and B obviously commute AB = A(-A) = -A² = (-A)A = BA. In this case, e^{A + B} = e^Ae^B = e^Ae^-A = e⁰ = I.
3. Matrices where B = A^-1. If B = A^-1 and A is invertible, then A and B commute because AB = AA^-1 = I = A^-1A = BA.

Methods for Computing e^At

Computing the matrix exponential e^At is essential for solving linear system of differerential equations of the form: $\vec{x}’(t) = A \vec{x}(t)$. There are serveral methods to compute e^At:

• Using the Series Definition. One method is to directly compute the power series expansion for e^At = $e^{At} = I_2 + At + A^2\frac{t^2}{2!} + A^3\frac{t^3}{3!} + ···$, as we did above, which is practical for small matrices or those with repeating patterns. However, this can become cumbersome, impractical, and very difficult for larger matrices, those without symplifying patterns or more complex systems.
• Decomposition Method. In some cases, you can decompose a matrix into simpler parts. For example, if the matrix can be written as $(\begin{smallmatrix}a & b\\ b & a\end{smallmatrix}) = (\begin{smallmatrix}a & 0\\ 0 & a\end{smallmatrix}) + (\begin{smallmatrix}0 & b\\ b & 0\end{smallmatrix})$ = D + N. D is a diagonal matrix, N is a symmetric. If D and N commute (and that’s a very big “if”), then you can use the exponential law to simplify the computation: $e^{At}=e^{(D+N)t}=e^{Dt}e^{Nt}$
• Fundamental Matrix Method. To construct the matrix exponential e^At, you can use the following method:

1. Find a fundamental matrix X(t). Solve for two linearly independent solutions $\vec{x_1}(t)$ and $\vec{x_2}(t)$, and form the matrix $X(t) = (\vec{x_1}(t)~\vec{x_2}(t))$.
2. Evaluate X(0) = $(\vec{x_1}(0)~\vec{x_2}(0))$ This is the value of the fundamental matrix at t = 0.
3. Compute X(0)^-1. X is a fundamental matrix and det(X) ≠ 0 for any t.
4. Construct e^At: The matrix exponential e^At can be written as (Recall X(t) = e^AtC = e^AtX(0)), e^At = X(t)·X(0)^-1. e^At solve X’(t) = AX(t) (just like any fundamental matrix: $\vec{x}(t) = X(t)C$ where C = X(0)^-1). At t = 0, X(0)·X(0)^-1 = I₂ = e^A·0. Conclusion: e^At = X(t)·X(0)^-1.

This method: (i) avoids directly calculating the matrix exponential from the infinite series definition, which can be complex for larger matrices; (ii) provides a systematic approach applicable to various matrices; (iii) guarantees that e^At satisfies both the differential equation and the initial condition at t = 0. Once you find a fundamental matrix X(t), the calculation of e^At becomes straightforward using X(0)^-1.

Applying the Fundamental Matrix Method

Consider the system of firt-order linear differential equations: $\begin{cases} x’ = y \\ y’ = x \end{cases}$

This can be written in matrix form as $\vec{x’} = A\vec{x}$ where $\vec{x} =(\begin{smallmatrix} x\\ y\end{smallmatrix})$ and $A = (\begin{smallmatrix}0 & 1\\ 1 & 0\end{smallmatrix})$.

Our goal is to find the general solution $\vec{x}$. One method to solve such systems is to use the fundamental matrix and the matrix exponential.

Step 1: Find Eigenvalues and Eigenvectors

We start by finding the eigenvalues of A. The characteristic equation is: det(A -λI) = 0 ↭ $det(\begin{smallmatrix}-λ & 1\\ 1 & -λ\end{smallmatrix}) = 0 ↭ λ^2-1=0↭ λ_1 = 1, λ_2 = 2.$

Now, find the eigenvectors

• For λ₁ = 1, $(A -λI)\vec{v} = \vec{0} ↭ (A -I)\vec{v} = \vec{0} ↭ (\begin{smallmatrix}-1 & 1\\ 1 & -1\end{smallmatrix})(\begin{smallmatrix}a_1\\ a_2\end{smallmatrix}) = 0$

$\begin{cases} -a_1 + a_2 = 0 \\ a_1 -a_2 = 0 \end{cases}$

$a_1 = a_2 =[\text{Simplicity}] 1 ⇒ \vec{v} = (\begin{smallmatrix}1\\ 1\end{smallmatrix})$

• For λ = -1, $(A -λI)\vec{v} = \vec{0} ↭ (A +I)\vec{v} = \vec{0} ↭ (\begin{smallmatrix}1 & 1\\ 1 & 1\end{smallmatrix})(\begin{smallmatrix}a_1\\ a_2\end{smallmatrix}) = 0$

$\begin{cases} a_1 + a_2 = 0 \\ a_1 + a_2 = 0 \end{cases}$

$a_1 = -a_2, a_1 =[\text{Simplicity}] 1, a_2 = -1 ⇒ \vec{v} = (\begin{smallmatrix}1\\ -1\end{smallmatrix})$

Step 2: Form the General Solution and the Fundamental Matrix X(t)

The general solution to $\vec{x’} = A\vec{x}$ is: $\vec{x}(t) = c_1e^{t}(\begin{smallmatrix}1\\ 1\end{smallmatrix}) + c_2e^{-t}(\begin{smallmatrix}1\\ -1\end{smallmatrix})$

Thus, the fundamental matrix is formed by placing independent solutions as columns into a matrix: $X(t) = (\vec{x_1}(t)~\vec{x_2}(t)) = (\begin{smallmatrix}e^{t} & e^{t}\\ e^{t} & -e^{-t}\end{smallmatrix})$

The fundamental matrix X(t) is constructed from a set of linearly independent solutions.

Step 3: Compute X(0) and X(0)^-1

At t = 0, X(0) = $(\begin{smallmatrix}e^{0} & e^{-0}\\ e^{-0} & -e^{-0}\end{smallmatrix}) = (\begin{smallmatrix}1 & 1\\ 1 & -1\end{smallmatrix})$

det(X(0)) = $det(\begin{smallmatrix}1 & 1\\ 1 & -1\end{smallmatrix}) = -1 -1 = -2 ≠ 0$. Since det(X(0)) = -2, X(0) is invertible.

$X(0)^{-1} = \frac{1}{-2}(\begin{smallmatrix}-1 & -1\\ -1 & 1\end{smallmatrix}) = \frac{1}{2}(\begin{smallmatrix}1 & 1\\ 1 & -1\end{smallmatrix})$

Step 4: Compute e^At = X(t)X(0)^-1

$e^{At} = X(t)X(0)^{-1} = (\begin{smallmatrix}e^{t} & e^{-t}\\ e^{t} & -e^{-t}\end{smallmatrix})·\frac{1}{2}(\begin{smallmatrix}1 & 1\\ 1 & -1\end{smallmatrix}) = \frac{1}{2}(\begin{smallmatrix}e^{t} + e^{-t} & e^{t} - e^{-t}\\ e^{t} -e^{-t} & e^{t} + e^{-t}\end{smallmatrix})$ This result matches the earlier expression for e^At.

We have found the solution to the system: $\vec{x}(t) = e^{At}\vec{x_0} = (\begin{smallmatrix}cosh(t) & sinh(t)\\ sinh(t) & cosh(t)\end{smallmatrix})\vec{x_0}$. This method confirms that the matrix exponentials provides a powerful tool for solving linear systems of differential equations.

Bibliography

1. NPTEL-NOC IITM, Introduction to Galois Theory.
2. Algebra, Second Edition, by Michael Artin.
3. LibreTexts, Calculus and Calculus 3e (Apex). Abstract and Geometric Algebra, Abstract Algebra: Theory and Applications (Judson).
4. Field and Galois Theory, by Patrick Morandi. Springer.
5. Michael Penn, and MathMajor.
6. Contemporary Abstract Algebra, Joseph, A. Gallian.
7. YouTube’s Andrew Misseldine: Calculus. College Algebra and Abstract Algebra.
8. MIT OpenCourseWare [18.03 Differential Equations, Spring 2006], YouTube by MIT OpenCourseWare.
9. Calculus Early Transcendentals: Differential & Multi-Variable Calculus for Social Sciences.