Gradient vector

When the world says, ‘Give up,’ hope whispers, ‘Try it one more time.’ Lyndon B. Johnson.

Learn everything you can, anytime you can, from anyone you can – there will always come a time when you will be grateful you did, Sarah Caldwell.

Recall

Definition. A vector $\vec{AB}$ is a geometric object or quantity that has both magnitude (or length) and direction. Vectors in an n-dimensional Euclidean space can be represented as coordinates vectors in a Cartesian coordinate system.

Definition. The magnitude of the vector $\vec{A}$, also known as length or norm, is given by the square root of the sum of its components squared, $|\vec{A}|~ or~ ||\vec{A}|| = \sqrt{a_1^2+a_2^2+a_3^2}$, e.g., $||< 3, 2, 1 >|| = \sqrt{3^2+2^2+1^2}=\sqrt{14}$, $||< 3, -4, 5 >|| = \sqrt{3^2+(-4)^2+5^2}=\sqrt{50}=5\sqrt{2}$, or $||< 1, 0, 0 >|| = \sqrt{1^2+0^2+0^2}=\sqrt{1}=1$.

• The sum of two vectors is the sum of their components, $\vec{A}+ \vec{B} = (a_1+b_1)\vec{i}+(a_2+b_2)\vec{j}+(a_3+b_3)\vec{k}$ = < (a₁+b₁), (a₂+b₂), (a₃+b₃) >.

• The subtraction of two vectors is similar to addition and is also done component-wise, it is given by simply subtracting their corresponding components (x, y, and z in 3D), $\vec{A} - \vec{B} = (a_1-b_1)\vec{i}+(a_2-b_2)\vec{j}+(a_3-b_3)\vec{k}$ = < (a₁-b₁), (a₂-b₂), (a₃-b₃) >.

• Scalar multiplication is the multiplication of a vector by a scalar, a real number, changing its magnitude without altering its direction. It is effectively multiplying each component of the vector by the scalar value, $c\vec{A} = (ca_1)\vec{i}+(ca_2)\vec{j}+(ca_3)\vec{k} = < ca_1, ca_2, ca_3>$.

• The dot or scalar product is a fundamental operation between two vectors. It produces a scalar quantity that represents the projection of one vector onto another. The dot product is the sum of the products of their corresponding components: $\vec{A}·\vec{B} = \sum a_ib_i = a_1b_1 + a_2b_2 + a_3b_3.$, e.g. $\vec{A}·\vec{B} = \sum a_ib_i = ⟨2, 2, -1⟩·⟨5, -3, 2⟩ = a_1b_1 + a_2b_2 + a_3b_3 = 2·5+2·(-3)+(-1)·2 = 10-6-2 = 2.$

  It is the product of their magnitudes multiplied by the cosine of the angle between them, $\vec{A}·\vec{B}=||\vec{A}||·||\vec{B}||·cos(θ).$

• The cross product, denoted by $\vec{A}x\vec{B}$, is a binary operation on two vectors in three-dimensional space. It results in a vector that is perpendicular to both of the input vectors and has a magnitude equal to the area of the parallelogram formed by the two input vectors.

  The cross product $\vec{A}x\vec{B}$ can be computed using the following formula, $\vec{A}x\vec{B} = det(\begin{smallmatrix}i & j & k\\ a_1 & a_2 & a_3\\ b_1 & b_2 & b_3\end{smallmatrix}) =|\begin{smallmatrix}a_2 & a_3\\ b_2 & b_3\end{smallmatrix}|\vec{i}-|\begin{smallmatrix}a_1 & a_3\\ b_1 & b_3\end{smallmatrix}|\vec{j}+|\begin{smallmatrix}a_1 & a_2\\ b_1 & b_2\end{smallmatrix}|\vec{k}$

• Matrices offers an efficient way for solving systems of linear equations. Utilizing matrices, we can represent these equations more concisely and conveniently.

  For a system like,

  $(\begin{smallmatrix}2 & 3 & 3\\ 2 & 4 & 4\\ 1 & 1 & 2\end{smallmatrix}) (\begin{smallmatrix}x_1\\ x_2\\x_3\end{smallmatrix}) = (\begin{smallmatrix}u_1\\ u_2\\u_3\end{smallmatrix})$, A · X = U provides a more convenient and concise notation and an efficient way to solve system of linear equations where A is a 3 x 3 matrix representing coefficients, X is a 3 x 1 column vector representing the results where we basically do dot products between the rows of A (a 3 x 3 matrix) and the column vector of X (a 3 x 1 matrix).

• Given a square matrix A (it is a matrix that has an equal number of rows and columns), an inverse matrix A^-1 exists if and only if A is non-singular, meaning its determinant is non-zero (det(A)≠ 0).

  The inverse matrix of A, denoted as A^-1, has the property that when multiplied by A, it results in the identity matrix, i.e., $A \times A^{-1} = A^{-1} \times A = I$. Essentially, multiplying a matrix by its inverse reverses (“undoes”) the effect of the original matrix.

• Consider a system of linear equations represented in matrix form as: AX = B, where A is a n×n matrix (coefficient matrix), X is an n×1 matrix (column vector of variables), and B is an n×1 matrix (column vector of constants).

  Solving a system of linear equations expressed as AX = B involves finding or isolating the matrix X when we are given both matrices A and B. AX = B ⇒[A should be non-singular, meaning its determinant is non-zero, det(A)≠0, then we can multiply both side by A^-1]⇒ X = A^-1B.. $A^{-1} = \frac{1}{\text{det}(A)} \times \text{adj}(A)$. adj(A) = C^T where $C_{ij} = (-1)^{i+j} \times \text{minor}(A_{ij})$ and det(A) =[A is a 3x3 matrix] a(ei - fh) - b(di - fg) + c(dh - eg)

• A function of two variables f: ℝ x ℝ → ℝ assigns to each ordered pair in its domain a unique real number, e.g., Area = $\frac{1}{2}b·h$, z = f(x, y) = 2x + 3y, f(x, y) = x² + y², e^x+y, etc.

• Partial derivatives are derivatives of a function of multiple variables, say f(x₁, x₂, ···, x_n) with respect to one of those variables, with the other variables held constant. They measure how the function changes as one variable changes, while keeping the others constant. $\frac{\partial f}{\partial x}(x_0, y_0) = \lim_{\Delta x \to 0} \frac{f(x_0+\Delta x, y_0)-f(x_0, y_0)}{\Delta x}$

Geometrically, the partial derivative $\frac{\partial f}{\partial x}(x_0, y_0)$ at a point (x₀, y₀) can be interpreted as the slope of the tangent line to the curve formed by the intersection of the surface defined by f(x,y) and the plane y = y₀. Similarly, $\frac{\partial f}{\partial y}(x_0, y_0)$ represents the slope of the tangent line to the curve formed by the intersection of the surface defined by f(x,y) and the plane x = x₀

• $\frac{dw}{ds}\bigg|_{\vec{u}}$ is the directional derivative in the direction of ${\vec{u}}$, the rate of change of the function w as we move in the direction of $\vec{u}$.

$\frac{dw}{ds}\bigg|_{\vec{u}} = ∇w·\vec{u} = |∇w|·|\vec{u}|cos(θ) = |∇w|·cos(θ)$ where θ is the angle between the gradient and the given unit vector.

• The directional derivative is maximized when cos(θ) = 1 ↭ θ = 0, that means, $\vec{u}$ is in the same direction as the gradient ∇w, $\vec{u}=dir(∇w).$

The gradient is the direction in which the function increases fastest (the direction of the steepest ascent) at a given point, and |∇w| = $\frac{dw}{ds}\bigg|_{\vec{u}=dir(∇w)}$.

The directional derivative is minimized when cos(θ) = -1 ↭ θ = 180 ↭ $\vec{u}$ is in the opposite direction of the gradient ∇w. Futhermore, $\frac{dw}{ds}\bigg|_{\vec{u}} = 0$ ↭ cos(θ) = 0 ↭ θ = 90° ↭ $\vec{u}$ ⊥ ∇w.

• The Chain Rule. Let w = f(x, y, z) where x = x(t), y = y(t), and z = z(t) are functions of another variable t, then the derivate of f with respect to t is given by $\frac{dw}{dt} = f_x\frac{dx}{dt} +f_y\frac{dy}{dt}+f_z\frac{dz}{dt}, \frac{dw}{dt} = w_x\frac{dx}{dt} +w_y\frac{dy}{dt}+w_z\frac{dz}{dt} = ∇w·\frac{d\vec{r}}{dt}~$ where ∇w = ⟨w_x, w_y, w_z⟩ and $\frac{d\vec{r}}{dt}=⟨\frac{d\vec{x}}{dt}, \frac{d\vec{y}}{dt}, \frac{d\vec{z}}{dt}⟩$

Gradient vector

When we have a function w = f(x, y, z) where x = x(t), y = y(t), and z = z(t) are functions of another variable t, then we can calculate the derivate of w with respect to t using the chain rule. The chain rule in this context is given by given by $\frac{dw}{dt} = \frac{∂w}{∂x}\frac{dx}{dt} +\frac{∂w}{∂y}\frac{dy}{dt}+\frac{∂w}{∂z}\frac{dz}{dt} =[\text{Using other notation:}] w_x\frac{dx}{dt} +w_y\frac{dy}{dt}+w_z\frac{dz}{dt}, \frac{dw}{dt}$.

This can be compactly written using the gradient vector ∇w and the vector $\frac{d\vec{r}}{dt}$ where $\vec{r}(t) = ⟨x(t), y(t), z(t)⟩$: $\frac{dw}{dt} = ∇w·\frac{d\vec{r}}{dt}$.

The gradient vector is defined as follows: ∇w = ⟨w_x, w_y, w_z⟩ = $⟨\frac{\partial w}{\partial x}, \frac{\partial w}{\partial y}, \frac{\partial w}{\partial z}⟩$. And $\frac{d\vec{r}}{dt}$ is the vector of derivatives of x, y, and z with respect to t: $\frac{d\vec{r}}{dt} = ⟨\frac{dx}{dt}, \frac{dy}{dt}, \frac{dz}{dt}⟩$.

The gradient vector is a vector that points in the direction of the steepest increase of the function at a given point. For example, if w = a₁x + a₂y + a₃z, then: ∇w = ⟨a₁, a₂, a₃⟩.

The Gradient Vector is Perpendicular to Level Surfaces

Theorem. The gradient vector is perpendicular (orthogonal) to the level surfaces of the function. This means that the gradient points in the direction of the steepest ascent or increase, and there is no change in the function’s value along the level surface, making the gradient perpendicular to it.

For a given constant c, the level surface of a multivariable function f is the set of points (x, y, z) such that f(x,y,z) = c.

For example, consider the linear function w = f(x, y, z) = a₁x + a₂y + a₃z. The level surface for w = c is: a₁x + a₂y + a₃z = c.

a₁x + a₂y + a₃z is a plane, and its normal vector is ⟨a₁, a₂, a₃⟩, which is also the gradient vector ∇w = ⟨$\frac{\partial w}{\partial x}, \frac{\partial w}{\partial y}, \frac{\partial w}{\partial z}$⟩

Example

Consider the function w = x² + y². The level surfaces are given by: w = c = x² + y². These are circles in the xy-plane.

The gradient of w is: $\nabla w = \langle 2x, 2y \rangle $. This gradient vector is perpendicular to the level surfaces, which are circles (Figure A).

Proof.

To prove that the gradient vector is perpendicular to the level surface, consider a curve $\vec{r}(t)$ that lies on the level surface w(x, y, z) = c (Figure B). The velocity vector of this curve or motion, $\vec{v} = \frac{d\vec{r}}{dt}$, is tangent to the level surface w = c.

By the chain rule, we have: $\frac{dw}{dt} = ∇w·\frac{d\vec{r}}{dt} = ∇w·\vec{v}$ where $\vec{v} = \frac{d\vec{r}}{dt}=⟨\frac{d\vec{x}}{dt}, \frac{d\vec{y}}{dt}, \frac{d\vec{z}}{dt}⟩$.

Since the curve $\vec{r}$ stays on the level surface w = c, the function value w does not change, and thus $\frac{dw}{dt} = 0 $.

This implies $\nabla w \cdot \vec{v} = 0$.

If the dot product of two vectors is zero, the vectors are perpendicular (orthogonal), ∇w ⊥ $\vec{v}$, and this is true for any motion on w = c. Therefore, ∇w is perpendicular to $\vec{v}$, ∇w ⊥ $\vec{v}$, and since $\vec{v}$ is any vector tangent to the level surface w = c (Figure C), the gradient vector is perpendicular to the level surface ∎

Solved exercises

• Find the gradients of the functions f(x, y) = x² -xy + 3y² and g(x, y) = sin(3x)cos(3y).

The gradient of a function f(x, y) is a vector that points in the direction of the steepest ascent. It is calculated by taking the partial derivatives of f with respect to x and y.

Calculate the gradient of f: ∇f = $⟨\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}⟩ = ⟨2x -y, -x +6y⟩$.

Calculate the gradient of g: ∇g = $⟨\frac{\partial g}{\partial x}, \frac{\partial g}{\partial y}⟩ = ⟨3cos(3x)cos(3y), -3sin(3x)sin(3y)⟩$.

• Imagine a mountain whose altitude at any point (x, y) is described by the function f(x, y) = x² -y. You are standing at the pont (2, 3) on this mountain. You want to determine the direction you should walk to climb the steepest uphill and also the direction you should walk to descend the steepest downhill.

The gradient of a function f(x, y) is a vector that points in the direction of the steepest ascent. It is calculated by taking the partial derivatives of f with respect to x and y.

Calculate the gradient of f: ∇f = $⟨\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}⟩ = ⟨2x, -1⟩$.

Evaluate the gradient at the point (2, 3). Next, we evaluate the gradient at the specific point (2, 3) by substituting x = 2 and y = 3 into the gradient expression: ∇f(2, 3) = ⟨2·2, -3⟩ = ⟨4, -3⟩.

Determine the Direction of the steepest ascent. The gradient vector ∇f at any point gives the direction of the steepest ascent (i.e., the direction in which the altitude increases most rapidly). Therefore, at the point (2, 3), the direction of the steepest ascent is ∇f = ⟨4, -3⟩.

Determine the direction of the steepest descent. The direction of the steepest descent or downhill is the opposite direction of the gradient vector, that is, ⟨-4, 3⟩.

• Determine the normal vector of the following equation of a plane: 2x +3y −6z +3 = 0.

The normal vector to a plane is a vector that is perpendicular to every vector lying on that plane.

Consider the plane given by the equation: 2x + 3y - 6z + 3 = 0. To determine the normal vector of this plane, we look at the coefficients of x, y, and z in the equation. Specifically, $\vec{n} = \langle a, b, c \rangle = \langle 2, 3, -6 \rangle$

The normal vector points directly away from the plane, indicating the direction in which the plane is “facing”. If we consider any point $P_0(x_0, y_0, z_0)$ on the plane, then any vector $\vec{P_0P}$, that lies on the plane will have the property that its dot product with the normal vector is zero. This is because the normal vector is perpendicular to every vector in the plane.

The dot product of the normal vector $\vec{n} = \langle 2, 3, -6 \rangle$ with any vector $\vec{P_0P}$ lying on the plane is given by: $\vec{n}·\vec{P_0P} = 0 ↭[\text{Expanding this, we get}] ⟨2, 3, -6⟩·⟨x -x_0, y -y_0, z-z_0⟩ = 0$. This equation must hold for any point (x, y, z) on the plane.

To verify, let’s consider a specific point $P_0(0, -1, 0)$ that lies on the plane. Substituting P₀ into the plane equation 2·0 + 3·(-1)-6·0+3 =[Simplifies to:] 0 −3 +3 = 0. To verify the perpendicularly, we check: $2(x-0) +3(y+1)-6(z-0) = 0 ↭ 2x + 3y -6z +3 = 0$.

• Given three points P(1, 4, 2), Q(−10, 4, 3), and R(2, 2, 4), find the equation of the plane that passes through these points.

Here’s a step-by-step explanation:

1. Find two vectors that lie on the plane. We can find vectors $\vec{PQ}$ and $\vec{PR}$ by subtracting the coordinates of point P from points Q and R respectively: $\vec{PQ} = Q - P = \langle -10 - 1, 4 - 4, 3 - 2 \rangle = \langle -11, 0, 1 \rangle, \vec{PR} = R - P = \langle 2 - 1, 2 - 4, 4 - 2 \rangle = \langle 1, -2, 2 \rangle$

2. Compute the Normal Vector to the plane. It is found by taking the cross product of $\vec{PQ}$ and $\vec{PR}$, $\vec{n} = \vec{PQ} \times \vec{PR} = \begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ -11 & 0 & 1 \\ 1 & -2 & 2 \end{vmatrix} = \mathbf{i}(0 \cdot 2 - 1 \cdot (-2)) - \mathbf{j}((-11) \cdot 2 - 1 \cdot 1) + \mathbf{k}((-11) \cdot (-2) - 0 \cdot 1) = \mathbf{i}(0 + 2) - \mathbf{j}(-22 - 1) + \mathbf{k}(22 - 0) = 2\mathbf{i} + 23\mathbf{j} + 22\mathbf{k} = ⟨2, 23, 22⟩$

3. Write the Point-Normal Form of the Plane. The point-normal form is given by: $\vec{n} \cdot (\vec{P_oP}) = 0$.

Using the normal vector $\vec{n} = ⟨2, 23, 22⟩$ and the point P(1, 4, 2), the equation becomes:

$⟨2, 23, 22⟩·⟨x-1, y-4, z-2⟩ =0$ ↭[Expanding the dot product:] 2(x-1) + 23(y-4) + 22(z-2) = 0 ↭[Simplifying and combining like terms] 2x + 23y + 22z - 138 = 0. So, the equation of the plane in point-normal form is: 2x + 23y + 22z - 138 = 0.

• Find the tangent plane to the surface x² + y² -z² = 4 at (2, 1, 1).

To find the equation of the tangent plane to the surface x² + y² -z² = 4 at the point (2, 1, 1), we can follow these steps:

1. Define the function and calculate its gradient. We define the function w(x, y, z) = x² + y² -z². The surface is given by the level set w(x, y, z) = 4. Next, we calculate the gradient of w, $\nabla w = \left\langle \frac{\partial w}{\partial x}, \frac{\partial w}{\partial y}, \frac{\partial w}{\partial z} \right\rangle = \left\langle 2x, 2y, -2z \right\rangle$.

2. Evaluate the gradient at the point (2, 1, 1): $⟨2x, 2y, -2z⟩\bigg|_{(2, 1, 1)}$ = ⟨4, 2, -2⟩. This gradient vector ⟨4, 2, -2⟩ is normal (perpendicular) to the surface at the point ⟨2, 1, 1⟩.

3. Write the equation of the Tangent Plane. The equation of the tangent plane to the surface at (x₀, y₀, z₀) can be written using the point-normal form of the plane equation: $\nabla w \cdot \langle x - x_0, y - y_0, z - z_0 \rangle = 0 $. Substituting $\nabla w = \langle 4, 2, -2 \rangle$ and the point (2, 1, 1), the equation becomes: 4(x - 2) + 2(y - 1) - 2(z - 1) = 0.

   Simplifying the equation: 4x - 8 + 2y - 2 - 2z + 2 = 0 ↭[Combining like terms:] 4x + 2y - 2z - 8 = 0. Thus, the equation of the tangent plane is: 4x + 2y - 2z = 8.

   The gradient vector of a function f(x,y,z) is a vector that points in the direction of the steepest increase of the function. It is perpendicular to the level surfaces of the function (w = f(x, y, z) = c).

   The tangent plane to a surface at a given point is the plane that just touches the surface at that point and is perpendicular to the gradient vector at that point (Figure 3). The equation of the tangent plane to the surface f(x, y, z) = c at a point (x₀, y₀, z₀) can be written using the point-normal form of the plane equation. The gradient vector ∇f at (x₀, y₀, z₀) serves as the normal vector to the tangent plane. The equation is: ∇f(x₀, y₀, z₀)·(x -x₀, y -y₀, z -z₀) = 0.

   

4. Alternative Approach Using Differentials

Another way to find the tangent plane is by using differentials. For the function $w = x^2 + y^2 - z^2$, the differential dw is given by: $dw = \frac{\partial w}{\partial x} dx + \frac{\partial w}{\partial y} dy + \frac{\partial w}{\partial z} dz ↭ dw = 2x dx + 2y dy - 2z dz$

Evaluating at the point (2, 1, 1): dw = 2(2) dx + 2(1)dy -2(1)dz, dw = 4dx + 2dy - 2dz.

For small changes near the point (2, 1, 1), we approximate: $\Delta w \approx 4 \Delta x + 2 \Delta y - 2 \Delta z$.

Since we are on the level surface w = 4, $\Delta w = 0$: $4 \Delta x + 2 \Delta y - 2 \Delta z = 0$

This corresponds to the tangent plane equation: 4(x - 2) + 2(y - 1) - 2(z - 1) = 0 ↭ 4x + 2y - 2z = 8.

• You are climbing a hill described by the equation z = 1000 -0.01x² -0.02y², where x, y, and z are measured in meters. The positive x-axis points due east, and the positive y-axis points due north. You are at the point (50, 80, 847) (Figure 2).

A. If you walk due south, will you start to ascend or descend? At what rate?

Determine the unit vector for south direction. South is the negative y-direction, $\vec{u} = ⟨0, -1⟩.$

Calculate the gradient of f: ∇f = $⟨\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}⟩ = ⟨-0.02x, -0.04y⟩$.

Evaluate the gradient at P(50, 80): ∇f(50, 80) = ⟨-0.02·50, -0.04·80⟩ = ⟨-1, -3.2⟩.

Calculate the directional derivative in the south direction: $D_{\vec{u}}(50, 80) = ∇f·\vec{u} = ⟨-1, -3.2⟩·⟨0, -1⟩ = 3.2$. Since the directional derivative is positive, you will start to ascend at a rate of $3.2 \frac{\text{meters of elevation}}{\text{meters of distance}}$.

B. If you walk northwest, will you start to ascend or descend. At what rate?

Determine the unit vector for northwest direction. Northwest direction is equally negative in x and positive in y, the unit vector $\vec{u}$ is $\frac{1}{\sqrt{2}}⟨-1, 1⟩$.

Calculate the directional derivative in the northwest direction: $D_{\vec{u}}(50, 80) = ∇f·\vec{u} = ⟨-1, -3.2⟩·\frac{1}{\sqrt{2}}⟨-1, 1⟩ = \frac{1-3.2}{\sqrt{2}} = \frac{-2.2}{\sqrt{2}}$. Since the directional derivative is negative, you will start to descend at a rate of $\frac{-2.2}{\sqrt{2}} \frac{\text{meters of elevation}}{\text{meters of distance}}$.

C. In which direction is the slope largest? What is the rate of ascent in that direction? At what angle above the horizontal does the path in that direction begin?

Direction of the steepest ascent. The direction of the steepest ascent is the direction of the gradient ∇f = ⟨-1, -3.2⟩.

Rate of ascent. The rate of ascent is the magnitude of the gradient: |∇f| = $\sqrt{(-1)^2+(-3.2)^2}≈3.35\frac{\text{meters of elevation}}{\text{meters of distance}}$

Angle of ascent. The angle θ above the horizontal can be found using the tangent function: tan(θ) = $\frac{\text{rate of ascent}}{1}$ = 3.35 ⇒ θ = tan^-1(3.35) ≈ 73.4°.

Bibliography

1. NPTEL-NOC IITM, Introduction to Galois Theory.
2. Algebra, Second Edition, by Michael Artin.
3. LibreTexts, Calculus and Calculus 3e (Apex). Abstract and Geometric Algebra, Abstract Algebra: Theory and Applications (Judson).
4. Field and Galois Theory, by Patrick Morandi. Springer.
5. Michael Penn, and MathMajor.
6. Contemporary Abstract Algebra, Joseph, A. Gallian.
7. YouTube’s Andrew Misseldine: Calculus. College Algebra and Abstract Algebra.
8. MIT OpenCourseWare 18.01 Single Variable Calculus, Fall 2007 and 18.02 Multivariable Calculus, Fall 2007.
9. Calculus Early Transcendentals: Differential & Multi-Variable Calculus for Social Sciences.