Gradient vector

When the world says, ‘Give up,’ hope whispers, ‘Try it one more time.’ Lyndon B. Johnson.

Learn everything you can, anytime you can, from anyone you can – there will always come a time when you will be grateful you did, Sarah Caldwell.

Functions, Partials, and Gradients

• A function of two variables $f: \mathbb{R} \times \mathbb{R} \to \mathbb{R}$ assigns to each ordered pair in its domain a unique real number. Examples include geometric formulas like Area = $\frac{1}{2}bh$, or algebraic surfaces like $z = f(x, y) = 2x + 3y$, $f(x, y) = x^2 + y^2$, and $f(x,y) = e^{x+y}$.

• Partial derivatives differentiate a multivariable function $f(x_1, x_2, \dots, x_n)$ with respect to one target variable while treating all other variables as constants. They measure how the function changes in a single specific direction.
  If $f(x, y) = 2x^2y^3$, then $\frac{\partial f}{\partial x} = f_x = 4xy^3$ and $\frac{\partial f}{\partial y} = f_y = 6x^2y^2$.
  If $f(x, y) = x^3y + y^2$, then $f_x = 3x^2y$ and $f_y = x^3+2y$.

• A point $(x_1, x_2, \dots, x_n)$ is considered a critical point if the gradient of $f$ at that point is the zero vector, or if the gradient does not exist.

Gradients and Level Surfaces

• The gradient $\nabla f$ is a vector containing all first-order partial derivatives of $f$. It geometrically points in the direction of the function’s steepest ascent. $\nabla f = \left\langle \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \dots, \frac{\partial f}{\partial x_n} \right\rangle$.

• In 2D space ($n = 2$), finding a critical point $(x_0, y_0)$ requires solving the system: $\frac{\partial f}{\partial x}(x_0, y_0) = 0$ and $\frac{\partial f}{\partial y}(x_0, y_0) = 0$.

• Theorem: The gradient vector is perpendicular (orthogonal) to the level curves or level surfaces of the function at any given point. This means if you walk along a contour line where the function’s value is constant, the direction of steepest uphill is exactly 90 degrees to your path.

The Chain Rule and Directional Derivatives

The directional derivative $\frac{dw}{ds}\bigg\vert{}_{\vec{u}}$ represents the rate of change of a function $w$ as you move in the direction of a unit vector $\vec{u}$. It can be expressed or compactly written via the dot product of the gradient and the direction vector: $\frac{dw}{ds}\bigg\vert{}_{\vec{u}} = \nabla w \cdot \vec{u} = \vert{}\nabla w\vert{}\vert{}\vec{u}\vert{}\cos(\theta) = \vert{}\nabla w\vert{}\cos(\theta)$ where $\theta$ is the angle between the gradient $\nabla w$ and the unit vector $\vec{u}$.

• Maximum Increase: The directional derivative is maximized when $\cos(\theta) = 1 \implies \theta = 0^\circ$. This occurs when $\vec{u}$ points in the exact same direction as the gradient $\nabla w$. The gradient is the direction in which the function increases fastest (the direction of steepest ascent) at a given point. The maximum rate of change is $\vert{}\nabla w\vert{}$.
• Maximum Decrease: The derivative is minimized when $\cos(\theta) = -1 \implies \theta = 180^\circ$. This occurs when $\vec{u}$ points in the exact opposite direction of the gradient.
• Zero Change: The derivative is zero when $\cos(\theta) = 0 \implies \theta = 90^\circ$. This means moving perpendicular to the gradient ($\vec{u} \perp \nabla w$) results in no change in the function’s value.

This behavior is underpinned by the Multivariable Chain Rule. If $w = f(x, y, z)$ and the coordinates are parametrized by time $t$ such that $\vec{r}(t) = \langle x(t), y(t), z(t) \rangle$, the rate of change of $w$ with respect to $t$ is: $\frac{dw}{dt} = \frac{\partial w}{\partial x}\frac{dx}{dt} + \frac{\partial w}{\partial y}\frac{dy}{dt} + \frac{\partial w}{\partial z}\frac{dz}{dt} = \nabla w \cdot \frac{d\vec{r}}{dt}$

Gradient Vectors and Level Surfaces

For a constant $c$, a level surface of a multivariable function $f(x, y, z)$ is the set of all points satisfying $f(x,y,z) = c$.

Consider the function $w = x^2 + y^2$. The level curves are given by: $w = c = x^2 + y^2$. These are circles in the xy-plane.

The gradient of w is: $\nabla w = \langle 2x, 2y \rangle$. This gradient vector is perpendicular to the level surfaces, which are circles (Figure A).

Theorem: The gradient vector is perpendicular (orthogonal) to the level surfaces of the function.

Proof:

1. Consider a curve $\vec{r}(t) = \langle x(t), y(t), z(t) \rangle$ that lies entirely on the level surface $w(x, y, z) = c$ (Figure B).
2. Because the curve never leaves the surface, the value of $w$ along the curve is constant, meaning its derivative with respect to $t$ is zero: $\frac{dw}{dt} = 0$.
3. By the chain rule, $\frac{dw}{dt} = \nabla w \cdot \frac{d\vec{r}}{dt} = \nabla w \cdot \vec{v}$, where $\vec{v}$ is the tangent velocity vector of the curve.
4. Therefore, $\nabla w \cdot \vec{v} = 0$. Since their dot product is zero, the gradient $\nabla w$ must be perpendicular to any tangent vector $\vec{v}$ on the surface, proving it is orthogonal to the surface itself (Figure C).

Solved Exercises

• Find the gradients of $f(x, y) = x^2 - xy + 3y^2$ and $g(x, y) = \sin(3x)\cos(3y)$.

1. Recall. The gradient of a function f(x, y) is a vector that points in the direction of the steepest ascent. It is calculated by taking the partial derivatives of f with respect to x and y.
2. Calculate the gradient of f: $\nabla f = \left\langle \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y} \right\rangle = \langle 2x - y, -x + 6y \rangle$
3. Calculate the gradient of g: $\nabla g = \left\langle \frac{\partial g}{\partial x}, \frac{\partial g}{\partial y} \right\rangle = \langle 3\cos(3x)\cos(3y), -3\sin(3x)\sin(3y) \rangle$

• Imagine a mountain whose altitude at any point (x, y) is described by the function $f(x, y) = x^2 - y$. You are standing at the point $(2, 3)$ on this mountain. You want to determine the direction you should walk to climb the steepest uphill and also the direction you should walk to descend the steepest downhill.

1. Recall. The gradient of a function f(x, y) is a vector that points in the direction of the steepest ascent. It is calculated by taking the partial derivatives of f with respect to x and y.
2. Calculate the gradient of f: $\nabla f = \left\langle \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}\right\rangle = \left\langle 2x, -1\right\rangle$.
3. Evaluate the gradient at the point (2, 3). Next, we evaluate the gradient at the specific point (2, 3) by substituting x = 2 and y = 3 into the gradient expression: $\nabla f(2, 3) = \langle 2(2), -1 \rangle = \langle 4, -1 \rangle$.
4. Determine the Direction of the steepest ascent. The gradient vector $\nabla f$ at any point gives the direction of the steepest ascent (i.e., the direction in which the altitude increases most rapidly). Therefore, at the point (2, 3), the direction of the steepest ascent is $\langle 4, -1 \rangle$.
5. Determine the direction of the steepest descent. The direction of the steepest descent or downhill is the opposite direction of the gradient vector, that is, $\langle -4, 1 \rangle$.

• Determine the normal vector of the following equation of a plane: $2x + 3y - 6z + 3 = 0$.

1. Recall. The normal vector to a plane is a vector that is perpendicular to every vector lying on that plane. A plane written in the form $ax + by + cz + d = 0$ has a normal vector $\vec{n} = \langle a, b, c \rangle$.
2. Consider the plane given by the equation: $2x + 3y - 6z + 3 = 0$. To determine the normal vector of this plane, we look at the coefficients of x, y, and z in the equation. Specifically, $\vec{n} = \langle a, b, c \rangle = \langle 2, 3, -6 \rangle$
3. 🔍 Geometric check. If we consider any point $P_0(x_0, y_0, z_0)$ on the plane, then any vector $\vec{P_0P}$, that lies on the plane will have the property that its dot product with the normal vector is zero. This is because the normal vector is perpendicular to every vector in the plane.
4. The dot product of the normal vector $\vec{n} = \langle 2, 3, -6 \rangle$ with any vector $\vec{P_0P}$ lying on the plane is given by: $\vec{n} \cdot \vec{P_0P} = 0 ↭[\text{Expanding this, we get}] \left\langle 2, 3, -6\right\rangle \cdot \left\langle x -x_0, y -y_0, z-z_0\right\rangle = 0$. This equation must hold for any point (x, y, z) on the plane.
   To verify, let’s consider a specific point $P_0(0, -1, 0)$ that lies on the plane. Substituting $P_0$ into the plane equation $2\cdot 0 + 3\cdot (-1)-6\cdot 0+3$ =[Simplifies to:] 0 −3 +3 = 0. To verify the perpendicularity, we check: $2(x-0) +3(y+1)-6(z-0) = 0 \iff 2x + 3y -6z +3 = 0$. And this expression is exactly the plane equation.

• Given three points P(1, 4, 2), Q(−10, 4, 3), and R(2, 2, 4), find the equation of the plane that passes through these points.

Here’s a step-by-step explanation:

1. Find two vectors that lie on the plane. We can find vectors $\vec{PQ}$ and $\vec{PR}$ by subtracting the coordinates of point P from points Q and R respectively:
   $\vec{PQ} = Q - P = \langle -10 - 1, 4 - 4, 3 - 2 \rangle = \langle -11, 0, 1 \rangle, \vec{PR} = R - P = \langle 2 - 1, 2 - 4, 4 - 2 \rangle = \langle 1, -2, 2 \rangle$
2. Compute the Normal Vector to the plane.
   It is found by taking the cross product of $\vec{PQ}$ and $\vec{PR}$:
   $\vec{n} = \vec{PQ} \times \vec{PR} = \begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ -11 & 0 & 1 \\ 1 & -2 & 2 \end{vmatrix} = \mathbf{i}(0 \cdot 2 - 1 \cdot (-2)) - \mathbf{j}((-11) \cdot 2 - 1 \cdot 1) + \mathbf{k}((-11) \cdot (-2) - 0 \cdot 1) = \mathbf{i}(0 + 2) - \mathbf{j}(-22 - 1) + \mathbf{k}(22 - 0) = 2\mathbf{i} + 23\mathbf{j} + 22\mathbf{k} = \left\langle 2, 23, 22\right\rangle$
3. Write the Point-Normal Form of the Plane. The point-normal form is given by: $\vec{n} \cdot (\vec{P_oP}) = 0$.
   Using the normal vector $\vec{n} = ⟨2, 23, 22⟩$ and the point P(1, 4, 2), the equation becomes: $\left\langle 2, 23, 22\right\rangle \cdot \left\langle x-1, y-4, z-2\right\rangle =0$ ↭[Expanding the dot product:] $2(x-1) + 23(y-4) + 22(z-2) = 0$ ↭[Simplifying and combining like terms] $2x + 23y + 22z - 138 = 0$.
4. Conclusion. So, the equation of the plane in point-normal form is: $2x + 23y + 22z - 138 = 0$.

• Find the tangent plane to the surface $x^2 + y^2 - z^2 = 4$ at $(2, 1, 1)$.

1. Define the function and calculate its gradient. We define the function $w(x, y, z) = x^2 + y^2 - z^2$. The surface is given by the level set $w(x, y, z) = 4$. Next, we calculate the gradient of w, $\nabla w = \left\langle \frac{\partial w}{\partial x}, \frac{\partial w}{\partial y}, \frac{\partial w}{\partial z} \right\rangle = \left\langle 2x, 2y, -2z \right\rangle$.

2. Evaluate the gradient at the point (2, 1, 1): $\left\langle 2x, 2y, -2z \right\rangle\bigg|_{(2, 1, 1)} = \left\langle 4, 2, -2 \right\rangle$. This gradient vector $\left\langle 4, 2, -2 \right\rangle$ is normal (perpendicular) to the surface at the point $(2, 1, 1)$.

3. Write the equation of the Tangent Plane. The equation of the tangent plane to the surface at $(x_0, y_0, z_0)$ can be written using the point-normal form of the plane equation: $\nabla w \cdot \langle x - x_0, y - y_0, z - z_0 \rangle = 0$. Substituting $\nabla w = \langle 4, 2, -2 \rangle$ and the point $(2, 1, 1)$, the equation becomes: $4(x - 2) + 2(y - 1) - 2(z - 1) = 0$.
   Simplifying the equation: $4x - 8 + 2y - 2 - 2z + 2 = 0 \iff [\text{Combining like terms:}] 4x + 2y - 2z - 8 = 0$. Thus, the equation of the tangent plane is: 4x + 2y - 2z = 8.

   The gradient vector of a function f(x,y,z) is a vector that points in the direction of the steepest increase of the function. It is perpendicular to the level surfaces of the function (w = f(x, y, z) = c).

   The tangent plane to a surface at a given point is the plane that just touches the surface at that point and is perpendicular to the gradient vector at that point (Figure 3). The equation of the tangent plane to the surface f(x, y, z) = c at a point $(x_0, y_0, z_0)$ can be written using the point-normal form of the plane equation. The gradient vector $\nabla f$ at $(x_0, y_0, z_0)$ serves as the normal vector to the tangent plane. Therefore, the equation is: $\nabla f(x_0, y_0, z_0) \cdot (x -x_0, y-y_0, z-z_0) = 0$.

4. Alternative Approach Using Differentials.
   For the function $w = x^2 + y^2 - z^2$, the differential dw is given by: $dw = \frac{\partial w}{\partial x} dx + \frac{\partial w}{\partial y} dy + \frac{\partial w}{\partial z} dz ↭ dw = 2x dx + 2y dy - 2z dz$. This formula tells you how w changes when you make tiny changes in x, y, and z.
   Evaluating at the point (2, 1, 1): $dw = 2(2) dx + 2(1)dy -2(1)dz, dw = 4dx + 2dy - 2dz$. This is the local linear behavior of the function at that point.
   For small changes near the point (2, 1, 1), we approximate: $\Delta w \approx 4 \Delta x + 2 \Delta y - 2 \Delta z$. 🎯 Near a point, the surface behaves almost linearly, and the differential dw gives the best linear approximation of how w changes.
   Since we are on the level surface w = 4, w stays constant, so: $\Delta w = 0$: $4 \Delta x + 2 \Delta y - 2 \Delta z = 0$
   This corresponds to the tangent plane equation: $4(x - 2) + 2(y - 1) - 2(z - 1) = 0 \iff 4x + 2y - 2z = 8$.

• You are climbing a hill described by the equation $z = 1000 - 0.01x^2 - 0.02y^2$, where x, y, and z are measured in meters. The positive x-axis points due east, and the positive y-axis points due north. You are at the point $(50, 80, 847)$ (Figure 2).

A. If you walk due south, will you start to ascend or descend? Determine the unit vector for south direction. South is the negative y-direction, $\vec{u} = \langle 0, -1 \rangle.$
Calculate the gradient of f: $\nabla f = \langle \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y} \rangle = \langle -0.02x, -0.04y \rangle$.
Evaluate the gradient at P(50, 80): $\nabla f(50, 80) = \langle -0.02(50), -0.04(80) \rangle = \langle -1, -3.2 \rangle$.
Calculate the directional derivative in the south direction: $D_{\vec{u}}f(50, 80) = \nabla f \cdot \vec{u} = \langle -1, -3.2 \rangle \cdot \langle 0, -1 \rangle = 3.2$. Since the directional derivative is positive, you will start to ascend at a rate of $3.2 \frac{\text{meters of elevation}}{\text{meters of distance}}$.

B. If you walk northwest, will you start to ascend or descend? Determine the unit vector for northwest direction. Northwest direction is equally negative in x and positive in y, the unit vector $\vec{u}$ is $\vec{u} = \langle -\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}} \rangle$.
Calculate the directional derivative in the northwest direction:$D_{\vec{u}}f(50, 80) = \langle -1, -3.2 \rangle \cdot \langle -\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}} \rangle = \frac{1}{\sqrt{2}} - \frac{3.2}{\sqrt{2}} = \frac{-2.2}{\sqrt{2}}$. Since the directional derivative is negative, you will start to descend at a rate of $\frac{-2.2}{\sqrt{2}} \frac{\text{meters of elevation}}{\text{meters of distance}}$.

C. In which direction is the slope largest? What is the rate of ascent in that direction? At what angle above the horizontal does the path in that direction begin?
Direction of the steepest ascent. The direction of the steepest ascent is the direction of the gradient $\nabla f = \langle -1, -3.2 \rangle$.
Rate of ascent. The rate of ascent is the magnitude of the gradient: $\vert{}\nabla f\vert{} = \sqrt{(-1)^2 + (-3.2)^2} = \sqrt{1 + 10.24} = \sqrt{11.24} \approx 3.35 \frac{\text{meters of elevation}}{\text{meters of distance}}$
Angle of ascent. The angle $\theta$ above the horizontal can be found using the tangent function: $\tan(\theta) = \frac{\text{rate of ascent}}{1} = 3.35 \implies \theta = \tan^{-1}(3.35) \approx 73.4^\circ$.

Bibliography

1. NPTEL-NOC IITM, Introduction to Galois Theory.
2. Algebra, Second Edition, by Michael Artin.
3. LibreTexts, Calculus and Calculus 3e (Apex). Abstract and Geometric Algebra, Abstract Algebra: Theory and Applications (Judson).
4. Field and Galois Theory, by Patrick Morandi. Springer.
5. Michael Penn, and MathMajor.
6. Contemporary Abstract Algebra, Joseph, A. Gallian.
7. YouTube’s Andrew Misseldine: Calculus. College Algebra and Abstract Algebra.
8. MIT OpenCourseWare 18.01 Single Variable Calculus, Fall 2007 and 18.02 Multivariable Calculus, Fall 2007.
9. Calculus Early Transcendentals: Differential & Multi-Variable Calculus for Social Sciences.