Directional derivatives

It ain’t about how hard you can hit. Its about how hard you can get hit, and how much you can take, and keep moving forward, Rocky Balboa.

And yet despite the look on my face, you’re still talking and thinking that I care, Anonymous.

Functions, Partials, and Gradients

• A function of two variables $f: \mathbb{R} \times \mathbb{R} \to \mathbb{R}$ assigns to each ordered pair in its domain a unique real number. Examples include geometric formulas like Area = $\frac{1}{2}bh$, or algebraic surfaces like $z = f(x, y) = 2x + 3y$, $f(x, y) = x^2 + y^2$, and $f(x,y) = e^{x+y}$.

• Partial derivatives differentiate a multivariable function $f(x_1, x_2, \dots, x_n)$ with respect to one target variable while treating all other variables as constants. They measure how the function changes in a single specific direction.
  If $f(x, y) = 2x^2y^3$, then $\frac{\partial f}{\partial x} = f_x = 4xy^3$ and $\frac{\partial f}{\partial y} = f_y = 6x^2y^2$.
  If $f(x, y) = x^3y + y^2$, then $f_x = 3x^2y$ and $f_y = x^3+2y$.

• A point $(x_1, x_2, \dots, x_n)$ is considered a critical point if the gradient of $f$ at that point is the zero vector, or if the gradient does not exist.

Gradients and Level Surfaces

• The gradient $\nabla f$ is a vector containing all first-order partial derivatives of $f$. It geometrically points in the direction of the function’s steepest ascent. $\nabla f = \left\langle \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \dots, \frac{\partial f}{\partial x_n} \right\rangle$.

• In 2D space ($n = 2$), finding a critical point $(x_0, y_0)$ requires solving the system: $\frac{\partial f}{\partial x}(x_0, y_0) = 0$ and $\frac{\partial f}{\partial y}(x_0, y_0) = 0$.

• Theorem: The gradient vector is perpendicular (orthogonal) to the level curves or level surfaces of the function at any given point. This means if you walk along a contour line where the function’s value is constant, the direction of steepest uphill is exactly 90 degrees to your path.

Directional derivatives

In single-variable calculus, the derivative measures the rate of change of a function with respect to one variable. For a function of two variables, $\frac{\partial f}{\partial x}$ and $\frac{\partial f}{\partial y}$ measure the rate of change moving strictly parallel to the $x$-axis or $y$-axis.

However, in many situations, we are interested in how the function changes in an arbitrary direction, not just along the coordinate axes.

A directional derivative represents how a function $w = f(x, y)$ changes as you move in a specific direction from a point in its domain.

The Geometric Intuition

Imagine standing on the side of a mountain ($f(x, y)$) 🧭.

• Facing East ($x$-axis) and walking one step tells you the partial derivative $f_x$.
• Facing North ($y$-axis) and walking one step tells you the partial derivative $f_y$.
• Facing North-East and walking one step tells you the directional derivative in that specific direction.

Geometrically, computing a directional derivative is like slicing the 3D surface with a vertical plane oriented in your chosen direction $\vec{u}$. The directional derivative is the slope of the resulting 2D curve at that point.

Computing the Directional Derivative

Given a function $f: \mathbb{R}^n \to \mathbb{R}$ and a point $P(x_0, y_0)$ in its domain, the directional derivative of $f$ at $P$ in the direction of a unit vector $\vec{u} = \left\langle a, b \right\rangle$ (a vector in $\mathbb{R}^n$ that has a length of 1) can be found as follows.

1. Suppose we want to move linearly in the direction of $\vec{u}$. If $\vec{u} = \left\langle a, b \right\rangle$, then we can describe the movement in the direction of $\vec{u}$ using the position vector $\vec{r}(s) = \left\langle x_0 +sa, y_0 + sb \right\rangle$, where s is a parameter representing how far we move in that direction.
2. Normalize the direction vector: The formula requires a unit vector $\vec{u}$ (length of 1). If given $\vec{v}$, calculate $\vec{u} = \frac{\vec{v}}{\vert{}\vec{v}\vert{}} = \langle a, b \rangle$.
3. Compute the Gradient of the function. The gradient of w, denoted by $\nabla f$, is a vector that contains all the partial derivatives of w. For a function w(x, y), it is given by: $\nabla f = \langle f_x, f_y \rangle$.
4. Directional Derivative Formula. The directional derivative in the direction of $\vec{u}$ is given by the dot product of the gradient $\nabla f$ and the unit vector $\vec{u}$: $\frac{dw}{ds}\bigg|_{\vec{u}} = D_{\vec{u}}f =\nabla f \cdot \vec{u}$, that is, the component of gradient $\nabla f$ in the direction of $\vec{u}$.

   $\nabla f \cdot \hat{\mathbf{i}} = f_x$ represents the rate of change of the function f(x, y) as we vary x and hold y fixed. $\nabla f \cdot \hat{\mathbf{j}} = f_y$ represents the rate of change of the function f(x, y) as we vary y and hold x fixed. This enables us to calculate the directional derivative in an arbitrary direction, by taking the dot product of $\nabla f$ with a unit vector, $\vec{u}$, in the desired direction.

Imagine a path in space: $\vec{r(s)} =$ position as a function of arc length $s$, $\frac{d\vec{r}}{ds}=\vec{u}$, meaning:
(1) You are walking in the direction $\vec{u}$.
(2) You measure distance along your path using arc length $s$.
(3) At each point, your instantaneous direction of motion is exactly $\frac{d\vec{r}}{ds}=\vec{u}$.

If your motion is constrained to follow the direction $\vec{u}$ (Figures A and B), then $\frac{dw}{ds}\bigg|_{\vec{u}}= D_{\vec{u}}f$.
🎯 This is the directional derivative in the direction of ${\vec{u}}$. It is the rate of change of the function w as we move in the direction of $\vec{u}$. Geometrically, it is the slope of the tangent line obtained by slicing the graph with a vertical plane parallel to ${\vec{u}}$.

Because the dot product of two vectors $\vec{A} \cdot \vec{B} = \vert{}\vec{A}\vert{}\vert{}\vec{B}\vert{}\cos(\theta)$, we can rewrite the directional derivative as: $D_{\vec{u}}f = \vert{}\nabla f\vert{} \vert{}\vec{u}\vert{} \cos(\theta) = \vert{}\nabla f\vert{} \cos(\theta)$ where $\theta$ is the angle between the gradient and the given unit vector.

This geometric interpretation reveals three crucial facts:

• Maximum Rate of Change: $D_{\vec{u}}f$ is maximized when $\cos(\theta) = 1$ (meaning $\theta = 0^\circ$). The function increases fastest when you move in the exact same direction as the gradient, $\vec{u}=dir(\nabla f).$
  The gradient is the direction in which the function increases fastest (the direction of the steepest ascent) at a given point, and $\vert \nabla f \vert = \frac{dw}{ds}\bigg|_{\vec{u}=dir(\nabla f)}$.
• Minimum Rate of Change: $D_{\vec{u}}f$ is minimized when $\cos(\theta) = -1$ (meaning $\theta = 180^\circ$). The function decreases fastest in the exact opposite direction of the gradient.
• Zero Rate of Change: $D_{\vec{u}}f = 0$ when $\cos(\theta) = 0$ (meaning $\theta = 90^\circ$). Moving perpendicular to the gradient (direction of maximum change) means you are walking along a surface level or contour line where elevation does not change.

Formal Definition and Proof

Definition. Let $z = f(x, y)$ be a function of two variables with domain $D$. Let $(x_0, y_0) \in D$ and define the unit vector $\vec{u} = \langle \cos(\theta), \sin(\theta) \rangle$. The directional derivative of $f$ in the direction of $\vec{u}$ is defined as: $D_{\vec{u}}f(x_0, y_0) = \lim_{h \to 0} \frac{f(x_0 + h\cos(\theta), y_0 + h\sin(\theta)) - f(x_0, y_0)}{h}$ provided the limit exists.

This is just the ordinary derivative of $f$ along a straight-line path pointing in the direction of $\vec{u}$.

Theorem (Computation via Partials): If the partial derivatives $f_x$ and $f_y$ exist, then the directional derivative of f in the direction of $\vec{u}$ is given by $D_{\vec{u}}f(x_0, y_0) = f_x(x_0, y_0)\cos(\theta) + f_y(x_0, y_0)\sin(\theta)$

🧱 Proof.

1. Parameterize the straight-line motion in the direction of $\vec{u}$: Define a line passing through $(x_0, y_0)$ in the direction of $\vec{u} = \langle \cos(\theta), \sin(\theta) \rangle$:
   $x(t) = x_0 + t\cos(\theta) \implies[\text{Differentiating}] \frac{dx}{dt} = \cos(\theta)$ and $y(t) = y_0 + t\sin(\theta) \implies[\text{Differentiating}] \frac{dy}{dt} = \sin(\theta)$
   🎯 We are slicing the surface $z = f(x, y)$ with a vertical plane parallel to $\vec{u}$ and walking along the intersection curve.
2. Define the single-variable function along the path: Let $g(t) = f(x(t), y(t))$. This is the height of the surface restricted to the line.
   The directional derivative is exactly the slope of this restricted function at $t = 0$: $D_{\vec{u}}f(x_0,y_0)=g'(0).$
   This matches the limit definition: $g'(0)=\lim_{h\to 0}\frac{f(x_0+h\cos\theta, y_0+h\sin\theta)-f(x_0,y_0)}{h}.$
   So the limit definition and the derivative of 𝑔 are the same object.
3. Apply the Multivariable Chain Rule:
   $g'(t) = \frac{\partial f}{\partial x}\frac{dx}{dt} + \frac{\partial f}{\partial y}\frac{dy}{dt}$
   Substitute the derivatives of x(t) and y(t): $g'(t) = f_x(x, y)\cos(\theta) + f_y(x, y)\sin(\theta)$
4. Evaluate at $t=0$: At $t=0$, the point on the path is $x = x_0$ and $y = y_0$.
   Thus, $g'(0) = D_{\vec{u}}f(x_0, y_0) = f_x(x_0, y_0)\cos(\theta) + f_y(x_0, y_0)\sin(\theta)$.
5. Finally, recognize the dot product: $D_{\vec{u}}f(x_0,y_0) = \langle f_x(x_0,y_0), f_y(x_0,y_0)\rangle \cdot \langle \cos\theta, \sin\theta\rangle = \nabla f(x_0,y_0)\cdot \vec{u}$.

🌄 Geometric meaning

1. The graph $z = f(x, y)$ is a surface.
2. The direction $\vec{u}$ determines a vertical slicing plane.
3. That plane cuts the surface in a curve.
4. The directional derivative is the slope of the tangent line to that curve at $(x_0, y_0, f(x_0, y_0))$. This is the exact analogue of partial derivatives, except the slicing plane is rotated to match $\vec{u}$.

Solved exercises

• Verification of coordinate directions.

The directional derivative of $w = f(x,y,z)$ in the direction of the unit vector $\hat{\mathbf{i}} = \langle 1, 0, 0 \rangle$ is: $D_{\hat{\mathbf{i}}}w = \nabla w \cdot \hat{\mathbf{i}} = \langle f_x, f_y, f_z \rangle \cdot \langle 1, 0, 0 \rangle = f_x$.

This confirms that the standard partial derivative $f_x$ is just a specific case of the directional derivative where the chosen direction is strictly along the $x$-axis.

• Evaluate a 2D directional derivative.

Find the directional derivative of $f(x, y) = 2 - x^2 - y^2$ at $\left(\frac{1}{2}, -\frac{1}{2}\right)$ in the direction of $\vec{u} = \left\langle \frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}} \right\rangle$.

1. First, calculate the gradient: $\nabla f = \left\langle \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y} \right\rangle = \left\langle -2x, -2y \right\rangle$.
2. Evaluate the gradient at $(\frac{1}{2}, \frac{-1}{2})$: $\nabla f\left(\frac{1}{2}, -\frac{1}{2}\right) = \langle -1, 1 \rangle$.
3. Calculate the dot product: $D_{\vec{u}}f = \langle -1, 1 \rangle \cdot \left\langle \frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}} \right\rangle = \left(-1 \cdot \frac{1}{\sqrt{2}}\right) + \left(1 \cdot \frac{1}{\sqrt{2}}\right) = 0$.
4. Interpretation. This result 0 indicates that at the point $(\frac{1}{2}, \frac{-1}{2})$, the function does not change as we move in the direction of $\vec{u}$. This implies that we are traveling along the surface’s level curve (contour line) at that exact instant.

• Evaluate a 3D directional derivative with normalization. Find the directional derivative of $w = x^2z + y^3z^2 - xyz$ in the direction of $\vec{v} = \langle -1, 0, 3 \rangle$.

1. First, we normalize $\vec{v}$ to get a unit vector $\vec{u}.$ The magnitude is $\vert{}\vec{v}\vert{} = \sqrt{(-1)^2 + 0^2 + 3^2} = \sqrt{10}$.
   The unit vector is $\vec{u} = \frac{1}{\sqrt{10}}\langle -1, 0, 3 \rangle$.
2. Calculate the Gradient: $w_x = 2xz - yz$, $w_y = 3y^2z^2 - xz$, and $w_z = x^2 + 2y^3z - xy$.
   Thus, $\nabla w = \langle 2xz - yz, 3y^2z^2 - xz, x^2 + 2y^3z - xy \rangle$
3. The directional derivative is $D_{\vec{u}}w = \nabla w \cdot \vec{u} = \langle 2xz - yz, 3y^2z^2 - xz, x^2 + 2y^3z - xy \rangle \cdot \left\langle \frac{-1}{\sqrt{10}}, 0, \frac{3}{\sqrt{10}} \right\rangle$
   $D_{\vec{u}}w = \frac{1}{\sqrt{10}} \left[ -1(2xz - yz) + 0 + 3(x^2 + 2y^3z - xy) \right]$
   $D_{\vec{u}}w = \frac{1}{\sqrt{10}} (-2xz + yz + 3x^2 + 6y^3z - 3xy)$. This expression gives the rate of change of w in the direction of $\vec{u}$.

• Evaluating a Directional Derivative with an Angle

Calculate the directional derivative $D_{\vec{u}}f(x, y)$ of the function $f(x, y) = 3x^2y - 4xy^3 + 3y^2 - 4x$ at the point $(3,4)$ in the direction of $\vec{u} = \cos(\frac{\pi}{3})\hat{\mathbf{i}} + \sin(\frac{\pi}{3})\hat{\mathbf{j}}$.

1. Recall. The directional derivative of f at a point (x, y) in the direction of a unit vector $\vec{u} = \cos(\theta)\hat{\mathbf{i}}+\sin(\theta)\hat{\mathbf{j}}$ is given by $D_{\vec{u}}f(a, b)=f_x(a, b)\cos(\theta) + f_y(a, b)\sin(\theta)$ where $f_x$ and $f_y$ are the partial derivatives of f with respect to x and y, respectively.
2. Identify the unit vector $\vec{u}$.
   The direction is given by the angle $\theta = \frac{\pi}{3}$, so $\vec{u} = \langle \cos(\frac{\pi}{3}), \sin(\frac{\pi}{3}) \rangle = \left\langle \frac{1}{2}, \frac{\sqrt{3}}{2} \right\rangle$
3. We need to find the partial derivatives $f_x$ and $f_y$.
   $f_x(x, y) = 6xy - 4y^3 - 4$ and $f_y(x, y) = 3x^2 - 12xy^2 + 6y$
4. Evaluate the gradient at the point $(3, 4)$. $f_x(3, 4) = 6(3)(4) - 4(4^3) - 4 = 72 - 256 - 4 = -188$, $f_y(3, 4) = 3(3^2) - 12(3)(4^2) + 6(4) = 27 - 576 + 24 = -525$.
   Thus, $\nabla f(3, 4) = \langle -188, -525 \rangle$
5. Calculate the dot product $\nabla f \cdot \vec{u}$.
   $D_{\vec{u}}f(3, 4) = -188\left(\frac{1}{2}\right) + (-525)\left(\frac{\sqrt{3}}{2}\right) = -94 - \frac{525\sqrt{3}}{2}$

• Calculate the directional derivative $D_{\vec{u}}f(x, y)$ of the function $f(x, y) = xe^{xy} + y$ at the point $(2, 0)$ where $\vec{u}$ is the unit vector in the direction of $\theta = \frac{2\pi}{3}$.

1. Identify the unit vector $\vec{u}$.
   $\vec{u} = \langle \cos(\frac{2\pi}{3}), \sin(\frac{2\pi}{3}) \rangle = \left\langle -\frac{1}{2}, \frac{\sqrt{3}}{2} \right\rangle$
2. Calculate the partial derivatives $f_x$ and $f_y$.
   Applying the product rule for $f_x$: $f_x(x, y) = (1)e^{xy} + x(ye^{xy}) = e^{xy} + xye^{xy}$
   Similarly, $f_y(x, y) = x(xe^{xy}) + 1 = x^2e^{xy} + 1$
3. Evaluate the gradient at the point $(2, 0)$.
   $f_x(2, 0) = e^0 + (2)(0)e^0 = 1 + 0 = 1$ and $f_y(2, 0) = (2^2)e^0 + 1 = 4 + 1 = 5$.
   Thus, $\nabla f(2, 0) = \langle 1, 5 \rangle$.
4. Calculate the directional derivative. $D_{\vec{u}}f(2, 0) = 1\left(-\frac{1}{2}\right) + 5\left(\frac{\sqrt{3}}{2}\right) = \frac{5\sqrt{3} - 1}{2}$

• Prove that the directional derivative $D_{\vec{u}}f(x, y)$ in the direction of the unit vector $\vec{u} = \langle a, b \rangle$ is $f_x a + f_y b$.

1. Setup: To find how $f(x,y)$ changes as we move from a starting point $(x_0, y_0)$ in the direction of $\vec{u} = \langle a, b \rangle$, we parameterize a line using a single variable $t$: $x(t) = x_0 + at$ and $y(t) = y_0 + bt$. Here $(x_0, y_0)$ is the point from which we start moving, and $\vec{u} = \langle a, b \rangle$ is the direction of movement (a unit vector).
2. The Single-Variable Function: Define a new function $g(t)$ that represents the value of $f$ along (or restricted to) this line: $g(t) = f(x(t), y(t)) = f(x_0 + at, y_0 + bt)$
3. The Limit Definition: The derivative of $g(t)$ at $t=0$ exactly mirrors the limit definition of the directional derivative: $g'(0) = \lim_{h \to 0}\frac{g(h)-g(0)}{h} = \lim_{h \to 0}\frac{f(x_0 + ah, y_0 + bh)-f(x_0, y_0)}{h} = D_{\vec{u}}f(x_0, y_0)$. \
4. Applying the Chain Rule: Differentiating $g(t)$ with respect to $t$ yields: $\frac{dg}{dt} = \frac{\partial f}{\partial x}\frac{dx}{dt} + \frac{\partial f}{\partial y}\frac{dy}{dt}$
   Since $\frac{dx}{dt} = a$ and $\frac{dy}{dt} = b$, this becomes $f_x(x,y)a + f_y(x,y)b$.
5. Conclusion: Evaluating at $t=0$ (which corresponds to the original point $(x_0, y_0)$): $D_{\vec{u}}f(x_0, y_0) = g'(0) = f_x(x_0, y_0)a + f_y(x_0, y_0)b$.

• Calculate the directional derivative of $f(x, y) = x\cos(y)$ in the direction of $\vec{v} = \langle 2, 1 \rangle$. In which direction does the function increase fastest, and what is that rate?

1. Normalize the direction vector $\vec{v}$.
   The magnitude is calculated as follows $\vert{}\vec{v}\vert{} = \sqrt{2^2 + 1^2} = \sqrt{5}$.
   The unit vector is $\vec{u} = \left\langle \frac{2}{\sqrt{5}}, \frac{1}{\sqrt{5}} \right\rangle$.
2. Find the partial derivatives of $f$.
   $f_x(x, y) = \cos(y)$ and $f_y(x, y) = -x\sin(y)$.
3. Calculate the directional derivative.
   $D_{\vec{u}}f(x, y) = f_x(x, y)a + f_y(x, y)b =[\text{Substitute the partial derivatives}] = \cos(y)\left(\frac{2}{\sqrt{5}}\right) - x\sin(y)\left(\frac{1}{\sqrt{5}}\right) = \frac{1}{\sqrt{5}}(2\cos(y) - x\sin(y))$
4. Maximum Rate of Increase.
   The function increases fastest in the exact direction of the gradient vector: $\nabla f = \langle \cos(y), -x\sin(y) \rangle$.
   The maximum rate of increase is the magnitude of the gradient: $\vert{}\nabla f\vert{} = \sqrt{\cos^2(y) + x^2\sin^2(y)}$.

• The temperature at a point $(x, y, z)$ is $T(x, y, z) = 2xy - yz$ in $^{\circ}\text{C}$, with coordinates in meters.

1. Find the rate of change in temperature at the point P(2, -1, 2) in the direction towards the point Q(3, -3, 3).
2. Calculate in which direction the temperature increase fastest at P? What is this rate of increase?
3. Determine if there is a direction at point P where the rate of change in temperature is $-5^{\circ}\text{C/m}$. If so, find the direction; if not, explain why.

Solution

1. Directional Derivative towards $Q$
   Find the directional vector $\vec{PQ}$: $\langle 3-2, -3-(-1), 3-2 \rangle = \langle 1, -2, 1 \rangle$.
   Normalize $\vec{PQ}$ to get unit vector $\vec{u}$: $\vert{}\vec{PQ}\vert{} = \sqrt{1^2 + (-2)^2 + 1^2} = \sqrt{6}$.
   $\vec{u} = \frac{1}{\sqrt{6}}\langle 1, -2, 1 \rangle$.
   Calculate gradient of the temperature function $\nabla T = \langle T_x, T_y, T_z \rangle = \langle 2y, 2x - z, -y \rangle$.
   Evaluate gradient at $P(2, -1, 2)$: $\nabla T(2, -1, 2) = \langle -2, 4-2, 1 \rangle = \langle -2, 2, 1 \rangle$.
   The directional derivative at P in the direction of $\vec{u}$ is given by: $D_{\vec{u}}T(2, -1, 2) = \langle -2, 2, 1 \rangle \cdot \frac{1}{\sqrt{6}}\langle 1, -2, 1 \rangle = \frac{1}{\sqrt{6}}(-2 - 4 + 1) = -\frac{5}{\sqrt{6}}$.
   So, the rate of change of temperature at P(2, -1, 2) in the direction towards the point Q(3, -3, 3) is $\frac{-5}{\sqrt{6}}\frac{°C}{m}$.

2. Maximum Rate of Increase
   The temperature increases fastest when the unit vector is parallel to the gradient vector. In other words, it changes most rapidly in the direction of the gradient vector: $\langle -2, 2, 1 \rangle$.
   The maximum rate of increase is the magnitude of the gradient: $\vert{}\nabla T\vert{} = \sqrt{(-2)^2 + 2^2 + 1^2} = \sqrt{9} = 3$.
   This means the maximum rate of temperature increase is $\mathbf{3 ^{\circ}\text{C/m}}$

3. Evaluating the $-5^{\circ}\text{C/m}$ claim
   The rate of change is bounded by $-\vert{}\nabla T\vert{} \le D_{\vec{u}}T \le \vert{}\nabla T\vert{}$.
   Because the maximum possible rate of increase is $3 ^{\circ}\text{C/m}$, the maximum possible rate of decrease is $-3 ^{\circ}\text{C/m}$ (occurring when moving in the exact opposite direction of the gradient).
   Since the maximum possible rate of decrease is $-3 ^{\circ}\text{C/m}$, it is impossible to have a rate of change of $-5 ^{\circ}\text{C/m}$.

Bibliography

1. NPTEL-NOC IITM, Introduction to Galois Theory.
2. Algebra, Second Edition, by Michael Artin.
3. LibreTexts, Calculus and Calculus 3e (Apex). Abstract and Geometric Algebra, Abstract Algebra: Theory and Applications (Judson).
4. Field and Galois Theory, by Patrick Morandi. Springer.
5. Michael Penn, and MathMajor.
6. Contemporary Abstract Algebra, Joseph, A. Gallian.
7. YouTube’s Andrew Misseldine: Calculus. College Algebra and Abstract Algebra.
8. MIT OpenCourseWare 18.01 Single Variable Calculus, Fall 2007 and 18.02 Multivariable Calculus, Fall 2007.
9. Calculus Early Transcendentals: Differential & Multi-Variable Calculus for Social Sciences.