|
 |
|
|
|
|
|
|
To err is human, to blame it on someone else is even more human, Jacob’s Law.
To understand how these surfaces change, we use partial derivatives. The partial derivative of $f$ with respect to $x$ measures how the function changes as $x$ shifts, while strictly locking $y$ as a constant.
Consider a function $f(x_1, x_2, \cdots, x_n)$ that depends on n variables. The partial derivative of f with respect to one of these variables, say $x_i$, is the derivative of f while holding the other variables constant.
For a function f(x,y), the partial derivative with respect to x at a point $(x_0, y_0)$ is defined as: $\frac{\partial f}{\partial x}(x_0, y_0) = \lim_{\Delta x \to 0} \frac{f(x_0+\Delta x, y_0)-f(x_0, y_0)}{\Delta x}$
Using a different notation, $\frac{\partial f}{\partial x}(x, y) = \lim_{h \to 0} \frac{f(x+h, y)-f(x, y)}{h}$
The tangent plane to the surface defined by the function $f(x,y)$ at the point $(x_0, y_0)$ is given by: $z \approx f(x_0, y_0) + \frac{\partial f}{\partial x}(x_0, y_0)(x -x_0)+ \frac{\partial f}{\partial y}(x_0, y_0)(y -y_0)$
This can be rewritten as: $\Delta z \approx \frac{\partial f}{\partial x}(x_0, y_0)(x -x_0)+ \frac{\partial f}{\partial y}(x_0, y_0)(y -y_0) =f_x\Delta x + f_y\Delta y$ where $\Delta z = z - f(x_0, y_0)$, $\Delta x = x - x_0$, and $\Delta y = y - y_0$.
This equation states that the graph of $f$ is close to its tangent plane in the vicinity of the point $(x_0, y_0)$. The tangent plane provides a highly accurate approximation of the surface near this anchor point.
Least squares interpolation is a method used to fit a curve to a set of data points in such a way that the sum of the squares of the differences between the observed and predicted values is minimized. This technique aims to find the best-fitting curve by minimizing the total squared error between the actual data points and the curve itself.
Let’s illustrate this by fitting a linear model to a set of data points $(x_i, y_i)$. We choose a linear model of the form $y = ax + b$ to represent the relationship between the independent and dependent variables, where $a$ is the slope and $b$ is the y-intercept.
Our objective is to minimize the total squared deviation between the observed values $y_i$ and the predicted value $ax_i + b$. The deviation for each data point is given by $y_i - (ax_i + b)$.
Therefore, our goal is to minimize the total square deviation or error function, i.e., $D(a, b) = \sum_{i=1}^n [y_i-(ax_i +b)]^2$. In other words, our aim is to find the values of a and b that minimize D(a, b).
To find the minimum of $D(a, b)$, we take the partial derivatives with respect to $a$ and $b$, and set them equal to zero:
$\frac{\partial D}{\partial a} = \sum_{i=1}^n 2(y_i-(ax_i +b))(-x_i)=0$
$\frac{\partial D}{\partial b} = \sum_{i=1}^n 2(y_i-(ax_i +b))(-1)=0$
By dividing by $-2$ and expanding the terms, we get the following system of equations:
$\begin{cases} \sum_{i=1}^n (y_i-(ax_i +b))(-x_i)=0 \\ \sum_{i=1}^n (y_i-(ax_i +b))(-1)=0 \end{cases}$
$\begin{cases} \sum_{i=1}^n (x_i^2a+x_ib-x_iy_i)= 0 \\ \sum_{i=1}^n (x_ia +b -y_i)=0 \end{cases}$
Rearranging these terms isolates our variables a and b, yielding the normal equations:
$\begin{cases} (\sum_{i=1}^n x_i^2)a + (\sum_{i=1}^nx_i)b = \sum_{i=1}^n x_iy_i \\ (\sum_{i=1}^n x_i)a + nb = \sum_{i=1}^n y_i \end{cases}$
This is a linear system of equations with two unknowns ($a$ and $b$). Solving this system provides the coefficients that minimize the total squared error.
Least squares is a method to find the best-fitting line to a given set of points. The line is defined by the equation y = ax +b. The goal is to minimize the sum of the squared differences between the observed values and the values predicted by the line.
We start with the normal equations derived from the least squares method: $\begin{cases} (\sum_{i=1}^n x_i^2)a + (\sum_{i=1}^nx_i)b = \sum_{i=1}^n x_iy_i \\ (\sum_{i=1}^n x_i)a + nb = \sum_{i=1}^n y_i \end{cases}$ where $n$ is the number of data points.
First, we calculate the required sums for our $n = 3$ data points: $\sum_{i=1}^n x_i = 0+2+3= 5, \sum_{i=1}^n y_i = 1+1+4 = 6, \sum_{i=1}^n x_i^2 = 0^2+2^2+3^2 = 0 + 4 + 9 = 13, \sum_{i=1}^n x_iy_i = 0 + 2 + 12 = 14$.
Plugging these into our normal equations forms the system: $\begin{cases} 13a + 5b = 14 \\\ 5a + 3b = 6 \end{cases}$
Next, we solve the system of equations:
If the data suggests a nonlinear trend, we can fit a quadratic function $y = ax^2 + bx + c$ using the same least squares principle.
Given some data points $(x_1, y_1), (x_2, y_2), \cdots, (x_n, y_n)$, we want to fit a quadratic function of the form $y = ax^2 + bx + c$ to these points using the least squares method. This means we want to minimize the sum of the squared differences between the observed $y_i$ values and the predicted y values given by the quadratic function.
First, we need to define our objective function. The sum of squared errors (SSE) is given by: $\sum_{i=1}^n (y_i- (ax_i^2+bx_i+c))^2.$ We want to find the values of a, b, and c that minimize this SSE.
As we have previously done, to minimize the SSE, we take the partial derivatives with respect to $a$, $b$, and $c$, and set them equal to zero.
$\frac{\partial}{\partial a}SSE = \sum_{i=1}^n 2(y_i - (ax_i^2 + bx_i + c))(-x_i^2) = 0$
Expanding and differentiating term by term: $\sum_{i=1}^n -x_i^2(y_i- (ax_i^2+bx_i+c)) = 0 \implies [\text{Simplify}] \sum_{i=1}^n x_i^2y_i = a\sum_{i=1}^nx_i^4 + b\sum_{i=1}^nx_i^3 + c\sum_{i=1}^nx_i^2$
$\frac{\partial}{\partial b}SSE = \sum_{i=1}^n 2(y_i - (ax_i^2 + bx_i + c))(-x_i) = 0$
Expanding and differentiating term by term: $\sum_{i=1}^n -x_i(y_i- (ax_i^2+bx_i+c)) = 0 \implies [\text{Simplify}] \sum_{i=1}^n x_iy_i = a\sum_{i=1}^nx_i^3 + b\sum_{i=1}^nx_i^2 + c\sum_{i=1}^nx_i$
$\frac{\partial}{\partial c}SSE = \sum_{i=1}^n 2(y_i - (ax_i^2 + bx_i + c))(-1) = 0$
Expanding and differentiating term by term: $\sum_{i=1}^n -(y_i- (ax_i^2+bx_i+c)) = 0 \implies [\text{Simplify}] \sum_{i=1}^n y_i = a\sum_{i=1}^nx_i^2 + b\sum_{i=1}^nx_i + c\sum_{i=1}^n1 = a\sum_{i=1}^nx_i^2 + b\sum_{i=1}^nx_i + cn$
The three equations derived from the partial derivatives are known as the normal equations for a quadratic fit. They are:
$\begin{cases} \sum_{i=1}^n x_i^2y_i = a\sum_{i=1}^nx_i^4 + b\sum_{i=1}^nx_i^3 + c\sum_{i=1}^nx_i^2 \\ \sum_{i=1}^n x_iy_i = a\sum_{i=1}^nx_i^3 + b\sum_{i=1}^nx_i^2 + c\sum_{i=1}^nx_i \\ \sum_{i=1}^n y_i = a\sum_{i=1}^nx_i^2 + b\sum_{i=1}^nx_i + cn \end{cases}$
For a better fit, especially if the data suggests a nonlinear trend, we can use a quadratic model: $y = ax^2 + bx + c.$ To find the coefficients a, b, and c, we minimize the function: $f(a, b, c) = \sum_{i=1}^3 (y_i- (ax_i^2+bx_i+c))^2.$
Let’s apply this to our data set. We calculate the required higher-order sums ($n = 3$): $\sum_{i=1}^3 x_i = 5, \sum_{i=1}^3 y_i = 6, \sum_{i=1}^3 x_i^2 = 0 + 4 +9 = 13, \sum_{i=1}^3 x_iy_i = 0 +2 + 12 = 14, \sum_{i=1}^3 x_i^3 = 0 + 8 + 27 = 35, \sum_{i=1}^3 x_i^4 = 0 + 16 + 81 = 97, \sum_{i=1}^3 x_i^2y_i = 0 + 4 + 36 = 40$
Plugging these into our normal equations yields a $3 \times 3$ system:
$\begin{cases} 40 = 97a + 35b + 13c \\ 14 = 35a + 13b + 5c \\ 6 = 13a + 5b + 3c \end{cases}$
The solution of this system is $a = 1$, $b = -2$, and $c = 1$. The quadratic function that best fits the data points (0, 1), (2, 1), and (3, 4) using the least square method is: $y = x^2 - 2x + 1$.
Let’s test both models against our original points $(0, 1)$, $(2, 1)$, and $(3, 4)$:
Linear Fit Output ($y = \frac{6}{7}x + \frac{4}{7}$):
For $x = 0$: $y = \frac{4}{7} \approx 0.571 \neq 1$
For $x = 2$: $y = \frac{12}{7} + \frac{4}{7} = \frac{16}{7} \approx 2.285 \neq 1$
For $x = 3$: $y = \frac{18}{7} + \frac{4}{7} = \frac{22}{7} \approx 3.142 \neq 4$
Quadratic Fit Output ($y = x^2 - 2x + 1$):
For $x = 0$: $y = 0^2 - 2(0) + 1 = 1 \implies (0, 1)$
For $x = 2$: $y = 2^2 - 2(2) + 1 = 1 \implies (2, 1)$
For $x = 3$: $y = 3^2 - 2(3) + 1 = 4 \implies (3, 4)$
The quadratic fit passes exactly through all the given points, so there is zero residual error. In conclusion, the quadratic fit perfectly matches all points, whereas the linear fit does not.
Moore's law is the observation (or the prediction) that the number of transistors in an integrated circuit (IC) doubles approximately every two years. This concept, first articulated by Intel co-founder Gordon Moore in 1965, has held remarkably true for several decades and is a key driver of technological advancement and computational scaling.
Because the transistor count doubles over a fixed period, the growth is exponential. This behavior can be mathematically represented by the function: $y = c·e^{ax}$ where:
Directly performing a least squares fit on an exponential curve is computationally difficult. However, we can transform this exponential relationship into a linear form. Taking the natural logarithm of both sides of the equation $y = c·e^{ax}$, we get: $\ln(y) = \ln(c \cdot e^{ax}) \implies \ln(y) = \ln(c) + \ln(e^{ax}) \implies \boxed{\ln(y) = \ln(c) + ax}$.
This algebraic manipulation reveals a highly useful property: $\ln(y)$ is a linear function of $x$.
In this linearized form, $a$ becomes the slope of the line, and $\ln(c)$ becomes the y-intercept. This allows us to use standard linear regression (least squares) on the transformed data to find the linear best fit.
To find the best straight-line fit for $\ln(y)$ over time, we use linear regression. Below is a Python script that applies this mathematical transformation to historical microprocessor data.
# Use Online Python. Then, Install Module: add numpy and matplotlib libraries.
import numpy as np
import matplotlib.pyplot as plt
import math
# Historical milestone data (Year vs. Transistor Count)
years = np.array([1971, 1974, 1978, 1982, 1989, 1993, 1997, 2000, 2004, 2006, 2008, 2012, 2016, 2018, 2020])
transistors = np.array([2300, 6000, 29000, 134000, 1180000, 3100000, 7500000, 42000000, 125000000, 291000000, 731000000, 2600000000, 7200000000, 18000000000, 57000000000])
# 1. Transform data to linearize the exponential growth: ln(y)
log_transistors = np.log(transistors)
# 2. Perform linear regression (Least squares polynomial fit of degree 1)
# Returns the slope (a) and intercept (ln(c))
slope, intercept = np.polyfit(years, log_transistors, 1)
# 3. Compute predicted values using the linear model
predicted_log_transistors = slope * years + intercept
# 4. Transform predictions back to exponential scale for plotting
predicted_transistors = np.exp(predicted_log_transistors)
# 5. Extract insights
# Note ⏱ Doubling time T = ln(2) / a
doubling_time = np.log(2) / slope
print(f"Calculated Growth Rate (slope): {slope:.4f}")
print(f"Historical Doubling Time: {doubling_time:.2f} years")
# 6. Plotting the data and the fit
plt.figure(figsize=(10, 6))
# Plot actual data on a logarithmic Y-axis
plt.scatter(years, transistors, label='Observed CPU Data', color='#ff79c6', zorder=5)
# Plot the best-fit exponential curve
plt.plot(years, predicted_transistors, label=f'Best Fit (Doubles every {doubling_time:.1f} yrs)', color='#8be9fd', linewidth=2)
plt.yscale('log')
plt.xlabel('Year', fontsize=12)
plt.ylabel('Number of Transistors (Log Scale)', fontsize=12)
plt.title("Moore's Law: Transistor Count Over Time", fontsize=14, fontweight='bold')
plt.legend(loc='upper left')
plt.grid(True, which="both", ls="--", alpha=0.3)
plt.tight_layout()
⏱ Doubling time derivation: $2 = e^{aT} \implies T = \frac{\ln(2)}{a}$. This is the standard formula for exponential doubling time. This makes the exponential trend appear linear, which visually confirms the regression.
JustToThePoint Copyright © 2011 - 2026 Anawim. ALL RIGHTS RESERVED. Bilingual e-books, articles, and videos to help your child and your entire family succeed, develop a healthy lifestyle, and have a lot of fun. Social Issues, Join us.
This website uses cookies to improve your navigation experience.
By continuing, you are consenting to our
use of cookies, in accordance with our Cookies Policy and
Website Terms and Conditions of use.