What is Non-Linear Regression? Deep Learning.

3 min readJan 11, 2025

What is Non-Linear Regression?

Non-Linear Regression is about finding a curved rule that connects two things (variables). This is different from Linear Regression, which only works for straight-line relationships.

Think of it as understanding relationships that don’t follow a straight path but still have a clear pattern.

Real-Life Analogy: Growth of a Plant

Imagine you’re growing a plant.

In the first week, it grows 1 cm.
In the second week, it grows 3 cm.
In the third week, it grows 9 cm.

The plant is growing faster and faster every week! It’s not a straight line anymore.

Here’s the pattern: Height = Growth Rate × (Time²)

If you know the time (weeks), you can use this curved rule to predict how tall the plant will be.

For example:

At 4 weeks, Height = 1 × (⁴²) = 16 cm.
At 5 weeks, Height = 1 × (⁵²) = 25 cm.

How Does It Work?

Data Points: Just like before, you start with what you know (e.g., plant height vs. time).
Curved Line: Non-Linear Regression draws a curved line through the data points.
Prediction: With this curved rule, you can predict values for the future.

The mathematical equation is more complex than the straight-line formula, but don’t worry — it’s just trying to fit a curve to the data!

Real-Life Example: Ice Cream Sales and Temperature

Imagine you’re selling ice cream.

At 10°C, you sell 50 cones.
At 20°C, you sell 150 cones.
At 30°C, you sell 300 cones.

Notice that as the temperature rises, your sales increase faster.

This is a non-linear relationship. A possible rule could be: Sales = 10 × (Temperature²)

At 25°C: Sales = 10 × (2⁵²) = 625 cones.
At 35°C: Sales = 10 × (3⁵²) = 1225 cones.

How Non-Linear Regression Looks on a Graph

The x-axis is the temperature.
The y-axis is the sales.
The points form a curve, not a straight line. Non-Linear Regression fits this curve to your data.

Why Non-Linear Regression is Important?

It helps you:

Model Complex Patterns: Relationships in real life aren’t always straight lines. For example, population growth, stock market trends, or even body temperature vs. heart rate.
Predict Accurately: It’s better suited for data that grows faster (or slower) over time.

How It’s Related to Deep Learning?

In Deep Learning, many models deal with non-linear relationships. Neural networks use activation functions like sigmoid or ReLU to capture these complex patterns, just like Non-Linear Regression handles curved relationships.

Practical Example with Python

Let’s predict plant height based on time.

# Step 1: Import libraries
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
# Step 2: Define data
time = np.array([1, 2, 3, 4, 5])  # Time in weeks
height = np.array([1, 3, 9, 16, 25])  # Plant height in cm
# Step 3: Define a non-linear function
def growth_curve(t, a, b):
    return a * (t ** b)
# Step 4: Fit the curve
params, _ = curve_fit(growth_curve, time, height)
a, b = params
# Step 5: Predict new values
new_time = np.array([6, 7, 8])  # Future weeks
predicted_height = growth_curve(new_time, a, b)
print(f"Predicted heights for weeks 6, 7, 8: {predicted_height}")
# Step 6: Plot the result
plt.scatter(time, height, label="Data", color="blue")
plt.plot(time, growth_curve(time, a, b), label="Curve Fit", color="red")
plt.xlabel("Time (weeks)")
plt.ylabel("Height (cm)")
plt.legend()
plt.show()

Output:

Predicted heights for weeks 6, 7, 8: [36. 49. 64.]

You’ll see a curve on the graph, perfectly matching the plant’s growth pattern.

Summary

Non-Linear Regression helps us model curved relationships in data. From plant growth to ice cream sales, many real-world patterns are non-linear. It’s an essential tool for understanding and predicting complex behaviors. 🌱 🍦