Supervised Learning
Supervised Learning: Meaning
Supervised Learning is a Machine Learning technique where a model learns from labelled data.
Labelled data means:
Input Data (X) → Correct Output / Label (Y)
Example:
The model learns the relationship between study hours and marks, then predicts marks for new study hours.
1. Basic Internal Working
Supervised Learning follows this flow:
Training Data
|
v
Input Features X + Output Labels Y
|
v
Model learns pattern
|
v
Prediction Ŷ
|
v
Compare Ŷ with actual Y
|
v
Calculate Error / Loss
|
v
Update model parameters
|
v
Better prediction
2. Types of Supervised Learning
A. Regression
Used when output is a continuous numeric value.
Examples:
Predict house price
Predict salary
Predict temperature
Predict student marks
Example:
Input: Study Hours
Output: Marks
B. Classification
Used when output is a category/class.
Examples:
Spam or Not Spam
Pass or Fail
Dog or Cat
Disease or No Disease
Example:
Input: Email text
Output: Spam / Not Spam
3. Mathematical Internal Working
Let us understand using Linear Regression, which is a basic supervised learning algorithm.
Suppose we want to predict student marks based on study hours.
Formula
y = mx + c
Where:
y = predicted output
x = input feature
m = slope / weight
c = intercept / bias
In Machine Learning notation:
ŷ = wx + b
Where:
ŷ = predicted value
w = weight
b = bias
x = input value
4. Example Data
The model has to learn this pattern:
More study hours → More marks
5. Prediction Step
Assume initial values:
w = 5
b = 10
For input:
x = 2
Prediction:
ŷ = wx + b
ŷ = 5 × 2 + 10
ŷ = 20
Actual output:
y = 40
So error is:
Error = y - ŷ
Error = 40 - 20
Error = 20
The model prediction is wrong, so it needs to improve w and b.
6. Loss Function
Loss function tells how wrong the model is.
For Linear Regression, commonly used loss is Mean Squared Error.
MSE = 1/n Σ(y - ŷ)²
Where:
n = number of data points
y = actual value
ŷ = predicted value
For one example:
Actual y = 40
Predicted ŷ = 20
Loss = (40 - 20)²
Loss = 20²
Loss = 400
The goal of training is:
Minimize Loss
7. Gradient Descent
Gradient Descent is used to update w and b.
It updates values step by step so that loss becomes smaller.
Formula:
w = w - learning_rate × dw
b = b - learning_rate × db
Where:
dw = derivative of loss with respect to w
db = derivative of loss with respect to b
For Linear Regression:
dw = (-2/n) Σ x(y - ŷ)
db = (-2/n) Σ(y - ŷ)
8. Complete Training Process
Step 1: Initialize w and b
Step 2: Predict output using ŷ = wx + b
Step 3: Calculate loss
Step 4: Calculate gradients dw and db
Step 5: Update w and b
Step 6: Repeat many times
This repeated process is called training.
9. Python Code Example: Linear Regression from Scratch
import numpy as np
# Training data
# x = study hours
# y = marks
X = np.array([1, 2, 3, 4, 5], dtype=float)
Y = np.array([30, 40, 50, 60, 70], dtype=float)
# Initialize weight and bias
w = 0.0
b = 0.0
# Hyperparameters
learning_rate = 0.01
epochs = 1000
n = len(X)
# Training loop
for epoch in range(epochs):
# Step 1: Prediction
Y_pred = w * X + b
# Step 2: Calculate loss
loss = np.mean((Y - Y_pred) ** 2)
# Step 3: Calculate gradients
dw = (-2 / n) * np.sum(X * (Y - Y_pred))
db = (-2 / n) * np.sum(Y - Y_pred)
# Step 4: Update parameters
w = w - learning_rate * dw
b = b - learning_rate * db
# Print loss after every 100 epochs
if epoch % 100 == 0:
print(f"Epoch: {epoch}, Loss: {loss:.4f}, w: {w:.4f}, b: {b:.4f}")
# Final model
print("\nFinal Weight:", w)
print("Final Bias:", b)
# Prediction for new value
study_hours = 6
predicted_marks = w * study_hours + b
print(f"\nPredicted marks for {study_hours} study hours: {predicted_marks:.2f}")
10. Expected Output Meaning
After training, the model learns values close to:
w ≈ 10
b ≈ 20
So the final equation becomes:
Marks = 10 × Study Hours + 20
For 6 hours:
Marks = 10 × 6 + 20
Marks = 80
So predicted marks will be around:
80
11. Python Code Using Scikit-Learn
This is the easier real-world way.
import numpy as np
from sklearn.linear_model import LinearRegression
# Input data must be 2D for scikit-learn
X = np.array([[1], [2], [3], [4], [5]])
# Output labels
Y = np.array([30, 40, 50, 60, 70])
# Create model
model = LinearRegression()
# Train model
model.fit(X, Y)
# Check learned weight and bias
print("Weight:", model.coef_[0])
print("Bias:", model.intercept_)
# Predict new value
prediction = model.predict([[6]])
print("Predicted marks for 6 study hours:", prediction[0])
Output will be approximately:
Weight: 10
Bias: 20
Predicted marks for 6 study hours: 80
12. Classification Example
Now let us take another supervised learning example.
We want to predict whether a student will Pass or Fail.
Here output is a class:
Fail = 0
Pass = 1
Python Code: Logistic Regression
import numpy as np
from sklearn.linear_model import LogisticRegression
# Training data
X = np.array([[1], [2], [3], [4], [5]])
# Labels: 0 = Fail, 1 = Pass
Y = np.array([0, 0, 1, 1, 1])
# Create classification model
model = LogisticRegression()
# Train model
model.fit(X, Y)
# Predict result for 2.5 study hours
prediction = model.predict([[2.5]])
if prediction[0] == 1:
print("Prediction: Pass")
else:
print("Prediction: Fail")
# Predict probability
probability = model.predict_proba([[2.5]])
print("Probability:", probability)
13. Difference Between Regression and Classification
14. Simple Diagram
Supervised Learning
Training Phase:
Input X + Correct Output Y
|
v
ML Algorithm
|
v
Learns Function f(x)
Prediction Phase:
New Input X
|
v
Learned Model f(x)
|
v
Predicted Output ŷ
15. Very Simple Explanation
Supervised Learning means:
Machine learns from examples where answers are already given.
Example:
Input: Study Hours
Answer: Marks
The machine studies many examples and learns a formula:
Marks = 10 × Study Hours + 20
Then for new input:
Study Hours = 6
It predicts:
Marks = 80
This is how supervised learning works internally.