Nexus Learn

Lecture 24: Machine Learning: Supervised, Unsupervised and Reinforcement

Lecture Notes CS-511 Artificial Intelligence 5 min read Updated 8 Nov 2026

↓ Download notes (Word)

On this page
  1. Learning Outcomes
  2. 1. What Is Machine Learning?
  3. 2. Supervised and Unsupervised Learning
  4. 3. Reinforcement Learning
  5. 4. Common Mistakes
  6. Word Bank
  7. Self-Check Questions

Learning Outcomes

SLO

After this lecture you will be able to...

CLO

Bloom

SLO 24.1

Explain machine learning and differentiate supervised, unsupervised and reinforcement learning with examples

CLO3

C2 Understand

SLO 24.2

Apply k-nearest neighbours to labelled data and one k-means step to unlabelled data

CLO3

C3 Apply

SLO 24.3

Apply the Q-learning update rule to a small reinforcement learning problem

CLO3

C3 Apply

1. What Is Machine Learning?

§1.1 Traditional programming vs machine learning

Slide L24-S06

Traditional: a person writes the rules (L19 expert system, L23 fan rules). Machine learning (ML): we give data and answers, and the computer finds the rules - a model - which then answers new cases. Arthur Samuel (1959): 'the ability to learn without being explicitly programmed'.

Traditional: rules + data -> answers

Machine learning: data + answers -> rules (a model)

model + new data -> answers

Mitchell (1997): a program learns from experience E for task T, measured by P, if its performance on T (by P) improves with E. Spam filter: T = mark spam, E = emails users marked, P = % marked correctly.

§1.2 Three kinds of learning

Slide L24-S07

Supervised

Unsupervised

Reinforcement

Feedback

Correct answer (label)

None

Reward / penalty

Output

Prediction

Groups, structure

Policy (what to do)

Everyday picture

Solved past paper

Sorting mixed screws

Learning to ride a bicycle

Example

Pass/fail, wheat yield

Customer groups

Robot walking, AlphaGo

2. Supervised and Unsupervised Learning

Data used all lecture: 12 BSCS students (study hours per week, attendance %).

Result

Students (hours, attendance %)

fail

(2, 55) (3, 60) (4, 50) (5, 65) (3, 70) (6, 58)

pass

(8, 80) (9, 75) (7, 90) (10, 85) (12, 88) (9, 95)

§2.1 Supervised learning

Slide L24-S09

Each example has features (inputs) and a label (answer). Classification: the label is a category (pass/fail, digit). Regression: the label is a number (electricity units). Workflow: collect labelled data, split into training and test sets, train, test on unseen data, predict new cases. k-nearest neighbours (k-NN): find the k closest known examples (distance √(Δx² + Δy²)) and take a majority vote.

trained on 8 students, tested on 4

9 h, 75 % true pass predicted pass

6 h, 58 % true fail predicted fail

10 h, 85 % true pass predicted pass

5 h, 65 % true fail predicted fail

test accuracy = 1.00

new student 6 h, 72 % -> fail

new student 11 h, 60 % -> fail

Why is '11 h, 60 %' predicted fail? Attendance numbers (50-95) are much bigger than hours (2-12), so they dominate the distance. Feature scaling (StandardScaler, P04b) fixes this.

§2.2 Unsupervised learning

Slide L24-S10

group 0: 6 students

centre 9.2 h, 85.5 %

group 1: 6 students

centre 3.8 h, 59.7 %

labels:

1 1 1 1 1 1 0 0 0 0 0 0

k-means (P06a): choose k centres; repeat - assign each point to the nearest centre, move each centre to the mean of its points - until nothing changes. It finds groups but cannot name them; people give the groups a meaning. Uses: customer groups (Jazz, Zong), farms grouped by soil tests, unusual transactions (fraud).

§2.3 Links to the practicals

Slide L24-S11

Practical

What it learned

Kind

P02, P03

Perceptron / MLP classifier

Supervised (classification)

P04a, P04b

Curve, electricity units

Supervised (regression)

P05a, P05b

Handwritten digits

Supervised (classification)

P06a

Household groups (k-means)

Unsupervised

P06b

Tomorrow's electricity use

Supervised (regression)

P11a, P11b

ANFIS Sugeno system

Supervised (regression)

Overfitting: memorising the training data and failing on new data - always test on unseen data.

3. Reinforcement Learning

§3.1 The RL loop

Slide L24-S13

Word

Meaning

Delivery road

Agent

The learner

The rider

Environment

The world it acts in

The road

State s

The situation now

Cell 2

Action a

What it does

Left or right

Reward r

Number: + good, - bad

-1 per move, +10 at goal

Policy

Best action in each state

Always right

The aim is the largest total reward over time. Explore (try new actions) vs exploit (use the best known action).

§3.2 Q-learning

Slide L24-S14

Q(s, a) <- Q(s, a) + α · [ r + γ · max Q(s', a') - Q(s, a) ]

α = learning rate (step size); γ = discount (how much the future counts); the bracket is the 'surprise'. At the goal there is no future, so max Q(s', a') = 0.

Q = 0.0 + 0.5 x (-1 + 0.9 x 4.0 - 0.0) = 1.30

§3.3 Worked example: the delivery road

Slide L24-S15

steps per episode: 10 13 6 4 4 6 6 4 4 5 ... 4 5 4 4 7

learned Q table left right

cell 0 2.71 4.58

cell 1 0.95 6.20

cell 2 1.67 8.00

cell 3 3.45 10.00

policy: 0:right 1:right 2:right 3:right

The reward at the goal flows back one cell at a time: 8.00 = -1 + 0.9 x 10.00, 6.20 = -1 + 0.9 x 8.00. Uses: AlphaGo (2016), robots learning to walk, traffic-light timing.

4. Common Mistakes

Wrong idea

Correct idea

Unsupervised learning has a teacher.

No labels; it finds groups itself.

Clustering names the groups.

People name them.

100 % training accuracy = good model.

Measure on unseen test data.

RL is told the correct action.

It only gets rewards.

Regression means going backwards.

Regression predicts a number.

Word Bank

Word

Simple meaning

Example

Model

Rules learned from data

k-NN, MLP

Feature

An input value

study hours

Label

The correct answer of an example

pass

Classification

Predicting a category

spam / not spam

Regression

Predicting a number

units used

Clustering

Grouping without labels

k-means

Overfitting

Memorising instead of learning

100 % train, 60 % test

Reward

Score for an action

+10 at goal

Policy

What to do in each state

always right

Self-Check Questions

  1. Give Mitchell's T, E, P for a wheat-yield predictor. (see §1.1)
  2. Name the kind of learning: grouping cotton farms by soil tests. (see §1.2)
  3. Classification or regression: predicting a petrol price? (see §2.1)
  4. k-NN, k = 1: predict 4 h, 72 %. (see §2.1)
  5. Why can k-means not tell us which group is 'pass'? (see §2.2)
  6. Q = 1, r = -1, γ = 0.9, best next 8, α = 0.5: new Q? (see §3.2)
  7. Why is Q(cell 1, right) = 6.20? (see §3.3)

Reading: Kaplan Ch 2; Russell & Norvig Ch 19 (learning from examples) and Ch 22 (reinforcement learning); scikit-learn user guide.

← All CS-511 material