Nexus Learn

Lecture 26: Neural Networks and the Perceptron

Lecture Notes CS-511 Artificial Intelligence 7 min read Updated 8 Nov 2026

↓ Download notes (Word)

On this page
  1. Learning Outcomes
  2. 1. From the Brain to an Artificial Neuron
  3. 2. The Perceptron Learning Rule
  4. 3. What One Perceptron Cannot Do
  5. 4. Common Mistakes
  6. Word Bank
  7. Self-Check Questions

Learning Outcomes

SLO

After this lecture you will be able to...

CLO

Bloom

SLO 26.1

Explain the parts of an artificial neuron and compute its output with a step activation

CLO3

C2 Understand

SLO 26.2

Apply the perceptron learning rule to train a neuron on a small data set (AND, OR)

CLO3

C3 Apply

SLO 26.3

Explain linear separability and why a single perceptron cannot learn XOR

CLO3

C2 Understand

1. From the Brain to an Artificial Neuron

§1.1 Biological and artificial neurons

Slide L26-S06

A biological neuron receives signals through its dendrites, adds them in the cell body and fires along its axon if the total passes a threshold; synapses (connections) can be strong or weak. McCulloch and Pitts (1943) turned this into a simple mathematical neuron; Frank Rosenblatt (1958) added a way to learn the weights - the perceptron.

Brain

Artificial neuron

Meaning

Dendrites

Inputs x1, x2, ...

The facts the neuron sees

Synapse strength

Weights w1, w2, ...

How important each input is (sign = direction)

Cell body adds

z = w·x + b

Weighted sum plus bias

Fires or not

Step: y = 1 if z >= 0 else 0

The decision

The bias b moves the cut-off: a negative bias makes the neuron harder to fire. It is a learned weight on a constant input 1.

§1.2 One neuron by hand

Slide L26-S07

Should a farmer run the tubewell tonight? Inputs are 0/1: soil is dry, hot day, rain forecast. Rain argues against pumping, so its weight is negative.

soil is dry (0/1) x = 1 w = +0.6 w*x = +0.6

hot day (0/1) x = 1 w = +0.4 w*x = +0.4

rain forecast (0/1) x = 0 w = -0.9 w*x = +0.0

bias b = -0.5

z = w.x + b = +0.5 -> step(z) = 1 (1 = run the pump)

same day with rain forecast: z = -0.4 -> 0

2. The Perceptron Learning Rule

§2.1 The rule

Slide L26-S09

error = t - y (t = target, y = output; error is 0, +1 or -1)

w <- w + lr · error · x

b <- b + lr · error

t

y

t - y

What happens

1

1

0

Nothing

0

0

0

Nothing

1

0

+1

Add lr·x to w and lr to b (raise z)

0

1

-1

Subtract lr·x from w and lr from b (lower z)

Learning rate (lr): the step size. Epoch: one pass over all training examples. Stop when a whole epoch has no mistakes. Convergence theorem: if one straight line can separate the classes, the rule always finds one in a finite number of steps.

§2.2 Worked example: AND

Slide L26-S10

Learning rate 1, start w = (0, 0), b = 0, examples in the order (0,0), (0,1), (1,0), (1,1). Epoch 1 is shown in full, then the weights at the end of each epoch.

x=(0, 0) t=0 y=1 -> w=(+0, +0) b=-1

x=(0, 1) t=0 y=0 -> w=(+0, +0) b=-1

x=(1, 0) t=0 y=0 -> w=(+0, +0) b=-1

x=(1, 1) t=1 y=0 -> w=(+1, +1) b=+0

epoch 1: 2 mistakes w = (1, 1), b = 0

epoch 2: 3 mistakes w = (2, 1), b = -1

epoch 3: 3 mistakes w = (2, 1), b = -2

epoch 4: 2 mistakes w = (2, 2), b = -2

epoch 5: 1 mistake w = (2, 1), b = -3

epoch 6: 0 mistakes w = (2, 1), b = -3

learned: w = (2, 1), b = -3 after 6 epochs

Epoch 1, first example: (0, 0) gives z = 0, so y = 1 but t = 0. Error -1, so b becomes -1. The weights do not change because both inputs are 0 (lr · error · 0 = 0). Check the result: (1, 1): 2 + 1 - 3 = 0 -> 1; (1, 0): -1 -> 0; (0, 1): -2 -> 0.

§2.3 A perceptron for the 12 students

Slide L26-S11

weights [1.32 0.69] bias 0.00

training accuracy 1.00 epochs 7

new student 8 h, 60 % -> pass

new student 4 h, 85 % -> fail

new student 11 h, 60 % -> pass

new student 7 h, 72 % -> pass

Features are scaled first (StandardScaler): a perceptron adds the features, so big numbers would dominate. The perceptron gives study hours the larger weight, so it predicts 8 h, 60 % as pass - the L25 tree said fail. Both are 100 % on 12 students; only more data and a test set can tell which to trust. A perceptron's boundary is always a straight line w1·x1 + w2·x2 + b = 0.

3. What One Perceptron Cannot Do

§3.1 Linear separability and XOR

Slide L26-S13

Linearly separable: one straight line can put all the 1s on one side and all the 0s on the other. AND and OR are; XOR (1 when exactly one input is 1 - like a staircase light with two switches) is not.

AND: target [0, 0, 0, 1] output [0, 0, 0, 1] w = (2, 1) b = -3 learned in 6 epochs

OR : target [0, 1, 1, 1] output [0, 1, 1, 1] w = (1, 1) b = -1 learned in 4 epochs

XOR: target [0, 1, 1, 0] output [1, 1, 0, 0] w = (-1, 0) b = 0 still 4 mistakes after 20 epochs

Minsky and Papert's book Perceptrons (1969) proved this limit. Research money for neural networks fell sharply for more than a decade (an 'AI winter').

§3.2 The way out: more than one layer

Slide L26-S14

x1

x2

OR

NAND

XOR = AND(OR, NAND)

0

0

0

1

0

0

1

1

1

1

1

0

1

1

1

1

1

1

0

0

OR, NAND and AND are each one perceptron (NAND: w = (-1, -1), b = 1.5). Two neurons in a hidden layer (OR, NAND) feeding one AND neuron compute XOR. The perceptron rule cannot train hidden neurons (we do not know their targets, and the step has no slope); smooth activations and back-propagation (Rumelhart, Hinton and Williams, 1986) solve this - Lecture 27. P02 built a perceptron; P03's MLPClassifier is a multilayer network.

4. Common Mistakes

Wrong idea

Correct idea

Weights change after every example.

Only after a mistake.

A weight changes even when its input is 0.

lr · error · 0 = 0.

A perceptron can learn any pattern.

Only linearly separable ones.

The bias is an input value.

It is a learned weight on a constant input 1.

The learned weights are unique.

Many weight sets work.

Word Bank

Word

Simple meaning

Example

Neuron

A unit that weighs inputs, adds them and decides

Tubewell neuron

Weight

How important an input is

rain -0.9

Bias

Shifts the cut-off

b = -0.5

Activation function

Turns the sum into an output

step: 1 if z >= 0

Learning rate

Size of each correction

lr = 1

Epoch

One pass over all training examples

4 examples of AND

Linearly separable

One straight line can split the classes

AND, OR

Hidden layer

Neurons between input and output

OR and NAND neurons for XOR

Self-Check Questions

  1. Tubewell neuron: output for (0, 0, 1)? (see §1.2)
  2. Why is the rain weight negative? (see §1.2)
  3. w = (2, 1), b = -3, x = (0, 1), t = 1, lr = 1. New w and b? (see §2.1)
  4. What is an epoch? (see §2.1)
  5. Why must features be scaled for a perceptron? (see §2.3)
  6. Draw the four XOR points and explain why no line separates them. (see §3.1)
  7. Give the weights of an OR neuron and a NAND neuron. (see §3.2)

Reading: NN Toolbox Guide Ch 1-3 (neuron model, perceptrons); Russell & Norvig Ch 21.1; Practical 02 handout (perceptron from scratch).

← All CS-511 material