Nexus Learn

Lecture 21: Bayesian Belief Networks

Lecture Notes CS-511 Artificial Intelligence 5 min read Updated 8 Nov 2026

↓ Download notes (Word)

On this page
  1. Learning Outcomes
  2. 1. Network Structure
  3. 2. Inference
  4. 3. Building and Using Networks
  5. 4. Common Mistakes
  6. Word Bank
  7. Self-Check Questions

Learning Outcomes

SLO

After this lecture you will be able to...

CLO

Bloom

SLO 21.1

Explain the structure of a Bayesian network (nodes, arrows, CPTs) and why it needs far fewer numbers than a full joint table

CLO3

C2 Understand

SLO 21.2

Apply the chain rule to compute a joint probability from a network

CLO3

C3 Apply

SLO 21.3

Apply inference by enumeration to a small network and explain explaining away

CLO3

C3 Apply

1. Network Structure

§1.1 Why Bayesian networks?

Slide L21-S06

A full joint table over n yes/no variables needs 2^n - 1 numbers. A Bayesian network is a directed acyclic graph (DAG): one node per variable, an arrow from each direct cause to its effect, no loops, and at each node a conditional probability table (CPT) P(node | parents). Each node looks only at its parents, so far fewer numbers are needed.

Numbers needed: network 10, full joint table 2^5 - 1 = 31

10 yes/no variables, max 2 parents each: network <= 40, full joint 1,023

20 yes/no variables, max 2 parents each: network <= 80, full joint 1,048,575

30 yes/no variables, max 2 parents each: network <= 120, full joint 1,073,741,823

§1.2 The shop alarm network (Russell & Norvig numbers)

Slide L21-S07

B = theft, E = earthquake, A = shop alarm rings, J = neighbour Junaid calls, M = neighbour Maryam calls. Arrows: B -> A, E -> A, A -> J, A -> M.

Node

Parents

CPT

B

-

P(b) = 0.001

E

-

P(e) = 0.002

A

B, E

P(a | b, e) 0.95; P(a | b, not e) 0.94; P(a | not b, e) 0.29; P(a | not b, not e) 0.001

J

A

P(j | a) 0.90; P(j | not a) 0.05

M

A

P(m | a) 0.70; P(m | not a) 0.01

Numbers: 1 + 1 + 4 + 2 + 2 = 10, instead of 31.

§1.3 The chain rule

Slide L21-S08

P(x1, ..., xn) = product of P(xi | parents(xi)). Write the product first, then substitute; use 1 - p for 'not'.

P(j, m, a, not b, not e)

= P(j|a) P(m|a) P(a|not b, not e) P(not b) P(not e)

= 0.90 x 0.70 x 0.001 x 0.999 x 0.998

= 0.000628

2. Inference

§2.1 Inference by enumeration

Slide L21-S10

P(query | evidence): for each query value, add the joint over all values of the hidden variables (chain rule), then normalise by dividing by the total (alpha). Exact, but the number of terms grows as 2^(hidden variables).

def enumerate_ask(net, query, evidence, order=None):

"""Exact inference: P(query | evidence) by summing the joint over all hidden variables, then normalising."""

order = order or list(net)

hidden = [v for v in order if v != query and v not in evidence]

dist = {}

for qv in (True, False):

total = 0.0

for vals in itertools.product((True, False), repeat=len(hidden)):

ev = dict(evidence); ev[query] = qv; ev.update(zip(hidden, vals))

total += joint(net, ev)

dist[qv] = total

z = dist[True] + dist[False]

return dist[True] / z, dist[True], dist[False]

P(theft | Junaid and Maryam call) = 0.284

P(theft | only Junaid calls) = 0.016

P(earthquake | both call) = 0.176

P(theft | alarm) = 0.374

P(theft | alarm, earthquake) = 0.003

Unnormalised: true 0.0005922, false 0.0014919

alpha = 1 / 0.0020841

§2.2 Three connections and explaining away

Slide L21-S11

Connection

Shape

Example

Knowing the middle node

Chain

A -> B -> C

Theft -> Alarm -> Junaid

Blocks the flow

Fork

A <- B -> C

Junaid <- Alarm -> Maryam

Blocks the flow

Collider

A -> B <- C

Theft -> Alarm <- Earthquake

Opens the flow: explaining away

Explaining away: P(theft | alarm) = 0.374 but P(theft | alarm, earthquake) = 0.003 - the earthquake explains the alarm.

§2.3 Approximate inference: rejection sampling

Slide L21-S12

Simulate many worlds by drawing each variable from its CPT (parents first), keep only samples that match the evidence and count the query. Fast but approximate; rare evidence wastes most samples.

10,000 samples: kept 24 estimate 0.250

100,000 samples: kept 213 estimate 0.286

1,000,000 samples: kept 2,109 estimate 0.275

exact (enumeration): 0.284

3. Building and Using Networks

§3.1 Steps and applications

Slide L21-S14

Step

What to do

1 Variables

Choose the variables

2 Order

Causes before effects

3 Arrows

Each node's direct causes are its parents

4 CPTs

From an expert or by counting data

Uses: medical diagnosis (e.g. Pathfinder for lymph-node diseases), printer and computer troubleshooters, spam filtering (Naive Bayes is the simplest network), crop-disease forecasting, credit risk.

4. Common Mistakes

Wrong idea

Correct idea

Loops are allowed.

A DAG has no loops.

A CPT lists all variables.

Only the node's parents.

Both calls mean a theft is likely.

Only 0.284 - the prior is tiny.

No need to normalise.

Divide by the sum (alpha).

Causes never affect each other.

In a collider they do: explaining away.

Word Bank

Word

Simple meaning

Example

Bayesian network

Graph of causes and effects with probabilities

Alarm network

DAG

Arrows with direction and no loops

B -> A -> J

Parent

A direct cause of a node

B and E for A

CPT

Table P(node | parents)

P(j | a) = 0.90

Hidden variable

Neither asked nor observed

E and A in P(B | j, m)

Explaining away

One cause makes another less likely

Earthquake vs theft

Sampling

Estimating by simulation

Rejection sampling

Self-Check Questions

  1. How many numbers does the alarm network need? Why 4 for Alarm? (see §1.2)
  2. Compute P(j, m, a, b, e). (see §1.3)
  3. What are hidden variables in P(B | j, m)? (see §2.1)
  4. Why is P(theft | both call) only 0.284? (see §2.1)
  5. Give an everyday example of explaining away. (see §2.2)
  6. Why does rejection sampling waste samples? (see §2.3)

Reading: Russell & Norvig (4e) 13.1-13.4 (Bayesian networks, exact and approximate inference).

← All CS-511 material