Learning Outcomes
SLO | After this lecture you will be able to... | CLO | Bloom |
|---|---|---|---|
SLO 21.1 | Explain the structure of a Bayesian network (nodes, arrows, CPTs) and why it needs far fewer numbers than a full joint table | CLO3 | C2 Understand |
SLO 21.2 | Apply the chain rule to compute a joint probability from a network | CLO3 | C3 Apply |
SLO 21.3 | Apply inference by enumeration to a small network and explain explaining away | CLO3 | C3 Apply |
1. Network Structure
§1.1 Why Bayesian networks?
Slide L21-S06
A full joint table over n yes/no variables needs 2^n - 1 numbers. A Bayesian network is a directed acyclic graph (DAG): one node per variable, an arrow from each direct cause to its effect, no loops, and at each node a conditional probability table (CPT) P(node | parents). Each node looks only at its parents, so far fewer numbers are needed.
Numbers needed: network 10, full joint table 2^5 - 1 = 31 10 yes/no variables, max 2 parents each: network <= 40, full joint 1,023 20 yes/no variables, max 2 parents each: network <= 80, full joint 1,048,575 30 yes/no variables, max 2 parents each: network <= 120, full joint 1,073,741,823 |
§1.2 The shop alarm network (Russell & Norvig numbers)
Slide L21-S07
B = theft, E = earthquake, A = shop alarm rings, J = neighbour Junaid calls, M = neighbour Maryam calls. Arrows: B -> A, E -> A, A -> J, A -> M.
Node | Parents | CPT |
|---|---|---|
B | - | P(b) = 0.001 |
E | - | P(e) = 0.002 |
A | B, E | P(a | b, e) 0.95; P(a | b, not e) 0.94; P(a | not b, e) 0.29; P(a | not b, not e) 0.001 |
J | A | P(j | a) 0.90; P(j | not a) 0.05 |
M | A | P(m | a) 0.70; P(m | not a) 0.01 |
Numbers: 1 + 1 + 4 + 2 + 2 = 10, instead of 31.
§1.3 The chain rule
Slide L21-S08
P(x1, ..., xn) = product of P(xi | parents(xi)). Write the product first, then substitute; use 1 - p for 'not'.
P(j, m, a, not b, not e) = P(j|a) P(m|a) P(a|not b, not e) P(not b) P(not e) = 0.90 x 0.70 x 0.001 x 0.999 x 0.998 = 0.000628 |
2. Inference
§2.1 Inference by enumeration
Slide L21-S10
P(query | evidence): for each query value, add the joint over all values of the hidden variables (chain rule), then normalise by dividing by the total (alpha). Exact, but the number of terms grows as 2^(hidden variables).
def enumerate_ask(net, query, evidence, order=None): """Exact inference: P(query | evidence) by summing the joint over all hidden variables, then normalising.""" order = order or list(net) hidden = [v for v in order if v != query and v not in evidence] dist = {} for qv in (True, False): total = 0.0 for vals in itertools.product((True, False), repeat=len(hidden)): ev = dict(evidence); ev[query] = qv; ev.update(zip(hidden, vals)) total += joint(net, ev) dist[qv] = total z = dist[True] + dist[False] return dist[True] / z, dist[True], dist[False] |
P(theft | Junaid and Maryam call) = 0.284 P(theft | only Junaid calls) = 0.016 P(earthquake | both call) = 0.176 P(theft | alarm) = 0.374 P(theft | alarm, earthquake) = 0.003 Unnormalised: true 0.0005922, false 0.0014919 alpha = 1 / 0.0020841 |
§2.2 Three connections and explaining away
Slide L21-S11
Connection | Shape | Example | Knowing the middle node |
|---|---|---|---|
Chain | A -> B -> C | Theft -> Alarm -> Junaid | Blocks the flow |
Fork | A <- B -> C | Junaid <- Alarm -> Maryam | Blocks the flow |
Collider | A -> B <- C | Theft -> Alarm <- Earthquake | Opens the flow: explaining away |
Explaining away: P(theft | alarm) = 0.374 but P(theft | alarm, earthquake) = 0.003 - the earthquake explains the alarm.
§2.3 Approximate inference: rejection sampling
Slide L21-S12
Simulate many worlds by drawing each variable from its CPT (parents first), keep only samples that match the evidence and count the query. Fast but approximate; rare evidence wastes most samples.
10,000 samples: kept 24 estimate 0.250 100,000 samples: kept 213 estimate 0.286 1,000,000 samples: kept 2,109 estimate 0.275 exact (enumeration): 0.284 |
3. Building and Using Networks
§3.1 Steps and applications
Slide L21-S14
Step | What to do |
|---|---|
1 Variables | Choose the variables |
2 Order | Causes before effects |
3 Arrows | Each node's direct causes are its parents |
4 CPTs | From an expert or by counting data |
Uses: medical diagnosis (e.g. Pathfinder for lymph-node diseases), printer and computer troubleshooters, spam filtering (Naive Bayes is the simplest network), crop-disease forecasting, credit risk.
4. Common Mistakes
Wrong idea | Correct idea |
|---|---|
Loops are allowed. | A DAG has no loops. |
A CPT lists all variables. | Only the node's parents. |
Both calls mean a theft is likely. | Only 0.284 - the prior is tiny. |
No need to normalise. | Divide by the sum (alpha). |
Causes never affect each other. | In a collider they do: explaining away. |
Word Bank
Word | Simple meaning | Example |
|---|---|---|
Bayesian network | Graph of causes and effects with probabilities | Alarm network |
DAG | Arrows with direction and no loops | B -> A -> J |
Parent | A direct cause of a node | B and E for A |
CPT | Table P(node | parents) | P(j | a) = 0.90 |
Hidden variable | Neither asked nor observed | E and A in P(B | j, m) |
Explaining away | One cause makes another less likely | Earthquake vs theft |
Sampling | Estimating by simulation | Rejection sampling |
Self-Check Questions
- How many numbers does the alarm network need? Why 4 for Alarm? (see §1.2)
- Compute P(j, m, a, b, e). (see §1.3)
- What are hidden variables in P(B | j, m)? (see §2.1)
- Why is P(theft | both call) only 0.284? (see §2.1)
- Give an everyday example of explaining away. (see §2.2)
- Why does rejection sampling waste samples? (see §2.3)
Reading: Russell & Norvig (4e) 13.1-13.4 (Bayesian networks, exact and approximate inference).