Recap of Lecture 19 and Homework Check
LAST LECTURE · 5 MIN
Today: your dengue test is positive. The test is '90% accurate'. Are you 90% likely to have dengue? (Answer at S11 - most people are wrong.)
Today's Plan (50 minutes)
ROADMAP
0-5
Recap of Lecture 19 and homework check
S02
5-8
Plan and learning outcomes
S03-S04
8-17
Module 1: Probability basics - joint, marginal, conditional, independence
S05-S08
17-28
Module 2: Bayes' rule - the dengue test and the base rate
S09-S12
28-35
Module 3: Naive Bayes - an SMS spam filter
S13-S16
35-43
Class activity: conditional probability and Bayes
S17
43-48
Common mistakes and recap quiz
S18-S19
48-50
Homework and next lecture
S20
Learning Outcomes (SLOs)
BY THE END OF THIS LECTURE YOU WILL BE ABLE TO
SLO 20.1
Compute joint, marginal and conditional probabilities from a table and test independence
→ CLO3 · C3 Apply
SLO 20.2
Apply Bayes' rule to update a belief from evidence and explain the effect of the prior
→ CLO3 · C3 Apply
SLO 20.3
Apply a Naive Bayes classifier to a small example and explain its independence assumption
→ CLO3 · C3 Apply
- Course Learning Outcome(s) served
- CLO3: Apply various techniques and algorithms of AI used in general problem solving, optimization problems, constraint satisfaction problems, and game programming.
Slide 5
MODULE 1
Probability Basics
A number for 'how sure am I?'
Why Do AI Systems Need Probability?
MODULE 1 · UNCERTAINTY
Hidden causes
Noisy sensors
Too many exceptions
A doctor sees fever, not the virus. A farmer sees yellow stripes, not the fungus.
A cheap thermometer or a blurry camera gives wrong readings sometimes.
'Fever means dengue' - except flu, typhoid, malaria ... rules cannot list them all.
Joint, Marginal and Conditional Probability
MODULE 1 · FROM A TABLE
REAL OUTPUT: L20/JOINT.PY (MADE-UP CLINIC DATA)
- fever no fever total
- dengue 45 5 50
- no dengue 190 760 950
- total 235 765 1000
- P(dengue) = 0.050 P(fever) = 0.235
- P(dengue AND fever) = 0.045
- P(dengue | fever) = 0.045 / 0.235 = 0.191
- P(fever | dengue) = 0.045 / 0.050 = 0.900
- P(dengue) x P(fever) = 0.0118, not 0.045
- Independent? no
Not the same: P(fever | dengue) = 0.90 but P(dengue | fever) = 0.19. Most fever patients do NOT have dengue.
Rules of Probability and Independence
MODULE 1 · RULES
Independent
Dependent
Two coin tosses; rain in Burewala and a cricket result in Sydney.
Dengue and fever; clouds and rain; load-shedding and a hot day.
Slide 9
MODULE 2
Bayes' Rule
Turning 'symptom given disease' into 'disease given symptom'
Bayes' Rule
MODULE 2 · THE FORMULA
BAYES' RULE
- P(E | H) x P(H)
- P(H | E) = -----------------
- P(E)
- P(E) = P(E | H) P(H) + P(E | not H) P(not H)
In words: posterior = likelihood x prior / evidence. New belief = old belief, re-weighted by how well it explains what we saw.
Why: studies give P(symptom | disease); the patient wants P(disease | symptom). Bayes turns one into the other.
The Dengue Test: Positive - So How Worried?
MODULE 2 · WORKED EXAMPLE
18 test + (90%)
REAL OUTPUT: L20/BAYES.PY
- Prior P(D) = 0.02
- Sensitivity P(+|D) = 0.9
- False positive P(+|not D) = 0.05
- P(+) = 0.9 x 0.02 + 0.05 x 0.98
- = 0.067
- P(D|+) = 0.9 x 0.02 / 0.067 = 0.269
- 20 dengue
- (2%)
2 test -
- 1,000
- people
49 test + (5%)
Count: 67 positives, only 18 really ill: 18 / 67 = 0.27.
- 980 healthy
- (98%)
931 test -
Answer to S02: a '90% accurate' test that is positive means about 27% chance of dengue here - not 90%.
The Prior Matters: Base Rate and Repeated Tests
MODULE 2 · BASE RATE
REAL OUTPUT: L20/BAYES.PY
- Second positive test: 0.269 -> 0.869
- Effect of the prior (same test):
- prior 0.001 -> P(D|+) = 0.018
- prior 0.02 -> P(D|+) = 0.269
- prior 0.2 -> P(D|+) = 0.818
- prior 0.5 -> P(D|+) = 0.947
Base-rate fallacy: judging a test result without the prior. A rare disease stays fairly unlikely after one positive test.
Sequential: the posterior becomes the next prior: 0.02 -> 0.269 -> 0.869 after two positive tests.
Slide 13
MODULE 3
Naive Bayes
Many clues, one simple assumption
The Naive Bayes Classifier
MODULE 3 · IDEA
NAIVE BAYES WITH LAPLACE SMOOTHING
- score(C) = P(C) x P(w1|C) x P(w2|C) x ... x P(wn|C)
- P(w | C) = (count of w in class C + 1)
- / (total words in C + vocabulary size)
- choose the class with the larger score
Naive assumption
Clues are independent once the class is known.
Laplace (+1)
No word gets probability 0 just because it was not seen.
Used in: spam filters, news and complaint sorting, sentiment of reviews, quick medical screening - fast, needs little data.
Why it works: even when the probabilities are not exact, the class with the highest score is usually right.
Worked Example: An SMS Spam Filter
MODULE 3 · EXAMPLE
REAL OUTPUT: L20/NAIVE_BAYES.PY (MADE-UP PRACTICE MESSAGES)
- Training: 4 spam, 6 ham; vocabulary 28 words; spam words 15, ham words 25
- P('free'|spam) = (3+1)/(15+28) = 0.0930 P('free'|ham) = (1+1)/(25+28) = 0.0377
- P('prize'|spam) = (2+1)/(15+28) = 0.0698 P('prize'|ham) = (1+1)/(25+28) = 0.0377
- P('now'|spam) = (2+1)/(15+28) = 0.0698 P('now'|ham) = (1+1)/(25+28) = 0.0377
- score spam = 1.811e-04 score ham = 3.224e-05
- P(spam | 'free prize now') = 0.849
- scikit-learn MultinomialNB: P(spam) = 0.849
- 'see you after class': P(spam) = 0.088
- 'win a free recharge': P(spam) = 0.937
Probability vs Certainty Factors
MODULE 3 · COMPARISON
Next lecture: Bayesian belief networks - a graph of causes and effects that makes probability practical for many variables.
Activity: Conditional Probability and Bayes
CLASS ACTIVITY · PAIRS · 8 MIN
Pairs, 6 min + 2 min check. Calculators allowed.
Clinic table (S07): P(no dengue | fever)?
Why does a 90% sensitive test give only 27% here?
Pakistan won the toss in 20 of 40 matches; won 14 of those 20 and 8 of the other 20. P(win)? P(win | toss won)? P(toss won | win)?
SMS 'see you after class': spam or ham? (S15)
Stripe rust in 10% of fields; stripes seen in 80% of rust fields and 10% of healthy. P(rust | stripes)?
P(A) = 0.3, P(B) = 0.5, P(A and B) = 0.15. Independent?
Common Mistakes
WATCH OUT
WRONG IDEA
CORRECT IDEA
✗ P(dengue | fever) = P(fever | dengue).
✓ 0.19 vs 0.90 - swapping the condition changes everything.
✗ A '90% accurate' positive test means 90% ill.
✓ Depends on the prior: 27% at 2%, 95% in an outbreak ward.
✗ P(A and B) = P(A) x P(B) always.
✓ Only if A and B are independent.
✗ Naive Bayes thinks words really are independent.
✓ It assumes it to simplify; ranking is still usually right.
✗ An unseen word can be given probability 0.
✓ Use Laplace (+1) smoothing - a zero wipes out the product.
Recap Quiz
CHECK YOUR UNDERSTANDING · 3 MIN
- Q1. P(dengue | fever) means:
- (a) Fever given dengue (b) Dengue given fever (c) Dengue and fever (d) Dengue or fever
- Q2. Prior 0.02, sensitivity 0.9, false positive 0.05. P(D | +) is about:
- (a) 0.90 (b) 0.27 (c) 0.05 (d) 0.67
- Q3. A and B are independent if:
- (a) P(A and B) = P(A) + P(B) (b) P(A | B) = P(A) (c) P(A) = P(B) (d) P(A | B) = 1
- Q4. Laplace smoothing adds 1 to counts to avoid:
- (a) Negative numbers (b) Zero probabilities (c) Large vocabularies (d) Overfitting priors
Homework and Next Lecture
BEFORE THE NEXT CLASS
- Task 1: Over 30 days: rain and load-shedding 6, rain and none 2, no rain and load-shedding 9, no rain and none 13. Find P(LS), P(LS | rain), P(rain | LS). Independent?
- Task 2: A mango-sorting camera: 15% of mangoes are bruised. It flags 95% of bruised and 8% of good mangoes. P(bruised | flagged)?
- Task 3: With the S15 training messages, classify 'free notes' by Naive Bayes (Laplace +1). Spam or ham?
NEXT LECTURE
- Lecture 21:
- Bayesian Belief Networks