Recap of Lecture 16 and Homework Check
LAST LECTURE · 5 MIN
Today: until now nobody fought back. In a game, every second move is chosen by an opponent who wants you to lose.
Today's Plan (50 minutes)
ROADMAP
0-5
Recap of Lecture 16 and homework check
S02
5-8
Plan and learning outcomes
S03-S04
8-15
Module 1: Games as search - kinds of games, formal definition, game tree
S05-S08
15-27
Module 2: The minimax algorithm - values, sticks game, tic-tac-toe
S09-S14
27-35
Module 3: Cost of minimax, depth limit and evaluation functions
S15-S17
35-43
Class activity: back up minimax values
S18-S19
43-48
Common mistakes and recap quiz
S20-S21
48-50
Homework and next lecture
S22
Learning Outcomes (SLOs)
BY THE END OF THIS LECTURE YOU WILL BE ABLE TO
SLO 17.1
Describe a game as a search problem (players, actions, terminal test, utility) and classify kinds of games
→ CLO3 · C2 Understand
SLO 17.2
Apply the minimax algorithm to back up values in a game tree and choose the best move
→ CLO3 · C3 Apply
SLO 17.3
Explain the cost of minimax and use a depth limit with an evaluation function
→ CLO3 · C3 Apply
- Course Learning Outcome(s) served
- CLO3: Apply various techniques and algorithms of AI used in general problem solving, optimization problems, constraint satisfaction problems, and game programming.
Slide 5
MODULE 1
Games as Search
Two players, one wants the opposite of the other
Kinds of Games
MODULE 1 · CLASSIFICATION
Zero-sum: one player's gain is exactly the other's loss. Win = +1, loss = -1, draw = 0. The total is always zero.
Today: deterministic, perfect information, two players, zero-sum - chess, dama, tic-tac-toe.
A Game as a Search Problem
MODULE 1 · FORMAL DEFINITION
MAX and MIN: MAX (X, us) wants high utility; MIN (O, the opponent) wants low utility. They take turns.
The Game Tree of Tic-Tac-Toe
MODULE 1 · GAME TREE
9, 8, 7, ... moves
549,946 nodes
Solved: a draw
X has 9 choices, then O has 8, then X 7 ... One level = one ply.
255,168 complete games. 9! = 362,880 is the upper bound.
Full minimax gives the empty board the value 0.
REAL OUTPUT: L17/TTT.PY
- Empty board: minimax value 0 (0 = draw with perfect play); nodes visited 549,946
- Complete games (terminal leaves): 255,168
Ply: one move by one player. 'Look 4 plies ahead' = my move, your reply, my move, your reply.
Slide 9
MODULE 2
The Minimax Algorithm
Assume the opponent plays its best
Minimax: MAX Chooses High, MIN Chooses Low
MODULE 2 · THE IDEA
A
MAX
MAX (up triangle)
Us. Picks the child with the HIGHEST value.
B
C
D
MIN
MIN (down triangle)
Opponent. Picks the child with the LOWEST value.
leaves
Question: D has a 14 under it. Is D the best move for MAX? (No - MIN would answer with 2.)
Backing Up the Values
MODULE 2 · WORKED EXAMPLE
REAL OUTPUT: L17/TREE2.PY
A
- MIN B: min(3, 12, 8) = 3
- MIN C: min(2, 4, 6) = 2
- MIN D: min(14, 5, 2) = 2
- MAX A: max(3, 2, 2) = 3
- Best move: B
- Nodes visited: 13
- (1 root + 3 MIN + 9 leaves)
B
C
D
Minimax value 3: MAX plays B and is sure of at least 3, whatever MIN does. A mistake by MIN only helps MAX.
The Minimax Algorithm
MODULE 2 · ALGORITHM
MINIMAX (GAMELIB.PY)
- def minimax(state):
- if TERMINAL(state):
- return UTILITY(state)
- values = [minimax(RESULT(state, a))
- for a in ACTIONS(state)]
- if PLAYER(state) == MAX:
- return max(values)
- else:
- return min(values)
- # choose MAX's move at the root
- best = max(ACTIONS(s0),
- key=lambda a: minimax(RESULT(s0, a)))
Go down
Depth-first to the leaves (terminal states).
Score leaves
UTILITY: +1 win, -1 loss, 0 draw (or points).
Back up
MAX node: largest child. MIN node: smallest child.
A Game You Can Solve: Sticks
MODULE 2 · STICKS GAME
REAL OUTPUT: L17/STICKS.PY
- 1 sticks: value +1 (MAX wins), nodes 2
- 2 sticks: value +1 (MAX wins), nodes 4
- 3 sticks: value -1 (MIN wins), nodes 7
- 4 sticks: value +1 (MAX wins), nodes 12
- 5 sticks: value +1 (MAX wins), nodes 20
- 6 sticks: value -1 (MIN wins), nodes 33
- 7 sticks: value +1 (MAX wins), nodes 54
- 8 sticks: value +1 (MAX wins), nodes 88
- 9 sticks: value -1 (MIN wins), nodes 143
Rules
Take 1 or 2 sticks per turn. Whoever takes the LAST stick wins.
Pattern
3, 6, 9 sticks: the first player loses. Leave your opponent a multiple of 3.
Notice: minimax found the winning rule by itself, just by searching. Node counts grow fast: 2, 4, 7, 12, 20, 33, 54, 88, 143.
Minimax Picks a Move in Tic-Tac-Toe
MODULE 2 · TIC-TAC-TOE
REAL OUTPUT: L17/TTT.PY (X TO MOVE)
- X O .
- X . O
- . . .
- X plays square 2: value +0
- X plays square 4: value +1
- X plays square 6: value +1
- X plays square 7: value +0
- X plays square 8: value +1
- Squares:
- 0 1 2
- 3 4 5
- 6 7 8
Optimal: all +1 moves are winning; prefer the fastest win.
Slide 15
MODULE 3
When the Tree Is Too Big
Depth limits and evaluation functions
How Costly Is Minimax?
MODULE 3 · PROPERTIES
REAL OUTPUT: L17/GROWTH.PY
- Tic-tac-toe: at most 9! = 362,880 move sequences
- Chess: b = 35, m = 80 -> 35^80 = 10^124 nodes
- Go: b = 250, m = 150 -> 250^150 = 10^360 nodes
- Chess at 1 billion nodes per second: about 10^107 years
- Chess depth 2: 35^2 = 1,225 nodes
- Chess depth 4: 35^4 = 1,500,625 nodes
- Chess depth 6: 35^6 = 1,838,265,625 nodes
- Chess depth 8: 35^8 = 2,251,875,390,625 nodes
Problem: chess has about 10^123 nodes - no computer can search to the end. We must stop early and GUESS the value.
Depth Limit and Evaluation Function
MODULE 3 · EVALUATION
REAL OUTPUT: L17/DEPTHLIM.PY
- X in centre (square 4): eval = 4; depth-2 value = 1
- X in corner (square 0): eval = 3; depth-2 value = -1
- X in edge (square 1): eval = 2; depth-2 value = -2
- Example board:
- X . .
- . O .
- . . .
- open lines: X 4, O 5 -> eval = -1
- Nodes from the empty board: depth 2 -> 82, full depth -> 549,946
Like h(n): an evaluation function is a heuristic for games - fast, not exact.
Horizon effect: a disaster one ply beyond the depth limit cannot be seen. Fix: search captures and checks deeper.
Activity Trees
CLASS ACTIVITY · USE THESE TREES
Tree 1 (2 plies)
Tree 2 (3 plies: MAX, MIN, MAX)
A
A
B
C
D
B
C
D
Activity: Back Up the Values
CLASS ACTIVITY · PAIRS · 8 MIN
Pairs, 6 min + 2 min check. Write a value next to every triangle on the trees (previous slide).
Tree 1: value of B, C, D and A. Which move should MAX play?
Classify Ludo and Rang: deterministic or chance? Perfect or imperfect information?
Tree 2: values at all three levels. Best move for MAX?
Tic-tac-toe: X in the centre, O in a corner. Evaluate: open lines X - open lines O.
Sticks game with 4 sticks, you move first. Do you win? What do you take?
Chess to depth 6 (35^6 nodes) at 1 million nodes per second: how long?
Common Mistakes
WATCH OUT
WRONG IDEA
CORRECT IDEA
✗ MAX should head for the biggest leaf.
✓ MIN chooses too: plan for MIN's best reply (D gives 2, not 14).
✗ Back up the average of the children.
✓ MAX node = largest child, MIN node = smallest child.
✗ A MIN node is our move.
✓ MIN is the opponent's move; levels alternate MAX, MIN, MAX...
✗ Minimax searches the whole chess tree.
✓ Too big (10^123): stop at a depth, use an evaluation function.
✗ The evaluation function is the true value.
✓ It is an estimate, like h(n); the horizon effect can fool it.
Recap Quiz
CHECK YOUR UNDERSTANDING · 3 MIN
- Q1. At a MIN node, minimax returns:
- (a) The largest child value (b) The smallest child value (c) The average (d) The first child
- Q2. MIN nodes B = 3, C = 2, D = 2. The root (MAX) value is:
- (a) 2 (b) 3 (c) 7 (d) 14
- Q3. Ludo is a game of:
- (a) Chance, perfect info (b) Chance, imperfect info (c) Deterministic, perfect info (d) Deterministic, imperfect info
- Q4. Time complexity of minimax (branching b, depth m):
- (a) O(bm) (b) O(b + m) (c) O(b^m) (d) O(m^b)
Homework and Next Lecture
BEFORE THE NEXT CLASS
- Task 1: Back up the minimax values: MAX root; MIN nodes B [3, 8, 6], C [9, 1], D [4, 5, 7]. Best move?
- Task 2: Sticks game (take 1 or 2; last stick wins). With 5 sticks and with 6 sticks: who wins, and what should the first player take?
- Task 3: Tic-tac-toe: X in the centre (4), O on the top edge (1). Find open lines for X and O and the evaluation.
- Read: Russell & Norvig 5.1-5.2 and 5.4 (evaluation functions).
NEXT LECTURE
- Lecture 18:
- Alpha-Beta Pruning and Constraint Satisfaction