Learning Outcomes
SLO | After this lecture you can… | CLO | Bloom |
|---|---|---|---|
SLO 6.1 | Explain three design principles - parallelism, locality and focusing on the common case - with examples. | CLO1 | C2 Understand |
SLO 6.2 | Apply Amdahl's law to find the overall speedup of an enhancement and its upper limit. | CLO2 | C3 Apply |
SLO 6.3 | Use Amdahl's law to compare two design alternatives and choose the better one. | CLO2 | C3 Apply |
§1 Three Great Ideas
§1.1 Take advantage of parallelism
Slide L06-S06
Parallelism (doing several things at the same time) is used at every level of a computer:
Level | Example in computers | Real-world example |
|---|---|---|
System | Many servers or many cores share the requests of a website (throughput) | Many toll booths on the M-2 motorway |
Processor | Pipelining overlaps the steps of many instructions (Unit 3) | Washing, drying and ironing different loads at the same time |
Circuit | A carry-lookahead adder finds all carries at once instead of one by one | All students solving their own question at once |
Since clock rates stopped rising (Lecture 2), parallelism has become the main way to gain speed.
§1.2 The principle of locality
Slide L06-S07
Programs reuse the data and instructions they used recently. A common rule of thumb: a program spends about 90% of its execution time in only 10% of its code.
sum = 0; |
Kind | Meaning | In the loop above | Real-world example |
|---|---|---|---|
Temporal locality (time) | An item used now will probably be used again soon | sum, i and the loop instructions | Your daily WhatsApp chats stay at the top of the list |
Spatial locality (space) | Items near a used item will probably be used soon | a[0], a[1], a[2], ... one after another | After reading page 10 you read page 11 |
Caches exist because of locality. In Practical 5 you read array elements one after another - spatial locality in action.
§1.3 Focus on the common case
Slide L06-S08
When a designer must make a trade-off, favour the frequent case over the rare one. The frequent case is often simpler, so it can be made fast. Example: an addition rarely overflows, so the normal addition is made fast and the rare overflow case may be slower. In Practical 4, the 'Other' class (multiply and system calls) used 20 of the 42 cycles of the bill program - it is the common case in cycles, so it is the first thing to improve. Real-world: NADRA opens extra counters for CNIC renewal, the most common request.
How much does improving one part help the whole program? Amdahl's law answers this question.
§2 Amdahl's Law
§2.1 A trip from Lahore to Islamabad
Slide L06-S10
Part of the trip | Before | Car 1.25 times faster on the motorway | Car infinitely fast |
|---|---|---|---|
Lahore city traffic | 1.0 h | 1.0 h | 1.0 h |
M-2 motorway | 4.0 h | 3.2 h | 0 h |
Islamabad city | 0.5 h | 0.5 h | 0.5 h |
Total | 5.5 h | 4.7 h (speedup 5.5 / 4.7 = 1.17) | 1.5 h (speedup 5.5 / 1.5 = 3.67) |
The part you did not improve (the city traffic) limits the speedup of the whole trip.
§2.2 The law
Slide L06-S11
Speedup (overall) = Old execution time / New execution time |
Symbol | Meaning | Range |
|---|---|---|
F (fraction enhanced) | Fraction of the ORIGINAL execution time that can use the improvement | 0 to 1 |
S (speedup enhanced) | How many times faster that part becomes | greater than 1 |
Check with the trip: F = 4 / 5.5 = 0.727 and S = 1.25, so speedup = 1 / (0.273 + 0.727 / 1.25) = 1 / (0.273 + 0.582) = 1.17. Remember: F is a fraction of time, not a fraction of instructions or of lines of code.
§2.3 Worked example 1: a faster web server processor
Slide L06-S12
A web server spends 40% of its time computing and 60% waiting for disk and network (I/O). A new processor computes 10 times faster.
F = 0.4, S = 10 |
A processor that is 10 times faster gives a server that is only 1.56 times faster. To do better, the company must improve the I/O.
§2.4 The limit and diminishing returns
Slide L06-S13
As S becomes very large, F / S approaches 0, so the overall speedup can never exceed 1 / (1 - F).
F | S = 2 | S = 4 | S = 8 | S = 16 | S = 64 | Limit 1 / (1 - F) |
|---|---|---|---|---|---|---|
0.4 | 1.25 | 1.43 | 1.54 | 1.60 | 1.65 | 1.67 |
0.8 | 1.67 | 2.50 | 3.33 | 4.00 | 4.71 | 5 |
0.95 | 1.90 | 3.48 | 5.93 | 9.14 | 15.42 | 20 |
Each extra increase in S gives less and less benefit (diminishing returns). A designer should first make sure the improved part is a large fraction of the time (focus on the common case), and only then make it very fast.
§3 Using Amdahl's Law to Choose a Design
§3.1 Worked example 2: square root or all floating point?
Slide L06-S15
In a graphics program, floating-point square root (FPSQR) takes 20% of the time and all floating-point (FP) instructions together take 50%.
Option 1: FPSQR 10 times faster | Option 2: all FP 1.6 times faster | |
|---|---|---|
F | 0.2 | 0.5 |
S | 10 | 1.6 |
Working | 1 / (0.8 + 0.2 / 10) = 1 / 0.82 | 1 / (0.5 + 0.5 / 1.6) = 1 / 0.8125 |
Overall speedup | 1.22 | 1.23 |
A small speedup on a big fraction beats a big speedup on a small fraction. Before buying hardware, ask what fraction of the time it helps.
§3.2 Worked example 3: how many cores?
Slide L06-S16
A program is 90% parallel (it can be split across cores) and 10% serial (it must run on one core). With N cores: speedup = 1 / (0.1 + 0.9 / N).
Cores N | Working | Speedup | Efficiency (speedup / N) |
|---|---|---|---|
2 | 1 / (0.1 + 0.45) | 1.82 | 91% |
8 | 1 / (0.1 + 0.1125) | 4.71 | 59% |
64 | 1 / (0.1 + 0.0141) | 8.77 | 14% |
Infinite | 1 / 0.1 | 10 | - |
With 10% serial work, no number of cores can give more than 10 times. This is why reducing the serial part matters so much for multicore processors and GPUs (Units 4 and 5). Real-world: a wedding kitchen with 100 cooks still waits for the one person who decides the menu.
§3.3 Amdahl's law and the CPU performance equation agree
Slide L06-S17
Lecture 4, worked example 2: average CPI 2.2; loads are 20% of the instructions with CPI 5; a new cache makes loads CPI 2.
Time in loads F = (0.2 x 5) / 2.2 = 1.0 / 2.2 = 0.4545 (45% of the cycles) |
Both methods give the same answer. But if you wrongly take F = 0.2 (the fraction of instructions), you get 1 / (0.8 + 0.08) = 1.14, which is wrong. F must be the fraction of time (cycles).
§4 Common Mistakes
Slide L06-S19
Wrong idea | Correct idea |
|---|---|
A part 10 times faster makes the program 10 times faster. | Overall = 1 / ((1 - F) + F / S): the unimproved part limits the gain. |
F is the fraction of instructions. | F is the fraction of the original execution time. |
More cores always give proportional speed. | The serial part caps the speedup at 1 / (1 - F). |
Locality means data is close together on the disk. | Locality is about reuse in time and neighbours in address. |
Word Bank
Word | Simple meaning | Real-world example |
|---|---|---|
Parallelism | Doing several things at the same time | Many toll booths open together |
Pipelining | Overlapping the steps of different tasks | Washing one load while drying another |
Temporal locality | Something used now is used again soon | Your favourite chats at the top of WhatsApp |
Spatial locality | Neighbours of something used are used soon | Reading the next page of a book |
Common case | The situation that happens most often | CNIC renewal at the NADRA office |
Fraction enhanced (F) | Share of the old time that is improved | Motorway part of the trip |
Speedup enhanced (S) | How much faster the improved part becomes | Faster car on the motorway |
Serial part | Work that must be done by one worker alone | One person deciding the wedding menu |
Diminishing returns | Each extra effort gives less benefit | Fifth hour of study before an exam |
Efficiency | Speedup divided by the number of cores | How busy each of the cooks really is |
Self-Check Questions
- Give one example of parallelism at the system, processor and circuit level. (§1.1)
- Explain temporal and spatial locality using a loop that adds the elements of an array. (§1.2)
- What does 'focus on the common case' mean? Give a hardware example. (§1.3)
- Write Amdahl's law and define F and S. (§2.2)
- 40% of the time is made 10 times faster. Find the overall speedup and its limit. (§2.3)
- Why does a very large S give diminishing returns? (§2.4)
- FPSQR (20% of time) 10x faster, or all FP (50%) 1.6x faster - which is better? (§3.1)
- A program is 90% parallel. Find the speedup on 8 cores and on infinitely many cores. (§3.2)
- Why must F be a fraction of time and not of instructions? Use the Lecture 4 loads example. (§3.3)
Answers: 1.56 and 1.67; option 2 (1.23 vs 1.22); 4.71 and 10.
Further Reading
- Hennessy and Patterson (2012): Chapter 1, section 1.9 (quantitative principles of computer design, Amdahl's law, the processor performance equation).