Module 5: Safety, testing and cost · 10 min
Testing, cost and launch checklist
Agents are less predictable than normal software, so test them with a set of realistic tasks (called an evaluation or 'eval') and run the set after every change.
| Measure | Question |
|---|---|
| Success rate | How often does it finish the task correctly? |
| Steps and cost | How many calls and tokens did it use? |
| Speed | How long did the user wait? |
| Safety | Did it ever try something it should not? |
- Log every thought, tool call and result so you can replay failures.
- Use a cheaper, faster model for easy steps and a stronger one for hard steps.
- Cache repeated lookups.
- Start with a small group of users and a human reviewer, then widen slowly.
Capstone idea: build a small research agent with one search tool, a step limit of 8, a written plan and a final check against sources. Test it on 10 questions. You have now finished the lessons: take the final exam to earn your certificate.