Nexus Learn

Log in · Create free account

Module 5: Safety, testing and cost · 10 min

Testing, cost and launch checklist

Agents are less predictable than normal software, so test them with a set of realistic tasks (called an evaluation or 'eval') and run the set after every change.

MeasureQuestion
Success rateHow often does it finish the task correctly?
Steps and costHow many calls and tokens did it use?
SpeedHow long did the user wait?
SafetyDid it ever try something it should not?
  • Log every thought, tool call and result so you can replay failures.
  • Use a cheaper, faster model for easy steps and a stronger one for hard steps.
  • Cache repeated lookups.
  • Start with a small group of users and a human reviewer, then widen slowly.

Capstone idea: build a small research agent with one search tool, a step limit of 8, a written plan and a final check against sources. Test it on 10 questions. You have now finished the lessons: take the final exam to earn your certificate.

(free account needed to save progress)

← Prompt injection