Monitoring and instrumentation
A live service is sending signals about how it is doing all the time: which tasks people finish, where they get stuck, what breaks, how long things take. Monitoring is the work of collecting those signals and looking at the few that matter, so the team can see whether the service is actually working for the people who use it.
Two things make that possible, in order. First, instrument the service: build it so it records what happens as people use it. Then watch a small set of signals and act on them. The instrumentation comes first, because a beautiful dashboard built on signals the service never emitted shows nothing real.
This is not optional for a Government of Canada service. The Guideline on Service and Digital requires departments to measure service performance and to collect and use client feedback to improve. Monitoring is how a team meets that obligation, and how it notices a problem before it becomes a crisis.
A team cannot improve what it cannot see.
What good looks like
Signals come from the service itself, generated as people use it, rather than typed into a spreadsheet by hand.
The team watches a small set of signals tied to real user experience and system health.
Every signal that matters has a target set in advance, so the team can tell whether a number is good or bad.
Dashboards are readable, trusted, and visible to the bodies that review the service.
Every signal that matters has an owner who acts on it, and an all-green dashboard is allowed to generate nothing.
Instrumentation is built into the service before go-live, while it is still cheap to add.
The Canadian how-to for this, at a business-owner level, is Monitoring and measuring task success on design.canada.ca.
The cost of skipping it
Collecting data is easy. Acting on it is the part that gets skipped, and a dashboard no one acts on is just decoration.
When monitoring is missing or ignored, the cost is real:
- The first news of a problem comes from complaints. Without its own signals, a team hears about a broken service from users, from the minister's office, or from the media, long after it started.
- No one can tell whether a change helped. With nothing measured before and after, every release is a guess and arguments get settled by whoever is most senior.
- Vanity metrics hide the truth. A dashboard full of big green numbers no one chose can look healthy while people fail at the one task that matters.
- Reports pile up and nothing changes. Data that is collected but never turned into work is a cost with no return.
A closer look
Monitoring works as a loop with three stages. Skip the first and there is nothing real to see; skip the last and nothing changes.
Whose job it is
Monitoring is a team activity, and it fails when the dashboard is someone's side project that no one reads.
- The service team builds the instrumentation into the service and stands up the dashboards.
- The product manager chooses the few signals that matter, sets their targets, and brings the readings to the backlog review.
- Whoever runs the service in production watches it live and responds when a signal crosses a line.
- The business owner of the application owns the targets, answers for the service's performance to the bodies that review it, and makes sure signals turn into work rather than reports.
Comparison
Two ways to monitor a service
Pax
Meet Pax, a service manager. They treated measurement as part of building the service:
- instrumented the service before launch, so it recorded each step people attempted and completed
- picked five signals tied to getting a licence, each with a target
- reviewed the signals at every backlog session and turned the weak ones into work
- caught an early drop-off on one step and fixed it before it grew
The result: the team saw the service the way its users experienced it, and fixed the real problems while they were still small.
What Monitoring and instrumentation looks like in each phase
Monitoring and instrumentation changes shape across the life of a service.
Decide what to measure before go-live.
Choose the few signals that matter and build the instrumentation into the first real version from the start.
Set a target for each one, so that from the first day live the team can tell a good number from a bad one.
The official instruments behind monitoring and instrumentation
Everything official this subject brings with it, and where in a service's life each one comes up. The full detail, including who does the work and what the business owner personally does, is in the table on the home page.
The written statement of what good this project is supposed to do, and the later report confirming what was actually delivered and whether the promised benefits arrived.
- DiscoveryFill
- StabilizationSubmit
- MaturityKeep current
The duty to have a way of spotting, containing and reporting a cyber incident before one happens, and to report it up the government-wide chain when it does. The government-wide plan sets who is told, in what order, and how an event escalates into a coordinated response.
- BetaFill
- StabilizationKeep current
- GrowthKeep current
- MaturityKeep current
Further reading
See also
Assumptions this page makes
You are already working to the Government of Canada Digital Standards, design with users, iterate and improve frequently, work in the open, use open standards, address security and privacy, build in accessibility, empower staff, be good data stewards, design ethical services, and collaborate widely, and to the law on privacy, security, official languages, and accessibility. The standards say how the government works in the digital world. The six Government of Canada digital competencies say what every public servant has to be able to do to work that way, and the team page covers them. This guide builds on those.