The Context Problem

Dashboards show what happened. Alerts say something changed. Incidents organize investigation. But explanation is still manual work.

June 20, 20266 min readBy heramb s.
analyticsmonitoringengineering

The Context Problem

A metric changes. Someone has to figure out why.

This is where the real work lives, and it's almost entirely manual.

A conversion rate drops. Someone opens the analytics dashboard, then the ad platform, then the CRM. They filter by segment, region, cohort. They check whether the drop is isolated or system-wide. They cross-reference deployment logs against support ticket volume. They write a Slack summary of what they found. Then they repeat this, for different metrics, different teams, different weeks.

This is not analysis. It's investigative labor. It doesn't compound. When the person who did it leaves, the knowledge of how to do it leaves with them.

The data was never the bottleneck. Most organizations have more instrumentation than they know what to do with. The bottleneck is connecting the signals into an explanation someone can act on.

Analyst Agent


What Dashboards, Alerts, and Incidents Actually Do

Dashboards answer what. They show metrics at a point in time. They're necessary for monitoring but useless for understanding. A dashboard will tell you revenue is down 18%. It won't tell you that 63% of the decline came from healthcare enterprise accounts, that marketing spend was flat, and that pipeline volume for that segment hasn't changed — meaning the drop is conversion-related, not acquisition-related.

Alerts answer when. They tell you something crossed a threshold. The problem is they fire without context. An alert that triggers once without explanation is noise. An alert that triggers repeatedly without consequence trains people to ignore it. Most organizations have alert fatigue because they invested in detection but not in explanation.

Incident systems answer who and how long. They assign ownership, track resolution, document what happened after the fact. They don't reduce the time it takes to understand what went wrong. The investigation still happens in someone's head, across five browser tabs, with notes buried in a thread.

Each of these tools solved one piece of the puzzle. None of them solved interpretation.


What They Don't Do

None of these systems connect signals automatically.

A dashboard won't correlate a deployment timestamp with a spike in a specific error code in support tickets. An alert won't notice that the churned accounts share a usage pattern — high activation followed by a drop in weekly active sessions around day 45. An incident system won't surface that 71% of the ticket increase came from one error that appeared after last Thursday's release.

These connections require a person. Someone has to know which data sources matter, how they relate to each other, what normal looks like, and when to care. That person is usually the most expensive person on the team, and they're spending most of their time doing work that follows the same logic every time.

The repetitive part of analytical work is not a small fraction of the job. For most teams, it is most of the job.


What an Analyst Agent Does

An Analyst Agent does the connective work automatically. It doesn't replace dashboards, alerts, or incident systems. It sits on top of them and does the part that's currently manual: noticing what matters, gathering relevant context, identifying likely causes, and producing an explanation.

A dashboard says:

Revenue is down 18%.

An Analyst Agent says:

Revenue declined 18% week-over-week. The largest contributor is a reduction in enterprise conversions from the healthcare segment, accounting for 63% of the decline. Marketing spend remained constant. Pipeline volume from that segment hasn't changed, which suggests the issue is conversion-related rather than acquisition-related. The drop likely reflects something happening during the sales cycle rather than earlier.

This is the output a good analyst produces after two hours of work. The difference is it's produced in seconds, continuously, across every metric the organization tracks.

The value isn't speed. It's coverage. The Analyst Agent catches things nobody was looking at because nobody had time to look.


Concrete Examples

Product. Feature X usage dropped 22%. The drop is concentrated in users who onboarded in the last 30 days. A UI change shipped three days before the decline correlates with an increase in drop-off at the same step in the onboarding flow.

Retention. Churn increased this month. The churned accounts share a pattern: high initial activation followed by a decline in weekly active use around day 45. No automated intervention exists at that point in the lifecycle.

Support. Ticket volume is up 34%. 71% of the increase comes from a single error code that appeared after last Thursday's deployment. Resolution time for those tickets is 4x baseline.

In each case, the data was already there. What was missing was the work of connecting it.


Why This Is Buildable Now

Analyst Agents aren't a new idea. What's changed is that the components needed to build them actually work in production.

Context windows are large enough to hold meaningful slices of business history — not just individual data points, but the relationships between them. Models can reason over structured business context and produce coherent narratives from it. Event-driven architectures make it practical to trigger analysis when something changes rather than running scheduled batch jobs. Retrieval layers let agents pull in relevant historical context, segment definitions, and business rules without inventing them.

Two years ago the failure rate on each of these steps was too high to ship. That's no longer true.

The bottleneck was never the idea. It was the execution cost. That cost has dropped sharply.


What This Actually Changes

Dashboards aren't going away. Neither are analysts. The part that's going away is the labor between "something changed" and "here's what it means."

The filtering by segment and cohort. The cross-referencing across five different tools. The writing of the same summary in a slightly different format for the seventh week in a row. This is pattern matching and narrative generation. It's repetitive, it's expensive, and it's now automatable.

The work that remains is judgment. Knowing which questions to ask. Understanding organizational context that no system has been trained on. Navigating ambiguity. Deciding what to do with the explanation once you have it.

That work was always the valuable part. It just used to be buried under everything else.

Heramb

20 y/o engineering student by day, and indie hacker by night.

Design & Developed by heramb

© 2025. All rights reserved.