The chart that took thirty seconds to draw
We'd shipped a fix for chart zoom a few days earlier and thought the loop was closed. Then dragging to zoom on a chart with a forecast and an anomaly band on it locked the tab for what felt like forever before anything drew.
The first guess was the natural one, and it was wrong, which is worth saying because it's the guess anyone would make. The chart in question was a combo: two metrics, one drawn as bars. Bars looked like the obvious cost. They weren't. Timed directly, the same chart with a real forecast and anomaly overlay took 4 milliseconds to build with no overlays on, and 888 milliseconds with them on, at 700 points. At the point count a zoom onto a one-minute widget actually reaches, 4,320 points, the same build took 30.5 seconds. The freeze had nothing to do with bars. It was in the code that lines up an overlay's points with the chart's axis, and it scanned the whole overlay for every point on that axis, parsing two dates on every comparison. Quadratic cost, hiding behind a chart that had nothing wrong with its drawing.
The fix was one data structure: index the overlay's points once, by time, then look each axis value up by key instead of scanning for it. Same output. The same chart, doing twice the forecast and anomaly work the slow version did, now builds in about 53 milliseconds. We added a test that fails on purpose if that scan ever comes back, because a green suite had never once caught the original one.
Fixing the freeze surfaced the real question
While we were in there, we finally answered something worth asking properly: when a widget set to the minute is shown at a coarser grain because a month of minutes is too much to draw, and you zoom in, does the grain actually step down, and does the forecast follow it?
Both, now, and they're answered separately on purpose. How fine a tile draws is bounded by how many points the panel can render, so a wide window starts coarse and narrows a step at a time as you zoom, never past the grain you configured. The forecast and the anomaly band are a different decision: computed once, at the grain you set, over the widget's whole period. Zooming changes what's on screen, not what the model believes. A forecast that quietly refit itself to whatever window happened to be visible would let the same chart say two different things about the same near future, and neither answer would be wrong exactly, just disagreeing with the other.
A rule we shipped and un-shipped the same day
The first version of the anomaly rule was simple: flag the bucket on screen if any of the finer buckets inside it is anomalous. It tested clean. Then we looked at an actual chart, a one-minute widget viewed over a month, and every single day was flagged. Thirty-one out of thirty-one.
The math explains it once you look for it. At the threshold this uses, about a quarter of a percent of individual minutes fall outside the expected band by chance alone, in data behaving exactly as it should. That's roughly four minutes a day. "Any minute anomalous" and "a day worth your attention" are different claims, and we'd shipped the wrong one. The rule now asks whether the share of anomalous minutes inside a day is well above what chance alone would produce, not just above zero. A genuinely bad hour still flags its day. Four unremarkable minutes out of fourteen hundred don't.
No test caught that one either. The suite was green the whole time, because nothing in it knew what "too many days flagged" looked like until a chart full of markers said so.