Engineering & product
What we're learning while building the data stack we wish we'd had.
Engineering decisions. Measurements. Security tests. Things that broke. Things we got wrong. And the reasoning behind the product.
Featured themes
01
Building analytics without building a data department
Why collecting data is easy, while making it useful is not.
02
Asking data questions in plain language
How natural-language analytics can work without giving an AI unrestricted access to your warehouse.
03
The infrastructure behind customer-facing analytics
The ingestion, storage, isolation and query problems hiding underneath “just give customers a dashboard.”
04
What the measurements say
Performance experiments, benchmarks and the boundaries around the numbers.
05
Security you can inspect
Red-team findings, tenant isolation and the controls we actually have.
No black box. No perfect story.
We publish the decisions, measurements and mistakes because the product should be understandable before it is trusted.
-
Put your dashboards on your own page
A read-only public API for the dashboards you share, an open source library that draws them on any web page, and short-lived sessions so that no credential of ours ever sits in your HTML. Here is how it works, and the decisions that made it safe to hand to a stranger.
engineeringapisecurity -
We pointed Event Horizon at Event Horizon
Our own servers, containers, ClickHouse and API requests now flow through the same pipelines, builder, anomaly bands and forecasts our customers use. Here is what that looks like, and how to build the same thing for your stack.
engineeringobservability -
The price that changed twice
Most ecommerce data lives in MySQL or Postgres, and a row doesn't always mean an event. Here's what we built so a changing product still has one current answer, and a real bug we found chasing the fix.
engineeringecommerce -
The chart that took thirty seconds to draw
A zoom fix we'd shipped days earlier turned out to be incomplete, and the bug we found chasing it down had nothing to do with what we blamed first. Here's the mistake, the fix, and a rule we shipped and un-shipped the same day.
engineeringperformance -
Compare any period to the one before
Month over month, quarter over quarter, year over year. The most common question about any number is "compared to what?", and it should not need SQL or a calculator. Here is where it lives in the builder and how to read it.
product -
Ask your data a question, in your own words
Type the question the way you'd say it out loud, and get a query matched to the values that are really in your data. On our own machines, and only ever showing you what it can prove.
productai -
A blank canvas is a question
Our ingestor opened on an empty grid, which felt like freedom and was actually an interrogation. Fixing it meant admitting that every question we asked first was about our architecture, and none of them were about the customer.
engineeringproduct -
The reasons I gave myself were wrong
I spent years being sure about how analytics should scale. A database in a cluster, OLAP cubes, then a columnar engine. Every one of those certainties died against a measurement, including the ones I used to justify the migration I still think was right.
engineeringarchitecture -
SQL and notebooks, without a warehouse to run
The default stays no-SQL, for the people who have the question but not the query. But some of your team writes SQL, and lives in a notebook. So we added both, on rails: you write against your own tables, never our warehouse, and every query is cost-checked before it runs.
productsqlanalytics -
Security you can check, not just trust
Multi-tenant isolation is easy to claim and hard to prove. So we red-team our own, against the live system, and publish the result. Here's how your data stays yours, and where to check us.
securitytrust -
Turning events into decisions
Every team collects event data. Far fewer turn it into answers. Here's the gap we set out to close, and why we built the whole stack, not another dashboard.
analyticsproduct