Jeff Dean’s 1% Rule for AI Founders

Source: Y Combinator · Speaker: Jeff Dean, interviewed by Diana Hu · Video ID: CxXgV54KzpQ · Original title: “Jeff Dean: The 1% Rule for Building in AI”

The best startup opportunities in AI are not where frontier models are already limping toward competence. Jeff Dean’s sharpest advice is to hunt for domains where the general model succeeds 0% or 1% of the time — not 20% — then build the surface, data, skills, evaluators, and systems that make that domain suddenly work.

Who Is Jeff Dean?

Jeff Dean is Google’s Chief Scientist and one of the central builders of modern internet and AI infrastructure. His fingerprints are on MapReduce, Bigtable, TensorFlow, TPUs, Gemini, distillation work, and the engineering culture that made Google’s large-scale systems possible.

That history gives his advice unusual weight. Dean is not offering abstract futurism; he is explaining how consequential systems often begin as back-of-the-envelope calculations, first-principles bottleneck hunting, and tiny teams willing to question assumptions everyone else treats as fixed.

The 1% Rule: Build Where General Models Still Fail

Dean’s clearest startup filter is brutally practical. General models are getting better across a broader range of tasks. If they already perform moderately well in a domain — if they succeed 20% of the time — that may be a warning sign rather than an invitation. The capability is present, and more data, scale, and post-training may soon close the gap.

The better opening is where the general model fails almost completely. Look for a problem where the model succeeds 0% or 1% of the time, especially because it lacks access to the right private data, domain-specific tools, high-quality evaluators, or specialized workflow context. That is where a small team can still build something delightful, accurate, and durable while frontier labs remain focused on broad capability.

The founder’s question is not “Can a general model sort of do this?” It is “What can become high accuracy only when wrapped in the right product, data, skills, and evaluation loop?”

Dean names two promising shapes. The first is private or user-specific data: Google may organize the world’s information, but organizing a person’s own information remains open because the model does not automatically have access to it. The second is a specialized domain where a niche model can outperform a general one, as AlphaFold did for protein folding and as similar approaches may do in material science or chip design.

Napkin Math Is Strategy, Not Trivia

Dean’s career repeatedly turns on “it fits” moments. In 2001, Google search still ran on hard drives. Dean and Sanjay Ghemawat did the math and realized Google’s full search index could fit in the RAM across the machines they already had. Within days, they shipped a new in-memory search system, dramatically improving speed.

The TPU began from a different napkin calculation. Around 2013, deep-learning-based speech recognition had cut error rates roughly in half — the equivalent of about 20 years of progress in a few months — but it was much more computationally expensive than the old system. If every Google user spoke to their phone for only three minutes per day, Google would need to roughly double its server fleet. The answer was not “buy more CPUs.” It was specialized hardware for low-precision dense linear algebra.

The first TPU generations were 30 to 80 times more energy efficient than CPUs and GPUs of the day, with 20 to 30 times lower latency. Crucially, the TPU was specialized but not over-specialized. It could not run Chrome or Word, but it could run the linear algebra primitives that kept showing up across evolving machine-learning algorithms, including transformers that arrived later.

The New Latency Numbers Every AI Engineer Should Know

Dean’s famous “latency numbers every programmer should know” helped distributed-systems engineers develop intuition for cache misses, disk seeks, and network hops. The AI-era version shifts the mental model toward memory bandwidth, interconnects, precision, and energy.

Founders building AI systems should understand the bandwidth between accelerator memory and on-chip memory, the energy cost of a multiplier operation, the interconnect bandwidth between chips, and how performance falls off when a system expands from hundreds to thousands or tens of thousands of chips. These numbers are not hardware trivia. They determine latency, cost, user experience, and which product architectures are possible.

The most important unit may be energy. Diana Hu frames a striking gap: doing a calculation can cost roughly one picojoule, while moving the data needed for that calculation can cost about a thousand times more. Dean points out that this difference shapes batching, model serving, and training. Batching exists partly to amortize data movement. But batching conflicts with low latency, which is why Dean is especially interested in inference hardware: minimizing movement, using extremely low precision where possible, and specializing around the needs of real-time agent systems.

Context Engineering Is the New Product Surface

Dean pushes against the idea that AI progress is only model size, parameters, and training data. The model is one component inside a larger system. Real capability comes from retrieval, tools, memory, prior work history, decomposition strategies, multi-agent orchestration, and evaluation.

Context is powerful because it is visible to the model in a way training data is not. Training data is “trillions of tokens stirred together into a soup” of parameters. Context, by contrast, is the current problem, the relevant data, the available tools, and the explicit instructions placed directly in front of the model. That makes context engineering one of the most accessible frontiers for founders: it does not require training a frontier model, only building better harnesses, tools, skills, and product workflows around existing APIs.

Dean’s example is concrete. He and Sanjay work on low-level performance improvements for libraries used across millions of Google processes. They have a microbenchmark library for measuring operations, cache footprints, and data-structure performance. They wrote a skill that taught an agent their human optimization loop: measure benchmarks, change code, rerun, inspect broader benchmarks, evaluate cache footprint, and iterate. They also published a 30-page “performance hints” document that others have fed into models to improve performance reasoning.

Long-Running Agents Need Skills, Search, and Evaluators

Dean believes people have not fully internalized how long agentic systems can run. In some domains, capable models can work for days or weeks on complicated tasks, such as implementing a new version of software in a different programming language for better safety or performance. But current agents still frequently go off the rails after dozens of tool interactions.

His diagnosis is classic machine learning: the model drifts off distribution. It starts doing something it has not seen much of, performance degrades, errors compound, and an open-loop process loses reliability. The fixes are also systems-oriented: give the model skills and hints that keep it on a brightly lit path, run multiple agents against the same problem, evaluate which approaches look promising, discard bad branches, and use inference-time compute to search over plausible solutions.

This is the deeper meaning of “AI-native founder.” It is not merely using agents to write code. It is learning to manage computation as a fleet: crisp specs, strong evaluators, explicit skills, parallel attempts, and taste about what is worth asking the agents to do.

Taste Becomes the Scarce Skill

If everyone can run 50 or 100 agents, the scarce skill becomes deciding what those agents should work on. Dean compares this to research: brilliant execution on a boring problem matters less than choosing a problem that, if solved, changes what is possible.

He offers a practical way to build taste. Write down many things you think might become important in the next 12 months. Pick one to work on, but revisit the entire list a year later. Which predictions mattered? Which did the world build? Which still remain open? That exercise creates more samples for your internal judgment.

He also argues for crazy thought experiments. Question assumptions that have been stable for decades. What if chips did not require every transistor to be nearly perfect? What if systems were built from unreliable transistor-level components the way distributed file systems are built from unreliable disks? The point is not that every wild premise should become a company. Most will fail for good reasons. The point is that MapReduce and TPUs came from this same habit of squinting at a familiar system and asking whether the abstraction boundary was wrong.

The Faster Scientific Method

Dean’s 2027 prediction is that AI systems will increasingly improve ML systems themselves. The loop is scientific: propose an experiment, implement what is needed to run it, evaluate the result, and incorporate what worked. When that loop is automated and low latency, one system can decompose high-level objectives into subproblems, run many experimental loops, and combine the best results into a stronger overall solution.

The same pattern applies beyond ML. In quantum chemistry, Dean describes colleagues replacing a slow density-functional-theory simulator — something that might take a night of computation for one molecule — with a neural approximation that was 300,000 times faster and nearly as accurate. That changes science. Ten million candidates can be screened during lunch instead of over months of expensive compute. Faster validators create faster discovery loops.

Key Lessons

Why This Matters for Diffie

For Anand and Diffie, Dean’s 1% rule is a direct ICP filter. The best customers are not teams where generic agents can already reliably inspect a UI, infer user intent, navigate edge cases, and produce trustworthy browser-testing evidence. The best customers are teams where today’s general agents fail almost completely because the surface is private, stateful, product-specific, and full of implicit frontend knowledge.

That points Diffie toward complex web products with proprietary workflows: internal tools, B2B SaaS dashboards, healthcare or fintech flows, AI product interfaces, admin consoles, design-system migrations, and role-dependent experiences. A general model may understand HTML and screenshots, but it does not know the customer’s release intent, historical regressions, authenticated flows, expected visual behavior, or which failure modes are expensive. Diffie can win by wrapping the model in the right context, browser tools, memory, and evaluation loop.

The outbound message should be framed around systems leverage, not generic AI automation. Dean’s TPU story is about finding the bottleneck and changing the architecture. Diffie’s equivalent bottleneck is engineering attention lost to slow, brittle, under-specified frontend QA. The claim should be concrete: Diffie reduces the cost of validating UI changes by turning private product context into repeatable browser checks and evidence that engineers can trust.

Product-wise, Dean’s emphasis on skills and evaluators is especially relevant. Diffie should not only run an agent through a browser; it should teach the agent the team’s testing skills, product invariants, design-system rules, and release-specific goals. It should use evaluators to separate promising findings from noise. It should write results back into the pull request, issue tracker, or release workflow so the decision trail survives beyond one run.

The strategic bar is Dean’s final question: if the best possible outcome happens, does the world get meaningfully better, or does it merely shrug? For Diffie, the meaningful version is not “another AI QA tool.” It is making frontend teams confident enough to ship faster without normalizing quality debt — a durable wedge where general models alone are still far below 1% reliable.