What happens when performance optimization becomes a continuous engineering process rather than a series of occasional fixes? Anthropic’s account of making the Claude web and desktop applications approximately three times faster in two weeks provides a useful case study. The most important insight is not a single optimization technique, but the development system behind it: measurable goals, AI agents that investigate and iterate, and engineers who retain responsibility for quality and release decisions.
This article looks beyond individual performance fixes to examine the engineering practices behind Anthropic’s results: how the team chose meaningful measurements, how agents investigated and verified improvements, and where human judgment remained essential. It then considers how those principles can be applied in practice, while clearly distinguishing ongoing evaluations from potential future initiatives.
1. What Anthropic Achieved in Two Weeks
During a two-week performance sprint in August 2026, Anthropic improved important user journeys in claude.ai and the Claude desktop app. This was an improvement to application responsiveness and user experience, not a claim that the underlying language model’s inference speed tripled.
- On a fresh web load, the 75th-percentile time until a user could type fell from 3.1 seconds to 0.55 seconds.
- Starting a new Claude Code session improved from 0.8 to 0.3 seconds.
- Loading a Claude Cowork cloud session improved from 2.6 to 0.73 seconds.
- Anthropic reported more than 3,000 merged changes during the sprint, with no customer-facing incidents or rollbacks.
Several improvements were straightforward in concept but demanding in execution. A static input composer embedded in HTML allowed users to begin typing before React finished initializing. The interface retained the composer when switching conversations, prefetched session data on hover, and reduced sidebar re-renders by 90%. For the desktop application, a precompiled V8 code cache reduced redundant startup compilation work.
Other optimizations addressed streaming responses. Completed content blocks could be reused instead of processed repeatedly, syntax highlighting for growing code blocks was moved to a worker, and tables were displayed incrementally. In one optimization effort, total main-thread blocking while rendering a long response fell from approximately 750 milliseconds to 200 milliseconds, with a 120 Hz benchmark used to verify smoothness.
These examples show the breadth of the engineering work. The more interesting question, however, is how the team found, tested, reviewed, and sustained so many improvements.
2. Measurement-Driven Optimization: Give Agents a Meaningful Target
Anthropic first identified four journeys responsible for approximately 95% of user activity: launching the application, starting a conversation, opening an existing conversation, and sending a message. Across different products and platforms, these became 13 measurements. Each measurement was designed to begin with a real user interaction and end when the relevant result appeared on screen, while distinguishing client-side and server-side work.
This is a different starting point from asking an agent to “make the application faster.” A clear baseline allows an agent to test a hypothesis, compare alternative implementations, and determine whether an optimization actually helped. Anthropic met 12 of its initial 13 performance targets by the third day, then expanded its search to newly measurable bottlenecks.
Why wall-clock time is not always enough
Response time is the outcome users experience, but small timing benchmarks can vary with machine load and execution conditions. To obtain stable signals for some JavaScript hot paths, Anthropic explored deterministic measurements such as instruction counts under Valgrind and Node.js’s --predictable mode. Other candidate metrics included React commits, function calls, style recalculations, and DOM mutations.
In two tested code paths, Claude reduced instruction counts by 48% and 31%. Measured execution time subsequently improved by 78% and 44%, respectively. These results mattered because they showed that, for those specific paths, the laboratory metric corresponded to real execution-time improvements.
The principle is important: a metric should be stable enough to guide automation and meaningful enough to reflect user value. If a benchmark was unreliable or failed to correlate with latency, Anthropic discarded it rather than allow an agent to optimize the wrong number.
Preventing regressions with a performance ratchet
Anthropic used a mechanism described as a performance ratchet. Once an instruction-count improvement was accepted, its benchmark became a CI guardrail. Later pull requests failed CI if they exceeded the accepted instruction-count ceiling; a daily process could lower the ceiling when further improvements were recorded.
This changes performance optimization from a one-time achievement into a property the development workflow actively protects. Each verified improvement becomes a stronger baseline for future work. The same idea can be applied to other measurable qualities, provided the checks are reliable and genuinely represent the intended behavior.
3. The Agent-Driven Improvement Loop
The sprint was not simply a matter of generating code faster. Anthropic used Claude Tag, an internal agent environment, alongside engineers in a shared Slack channel. Claude could trace execution flows, create benchmarks, propose changes, and examine post-deployment telemetry. Engineers remained involved in setting priorities, reviewing decisions, and approving changes.
The repeatable loop can be summarized as follows:
- Measure: Identify a slow user journey and establish a reliable baseline.
- Investigate: Trace the underlying execution path and formulate a testable explanation.
- Compare: Implement a targeted change and compare its benchmark result with the baseline.
- Validate: Review the change, deploy it safely, and examine real-user data.
- Preserve and repeat: Tighten regression checks when the change succeeds; otherwise, disable the feature if necessary, revise the hypothesis, and continue investigating.
At points, more than 150 Slack threads operated concurrently. Threads could continue discovering problems even after their original task was complete. Claude also began opening new investigations based on opportunities it identified, instead of waiting for a human to specify every next step.
A revealing example involved sidebar elements that shifted after the page had become usable. Existing high-level layout-shift metrics did not adequately expose the issue. The team introduced more specific telemetry by screen region and loading phase, while Claude created a reproducible integration test and used it to verify fixes. The process did not begin with a predetermined implementation; it began by making an undesirable user experience observable.
The broader lesson is that agents become more useful when they have a defined area to improve, instruments that show whether they are succeeding, and a way to continue the investigation without repeatedly requiring a new implementation instruction.
4. Human-in-the-Loop: Autonomy Needs Engineering Judgment
An agent can reduce a benchmark score without necessarily improving the product. A change may introduce edge cases, alter user-visible behavior, or create disproportionate maintenance costs. For this reason, Anthropic combined rapid agent-driven experimentation with explicit safety measures: automated review and at least one human approval for every pull request, tests before optimizations, feature flags for potentially disruptive changes, and gradual rollouts for high-risk features.
The original article groups the engineers’ steering responsibilities into three areas: Ambition, encouraging meaningful improvements; Taste, judging the experience and the trade-offs; and Direction, deciding what to prioritize, coordinate, or stop. One example makes this concrete: engineers declined a proposed 900-line change because saving roughly two milliseconds per message did not justify the added maintenance complexity.
A practical interpretation is to keep human approval for every change while adjusting the depth of human involvement according to risk. This is an application of the lessons, not a formal policy stated by Anthropic:
- Low-risk optimizations: Agents can investigate, implement, and run tests with substantial autonomy; a human evaluates the evidence and approves the change.
- Changes affecting behavior or user experience: Human reviewers also assess design assumptions, product expectations, and possible unintended consequences.
- Major architectural changes: Humans establish direction before implementation, review progress at appropriate checkpoints, and make the final approval decision.
This separation is not an argument against automation. It is a way to make automation sustainable. Agent-to-agent review can add another layer of scrutiny, but neither passing tests nor agreement between agents is a substitute for human accountability.
5. From Engineering Lessons to iFactiS Knowledge
Introducing iFactiS Knowledge
To see where these engineering lessons may be useful in practice, it is helpful to begin with the product and the problems it addresses. iFactiS Knowledge is an AI-powered knowledge management solution for manufacturing environments. It is designed to help teams find relevant information in internal documents and manuals, draw on past troubleshooting cases, support report creation, and share experienced workers’ operational knowledge. By making existing information easier to find and reuse, the product aims to reduce time spent searching and support more consistent decision-making on the factory floor.
Official product introduction: iFactiS Knowledge (Japanese).
With this product context in mind, Anthropic’s approach offers two areas worth exploring in the engineering process behind iFactiS Knowledge. The first is to strengthen development quality through agent-assisted workflows and cross-review, an area already being evaluated. The second is to consider how measurement-driven optimization could be extended to document retrieval and AI-generated answers in the future.
Current exploration: agent harnesses and cross-review for development quality
As part of ongoing development practices, the team is evaluating an agent harness that involves Claude Code and Codex. One area of investigation is how to keep the main CLAUDE.md project guidance concise (targeting 200 lines or fewer), move detailed guidance into smaller supporting documents where appropriate, and verify whether Codex correctly references and follows those instructions.
The objective is broader than instruction-file maintenance. The harness also has a quality improvement and cross-review dimension: using two agents to examine proposed changes from different perspectives, question assumptions, and identify potential inconsistencies with requirements or project conventions. Such mutual review can complement conventional testing and human code review; it does not guarantee correctness on its own.
These are ongoing evaluations, not claims that a fully autonomous multi-agent review system has already been established. The work includes checking expected behavior after restructuring guidance, investigating differences in agent responses, comparing outcomes, and validating them against project requirements. In this limited but practical sense, the process already incorporates a measure → investigate → compare → validate mindset.
A next step is to make these checks more repeatable and systematic: define representative evaluation cases, retain useful results as regression checks, and expand agent-assisted investigation and cross-review where the evidence supports it. The aim is to strengthen software quality while keeping engineering judgment and final approvals with people.
Future direction: optimize RAG performance without sacrificing answer quality
A further possibility, still under consideration rather than presented here as an implemented capability, is to use the same measurement-driven approach for iFactiS Knowledge’s retrieval-augmented generation (RAG) workflow. Potential steps include:
- Measure the experience: Establish baselines for document retrieval, search APIs, answer generation, and frontend rendering, including end-to-end latency as seen by users.
- Investigate bottlenecks: Let agents analyze search logic, data access, backend processing, and rendering to identify candidate improvements.
- Compare reproducibly: Benchmark changes against representative queries and a stable dataset rather than relying on isolated successful examples.
- Validate answer quality: Check retrieval relevance, factual correctness, grounding in source documents, and appropriate handling of questions without sufficient evidence.
- Improve safely: Add meaningful regression checks, request focused changes, and retain human review before merging and deploying them.
The central constraint is simple: a faster RAG response is not necessarily a better response. Retrieving fewer documents may save time yet omit critical evidence. Aggressively shortening generated answers may reduce latency while removing useful explanations or qualifications. An optimization should therefore satisfy both performance goals and answer-quality safeguards.
For a manufacturing knowledge system, this balance is especially important. Users may rely on retrieved procedures or past troubleshooting knowledge to decide what to check next. Speed matters, but accuracy, relevance, and trustworthiness must remain explicit evaluation criteria. A measured improvement in latency is valuable only if those qualities are preserved.
6. Engineering Takeaways: Build a System for Improvement
A central engineering lesson from Anthropic’s experience is that AI-assisted development can move beyond generating individual code changes. When agents have meaningful measurements, reproducible benchmarks, access to relevant diagnostic tools, and clearly defined boundaries, they can support a continuing cycle of investigation and optimization.
Equally important is what those measurements cannot tell us. A smaller number may not mean a better product. Automated checks can miss assumptions that were never encoded, and a technically valid patch can still be a poor choice for maintainability or user experience. This is why human oversight and agent-assisted cross-review should be treated as complementary elements of a quality-oriented engineering process.
The goal is not to remove engineers from the improvement loop. It is to let agents perform more of the measurable, repeatable work while people remain responsible for product quality, engineering trade-offs, and release decisions.
For day-to-day engineering, the most useful next step is not to imitate the scale of Anthropic’s sprint. It is to begin with a small, verifiable loop, improve the evaluation process itself, and gradually expand the work that agents can perform reliably. The ongoing harness evaluations around iFactiS Knowledge provide one opportunity to develop that discipline; the potential RAG performance-and-quality evaluation loop offers a further direction.
The longer-term objective is more agent-driven development and stronger quality assurance at the same time: faster discovery of issues, more consistent verification, and deliberate human judgment over what ultimately reaches users.
References
- Raymond Wang, Sam Attard, and Issac G., How we made claude.ai 3x faster in two weeks, claude.dev, September 23, 2026. Anthropic’s reported metrics, technical examples, and sprint practices are drawn from this article.
- iFactiS Knowledge — Official Product Introduction (Japanese), iFactiS YouTube channel, August 27, 2026.