← All insights

Hiring rubric

The skill shift: from writing code to designing the system around it.

AI writes the lines. Senior engineers earn their salary on what surrounds them — contracts, boundaries, eval loops, and the orchestration that holds it all together. Here's what that means for the rubric.

Nick Savov5 min read

A senior engineer in 2020 spent most of their week writing code. A senior engineer in 2026 spends most of their week deciding what code should exist, how the pieces should talk, where the system is allowed to fail, and how anyone will know whether it's actually working. The implementation has not gone away — but it has gotten dramatically cheaper, and the value-add has migrated upward.

We see this on every search we run. Hiring managers don't describe their best engineers as the fastest typists anymore. They describe them as the people who can take a fuzzy product brief, decompose it into a set of components with clear contracts, and reason about the failure modes before a single line is written. That used to be a principal-level skill. It's now table stakes for senior.

What's actually changing

The implementation layer is being absorbed by AI. Boilerplate is gone. Routine CRUD, glue, refactors, test scaffolds, simple algorithms — all of it is increasingly produced by an agent under human review. That doesn't make engineering easier; it moves the bottleneck. Three things now consume the time that used to go into typing:

  • System decomposition. What are the components? What does each one own? What are the contracts between them? Get this wrong and the AI happily generates thousands of lines of well-shaped code in the wrong shape.
  • Orchestration. How do the pieces fit together at runtime? What retries, what timeouts, what fallback paths, what idempotency guarantees, what backpressure? In an agent system, this is the entire product.
  • Evaluation.How will you know the system is working? What's the golden set, what's the grader, what regressions block a release? An LLM-shaped system without an eval harness is a vibes-driven product.

None of these are new ideas. What's new is that they used to be the last 20% of a senior engineer's job. Now they're the first 80%.

Why orchestration is the new core skill

When you build with agents, you're no longer writing a program in the classic sense. You're composing a graph of probabilistic components, each of which can hallucinate, time out, exceed a token budget, or call a tool wrong. The interesting engineering is in the seams: how state flows through the graph, where the human-in-the-loop checkpoints sit, how partial failures cascade or don't, how observability is structured so you can actually debug a misbehaving run an hour after it happened.

Engineers who try to brute-force this with one big prompt and a try/catch don't ship. Engineers who treat it as a distributed-systems problem do. The skill set that wins looks much more like a staff backend engineer's — queues, idempotency, circuit breakers, structured logs — than a frontend or ML engineer's. That surprises people, and it's a big part of why we recruit the way we do.

What this means for hiring

The screen has to follow the work. We've quietly retired a few signals that don't carry weight anymore:

  • LeetCode sprints. AI does these in seconds. Selecting for them selects for nothing meaningful about how someone designs a system.
  • Framework familiarity for its own sake. Whether someone has used LangGraph or Mastra or rolled their own loop matters less than whether they can articulate why a graph-shaped runtime is the right model for the problem.
  • Lines-shipped proxies.A PR count meant something when typing was the bottleneck. It doesn't now.

And we've leaned harder into a few signals that do carry weight:

  • System-decomposition fluency.Hand a candidate a fuzzy brief (“build the intake agent for an HR platform”) and watch what they ask first. Strong candidates ask about contracts, ownership, and failure modes before they ask about the model.
  • Eval thinking.“How would you know this is working in production?” is a one-question screen for seniority. Weak answers talk about unit tests; strong answers talk about graders, golden sets, regression gates, and what they'd page on.
  • Explicit reasoning about the human in the loop.Senior engineers building with agents have opinions about which decisions stay with humans and why. That's a product instinct as much as a technical one, and it's the difference between an agent that ships and one that gets quietly turned off.

What this means if you're an engineer

If you've been worried the AI is coming for your job, the honest answer is: it already came for the part of the job you probably didn't enjoy anyway. The part that's left — design, orchestration, evaluation, judgement — is the part senior engineers were always supposed to be paid for.

The fastest way to grow into this is to stop measuring yourself by lines shipped and start measuring by the systems you can hold in your head. Pick one production agent system you've worked on and write down its contracts, its failure modes, and its eval rubric, end to end, on one page. If you can't, that's the skill to build next. If you can, that's the work we recruit for.

What this means if you're hiring

Look at the JD you're about to post. If the bullets are mostly about languages, frameworks, and years of experience, you're going to attract people who optimize for those signals — and miss the people who actually build production agent systems. Rewrite the bullets around the work: what this engineer will own, what contracts they'll define, what they'll be on the hook for when something breaks at 2am.

That's the rubric we calibrate every search against. It's also why we only recruit for agentic AI roles — the screen doesn't generalize, and we don't pretend it does.

Hiring for, or growing into, this kind of work? We'd like to hear about it.