← All roles
AI Systems Engineer
Infrastructure engineer with an LLM specialty. Builds prompt caches, semantic routers, model gateways, and the platform pieces that make every other engineer in the company shipped faster and cheaper.
Indicative comp$200K – $290K base (US, senior/staff)
Ranges are indicative US base salary at senior level. Actual offers depend on company stage, equity, and candidate strength.
What this role actually owns
- Build the prompt and inference platform the rest of the company uses.
- Implement caching, batching, and routing strategies that move the cost curve.
- Define provider failover policy — when do we fall back, and to what.
- Set latency budgets and enforce them with concrete instrumentation.
- Partner with finance on per-feature cost reporting.
What we screen for
- 7+ years infrastructure or platform engineering.
- Has built a system that other engineers depend on — gateway, cache, queue.
- Comfortable measuring and moving p95 latency.
- Has worked with at least two LLM providers in production.
- Bonus: contributed to LiteLLM, Portkey, or built an internal equivalent.
Sample job description
A starting point you can paste into your ATS and adjust. The exact wording matters less than the rubric — the bullets above are what we'll calibrate against during search.
AI Systems Engineer
Lives at the seam between agents and the systems they depend on — caching, routing, prompt infrastructure, model gateways.
You'll own:
- Build the prompt and inference platform the rest of the company uses.
- Implement caching, batching, and routing strategies that move the cost curve.
- Define provider failover policy — when do we fall back, and to what.
- Set latency budgets and enforce them with concrete instrumentation.
We're looking for:
- 7+ years infrastructure or platform engineering.
- Has built a system that other engineers depend on — gateway, cache, queue.
- Comfortable measuring and moving p95 latency.
- Has worked with at least two LLM providers in production.
- Bonus: contributed to LiteLLM, Portkey, or built an internal equivalent.