Roles at frontier labs · Systems and infrastructure
Inference Engineer
55 recorded postings · Partial update · October 8, 2026
Recorded postings, not a live vacancy count. Partial update: some sources retain earlier observations; each listing has its own check date.
Inference engineers make models fast and cheap to serve: batching, caching, quantization, kernels and the serving systems behind every API call.
- Inference and serving 100%
- How language models work 67%
- Production systems 67%
- Python 64%
- GPU kernels 56%
- C++, Rust or Go 56%
taught here course coming not taught
The numbers
- Recorded postings at 18 labs
- 55
- Median base pay middle half $270k to $403k, 33 U.S. bands
- $325k
- No degree requirement stated 0% require a PhD outright
- 80%
- Years asked, median of the 18 that state a minimum
- 5
The recorded snapshot
Postings from 18 labs
In the labs' own words
In the labs' own words
“Experience: 5+ years in a technical role, with a strong focus on inference systems, open-source LLM deployment, or post-training workflows.”
“Real depth in at least one accelerator ecosystem (CUDA/GPU, TPU, or Trainium/AWS Neuron) and genuine appetite to learn the others”
“Proficiency in C++ for systems development, including concurrent programming, memory ownership, and resource lifetime management.”
Your way in
Three doors to this role
- Door 1
Finishing a degree
7% mention publications
The way in is a program: 32 recorded in the snapshot
- Door 2
A software engineer
80% have no degree requirement stated
What to add
- Door 3
A PhD
27% ask for distributed training
Engineering at scale
Asked often
The short answers
How much does a inference engineer at a frontier lab make?
The median U.S. base salary across 33 recorded inference engineer postings that publish a band is $325k; the middle half runs from $270k to $403k. Equity and bonus are not included.
Do you need a PhD to be a inference engineer?
80% of the 55 recorded postings do not state a degree requirement, 7% mention a PhD or master's, and 0% require a PhD outright.
What skills do inference engineer postings ask for?
Most often: inference and serving (100%), how language models work (67%), production systems (67%), python (64%), gpu kernels (56%).
Which labs posted inference engineer roles in this recorded board?
Recorded board: Together AI (12), Anthropic (11), OpenAI (8), Cohere (4), Google DeepMind (3), H Company (2), and 12 more.
Prepare for it here
A free account opens the courses and saves your progress.
Create your free accountNewsletter
One lesson by email each week.
One lesson a week. Free, and it stays free.
The board
Recorded postings, newest first
The newest 40 of 55. Each title opens the lab's own posting. By family: ml systems and infrastructure · software engineering at labs · research scientists and engineers · machine learning engineers · applied ai and forward-deployed · fellowships, residencies and internships
- Research Engineer, Inference Foundation ↗
Mistral AI · Research scientists and engineers · No degree stated
- Software Engineer, Model Routing & Inference ↗
Cursor (Anysphere) · ML systems and infrastructure · No degree stated
- Software Engineer, Inference ↗
Thinking Machines Lab · ML systems and infrastructure · No degree stated · $300k to $400k base
- Engineering Manager, Inference Infrastructure ↗
Anthropic · ML systems and infrastructure · No degree stated · $405k to $625k base
- Performance Engineer, Inference Engine ↗
Anthropic · ML systems and infrastructure · No degree stated · $350k to $850k base
- Performance Engineer, Inference Systems ↗
Anthropic · ML systems and infrastructure · No degree stated · $350k to $850k base
- Staff + Senior Software Engineer, Inference ↗
Anthropic · ML systems and infrastructure · No degree stated
- Staff + Senior Software Engineer, Inference Deployment ↗
Anthropic · ML systems and infrastructure · No degree stated · $320k to $485k base
- Staff + Senior Software Engineer, Inference Infrastructure ↗
Anthropic · ML systems and infrastructure · No degree stated · $320k to $485k base
- Staff + Sr. Software Engineer, Cloud Inference ↗
Anthropic · ML systems and infrastructure · No degree stated · $320k to $485k base
- Staff + Sr. Software Engineer, Cloud Inference Launch Engineering ↗
Anthropic · ML systems and infrastructure · No degree stated · $320k to $485k base
- Staff Software Engineer, Inference ↗
Anthropic · ML systems and infrastructure · No degree stated
- Staff+ Software Engineer, Inference Velocity ↗
Anthropic · ML systems and infrastructure · No degree stated · $405k to $485k base
- Staff+ Software Engineer, ML Inference Path ↗
Anthropic · ML systems and infrastructure · No degree stated · $320k to $485k base
- Inference Engineer ↗
Cartesia · ML systems and infrastructure · No degree stated · $180k to $250k base
- Audio Inference Engineer, Model Efficiency ↗
Cohere · ML systems and infrastructure · No degree stated
- Lead Member of Technical Staff, Inference Infrastructure ↗
Cohere · ML systems and infrastructure · No degree stated · $235k to $325k base
- Site Reliability Engineer, Inference Infrastructure ↗
Cohere · ML systems and infrastructure · No degree stated · $160k to $260k base
- Staff Software Engineer, Inference Infrastructure ↗
Cohere · ML systems and infrastructure · No degree stated · $190k to $325k base
- Senior Security Engineer, Serving Layer and Platform Security, DeepMind ↗
Google DeepMind · Software engineering at labs · Bachelor's or equivalent
- Software Engineer, Model Inference, DeepMind ↗
Google DeepMind · ML systems and infrastructure · Bachelor's or equivalent
- Staff Software Engineer, Inference Performance Optimization, GenAI, DeepMind ↗
Google DeepMind · ML systems and infrastructure · Bachelor's or equivalent
- Research Engineer, Model Inference & Serving - London ↗
H Company · Research scientists and engineers · Master's or PhD required
- Research Engineer, Model Inference & Serving - Paris ↗
H Company · Research scientists and engineers · Master's or PhD required
- Research Engineer, Training & Inference ↗
Harmonic · Research scientists and engineers · PhD preferred · $200k to $450k base
- Staff Software Engineer (Inference Platform), London ↗
Isomorphic Labs · ML systems and infrastructure · No degree stated
- Member of Technical Staff - Inference Systems ↗
Liquid AI · ML systems and infrastructure · No degree stated
- Software Engineer, Inference ↗
Luma AI · ML systems and infrastructure · No degree stated · $195k to $345k base
- Member of Technical Staff, Inference & RL Systems ↗
Magic · Research scientists and engineers · No degree stated · $275k to $550k base
- Engineering Manager, Inference ↗
Mistral AI · ML systems and infrastructure · No degree stated
- Data Scientist, Inference Capacity Optimization ↗
OpenAI · Machine learning engineers · Master's, PhD or equivalent · $293k to $325k base
- Inference Engineer, Robotics ↗
OpenAI · ML systems and infrastructure · No degree stated · $380k to $380k base
- Operating Systems Engineer, On-Device Inference | Consumer Devices ↗
OpenAI · ML systems and infrastructure · No degree stated · $230k to $385k base
- Software Engineer, Inference - Multi Modal ↗
OpenAI · ML systems and infrastructure · No degree stated · $295k to $555k base
- Software Engineer, Inference - Performance Optimization ↗
OpenAI · ML systems and infrastructure · No degree stated · $266k to $500k base
- Software Engineer, Model Deployment- ChatGPT Engineering ↗
OpenAI · Software engineering at labs · No degree stated
- Software Engineer, Model Inference ↗
OpenAI · ML systems and infrastructure · No degree stated · $266k to $500k base
- Software Engineer, Productivity - Inference Runtime ↗
OpenAI · ML systems and infrastructure · No degree stated · $230k to $385k base
- Member of Engineering (Inference Infrastructure) ↗
Poolside · ML systems and infrastructure · No degree stated
- Research Engineer, Infrastructure, Inference ↗
Thinking Machines Lab · Research scientists and engineers · Bachelor's or equivalent · $350k to $475k base
