Full-time
Model Serving Inference
San Francisco, California, United States · Hybrid
$250,000 - £320,000
ML Model Serving Engineer
Build the inference infrastructure behind the next generation of conversational AI.
Making AI conversations feel genuinely natural isn't just a model training problem. It's also about how quickly, efficiently and reliably those models can run in production.
We're working with an AI company developing a new generation of voice-based computing, where people can interact with intelligent systems through natural, human-like conversations.
Their team brings together founders and engineers with backgrounds at Oculus, Meta, Google and Apple, combining expertise across hardware, ML research and large-scale systems engineering.
They're now looking for an ML Model Serving Engineer to optimise the infrastructure serving their LLM, speech and vision models.
The opportunity
You'll work on some genuinely interesting inference challenges, where milliseconds matter and improvements in throughput, latency and compute efficiency directly influence the user experience.
This goes well beyond deploying models using existing frameworks.
You'll be modifying and extending frameworks such as vLLM and SGLang, exploring new inference optimisation techniques and working closely with ML researchers to improve model performance without compromising quality.
Your work will directly influence how their models are designed, deployed and scaled.
Your technical focus
Optimise LLM, speech and vision model serving for low latency, high throughput and reliability.
Modify and extend vLLM, SGLang and other serving frameworks to improve inference performance.
Implement techniques including in-flight batching, caching and custom kernels to reduce inference overhead.
Work closely with ML training engineers to identify model-level improvements and reduce model initialisation times.
What you'll bring
You'll need strong experience optimising ML models in production, combined with a solid understanding of the underlying systems.
Ideally, you'll have:
Deep PyTorch expertise and experience optimising inference performance at scale.
Strong systems programming skills, comfortable working with both complex ML codebases and high-performance serving infrastructure.
Experience profiling low-level systems, identifying bottlenecks and improving latency and throughput.
Knowledge of LLM serving frameworks such as vLLM or SGLang, ideally including their internals.
Experience with Kubernetes, Ray, cloud infrastructure or GPU-level optimisation would also be useful.
What's particularly interesting is the relationship between research and infrastructure. You'll have the opportunity to influence both how models are served and how they're developed to perform efficiently in real-world applications.
Package and location
Salary: $250,000 - $320,000
Location: SF/ NY/ Seattle
Benefits include employer-paid medical, dental and vision coverage, 401(k) matching, unlimited PTO and sick leave.
If you've spent time digging into the internals of model serving frameworks and enjoy solving difficult performance engineering problems, this is worth exploring.
Contact Fraser at Techire AI: fraser@techire.ai
All applicants will receive a response.
Salary
$250,000 - £320,000
Location
San Francisco, California, United States · Hybrid
Contract
Full-time
Your consultant
Fraser Moran
Your consultant
Fraser Moran

Full-time
Model Serving Inference
San Francisco, California, United States · Hybrid
$250,000 - £320,000
ML Model Serving Engineer
Build the inference infrastructure behind the next generation of conversational AI.
Making AI conversations feel genuinely natural isn't just a model training problem. It's also about how quickly, efficiently and reliably those models can run in production.
We're working with an AI company developing a new generation of voice-based computing, where people can interact with intelligent systems through natural, human-like conversations.
Their team brings together founders and engineers with backgrounds at Oculus, Meta, Google and Apple, combining expertise across hardware, ML research and large-scale systems engineering.
They're now looking for an ML Model Serving Engineer to optimise the infrastructure serving their LLM, speech and vision models.
The opportunity
You'll work on some genuinely interesting inference challenges, where milliseconds matter and improvements in throughput, latency and compute efficiency directly influence the user experience.
This goes well beyond deploying models using existing frameworks.
You'll be modifying and extending frameworks such as vLLM and SGLang, exploring new inference optimisation techniques and working closely with ML researchers to improve model performance without compromising quality.
Your work will directly influence how their models are designed, deployed and scaled.
Your technical focus
Optimise LLM, speech and vision model serving for low latency, high throughput and reliability.
Modify and extend vLLM, SGLang and other serving frameworks to improve inference performance.
Implement techniques including in-flight batching, caching and custom kernels to reduce inference overhead.
Work closely with ML training engineers to identify model-level improvements and reduce model initialisation times.
What you'll bring
You'll need strong experience optimising ML models in production, combined with a solid understanding of the underlying systems.
Ideally, you'll have:
Deep PyTorch expertise and experience optimising inference performance at scale.
Strong systems programming skills, comfortable working with both complex ML codebases and high-performance serving infrastructure.
Experience profiling low-level systems, identifying bottlenecks and improving latency and throughput.
Knowledge of LLM serving frameworks such as vLLM or SGLang, ideally including their internals.
Experience with Kubernetes, Ray, cloud infrastructure or GPU-level optimisation would also be useful.
What's particularly interesting is the relationship between research and infrastructure. You'll have the opportunity to influence both how models are served and how they're developed to perform efficiently in real-world applications.
Package and location
Salary: $250,000 - $320,000
Location: SF/ NY/ Seattle
Benefits include employer-paid medical, dental and vision coverage, 401(k) matching, unlimited PTO and sick leave.
If you've spent time digging into the internals of model serving frameworks and enjoy solving difficult performance engineering problems, this is worth exploring.
Contact Fraser at Techire AI: fraser@techire.ai
All applicants will receive a response.
Salary
$250,000 - £320,000
Location
San Francisco, California, United States · Hybrid
Contract
Full-time
Your consultant
Fraser Moran
Your consultant
Fraser Moran

Full-time
Model Serving Inference
San Francisco, California, United States · Hybrid
$250,000 - £320,000
ML Model Serving Engineer
Build the inference infrastructure behind the next generation of conversational AI.
Making AI conversations feel genuinely natural isn't just a model training problem. It's also about how quickly, efficiently and reliably those models can run in production.
We're working with an AI company developing a new generation of voice-based computing, where people can interact with intelligent systems through natural, human-like conversations.
Their team brings together founders and engineers with backgrounds at Oculus, Meta, Google and Apple, combining expertise across hardware, ML research and large-scale systems engineering.
They're now looking for an ML Model Serving Engineer to optimise the infrastructure serving their LLM, speech and vision models.
The opportunity
You'll work on some genuinely interesting inference challenges, where milliseconds matter and improvements in throughput, latency and compute efficiency directly influence the user experience.
This goes well beyond deploying models using existing frameworks.
You'll be modifying and extending frameworks such as vLLM and SGLang, exploring new inference optimisation techniques and working closely with ML researchers to improve model performance without compromising quality.
Your work will directly influence how their models are designed, deployed and scaled.
Your technical focus
Optimise LLM, speech and vision model serving for low latency, high throughput and reliability.
Modify and extend vLLM, SGLang and other serving frameworks to improve inference performance.
Implement techniques including in-flight batching, caching and custom kernels to reduce inference overhead.
Work closely with ML training engineers to identify model-level improvements and reduce model initialisation times.
What you'll bring
You'll need strong experience optimising ML models in production, combined with a solid understanding of the underlying systems.
Ideally, you'll have:
Deep PyTorch expertise and experience optimising inference performance at scale.
Strong systems programming skills, comfortable working with both complex ML codebases and high-performance serving infrastructure.
Experience profiling low-level systems, identifying bottlenecks and improving latency and throughput.
Knowledge of LLM serving frameworks such as vLLM or SGLang, ideally including their internals.
Experience with Kubernetes, Ray, cloud infrastructure or GPU-level optimisation would also be useful.
What's particularly interesting is the relationship between research and infrastructure. You'll have the opportunity to influence both how models are served and how they're developed to perform efficiently in real-world applications.
Package and location
Salary: $250,000 - $320,000
Location: SF/ NY/ Seattle
Benefits include employer-paid medical, dental and vision coverage, 401(k) matching, unlimited PTO and sick leave.
If you've spent time digging into the internals of model serving frameworks and enjoy solving difficult performance engineering problems, this is worth exploring.
Contact Fraser at Techire AI: fraser@techire.ai
All applicants will receive a response.
Salary
$250,000 - £320,000
Location
San Francisco, California, United States · Hybrid
Contract
Full-time
Your consultant
Fraser Moran
Your consultant
Fraser Moran


