Full-time

Model Serving Inference

San Francisco, California, United States · Hybrid

$250,000 - £320,000

ML Model Serving Engineer

Build the inference infrastructure behind the next generation of conversational AI.

Making AI conversations feel genuinely natural isn't just a model training problem. It's also about how quickly, efficiently and reliably those models can run in production.

We're working with an AI company developing a new generation of voice-based computing, where people can interact with intelligent systems through natural, human-like conversations.

Their team brings together founders and engineers with backgrounds at Oculus, Meta, Google and Apple, combining expertise across hardware, ML research and large-scale systems engineering.

They're now looking for an ML Model Serving Engineer to optimise the infrastructure serving their LLM, speech and vision models.

The opportunity

You'll work on some genuinely interesting inference challenges, where milliseconds matter and improvements in throughput, latency and compute efficiency directly influence the user experience.

This goes well beyond deploying models using existing frameworks.

You'll be modifying and extending frameworks such as vLLM and SGLang, exploring new inference optimisation techniques and working closely with ML researchers to improve model performance without compromising quality.

Your work will directly influence how their models are designed, deployed and scaled.

Your technical focus

  • Optimise LLM, speech and vision model serving for low latency, high throughput and reliability.

  • Modify and extend vLLM, SGLang and other serving frameworks to improve inference performance.

  • Implement techniques including in-flight batching, caching and custom kernels to reduce inference overhead.

  • Work closely with ML training engineers to identify model-level improvements and reduce model initialisation times.

What you'll bring

You'll need strong experience optimising ML models in production, combined with a solid understanding of the underlying systems.

Ideally, you'll have:

  • Deep PyTorch expertise and experience optimising inference performance at scale.

  • Strong systems programming skills, comfortable working with both complex ML codebases and high-performance serving infrastructure.

  • Experience profiling low-level systems, identifying bottlenecks and improving latency and throughput.

  • Knowledge of LLM serving frameworks such as vLLM or SGLang, ideally including their internals.

Experience with Kubernetes, Ray, cloud infrastructure or GPU-level optimisation would also be useful.

What's particularly interesting is the relationship between research and infrastructure. You'll have the opportunity to influence both how models are served and how they're developed to perform efficiently in real-world applications.

Package and location

Salary: $250,000 - $320,000
Location: SF/ NY/ Seattle

Benefits include employer-paid medical, dental and vision coverage, 401(k) matching, unlimited PTO and sick leave.

If you've spent time digging into the internals of model serving frameworks and enjoy solving difficult performance engineering problems, this is worth exploring.

Contact Fraser at Techire AI: fraser@techire.ai

All applicants will receive a response.

Salary

$250,000 - £320,000

Location

San Francisco, California, United States · Hybrid

Contract

Full-time

Your consultant

Fraser Moran

Your consultant

Fraser Moran

Full-time

Model Serving Inference

San Francisco, California, United States · Hybrid

$250,000 - £320,000

ML Model Serving Engineer

Build the inference infrastructure behind the next generation of conversational AI.

Making AI conversations feel genuinely natural isn't just a model training problem. It's also about how quickly, efficiently and reliably those models can run in production.

We're working with an AI company developing a new generation of voice-based computing, where people can interact with intelligent systems through natural, human-like conversations.

Their team brings together founders and engineers with backgrounds at Oculus, Meta, Google and Apple, combining expertise across hardware, ML research and large-scale systems engineering.

They're now looking for an ML Model Serving Engineer to optimise the infrastructure serving their LLM, speech and vision models.

The opportunity

You'll work on some genuinely interesting inference challenges, where milliseconds matter and improvements in throughput, latency and compute efficiency directly influence the user experience.

This goes well beyond deploying models using existing frameworks.

You'll be modifying and extending frameworks such as vLLM and SGLang, exploring new inference optimisation techniques and working closely with ML researchers to improve model performance without compromising quality.

Your work will directly influence how their models are designed, deployed and scaled.

Your technical focus

  • Optimise LLM, speech and vision model serving for low latency, high throughput and reliability.

  • Modify and extend vLLM, SGLang and other serving frameworks to improve inference performance.

  • Implement techniques including in-flight batching, caching and custom kernels to reduce inference overhead.

  • Work closely with ML training engineers to identify model-level improvements and reduce model initialisation times.

What you'll bring

You'll need strong experience optimising ML models in production, combined with a solid understanding of the underlying systems.

Ideally, you'll have:

  • Deep PyTorch expertise and experience optimising inference performance at scale.

  • Strong systems programming skills, comfortable working with both complex ML codebases and high-performance serving infrastructure.

  • Experience profiling low-level systems, identifying bottlenecks and improving latency and throughput.

  • Knowledge of LLM serving frameworks such as vLLM or SGLang, ideally including their internals.

Experience with Kubernetes, Ray, cloud infrastructure or GPU-level optimisation would also be useful.

What's particularly interesting is the relationship between research and infrastructure. You'll have the opportunity to influence both how models are served and how they're developed to perform efficiently in real-world applications.

Package and location

Salary: $250,000 - $320,000
Location: SF/ NY/ Seattle

Benefits include employer-paid medical, dental and vision coverage, 401(k) matching, unlimited PTO and sick leave.

If you've spent time digging into the internals of model serving frameworks and enjoy solving difficult performance engineering problems, this is worth exploring.

Contact Fraser at Techire AI: fraser@techire.ai

All applicants will receive a response.

Salary

$250,000 - £320,000

Location

San Francisco, California, United States · Hybrid

Contract

Full-time

Your consultant

Fraser Moran

Your consultant

Fraser Moran

Full-time

Model Serving Inference

San Francisco, California, United States · Hybrid

$250,000 - £320,000

ML Model Serving Engineer

Build the inference infrastructure behind the next generation of conversational AI.

Making AI conversations feel genuinely natural isn't just a model training problem. It's also about how quickly, efficiently and reliably those models can run in production.

We're working with an AI company developing a new generation of voice-based computing, where people can interact with intelligent systems through natural, human-like conversations.

Their team brings together founders and engineers with backgrounds at Oculus, Meta, Google and Apple, combining expertise across hardware, ML research and large-scale systems engineering.

They're now looking for an ML Model Serving Engineer to optimise the infrastructure serving their LLM, speech and vision models.

The opportunity

You'll work on some genuinely interesting inference challenges, where milliseconds matter and improvements in throughput, latency and compute efficiency directly influence the user experience.

This goes well beyond deploying models using existing frameworks.

You'll be modifying and extending frameworks such as vLLM and SGLang, exploring new inference optimisation techniques and working closely with ML researchers to improve model performance without compromising quality.

Your work will directly influence how their models are designed, deployed and scaled.

Your technical focus

  • Optimise LLM, speech and vision model serving for low latency, high throughput and reliability.

  • Modify and extend vLLM, SGLang and other serving frameworks to improve inference performance.

  • Implement techniques including in-flight batching, caching and custom kernels to reduce inference overhead.

  • Work closely with ML training engineers to identify model-level improvements and reduce model initialisation times.

What you'll bring

You'll need strong experience optimising ML models in production, combined with a solid understanding of the underlying systems.

Ideally, you'll have:

  • Deep PyTorch expertise and experience optimising inference performance at scale.

  • Strong systems programming skills, comfortable working with both complex ML codebases and high-performance serving infrastructure.

  • Experience profiling low-level systems, identifying bottlenecks and improving latency and throughput.

  • Knowledge of LLM serving frameworks such as vLLM or SGLang, ideally including their internals.

Experience with Kubernetes, Ray, cloud infrastructure or GPU-level optimisation would also be useful.

What's particularly interesting is the relationship between research and infrastructure. You'll have the opportunity to influence both how models are served and how they're developed to perform efficiently in real-world applications.

Package and location

Salary: $250,000 - $320,000
Location: SF/ NY/ Seattle

Benefits include employer-paid medical, dental and vision coverage, 401(k) matching, unlimited PTO and sick leave.

If you've spent time digging into the internals of model serving frameworks and enjoy solving difficult performance engineering problems, this is worth exploring.

Contact Fraser at Techire AI: fraser@techire.ai

All applicants will receive a response.

Salary

$250,000 - £320,000

Location

San Francisco, California, United States · Hybrid

Contract

Full-time

Your consultant

Fraser Moran

Your consultant

Fraser Moran

techire AI website footer image