Full-time
Inference Systems Engineer
San Francisco, California, United States · On-site
$250,000 - $320,000
Inference Systems Engineer
Build the distributed inference infrastructure behind safety-focused healthcare AI.
Scaling LLM inference isn't just about making individual models faster. It's about how efficiently GPUs communicate, how workloads are distributed across nodes, and how reliably the entire system performs under production traffic.
We're working with a well-funded healthcare AI company developing safety-focused foundation models designed to support real-world patient interactions.
With over $400M raised and a team bringing together researchers and engineers from Google, Meta, Microsoft, NVIDIA and leading healthcare institutions, they're building AI systems where performance and reliability have direct implications for patient experience.
They're now looking for an Inference Systems Engineer to develop the distributed serving and high-performance networking infrastructure behind their LLMs.
The opportunity
You'll work at the intersection of LLM inference, distributed systems and GPU networking, tackling the performance challenges that emerge when serving large models across multiple nodes.
This isn't simply about deploying models through vLLM or SGLang.
You'll be working much deeper in the stack, from prefill/decode disaggregation and KV-cache transfer through to GPU interconnects, network topology and communication bottlenecks.
Your work will directly influence inference latency, throughput, reliability and cost per token, helping the team deploy increasingly capable models efficiently.
Your technical focus
Design and optimise disaggregated LLM serving architectures, including independent prefill/decode pools, KV-cache transfer, routing and batching.
Improve multi-node GPU communication using technologies such as NVLink, NVSwitch, InfiniBand, RDMA, NCCL and UCX.
Build and optimise inference systems using SGLang, vLLM, NVIDIA Dynamo or TensorRT-LLM.
Profile performance across compute, memory and networking, improving time to first token, inter-token latency, throughput and tail latency.
You'll also develop benchmarking and capacity-planning approaches, troubleshoot complex distributed failures and work with ML researchers to bring new models and hardware into production.
What you'll bring
You'll need strong experience building high-performance distributed systems, ideally within large-scale ML inference or HPC environments.
We're particularly interested in engineers with:
Deep understanding of LLM inference, including parallelism strategies, continuous batching, KV-cache management and prefill/decode behaviour.
Hands-on experience with GPU networking and communication technologies such as NCCL, NVLink, InfiniBand, RDMA or UCX.
Strong Python skills, alongside C++ or another systems programming language.
Experience profiling distributed systems and deploying reliable inference workloads on Kubernetes.
A Master's or PhD is required.
Experience with disaggregated serving, NVIDIA Dynamo/NIXL, CUDA, Triton or low-level GPU optimisation would be particularly relevant.
Why consider it?
You'll be working on problems that sit right at the intersection of ML systems and high-performance computing, with the opportunity to influence how large-scale inference infrastructure is designed and operated.
Rather than focusing on one isolated component, you'll have visibility across the entire serving path, from model execution through to the GPU fabric.
And importantly, the systems you build will support healthcare AI applications where low latency, reliability and consistent performance genuinely matter.
Package and location
Location: Menlo Park, California, five days onsite
If you've worked on large-scale distributed inference or GPU networking and enjoy getting into the internals of high-performance systems, this could be worth a conversation.
Contact Fraser at Techire AI: fraser@techire.ai
All applicants will receive a response.
Salary
$250,000 - $320,000
Location
San Francisco, California, United States · On-site
Contract
Full-time
Your consultant
Fraser Moran
Your consultant
Fraser Moran

Full-time
Inference Systems Engineer
San Francisco, California, United States · On-site
$250,000 - $320,000
Inference Systems Engineer
Build the distributed inference infrastructure behind safety-focused healthcare AI.
Scaling LLM inference isn't just about making individual models faster. It's about how efficiently GPUs communicate, how workloads are distributed across nodes, and how reliably the entire system performs under production traffic.
We're working with a well-funded healthcare AI company developing safety-focused foundation models designed to support real-world patient interactions.
With over $400M raised and a team bringing together researchers and engineers from Google, Meta, Microsoft, NVIDIA and leading healthcare institutions, they're building AI systems where performance and reliability have direct implications for patient experience.
They're now looking for an Inference Systems Engineer to develop the distributed serving and high-performance networking infrastructure behind their LLMs.
The opportunity
You'll work at the intersection of LLM inference, distributed systems and GPU networking, tackling the performance challenges that emerge when serving large models across multiple nodes.
This isn't simply about deploying models through vLLM or SGLang.
You'll be working much deeper in the stack, from prefill/decode disaggregation and KV-cache transfer through to GPU interconnects, network topology and communication bottlenecks.
Your work will directly influence inference latency, throughput, reliability and cost per token, helping the team deploy increasingly capable models efficiently.
Your technical focus
Design and optimise disaggregated LLM serving architectures, including independent prefill/decode pools, KV-cache transfer, routing and batching.
Improve multi-node GPU communication using technologies such as NVLink, NVSwitch, InfiniBand, RDMA, NCCL and UCX.
Build and optimise inference systems using SGLang, vLLM, NVIDIA Dynamo or TensorRT-LLM.
Profile performance across compute, memory and networking, improving time to first token, inter-token latency, throughput and tail latency.
You'll also develop benchmarking and capacity-planning approaches, troubleshoot complex distributed failures and work with ML researchers to bring new models and hardware into production.
What you'll bring
You'll need strong experience building high-performance distributed systems, ideally within large-scale ML inference or HPC environments.
We're particularly interested in engineers with:
Deep understanding of LLM inference, including parallelism strategies, continuous batching, KV-cache management and prefill/decode behaviour.
Hands-on experience with GPU networking and communication technologies such as NCCL, NVLink, InfiniBand, RDMA or UCX.
Strong Python skills, alongside C++ or another systems programming language.
Experience profiling distributed systems and deploying reliable inference workloads on Kubernetes.
A Master's or PhD is required.
Experience with disaggregated serving, NVIDIA Dynamo/NIXL, CUDA, Triton or low-level GPU optimisation would be particularly relevant.
Why consider it?
You'll be working on problems that sit right at the intersection of ML systems and high-performance computing, with the opportunity to influence how large-scale inference infrastructure is designed and operated.
Rather than focusing on one isolated component, you'll have visibility across the entire serving path, from model execution through to the GPU fabric.
And importantly, the systems you build will support healthcare AI applications where low latency, reliability and consistent performance genuinely matter.
Package and location
Location: Menlo Park, California, five days onsite
If you've worked on large-scale distributed inference or GPU networking and enjoy getting into the internals of high-performance systems, this could be worth a conversation.
Contact Fraser at Techire AI: fraser@techire.ai
All applicants will receive a response.
Salary
$250,000 - $320,000
Location
San Francisco, California, United States · On-site
Contract
Full-time
Your consultant
Fraser Moran
Your consultant
Fraser Moran

Full-time
Inference Systems Engineer
San Francisco, California, United States · On-site
$250,000 - $320,000
Inference Systems Engineer
Build the distributed inference infrastructure behind safety-focused healthcare AI.
Scaling LLM inference isn't just about making individual models faster. It's about how efficiently GPUs communicate, how workloads are distributed across nodes, and how reliably the entire system performs under production traffic.
We're working with a well-funded healthcare AI company developing safety-focused foundation models designed to support real-world patient interactions.
With over $400M raised and a team bringing together researchers and engineers from Google, Meta, Microsoft, NVIDIA and leading healthcare institutions, they're building AI systems where performance and reliability have direct implications for patient experience.
They're now looking for an Inference Systems Engineer to develop the distributed serving and high-performance networking infrastructure behind their LLMs.
The opportunity
You'll work at the intersection of LLM inference, distributed systems and GPU networking, tackling the performance challenges that emerge when serving large models across multiple nodes.
This isn't simply about deploying models through vLLM or SGLang.
You'll be working much deeper in the stack, from prefill/decode disaggregation and KV-cache transfer through to GPU interconnects, network topology and communication bottlenecks.
Your work will directly influence inference latency, throughput, reliability and cost per token, helping the team deploy increasingly capable models efficiently.
Your technical focus
Design and optimise disaggregated LLM serving architectures, including independent prefill/decode pools, KV-cache transfer, routing and batching.
Improve multi-node GPU communication using technologies such as NVLink, NVSwitch, InfiniBand, RDMA, NCCL and UCX.
Build and optimise inference systems using SGLang, vLLM, NVIDIA Dynamo or TensorRT-LLM.
Profile performance across compute, memory and networking, improving time to first token, inter-token latency, throughput and tail latency.
You'll also develop benchmarking and capacity-planning approaches, troubleshoot complex distributed failures and work with ML researchers to bring new models and hardware into production.
What you'll bring
You'll need strong experience building high-performance distributed systems, ideally within large-scale ML inference or HPC environments.
We're particularly interested in engineers with:
Deep understanding of LLM inference, including parallelism strategies, continuous batching, KV-cache management and prefill/decode behaviour.
Hands-on experience with GPU networking and communication technologies such as NCCL, NVLink, InfiniBand, RDMA or UCX.
Strong Python skills, alongside C++ or another systems programming language.
Experience profiling distributed systems and deploying reliable inference workloads on Kubernetes.
A Master's or PhD is required.
Experience with disaggregated serving, NVIDIA Dynamo/NIXL, CUDA, Triton or low-level GPU optimisation would be particularly relevant.
Why consider it?
You'll be working on problems that sit right at the intersection of ML systems and high-performance computing, with the opportunity to influence how large-scale inference infrastructure is designed and operated.
Rather than focusing on one isolated component, you'll have visibility across the entire serving path, from model execution through to the GPU fabric.
And importantly, the systems you build will support healthcare AI applications where low latency, reliability and consistent performance genuinely matter.
Package and location
Location: Menlo Park, California, five days onsite
If you've worked on large-scale distributed inference or GPU networking and enjoy getting into the internals of high-performance systems, this could be worth a conversation.
Contact Fraser at Techire AI: fraser@techire.ai
All applicants will receive a response.
Salary
$250,000 - $320,000
Location
San Francisco, California, United States · On-site
Contract
Full-time
Your consultant
Fraser Moran
Your consultant
Fraser Moran


