Senior IA Inference Engineer (x/f/m)

About the role

<h2 class="pb-2 pt-4 heading-xl text-foreground"><strong class="font-semibold text-foreground">Set a new pulse for healthcare!</strong></h2> <div class="whitespace-pre-wrap break-words font-normal first:pt-0 last:pb-0 py-1 @sm:pt-2 @sm:pb-[10px] @sm:leading-relaxed text-base leading-relaxed text-foreground">We are looking for a <strong class="font-semibold text-foreground">Senior Inference Engineer</strong> to join the <strong class="font-semibold text-foreground">ML Platform (TIGER)</strong> team in <strong class="font-semibold text-foreground">AI &amp; Machine Learning</strong>.</div> <div class="whitespace-pre-wrap break-words font-normal first:pt-0 last:pb-0 py-1 @sm:pt-2 @sm:pb-[10px] @sm:leading-relaxed text-base leading-relaxed text-foreground">Your mission will be to power Doctolib's AI-driven healthcare products — from Consultation Assistant to Phone Assistant and Health Companion — by owning the self-hosted LLM serving platform that keeps the most sensitive patient data within infrastructure Doctolib controls. You will build and operate the shared foundations that every AI feature team relies on: deploying, tuning, and scaling open-weight models across our GPU fleet, contributing directly to faster, safer, and more cost-efficient AI for care teams and patients across Europe.</div> <div class="whitespace-pre-wrap break-words font-normal first:pt-0 last:pb-0 py-1 @sm:pt-2 @sm:pb-[10px] @sm:leading-relaxed text-base leading-relaxed text-foreground">Working in the tech team at Doctolib means building innovative products and features to improve the daily lives of care teams and patients.</div> <h2 class="pb-2 pt-4 heading-xl text-foreground"><strong class="font-semibold text-foreground">What you'll do</strong></h2> <div class="whitespace-pre-wrap break-words font-normal first:pt-0 last:pb-0 py-1 @sm:pt-2 @sm:pb-[10px] @sm:leading-relaxed text-base leading-relaxed text-foreground">Your responsibilities include but are not limited to:</div> <ul class="pb-2 flex flex-col gap-1 list-disc pl-6 text-foreground text-base leading-relaxed"> <li class="break-words text-foreground text-base leading-relaxed">Build the shared LLM serving foundations with MLOps engineers — define the serving stack, GPU capacity planning, and deployment automation for open-weight models (Gemma, Mistral, Llama, Qwen, and their fine-tuned variants) on Ray Serve / Anyscale, while shaping the evolution toward Kubernetes-based orchestration</li> <li class="break-words text-foreground text-base leading-relaxed">Own the model registry and promotion path — establish clear processes for versioning, validating, and promoting models into production, benchmark performance and output quality under realistic workloads, and maintain the curated model catalog together with the Agents Platform team</li> <li class="break-words text-foreground text-base leading-relaxed">Optimize inference performance and efficiency — operate and tune serving engines (vLLM, Triton Inference Server, TensorRT-LLM) to reduce time-to-first-token, inter-token latency, and cost per token through batching, caching, quantization, and parallelism strategies across our GPU fleet (L4 to H200)</li> <li class="break-words text-foreground text-base leading-relaxed">Own production reliability and disaster recovery — define SLOs, run load tests before launches, implement safe rollouts, rollbacks, and autoscaling, and participate in on-call and incident response</li> <li class="break-words text-foreground text-base leading-relaxed">Make serving performance and costs measurable — build Datadog dashboards, alerting, and cost reporting to track latency, GPU utilization, and inference spend, and identify opportunities to improve resource efficiency</li> <li class="break-words text-foreground text-base leading-relaxed">Keep the inference stack secure, compliant, and evolving — work with Security and Privacy on network isolation, data residency, and auditability, while continuously assessing new models, serving

Skills