Job overview: The ML Platform Engineer will focus on the systems engineering aspects of AI deployment, including request routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse model workloads. The ideal candidate brings strong distributed systems and performance engineering experience, has shipped serving systems at scale, and understands the trade-offs between latency, throughput, cost, and quality in ML serving.

Key Responsibilities: - Design and operate model serving platforms supporting diverse workloads including LLMs, vision models, and recommendation systems. - Optimize inference performance using continuous batching, paged attention, speculative decoding, and request multiplexing. - Implement multi-tenant routing, rate limiting, and quality-of-service policies across model endpoints. - Build autoscaling and capacity management systems that balance latency, throughput, and cost. - Tune GPU utilization, memory management, and KV cache strategies for LLM serving workloads. - Integrate model serving with API gateways, identity systems, and observability platforms.\

Job overview: The ML Platform Engineer will focus on the systems engineering aspects of AI deployment, including request routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse model workloads. The ideal candidate brings strong distributed systems and performance engineering experience, has shipped serving systems at scale, and understands the trade-offs between latency, throughput, cost, and quality in ML serving.

Key Responsibilities: - Design and operate model serving platforms supporting diverse workloads including LLMs, vision models, and recommendation systems. - Optimize inference performance using continuous batching, paged attention, speculative decoding, and request multiplexing. - Implement multi-tenant routing, rate limiting, and quality-of-service policies across model endpoints. - Build autoscaling and capacity management systems that balance latency, throughput, and cost. - Tune GPU utilization, memory management, and KV cache strategies for LLM serving workloads. - Integrate model serving with API gateways, identity systems, and observability platforms.\