A Typical Day: - Design and productionize scalable data systems that support AI evaluation, metric computation, observability, and feedback loops for agentic AI products. - Build data models, schemas, and processing pipelines for agentic AI interactions, enabling reliable logging, retrieval, metric computation, and long-term evaluation dataset management. - Work with Core Modeling engineers to address pain points in the LLM evaluation process, developing LLM-as-a-judge solutions and data pipelines to meet metric challenges at scale. - Collaborate with machine learning infrastructure teams to evolve evaluation frameworks for Airbnb’s Conversational AI products. - Lead all phases of software development, including architecture design, implementation, and testing. - Collaborate with product managers, operations, and data scientists to identify opportunities for business impact

A Typical Day: - Design and productionize scalable data systems that support AI evaluation, metric computation, observability, and feedback loops for agentic AI products. - Build data models, schemas, and processing pipelines for agentic AI interactions, enabling reliable logging, retrieval, metric computation, and long-term evaluation dataset management. - Work with Core Modeling engineers to address pain points in the LLM evaluation process, developing LLM-as-a-judge solutions and data pipelines to meet metric challenges at scale. - Collaborate with machine learning infrastructure teams to evolve evaluation frameworks for Airbnb’s Conversational AI products. - Lead all phases of software development, including architecture design, implementation, and testing. - Collaborate with product managers, operations, and data scientists to identify opportunities for business impact