Senior Technical Product Manager - AI Agents, Evals & Reliability

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Technical Product Manager - AI Agents, Evals & Reliability based in United States. This is a deeply technical product leadership role focused on turning advanced AI capabilities into reliable, user-facing experiences. You will shape systems designed to support long-running workflows, persistent context, multi-step reasoning, and real-world task completion. Working closely with ML, backend, mobile, and product teams, you will translate technical possibilities into clear product requirements and system decisions. A major focus will be building robust evaluation frameworks that measure quality, reliability, and user impact across AI systems. You’ll make high-leverage trade-offs involving quality, latency, cost, reliability, and user experience in a rapidly evolving environment. The role offers significant ownership within a high-talent-density, hands-on team where decisions move quickly and execution matters. Your work will directly influence how AI products behave, earn user trust, and deliver practical value at global scale. Define end-to-end requirements for AI-powered systems, connecting model capabilities and technical constraints to user needs and measurable product outcomes. Translate model performance, data limitations, evaluation results, and user feedback into clear product and system decisions. Partner closely with ML, backend, mobile, and other engineering teams on architecture, system design, evaluation, iteration, and product delivery. Establish and continuously improve evaluation frameworks spanning offline metrics, online experiments, human feedback, and real-world user outcomes. Make informed trade-offs across product quality, reliability, latency, cost, speed of execution, and overall user experience. Drive execution through clear product specifications, disciplined prioritization, strong technical judgment, and effective cross-functional alignment. Own product quality end-to-end, with particular emphasis on correctness, predictability, reliability, failure handling, and user trust. Identify weaknesses in AI behavior, including hallucinations, bias, model limitations, drift, and other sources of unreliable performance, and drive solutions to address them. Establish strong feedback loops that enable teams to learn quickly from production behavior and continuously improve AI-powered workflows. Ensure new AI capabilities are shipped efficiently and responsibly while maintaining high standards for quality and reliability. Help align product strategy, roadmaps, and priorities with user needs, AI capabilities, and broader business objectives. Bring structure to ambiguous problems and independently drive complex initiatives from definition through execution and measurable outcomes. Requirements: Strong foundation in computer science fundamentals, including algorithms, data structures, and system design, with the ability to engage deeply in technical discussions. Solid understanding of machine learning fundamentals and how modern AI systems behave in production environments. Significant experience owning complex, technically sophisticated products from strategy through execution. Hands-on experience with AI-powered products, particularly LLM-based systems, model evaluation, prompt or pipeline iteration, and AI feedback loops. Strong intuition for AI limitations and failure modes, including hallucinations, bias, model drift, and non-deterministic behavior. Ability to read, review, and discuss technical design documents and confidently collaborate with senior engineers and ML specialists. Demonstrated ability to make sound product and technical decisions in ambiguous, rapidly changing environments. Strong judgment and prioritization skills, with the ability to balance ambitious product goals against technical, operational, and commercial realities. Excellent communication and collaborati