Senior AI OPS Engineer

We are looking for a Senior AI OPS Engineer to help build an Enterprise AI Gateway from scratch. It is the one door every team in the company goes through to use large language models. It is also how we keep AI under control at this scale: who can use which models, what it costs and what gets logged. The aim is that it runs itself, with onboarding of teams, agents and MCP servers, along with keys, permissions, limits and guardrails, handled by automation rather than tickets. Very little of it exists yet, so you will shape it from the very first design decision. The team is small and focused, so decisions do not take long and your work is visible. We are looking for someone who automates by default, and who cares about the governance side of AI as much as the models themselves.

Responsibilities

• Build self-service onboarding for teams and agents, from request to working access
• Automate key provisioning, rotation and permission changes
• Adapt rate limits and quotas as usage grows
• Roll out and configure models: Vertex AI endpoints, vendor fallbacks, Model Armor
• Run MCP servers behind the gateway, under the same rules as models
• Write Python extensions that connect the gateway to other enterprise systems
• Manage the whole platform as code with Terraform, Helm and Jenkins
• Keep production healthy: availability, latency and cost, watched by alerts
• Support the teams using it, and turn every repeat question into automation

Requirements

• 3+ years of experience in Python for automation, extensions and integrations
• 3+ years of experience in SRE, with real experience keeping production systems healthy
• Deep expertise in Kubernetes, preferably on GKE
• Hands-on experience with Google Vertex AI, especially endpoints for model serving and Model Armor
• Proficiency in Terraform, Helm and CI/CD with Jenkins
• Working knowledge of GenAI/Agentic AI concepts (patterns, frameworks, protocols)
• Production experience with an AI gateway (LiteLLM, EPAM DIAL or similar) is highly appreciated
• Very good communication skills, with the ability to explain things clearly to teams using the platform every day
• English proficiency at B2 level or higher

Nice to have

• Familiarity with GCP beyond GKE and Vertex AI (BigQuery, Cloud Run, IAM)
• Experience building AI agents, for example with Google's Agent Development Kit (ADK)
• Experience with AWS Bedrock