Lead AI Engineer, Databricks

We are seeking a Lead AI Engineer, Databricks, to design and build data engineering frameworks, reusable components, and best practices that empower hundreds of SAP Data Engineers to build AI-ready Data Products at scale, while driving innovation in AI-assisted development and Agentic AI solutions.

Responsibilities

• Design and build the data engineering framework, reusable components and templates (e.g. CI/CD), and best practices to boost productivity of Data Engineers building AI-ready Data Products
• Extend the data engineering framework to make it easy for AI Agents to discover and consume Data Products at scale
• Provide a powerful AI-assistant to accelerate Data Product development and operations
• Establish and promote engineering excellence practices across agile team peers
• Set up robust, scalable pipelines leveraging CI/CD frameworks
• Collaborate with international teams to align on technical strategy and delivery
• Guide Data Product Engineers with actionable technical direction

Requirements

• 5+ years of experience in Data Engineering with a successful track record of building enterprise-quality Data Products, preferably in Lakehouse architecture
• Proficiency in Python development
• Proficiency in version control systems and CI/CD frameworks, such as GitHub and GitHub Actions
• Expertise in Databricks technologies, including PySpark, Delta Lake and Unity Catalog, or equivalent technology
• Expertise in Agentic AI and frameworks such as LangGraph, MCP and A2A
• Knowledge of industry standard microservices technologies, including Kubernetes and Docker
• Strong English communication skills, both written and spoken, for collaboration with international teams
• Capability to provide actionable technical guidance and implement engineering excellence practices among agile team peers