Lead Data Software Engineer

We are seeking a Lead Data Engineer (Python, Kubernetes, Snowflake) to own reliable data pipelines and production platforms across Cloud and on-prem. You will deploy and run data and ML workloads on OpenShift, improve CI/CD, troubleshoot complex issues and guide engineering teams as we scale an enterprise analytics and ML platform.

Responsibilities

• Design, build and maintain scalable data pipelines and platform components using Snowflake, Apache Spark (PySpark), SQL and Apache Airflow
• Deploy, operate and support data and ML workloads on Kubernetes and OpenShift in production
• Develop and maintain Python services and APIs using FastAPI or similar frameworks
• Monitor, troubleshoot and optimize performance, reliability and data quality across pipelines and platforms
• Build and improve continuous integration and continuous delivery (CI/CD) pipelines for safe, repeatable releases
• Partner with DevOps, Platform, ML, infrastructure and application teams to deliver production-ready solutions
• Drive root cause analysis for complex incidents and define preventative fixes and operational best practices
• Provide technical leadership through architecture decisions, reviews and mentoring

Requirements

• Proven experience in data engineering and enterprise-scale data platform delivery
• Hands-on expertise with Snowflake, Apache Spark (PySpark), SQL and Apache Airflow
• Strong Python development capability, including building services or APIs
• Production experience operating workloads on Kubernetes or Red Hat OpenShift
• Solid background in continuous integration and continuous delivery (CI/CD), Git workflows, containers and deployment automation
• Experience supporting solutions across Microsoft Azure and hybrid cloud environments
• Strength in monitoring, troubleshooting and production support for distributed systems
• Track record of technical ownership, clear communication and mentoring within engineering teams

Nice to have

• Experience with FastAPI, Databricks or Splunk
• Exposure to MLOps practices, ML platform operations or LLM and GenAI enablement platforms
• Infrastructure as Code experience (tooling aligned to your environment)