Platform Engineering Team Lead

We are seeking an experienced and visionary Platform Engineering Team Lead to lead a team of highly skilled platform experts. In this role, you will own the architecture, stability, and scaling of our enterprise data streaming and processing platforms.

You will act as a key technical leader, bridging the gap between Infrastructure, Data Engineering, and Data Science, ensuring high availability, continuous automation, and state-of-the-art platform observability.

Responsibilities

• Team Leadership: Provide professional and personal management, mentorship, and guidance to a team of platform engineering experts
• Confluent/Kafka Platform Ownership: End-to-end responsibility for the Confluent Suite, including Apache Kafka, Kafka Connect, Schema Registry, REST Proxy, Confluent Cloud, and KSQL
• Data Platforms: Own and manage enterprise data analytics platforms, including Azure Synapse
• DataOps, MLOps, & Compute: Manage end-to-end infrastructure supporting DataOps, MLOps, and specialized GPU compute resources
• Ingestion & Logging Infrastructure: Oversee streaming ingestion and logging pipelines based on Apache NiFi, PortX, Fluent Bit, and Filebeat
• Lifecycle & Projects: Lead complex architecture, installation, upgrading, and migration projects for all data platforms
• Incident Management: Drive deep troubleshooting and resolution of complex production issues
• Architectural Guidance: Provide professional architectural advisory and support to Data Engineering and Data Science development teams
• Vendor Management: Interface directly with external vendors and technology partners
• Automation & Observability: Drive automation initiatives and implement advanced system monitoring and observability frameworks
• System Resiliency: Lead initiatives for Capacity Planning, High Availability (HA), and Disaster Recovery (DR)

Requirements

• Leadership: At least 3 years of experience leading and managing a technology/engineering team
• Domain Expertise: At least 5 years of experience in provisioning, maintaining, and operating large-scale Data and/or Streaming platforms
• Operating Systems: Deep, hands-on experience working in Linux environments
• Kafka & Confluent: Significant hands-on experience with on-premises Apache Kafka / Confluent and all its core ecosystem components
• Production Operations: Proven track record in troubleshooting and root-cause analysis of complex issues in high-pressure Production environments
• Cross-Functional Collaboration: Extensive experience working closely with Software/Data Developers and Solutions Architects
• Big Data Ecosystem: Strong hands-on experience with at least one or more of the following: Hadoop, Cloudera, Databricks, Apache Spark, or Azure Synapse
• Configuration Management: Solid experience working with Ansible for automation
• Modern Paradigms: Hands-on experience in at least one of these domains: DataOps, MLOps, or AI/ML Platforms
• A systemic, holistic approach to system architecture and planning
• Excellent self-learning capabilities and adaptability to new technologies
• Outstanding interpersonal skills with a strong service-oriented mindset
• Proven ability to work effectively across multiple cross-functional departments (interfaces)

Nice to have

• Data Ingestion: Strong hands-on experience with Apache NiFi (Highly Advantageous)
• Cloud Infrastructure: Experience working within enterprise-grade Cloud environments (AWS, Azure, or GCP)
• Distributed Querying: Experience with Trino (Presto SQL)
• Infrastructure as Code (IaC): Experience with Terraform
• Containerization: Experience with containers and container orchestration (Docker, Kubernetes)
• Managed Streaming: Practical experience with Confluent Cloud
• Scripting & Big Data Development: Practical experience with Python and PySpark
• Scale: Prior experience working within a large Enterprise organization