Senior DevOps Engineer (DHDP)

We are looking for a Senior DevOps Engineer to support the Digital Health Data Platform (DHDP) by driving CI/CD, infrastructure automation, deployment, environment configuration, monitoring, and operational support across DEV, QA, PRE-PROD, and PROD. In this role, you will work closely with Data Engineering, Data Architecture, Cloud/Infrastructure, Security, and QA teams to ensure DHDP changes are deployed securely, consistently, and according to established architecture, security, and change-management requirements.

Responsibilities

• Design and maintain CI/CD pipelines across DHDP environments
• Automate deployment and controlled promotion across DEV, QA, PRE-PROD, and PROD
• Maintain environment-specific deployment configuration and parameters
• Ensure deployments are version-controlled, repeatable, traceable, and auditable
• Troubleshoot deployment and pipeline failures
• Develop and maintain Infrastructure as Code, preferably using Terraform, to automate provisioning and configuration of DHDP Azure resources
• Maintain reusable infrastructure modules and minimize configuration drift
• Support DHDP services including Azure Databricks, ADLS Gen2, Key Vault, Event Hubs, PostgreSQL, Cosmos DB integrations, Monitor / Log Analytics, and Purview components
• Collaborate with Cloud teams on networking, DNS, Private Endpoints, and shared infrastructure
• Automate deployment of Databricks jobs, workflows, notebooks, data pipelines, and configuration, and implement and maintain Databricks Asset Bundles
• Support compute and Unity Catalog configuration, as well as Service Principals and Managed Identities used for Databricks automation
• Integrate CI/CD with Microsoft Entra ID, Managed Identities, Service Principals, and Azure Key Vault, applying least-privilege principles to deployment identities
• Ensure credentials and secrets are securely managed and not stored in source code
• Support automation of DHDP access-control configuration where applicable
• Maintain deployment consistency across all DHDP environments and support release promotion, configuration validation, rollback, and recovery
• Ensure production data and credentials remain isolated from non-production environments
• Support DHP/DHDP change-control processes and maintain deployment evidence
• Implement monitoring and alerting using Azure Monitor and Log Analytics, and monitor Databricks jobs, deployments, and infrastructure
• Troubleshoot operational issues and support root-cause analysis
• Maintain Git-based development and Pull Request workflows

Requirements

• 3+ years of experience as a DevOps Engineer, with hands-on expertise in Microsoft Azure, Azure DevOps / GitHub CI/CD, and Terraform / Infrastructure as Code
• Experience with Azure Databricks and Databricks Asset Bundles
• Proficiency in Git and Pull Request workflows
• Knowledge of Azure Key Vault, Microsoft Entra ID, Managed Identities, and Service Principals
• Familiarity with Azure Monitor / Log Analytics, Azure networking, and Private Endpoints
• Skills in PowerShell, Bash, and/or Python
• Understanding of multi-environment deployment and cloud security practices
• Proficiency in English at a B2+ level

Nice to have

• Experience with Unity Catalog, ADLS Gen2, and PostgreSQL
• Familiarity with Cosmos DB and Event Hubs
• Knowledge of Microsoft Purview and Power BI integration