APPLYDJINN
Vacancy catalog
OLD

Long-running vacancy

This listing is older than 30 days but its source has not removed it. Verify availability on the original company page before applying.

SoftServe
Open role>1 month

Senior Big Data Software Engineer (Databricks + AWS)

SoftServePoland
Work model
Hybrid
Experience
4+ years
Employment
Full Time
Compensation
Not disclosed
Technology signal
10 tags

Technology context

10

Parsed from the vacancy text; ordered by relevance to this role.

AIAWSPythonCloudKafkaScalabilitySparkDatabricksSQLAirflow

Full listing

Role description

ABOUT THE ROLE

In this role, you will contribute to developing scalable cloud-based data solutions on AWS using Databricks and modern big data technologies. You will work on batch and streaming data pipelines, support optimization and modernization initiatives, and collaborate with cross-functional teams to deliver reliable and efficient data platforms across different stages of the project lifecycle.

RESPONSIBILITIES

  • Design, develop, and maintain scalable batch and streaming data pipelines
  • Build and optimize data processing solutions using Python (PySpark) and SQL
  • Develop and manage data workflows using Databricks on AWS
  • Work with Apache Spark and Databricks for large-scale data processing
  • Implement and maintain Delta Lake-based data architectures
  • Utilize orchestration tools such as Apache Airflow or MWAA to manage workflows
  • Support implementation of streaming solutions using Apache Kafka, Amazon MSK, or Kinesis
  • Collaborate with technical and business stakeholders to deliver effective data solutions
  • Participate in architecture discussions and contribute to continuous improvement initiatives within the team
  • Participate in the full project lifecycle, from PoC and MVP stages to production implementation

REQUIREMENTS

  • 4+ years of experience in Big Data or Data Engineering
  • Strong proficiency in Python (PySpark) and SQL
  • Hands-on experience with AWS cloud services for data engineering solutions
  • Practical experience with Databricks on AWS, including building and managing data pipelines
  • Strong knowledge of Apache Spark and large-scale data processing
  • Experience with batch and streaming data processing
  • Familiarity with Delta Lake and Databricks components such as Workflows and Jobs
  • Experience with orchestration tools such as Apache Airflow or MWAA
  • Knowledge of streaming technologies such as Apache Kafka, Amazon MSK, or Kinesis
  • Upper-intermediate or higher level of English