Data Engineering Lead

Job description

We are seeking a seasoned Data Engineering Lead to join our project delivery stream.

As a lead, you will direct a team of 2-4 Data Engineers, manage data processing operations for a particular market, and engage in hands-on engineering tasks utilizing Databricks on Azure. This role involves intensive collaboration with client stakeholders and contributes to the enhancement and development of our main platform.

Responsibilities

- Oversee the execution of ETL processes using Databricks notebooks

- Manage and mentor a team of Data Engineers

- Ensure the accuracy and quality of data processing and reporting

- Collaborate closely with client stakeholders to understand and meet their data needs

- Participate in the central team’s efforts to implement improvements and evolve the main platform

- Support the integration of reporting and simulation applications with Power BI

Requirements

- Minimum 5 years of relevant work experience in data engineering

- Proficiency in Databricks and SQL

- Expertise in Python / Pyspark and ETL processing

- Background in managing complex data engineering projects

- Capability to handle detailed and complex processes

- Strong collaboration and relationship-building skills

- Fluent English communication skills at a B2+ level

Nice to have

- Knowledge of CI/CD and DevOps methodologies