Senior Big Data Software Engineer [Databricks, Azure, AI]

Job description

We are seeking a skilled Big Data Engineer to become an essential part of this forward-looking and dynamic team.

In this role, you will leverage advanced cloud technologies to deliver impactful solutions.

If you excel at tackling complex challenges, creating robust cloud-based solutions, and driving meaningful business outcomes, this opportunity offers the chance to join a fast-paced, collaborative environment where your work shapes global success.

Responsibilities

- Build scalable, cloud-based systems in Azure using tools like Azure Data Factory, Azure Data Lake Storage, and Databricks to enhance seamless, high-performance data pipelines

- Migrate on-premises MS SQL databases to Azure Data Lake, utilizing the Delta Lake format to optimize operational performance

- Develop interfaces that connect non-Microsoft proprietary applications, enabling interoperability and unlocking valuable data insights

- Provide expertise in Data Lakes, Data Warehouses (DWH), and Delta Lakehouse architectures to guide transformative business solutions

- Assess new feature proposals, prioritize them based on business value, and collaborate to make impactful product development decisions

Requirements

- 4+ years of experience as a Data Engineer

- Background in working with Python or Scala

- Expertise with Databricks or Spark

- Experience with Clouds: AWS or Azure

- Background in working with Agile or XP methodologies, thriving in fast-moving, adaptive environments

- Experience in AI development would be a plus

- Good English communication skills (Upper-Intermediate/B2+ or higher) to engage effectively with diverse teams and stakeholders

Nice to have

- Proficiency in advanced Azure tools such as Synapse Analytics, Cosmos DB, or Apache Synapse

- Flexibility to use Core Python or Scala for automation and advanced data processing