Senior Data Engineer (Azure/Databricks)

Job description

Location: 100% remotely

Rate: up to 160 pln/h on b2b

We are looking for an experienced Senior Data Engineer (Azure/Databricks), who will support our global customers mainly in the pharmaceutical and FMCG industry in building innovative big data products and services.

You will be responsible for:

- Designing, developing, and maintaining scalable data solutions in Azure Databricks for analytics and BI

- Managing the full data lifecycle, from acquisition and integration to analysis and visualization

- Building and maintaining CI/CD pipelines using Azure DevOps and/or GitHub, automating testing, deployment, and operations

- Delivering end-to-end data and analytics projects, from requirements and architecture to implementation and deployment

- Developing and optimizing data pipelines, visualizations, analytics products, automated services, and APIs

- Ingesting and integrating data from diverse sources into scalable, maintainable data platforms

- Ensuring performance, reliability, scalability, and data quality of data solutions

- Collaborating with multidisciplinary teams to turn data into models, insights, and production-ready solutions

- Identifying opportunities to leverage data and models to improve products and customer solutions

- Advising clients and stakeholders on data, Databricks, and solution architecture, identifying optimization opportunities

What you will need:

- 4+ years of experience in Azure and 6+ years of industrial experience in large-scale data management, visualization, and analytics

- Strong hands-on experience with Azure Databricks, including data engineering, data processing, and analytics workloads

- Hands-on knowledge of other Azure data services and technologies, such as Azure Data Lake, Azure SQL, Azure Data Factory

- Good knowledge of SQL and Python

- Proactive approach, goal-oriented mindset, and strong problem-solving skills

- Great communication skills for discussing technical issues with end users and clients

- Very good knowledge of English, with the ability to communicate effectively and explain and simplify technical topics for the business

- Willingness to show up at the Warsaw office as needed (customer visits/workshops/important project periods requiring F2F teamwork).

Nice to have:

- Experience designing and implementing CI/CD pipelines for data and analytics solutions, preferably using Azure DevOps, and data orchestration through Databricks Workflows

- Good understanding of DevOps practices, version control, automated testing, and deployment processes