Lead Data Integration Engineer

Job description

We are seeking a Lead Data Integration Engineer to build and optimize data pipelines within the AWS ecosystem, enabling seamless integration and robust data reporting for critical decision-making processes.

Responsibilities

- Collaborate with business analysts to understand and implement data requirements

- Design and develop efficient data pipelines using AWS technologies

- Ensure reliability, performance, and scalability of data pipelines

- Troubleshoot and resolve data integration issues promptly

- Implement data mappings, aggregations, and quality checks based on business analyst input

- Maintain clear and comprehensive documentation of integration processes

- Support data asset transition processes using Iceberg

- Facilitate data querying and manipulation with SQL

- Leverage tools like DBT and Airflow for pipeline orchestration and data modeling

- Align data integration processes with compliance standards and best practices

Requirements

- 5+ years of proven experience in data integration and data warehousing

- Proficiency in DBT, Airflow, and SQL for data transformation, orchestration, and querying

- Skills in Python for API development and automation

- Expertise in Iceberg for data asset management and transition

- Knowledge of Snowflake and Power BI for data warehousing and visualization

- Demonstrated ability in data modeling and database performance tuning

- Familiarity with implementing data quality checks and aggregation processes

- Strong problem-solving, analytical, and communication skills

- Effective use of data integration tools in cloud-based environments (AWS)