Data DevOps Lead

Job description

The Data DevOps Lead owns the infrastructure and delivery processes for data: building and evolving the data platform, establishing CI/CD and DataOps practices, leading a small team of engineers, and setting technical standards. This role sits at the intersection of data engineering, DevOps, and team leadership.

Responsibilities

- Own the data platform architecture: design, evolve, and scale the infrastructure for data pipelines

- Build and maintain CI/CD processes for data pipelines, infrastructure, and ML models

- Implement Infrastructure as Code, versioning, automated testing, and data monitoring practices

- Define the technical roadmap and standards, and manage technical debt

- Ensure reliability, security, and cost-efficiency of the infrastructure (SLAs, observability, cost optimization)

- Partner with stakeholders: data engineers, analysts, DS/ML teams, product, and security

Requirements

- 5+ years in DevOps / Data Engineering, including technical or team leadership experience

- Hands-on experience with AI/ML infrastructure and supporting AI/ML workloads in production (MLOps)

- Strong command of one cloud platform (AWS / GCP / Azure) and its data services

- Docker, Kubernetes, Terraform (or equivalent IaC)

- Pipeline orchestration (Airflow, Dagster, Prefect, or equivalent)

- Experience with big data and streaming: Spark, Kafka, data lake / DWH (Snowflake, BigQuery, Redshift, etc.)

- Languages: Python and/or Scala, strong SQL, Bash

- Monitoring and observability setup (Prometheus, Grafana, ELK, etc.)

- Experience with LLM/GenAI infrastructure and serving

- Data governance, security, and data compliance

- Cloud cost optimization (FinOps)