Middle+ Data Engineer

Job description

Dataforest is looking for a Data Engineer to join an interesting software development project in the field of Cybersecurity. Our client’s platform offers Identity Insights drives real-time, multi-contextual digital forensics to protect new account opening workflows and expose fake accounts already in a customer database.

You'll be a great fit if you have:

- 3+ years of hands on in data engineering;

- 3+ years of commercial experience with Python;

- Advanced experience with SQL DBs (optimisations, monitoring, etc.);

- Advanced experience with PySpark;

- Solid understanding of ETL principles (architecture/ monitoring/ alerting/search and resolve bottlenecks);

- Familiar with AWS infrastructure (boto3, S3 buckets, etc);

- Experience working with large volumes of data;

- Understanding the principles of medallion architecture.

Would be a plus:

- Cassandra/Scylla

- PostgreSQL

- Pandas

- Experience with Kafka and Redis

What you'll do:

- Maintain data processing architecture using Python and PySpark that interacts with ScyllaDB & PostgreSQL.

- Work with Databricks infrastructure to ensure reliability, scalability, and cost efficiency.

- Proactively identify bottlenecks and suggest technical improvements.

We ensure your growth with:

- Working in a fast-growing company;

- Great networking opportunities with international clients, challenging tasks;

- Personal and professional development opportunities;

- Competitive salary fixed in USD;

- Paid vacation and sick leaves;

- Flexible work schedule;

- Friendly working environment with minimal hierarchy;

- Team building activities, corporate events.