Data Engineer – Scalable Data Platforms & Pipelines

At YORA, we bring together sharp people who want to build, challenge, and move technology forward.

We are now looking for an experienced Data Engineer to build and evolve scalable data platforms and pipelines that support the analysis and validation of complex technical systems.

You will work with Python, SQL, PySpark, and distributed data processing, transforming large volumes of data into structured, reliable, and accessible information.

This is the right role for someone who enjoys building robust data infrastructure and wants to see how well-designed data pipelines can enable smarter, faster, and more automated system validation.

About the role

As a Data Engineer, you will join a team focused on processing, structuring, and managing large volumes of data.

Your main responsibility will be to design and develop the data pipelines, processing capabilities, and data models needed to make complex datasets useful for analysis, system validation, and automated regression testing.

The role sits at the intersection of data engineering and system validation, where data quality, traceability, reproducibility, and performance are essential.

You will work with both batch and streaming data, modern data architectures, and distributed processing technologies.

While an understanding of testing and validation is valuable, the primary focus is on building the technical data infrastructure that makes large-scale, automated validation possible.

What you’ll do

• Design, build, and maintain scalable data pipelines for large and complex datasets

• Develop data processing solutions using Python, SQL, and PySpark

• Work with batch and streaming data processing

• Build and optimize ETL/ELT workflows for data transformation and integration

• Design data models that support analysis, comparison, and validation of large datasets

• Work with data lakehouse architectures and table formats such as Iceberg or Delta Lake

• Ensure high standards for data quality, lineage, traceability, and reproducibility

• Develop data flows that enable data-driven system validation and automated regression testing

• Collaborate with engineers and technical teams to improve how data is processed, structured, and used

What we’re looking for

You have a strong background in Data Engineering and enjoy solving challenges involving large datasets, distributed processing, and scalable data architectures.

We believe you bring:

• Several years of hands-on experience in Data Engineering and scalable data pipeline development

• Strong programming skills in Python, SQL, and PySpark

• Experience with distributed data processing

• Experience with both batch and streaming data

• Solid knowledge of ETL/ELT, data quality, and data lineage

• Good understanding of data lakehouse architecture, including Iceberg or Delta Lake

• Strong experience in data modeling for analysis and comparison of large datasets

• Experience developing data pipelines or models that support system validation and automated regression testing

• Understanding of reproducibility, traceability, versioning, and data quality in validation environments

Who you are

You are a technically curious engineer who enjoys turning complex data challenges into well-structured, scalable solutions.

You understand that good data engineering is about more than moving data — it is about building reliable foundations that make data useful, traceable, and trustworthy.

You enjoy working with complex systems, taking ownership of technical challenges, and collaborating with others to improve how data is processed and used.

About YORA

YORA is a bold, value-driven tech hub focused on AI, cloud, cybersecurity, and modern engineering.

We believe in technical excellence, freedom, and building long-term value together with our clients and consultants.

We care about strong engineering, meaningful assignments, and giving our people the trust and freedom to do great work.