Data Engineer

About Zoral

Zoral is an IT product and professional services company serving banks, insurers, wealth managers and fintechs. Alongside its flagship product, Zoral fOS (Financial Operating System), Zoral delivers data, analytics and AI programmes for regulated financial-services firms in Europe, the UK, the US and Asia.

About the role

We are looking for a Data Architect / Senior Data Engineer to define the target architecture and actively design, develop and deliver enterprise data platforms on Azure Databricks for regulated financial-services clients. Typical platforms integrate tens of source systems and millions of customer and policy records, support several hundred regulatory, management and operational reports, and are delivered by growing multidisciplinary teams of data engineers, architects, analysts and client specialists.

This is a hands-on architecture and engineering role within a collaborative delivery team. You will assess the client’s existing data landscape, help refine the target architecture, establish modelling and engineering standards, and act as a senior technical authority throughout delivery. You will work closely with other architects and engineers, providing direction, reviewing designs and supporting implementation rather than operating as a standalone architect.

At the same time, you will remain close to the engineering work: developing transformation logic and data models, profiling and cleansing data, implementing data-quality controls, and contributing to production engineering where your experience has the greatest impact. The successful candidate should be comfortable moving between architecture, technical leadership and hands-on implementation while enabling the wider team to deliver effectively.

Requirements

· Eight or more years in data architecture and/or senior data engineering, with substantial hands-on delivery responsibility, including at least three years designing and engineering Databricks platforms in production.

· Demonstrated ability to operate as both an architect and a senior engineer within a delivery team: define target-state designs and standards, guide other engineers, and implement or materially contribute to pipelines, transformations and data models.

· Experience providing technical leadership within multidisciplinary data teams, including design reviews, mentoring, engineering standards and collaborative delivery.

· Expert SQL and strong Python/PySpark skills, with experience developing, testing, optimising and supporting production data transformations.

· Hands-on knowledge of Azure Databricks, including Unity Catalog governance (grants, row filters, column masks, tags and lineage), Delta Lake, Lakeflow Declarative Pipelines, Lakeflow Jobs and Lakeflow Connect.

· Proven experience designing and developing layered lakehouse or data-warehouse architectures using Data Vault 2.0 and Kimball dimensional modelling, including slowly changing history and as-at data.

· Experience defining and implementing data-quality and cleansing strategies, including profiling, rule design, quality gates, monitoring, scorecards, quarantine, remediation and reconciliation controls.

· Experience with master data management, customer matching, de-duplication or entity resolution.

· Experience migrating data, pipelines and models from legacy warehouses or lakes, including coexistence, reconciliation, parallel running and decommissioning.

· Practical experience with Git, code review, automated testing, CI/CD and production support for data pipelines.

· Ability to write clear architecture and engineering documentation and present technical decisions to senior stakeholders.

· Excellent written and spoken English; willingness to work on site at client premises, including in the UK, for periods of several weeks.

Nice to have

· Experience in financial services, particularly banking, insurance, wealth or asset management, capital markets or trading, with an understanding of financial data, instruments, transactions, positions, valuations, risk, regulatory or management reporting.

· Azure networking and identity for Databricks, including Private Link, Entra ID, Key Vault and customer-managed keys.

· Infrastructure as code using Terraform and deployment with Databricks Asset Bundles.

· Power BI semantic-model standards, Databricks SQL and Microsoft Purview.

· Data Vault automation, metadata-driven pipeline development or model/code-generation frameworks.

· Experience with data-quality or entity-resolution tools such as Great Expectations, Soda, Splink, Zingg, Dedupe or comparable commercial platforms.

· Databricks or Azure architecture/data-engineering certifications; CDVP2 (Data Vault 2.0).

Tech stack

Azure Databricks (Unity Catalog, Delta Lake, Lakeflow Declarative Pipelines, Lakeflow Jobs, Lakeflow Connect, Auto Loader and Databricks SQL), Azure Data Lake Storage, Azure Data Factory, SQL, Python/PySpark, Data Vault 2.0, Kimball dimensional modelling, Great Expectations or Soda, Splink or similar, Power BI, Terraform, Git, Azure DevOps or GitHub Actions.

Відгукнутись на вакансію