DevOps Engineer

About Zoral. Zoral is an IT product and professional services company serving banks, insurers, wealth managers and fintechs. Alongside its flagship product, Zoral fOS (Financial Operating System), Zoral delivers data, analytics and AI programmes for regulated financial-services firms in Europe, the UK, the US and Asia.

About the role. We are looking for a Senior DevOps and Platform Engineer to build, secure and run the foundation of enterprise data platforms on Azure Databricks for regulated financial-services clients. Typical programmes turn an existing data lake into a governed, production-grade data platform: tens of source systems, millions of client records, several hundred regulatory, management and operational reports, and delivery in vertical slices to dozen of teams. You are part of the shared core team that serves every slice: you own the delivery pipelines, the platform’s security and resilience, cost governance and alerting, and you make sure the new platform is cleanly isolated from the legacy lake while both run side by side. This is a remote role based in Europe, with possible travel to client premises for workshops and key milestones.

Responsibilities

· Build the platform foundation as code with Terraform: workspaces, the Unity Catalog metastore and one catalog per environment (development, test, production, with production isolated from non-production), storage, identities and permissions.

· Implement the CI/CD route to live for data products with Declarative Automation Bundles (formerly Databricks Asset Bundles) and Azure DevOps or GitHub Actions: every pipeline, rule, semantic model and report change reviewed, tested and promoted through environments.

· Deliver the platform backlog with the Head of Data Platform: Hive metastore to Unity Catalog migration, high availability and disaster recovery, privileged access on demand, masking of personal data and remediation of penetration-test findings.

· Set up observability: run metrics, freshness, quality results and cost from Databricks system tables; audit events through Azure Monitor diagnostic settings; job, quality-gate and freshness alerts into ServiceNow and Teams with runbook references; service-level objectives for critical assets.

· Run cost governance (FinOps): compute policies, job and serverless compute, separate SQL warehouses for critical reporting, self-service and data science, budgets with alerts, usage policies and billing system tables as the source of truth.

· Support Power BI deployment and capacity planning together with the Power BI engineers; later, extend the same pipeline to MLOps (experiment tracking, model registry, serving endpoints).

· Configure and operate the platform controls for AI services and AI-assisted delivery, including secure model access, permissions, logging and deployment controls.

Requirements

· Five or more years in DevOps, platform or cloud engineering, including strong experience running Azure Databricks in production.

· Strong Terraform with the AzureRM and Databricks providers: modules, state management, code review and multi-environment layouts.

· Hands-on Databricks administration: workspaces, Unity Catalog (metastore, catalogs, grants, service principals, system tables), compute policies, jobs and secrets.

· CI/CD for data platforms in Azure DevOps or GitHub Actions, including automated tests, quality and security checks, approvals and promotion between environments.

· Azure networking and identity: VNet injection, Private Link and private DNS, Entra ID, managed identities and service principals, Key Vault, role-based access control.

· Monitoring and incident integration: Azure Monitor and Log Analytics, alert routing into an ITSM tool such as ServiceNow.

· Practical disaster-recovery and backup design for cloud data platforms, including recovery point and recovery time objectives.

· Solid Linux skills (command line, troubleshooting, self-hosted build agents, cluster init scripts); scripting in Python, PowerShell or Bash; Git.

· Understanding of security and compliance in regulated environments: least privilege, encryption, segregation of duties, change control, audit trails and GDPR.

· Understanding of how to run AI workloads and AI-assisted development securely: model endpoints, access control, logging and audit, and controls on AI-generated changes.

· Good written and spoken English.

Nice to have

· Policy as code (Azure Policy or Open Policy Agent) and security scanning of infrastructure code.

· MLOps on Databricks (MLflow, model serving).

· Microsoft Certified: DevOps Engineer Expert or Azure Administrator Associate, HashiCorp Certified: Terraform Associate; Databricks Platform Administrator accreditation.

· Experience of a Hive metastore to Unity Catalog migration (for example with UCX or SYNC).

· Entra Privileged Identity Management and the Databricks compliance security profile.

· Power BI administration, capacity planning and deployment automation.

· Databricks FinOps: budgets, serverless usage policies, billing system tables.

Tech stack: Microsoft Azure (Entra ID and Privileged Identity Management, Key Vault, VNet and Private Link, Azure Data Lake Storage, Azure Monitor and Log Analytics), Azure Databricks (Unity Catalog, system tables, Lakeflow Jobs, Databricks SQL, compliance security profile), Terraform, Declarative Automation Bundles, Azure DevOps or GitHub Actions, Git, Python, PowerShell or Bash, ServiceNow, Power BI.

Відгукнутись на вакансію