> DOUEdvantis
Data Engineer
format:Remotecompany:Outsource
databricksapache sparksqlci/cdterraformdata modeling
azureawskafkapower bitableau
> full description
We’re opening the position of an Expert Data Engineer to work on the project of our Customer — an international company that develops software solutions for small and mid-sized municipalities. The project focuses on a product suite that supports key local government operations, including finance, utility billing, HR, payroll, online payments, tax collection, and asset management.
Responsibilities:
- Own the target architecture: medallion layering, Unity Catalog structure (metastore, catalogs, schemas, naming, environments), and the storage layout
- Evaluate ingestion options (Auto Loader, Lakeflow Connect, Lakeflow Declarative Pipelines, ADF/Fivetran, streaming vs batch) and recommend an approach with cost, complexity and latency trade-offs
- Define the governance model: access control and grants; external locations and storage credentials; lineage, tagging, and PII handling
- Set the compute strategy: serverless vs classic, job clusters, SQL warehouses, cluster policies, and cost controls
- Define standards for orchestration (Jobs/Workflows), data maintenance (optimization, retention, vacuum, schema evolution) and data quality
- Design reporting-ready Gold models together with BI Engineers and business stakeholders
- Set up the engineering framework: CI/CD (Databricks Asset Bundles), environment promotion, and IaC
- Guide and review the work of Data Engineers, and stay hands-on for PoCs and critical components
Requirements:
- 7+ years in Data Engineering or architecture, with 3+ years designing Databricks platforms in Production
- Deep Unity Catalog expertise: metastore design, managed vs external tables and volumes, external locations, permissions model, lineage, and system tables
- Proven skills in end-to-end medallion lakehouse design and delivery
- Strong Delta Lake internals knowledge: MERGE/CDC patterns, liquid clustering or Z-order, CDF, time travel, and maintenance
- Broad knowledge of Databricks ingestion patterns and the ability to justify choices between them
- Understanding of analytical data modelling: dimensional/Kimball, SCD, and Data Vault awareness
- Strong PySpark and SQL skills, with the ability to build PoCs personally
- CI/CD and IaC expertise, covering Asset Bundles and Terraform
- A proven track record of producing architecture decision records and presenting options to stakeholders
- English level — Upper-Intermediate or higher
Will be a plus:
- Databricks Data Engineer Professional or Solutions Architect certification
- Ability to work with cloud architecture on Azure or AWS (networking, IAM, private endpoints)
- Skills with Structured Streaming and Kafka/Event Hubs
- Knowledge of data quality frameworks such as Lakeflow expectations, Great Expectations, and DQX
- Ability to work on BI integration, including Power BI or Tableau on Databricks SQL, metric views, and semantic layers
- Experience in consulting or regulated environments
We offer:
- Remote-first work model with flexible working hours (we provide all equipment)
- Comfortable and fully equipped offices in Lviv and Rzeszów
- Competitive compensation with regular performance reviews
- Referral Bonus Program
- 18 paid vacation days per year
- 12 days of paid sick leave per year
- Extra paid leave (blood donation, marriage, childbirth, etc.)
- Health & wellness support: either a monthly budget for medical insurance and sports activities, or a full medical insurance plan, depending on your cooperation model
- Mental Health Program — company-covered psychologist consultations, with an annual coverage limit
- English, German, and Polish language courses and Speaking Clubs
- Corporate merch gifts for your special dates
- Corporate subscription to learning platforms, regular meetups and webinars
- Friendly team that values accountability, innovation, teamwork, and customer satisfaction
- Maternity leave
- Inclusive environment where everyone feels valued and treated equally. We proudly partner with VeteranHub to support Ukrainian veterans
- We are committed to supporting Ukraine and actively participate in charity initiatives