Data Engineer
Grid Dynamics is looking for a Senior Data Engineer with GCP experience. Team is working on developing a new recommendation model.
Role Description
We are looking for a Data Engineer to develop and maintain ELT pipelines using Kubeflow and support the migration from Acxiom inputs to Amperity platform inputs. The role requires hands-on experience with Google Cloud, data pipeline development, and data integration/migration.
The ideal candidate should have strong expertise in Python, SQL, experience working with GCP is mandatory, and proficiency with data pipeline orchestration tools such as Airflow or Kubeflow.
The client is the largest department store company by retail sales in the United States.
Requirements to the candidate:
- Strong Python and SQL skills.
- Hands-on experience with Google Cloud Platform.
- Experience with Airflow and/or Kubeflow.
- Experience with Docker and Kubernetes.
- Experience building production data pipelines.
- Understanding of data integration and migration processes.
- GCP certification.
Responsibilities
- Design, develop, and maintain scalable ELT/data pipelines.
- Build and support workflows using Kubeflow and/or Airflow.
- Work with Python and SQL for data processing and transformation.
- Support migration of data inputs from Acxiom to Amperity.
- Perform data mapping, validation, reconciliation, and quality checks.
- Work with GCP services and cloud-based data infrastructure.
- Containerize and run workloads using Docker and Kubernetes.
- Monitor production pipelines and troubleshoot failures or performance issues.
- Collaborate with Data Engineers, Data Scientists, ML Engineers, and client teams.
- Support datasets and pipelines used by the recommendation model.
Team size:
15 engineers