Lead Data Engineer

Join our Data Engineering & Architecture team as a Lead Data Engineer. In this role, you will design, maintain, and improve the pipelines and infrastructure that power our data platform.

Responsibilities

• You will work across the full data lifecycle, collaborate with stakeholders, and help define how data should flow across systems.
• Additionally, you will monitor and refine pipeline performance, use Terraform for resource management, and support compliance with platform policies.
• Design, maintain, and improve the pipelines and infrastructure that power the data platform, working across the full data lifecycle.
• Collaborate with stakeholders to understand their needs, co-create data contracts, and guide teammates on how to get the most out of the platform.
• Monitor and refine pipeline performance, use Terraform to provision and manage resources on Google Cloud Platform (GCP), and review code to ensure good practices.

Requirements

• Proficient in programming languages, particularly Bash, Python, and SQL
• Experience with Spark, Flink, Kafka, and dbt Labs for data processing and transformation
• Bonus: Proven experience with data security and access controls for SOX Compliant data
• Knowledge of database systems, including OLAP, OLTP, Document, and Vector databases
• 8+ years of experience in data engineering or related roles
• Bonus: Experience and knowledge of the Elastic stack

Experience with Apache Airflow for orchestrating data workflows

• Experience with Google Cloud Platform (GCP) for cloud resource management
• Understanding of Apache Iceberg for data table formats and data warehouse and lakehouse strategies

Preferred

• Skilled in using Terraform for infrastructure-as-code deployments

Pay

USD 113,100 - 215,100

Benefits

• Enjoy benefits such as fully paid health coverage, flexible location and schedule, generous vacation days, parental leave, volunteer time off, and charitable donation matching.