jobapplier

Consultant-GCP Data Engineer

Evalueserve (Lighthouse)

This job is closed: the employer no longer lists it.

Skills

Required

  • data engineering
  • Google Cloud
  • Python
  • SQL
  • data processing
  • automation
  • BigQuery
  • Cloud Storage
  • Dataflow
  • Cloud Composer
  • Apache
  • Spark
  • Terraform
  • Cloud Build

Preferred

  • streaming
  • ETL/ELT pipelines
  • Apache Beam
  • Integrate data
  • APIs
  • into governed data warehouses
  • queries
  • storage patterns

Responsibilities

  • Design and implement scalable batch and streaming ETL/ELT pipelines using Dataflow, Apache Beam or Spark, BigQuery, and Cloud Storage.
  • Integrate data from SQL Server, Couchbase, APIs, files, and event-driven sources into governed data warehouses, data lakes, or lakehouse platforms.
  • Transform raw data into trusted, analytics-ready datasets using SQL, Python, dbt, and Dataflow; apply sound dimensional and analytical modeling practices.
  • Build, schedule, monitor, and support workflows using Cloud Composer and appropriate Google Cloud services.
  • Monitor and tune pipelines, queries, storage patterns, and compute usage for reliability, throughput, scalability, and cost efficiency.
  • Implement validation rules, reconciliation controls, observability, lineage, metadata, anomaly detection, and data-quality standards using Dataplex and related capabilities.
  • Apply least-privilege access, secure data handling, encryption, auditability, retention, and privacy controls in alignment with organizational and regulatory requirements.
  • Provision and manage Google Cloud resources with Terraform; create automated build, test, deployment, and release workflows using Cloud Build and source-control practices.
  • Use Pub/Sub, Cloud Functions, and Datastream where applicable for near-real-time ingestion, change data capture, and event-driven processing.
  • Partner with data scientists, analysts, application teams, security, platform engineering, and business stakeholders to translate requirements into dependable data solutions.
  • Maintain architecture diagrams, data mappings, runbooks, operational procedures, deployment documentation, and coding standards.
  • Track relevant Google Cloud data, DevOps, and AI platform developments and assess their practical application to enterprise data-engineering use cases.

Requirements

  • Qualifications 4+ years of relevant experience in data engineering, including production delivery on Google Cloud.
  • Strong proficiency in Python and SQL for data processing, automation, testing, and analytical querying.
  • Hands-on experience with BigQuery, Cloud Storage, Dataflow, Dataplex, and Cloud Composer.
  • Experience with Apache Beam or Spark and distributed data-processing patterns.
  • Practical knowledge of Terraform for reusable, version-controlled infrastructure as code.
  • Experience building CI/CD pipelines with Cloud Build, including automated validation, testing, packaging, and deployment.
  • Understanding of data warehouse, data lake, and lakehouse principles, including distributed data architectures and data modeling.
  • Experience integrating SQL Server, Couchbase, REST APIs, and file or object-storage sources.
  • Knowledge of data security, privacy, governance, observability, and compliance best practices.
  • Strong troubleshooting, problem-solving, documentation, stakeholder communication, and delivery skills for complex projects and tight timelines.
  • Google Cloud Professional Data Engineer certification.

Disclaimer

  • The following job description serves as an informative reference for the tasks you may be required to perform.
  • However, it does not constitute an integral component of your employment agreement and is subject to periodic modifications to align with evolving circumstances .

Please Note

  • We appreciate the accuracy and authenticity of the information you provide, as it plays a key role in your candidacy.
  • As part of the Background Verification Process, we may verify your employment, education, and other details.
  • Please ensure all information is factual and submitted on time.
  • For any assistance, your recruiter is available to support you.