jobapplier

Data Engineer

Sago

Skills

Required

  • SQL
  • Python / PySpark
  • data processing
  • ETL / ELT pipelines
  • data engineering
  • ETL development
  • common table expressions
  • window functions
  • data validation
  • query optimization
  • automation
  • upserts
  • Parquet
  • Git

Preferred

  • Apache
  • Spark / Delta Lake / Apache Airflow / Dremio / Trino-compatible
  • Power BI
  • AWS
  • Amazon S3
  • Athena
  • EMR
  • SharePoint
  • Airflow
  • cloud storage

We are looking for a Data Engineer with 2–5 years of experience to design, build, and maintain reliable data pipelines and analytics-ready datasets. The ideal candidate will have strong SQL and Python or PySpark skills, hands-on experience with cloud data platforms, and the ability to troubleshoot data-quality, performance, and production issues independently.

Responsibilities

  • Design, develop, and maintain scalable ETL and ELT pipelines for batch and scheduled data processing.
  • Build and optimize data transformations using SQL, Python, and PySpark.
  • Ingest and integrate data from databases, APIs, files, SharePoint, and cloud storage.
  • Create and maintain curated, analytics-ready datasets for Power BI, Superset, and other reporting platforms.
  • Implement data-quality checks, reconciliation controls, monitoring, logging, and failure alerts.
  • Troubleshoot pipeline failures, schema mismatches, performance bottlenecks, and data discrepancies.
  • Optimize large-scale data processing for runtime, memory, storage, and cost efficiency.
  • Automate recurring workflows and schedule production jobs using orchestration tools such as Apache Airflow.
  • Maintain technical documentation, data mappings, business rules, and operational runbooks.
  • Collaborate with BI analysts, application teams, and business stakeholders to deliver dependable data solutions.

Requirements

  • 2–5 years of professional experience in data engineering, ETL development, or a similar role.
  • Strong SQL skills, including joins, common table expressions, window functions, data validation, and query optimization.
  • Hands-on experience with Python and/or PySpark for data processing and automation.
  • Practical experience building and supporting ETL or ELT pipelines in production.
  • Experience working with large datasets, incremental loads, upserts, schema changes, and partitioned data.
  • Good understanding of relational databases, dimensional modeling, data warehousing, and file formats such as Parquet, CSV, and JSON.
  • Experience with Git or another version-control system and structured deployment practices.
  • Strong debugging, analytical, documentation, and stakeholder-communication skills.

Preferred

  • Experience with AWS services such as Amazon S3, Athena, EMR, and EMR Serverless.
  • Experience with Apache Spark, Delta Lake, Apache Airflow, Dremio, or Trino-compatible SQL engines.
  • Familiarity with Power BI, Apache Superset, or other downstream reporting tools.
  • Experience with REST APIs, SharePoint integrations, and automated file-delivery workflows.
  • Understanding of data security, access controls, privacy, and production monitoring practices.

Pay

  • Sago · Gurugram, India (Remote) · Full-time · Posted 2026-11-12
  • Salary: INR 400,000–600,000

Benefits

\* 5 Days working

Apply

[Apply at Sago](https://apply.workable.com/sago-group/j/D536B3361B/apply)