jobapplier

IN_Senior Associate_Azure Data Engineer_D&A_Advisory_Ahmedabad

PwC Asia

Skills

Required

  • Azure
  • integration
  • SQL
  • Delta Lake
  • Azure Data Factory
  • Azure data services
  • orchestration
  • data governance
  • CI/CD
  • data engineering
  • PySpark
  • Spark
  • Python
  • Git

Preferred

  • cloud
  • analytics
  • streaming
  • real-time
  • data processing
  • Stream Analytics / Kafka
  • machine learning pipeline support
  • ML
  • Docker
  • Kubernetes

PwC India is looking for a highly skilled Azure Data Engineer Senior Associate with hands-on experience in designing, developing, and managing scalable data solutions on the Azure cloud platform, with particular expertise in Databricks. This is a technical role focused on building robust data pipelines, optimizing cloud data infrastructures, and enabling advanced analytics for enterprise clients.

Responsibilities

IN_Senior Associate_Azure Data Engineer_D&A_Advisory_Ahmedabad

  • Design, develop, and optimize scalable data pipelines and ETL/ELT processes on Azure using Databricks, Azure Data Factory, and related services.
  • Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality data solutions.
  • Build and manage data lakes, data warehouses, and lakehouse architectures leveraging Azure Synapse Analytics, Azure Data Lake Storage, and Databricks Delta Lake.
  • Implement data ingestion, cleansing, transformation, and integration workflows from diverse data sources including SQL, NoSQL, and streaming platforms.
  • Monitor and fine-tune data pipelines and Databricks clusters to ensure performance, cost-efficiency, and scalability.
  • Apply data governance, security, and compliance best practices throughout data lifecycle management.
  • Support continuous integration and continuous deployment (CI/CD) of data workflows and infrastructure as code (IaC).
  • Prepare and maintain technical documentation including architecture diagrams, data flow designs, and operational guides.
  • Stay updated with latest Azure data services, Databricks features, and industry trends to recommend innovative solutions.

Requirements

  • Hands-on experience in designing and implementing data engineering solutions on Azure, specifically with Azure Databricks.
  • Strong skills in PySpark, Spark SQL, and Delta Lake for building and optimizing distributed data pipelines.
  • Proficiency in Azure Data Factory and other Azure data services for orchestration and integration.
  • Solid understanding of data warehousing, lakehouse concepts, and big data architectures.
  • Experience with scripting languages such as Python and SQL for data manipulation and automation.
  • Knowledge of data governance, security, and compliance standards in cloud environments.
  • Familiarity with version control systems (Git) and CI/CD tools like Azure DevOps.
  • Strong analytical and problem-solving capabilities with attention to detail.
  • 3 - 7 Years
  • BE, B.Tech, MCA, MBA, M.Tech

Preferred

  • Experience with real-time data processing and streaming technologies such as Azure Stream Analytics or Kafka.
  • Exposure to machine learning pipeline support and integration with Azure ML or similar platforms.
  • Working knowledge of containerization (Docker) and orchestration (Kubernetes) within cloud data environments.
  • Prior consulting experience delivering cloud data modernization and analytics projects.
Apply with Jobapplier