Senior Spark QA Engineer
infracloud
- Pune, India
- Senior Level
- 5+ years
- Posted
Skills
Required
- Spark
- Apache
- functional and performance testing
- Kubernetes
- distributed systems
- Spark architecture
- optimization techniques
- cloud-based
- Spark environments
- Python / Java
- automation frameworks
- CI/CD pipelines
Preferred
- Spark jobs
- SQL queries
- ETL
- YARN
- Mesos
- Databricks
- EMR
- Dataproc
We are looking for a Senior Spark QA Engineer with strong expertise in functional and performance testing of Apache Spark applications and distributed data platforms.
Responsibilities
- Perform manual and automated testing of Spark jobs, Spark SQL queries, and ETL pipelines.
- Execute functional, scalability, and performance testing for Spark workloads.
- Set up and manage Spark clusters on Standalone, YARN, Kubernetes, Mesos, Databricks, EMR, and Dataproc.
- Conduct benchmarking, validation, and performance analysis of Spark applications.
- Identify bottlenecks and troubleshoot distributed system issues.
- Lead QA initiatives and mentor team members.
Requirements
- 5+ years of QA/testing experience with strong hands-on expertise in Apache Spark.
- Experience in functional and performance testing of distributed systems.
- Strong understanding of Spark architecture and optimization techniques.
- Experience with Kubernetes and cloud-based Spark environments.
- Proficiency in Python or Java.
- Experience with automation frameworks and CI/CD pipelines.
Location: Remote
Experience: 5+ Years