Senior Data Platform Engineer

Komodo Health as a Senior Data Platform Engineer, where you will design, build, operate, and improve large-scale data pipelines and foundational data platforms. This hands-on engineering role focuses on processing complex healthcare data at scale, improving reliability and performance, and contributing to the technical direction of core data systems.

Responsibilities

• You will work with Python, SQL, Spark, and Airflow on AWS, and make critical decisions regarding data storage and layout.
• Additionally, you will strengthen data reliability through quality checks, validation, lineage, observability, and alerting.
• Design, build, operate, and improve large-scale data pipelines and foundational data platform.
• Make storage and data layout decisions to keep large datasets fast to query and cost-efficient.
• Strengthen reliability through data quality checks, validation, lineage, observability, and alerting.
• Strong hands-on experience building, owning, and debugging production data pipelines at multi-terabyte scale
• You will be expected to leverage AI-augmented engineering tools, such as ChatGPT, Gemini, or Claude, to improve productivity and technical decision-making. This may include using AI to generate and refine code, accelerate documentation, automate test case creation, debug complex issues, explore unfamiliar technical concepts, and assess architectural trade-offs and risks
• The ability to communicate technical trade-offs clearly with engineering, product, and data partners

Requirements

• Proven experience designing and operating data systems in AWS
• Comfort using AI-assisted engineering tools for coding, debugging, documentation, and technical exploration
• Advanced Python, including performant, memory-efficient code for large-scale data processing, plus strong SQL
• Deep production experience with Airflow, including operating and scaling it, not just writing DAGs
• A solid understanding of how data is physically stored: columnar vs. row formats, and open table formats like Iceberg, Delta Lake, or Hudi (updates, compaction, small files, partitioning)
• Strong instincts for data correctness, root-cause analysis, and production troubleshooting
• Experience with Spark or a comparable distributed processing framework

Preferred

• AI-Augmented Engineering Expectations:
• Containers and orchestration (Docker, Kubernetes)
• Strong core software engineering fundamentals and experience in more than one language

Data governance and lineage experience

• Experience applying AI or agentic workflows to engineering, data quality, delivery, or operations
• Experience delivering external-facing data products through APIs, serving layers, or customer access patterns
• Success in high-growth or ambiguous environments

Pay

USD 170,000 - 230,000

No healthcare background required. You’ll build that domain expertise here