Staff Data Platform Engineer

Join our team as a Staff Data Platform Engineer, where you will own the end-to-end data platform for AI research.

You will also complete and scale our Databricks lakehouse and distributed processing stack, build reusable infrastructure for in-house synthetic data generation, and ensure every dataset is reproducible and traceable. Additionally, you will partner with ML researchers, mentor engineers, work with security and compliance stakeholders, and help shape the long-term data platform roadmap.

About Reality Defender

• Posséder la plateforme de données de bout en bout pour la recherche en IA, y compris l'ingestion continue, le prétraitement, la gouvernance et l'exportation des données.
• Compléter et mettre à l'échelle notre lac de données Databricks et notre pile de traitement distribué, en prenant des décisions architecturales et de construction.

Responsibilities

• You will be responsible for continuous ingestion, preprocessing, governance, and export of high-volume image, video, audio, and text data.
• Construire une infrastructure réutilisable pour la génération de données synthétiques en interne, permettant aux chercheurs de produire de nouvelles variantes de deepfake.
• Experience building synthetic data generation pipelines or running generative models (diffusion, voice cloning/TTS, video generation) at scale. (nice to have)

Requirements

• Deep hands-on experience with distributed processing (Spark, Ray, or similar) and lakehouse platforms, ideally Databricks, on top of cloud object storage (S3, GCS, or Azure Blob)
• 8+ years building data platforms or data infrastructure, including staff- or lead-level ownership of production pipelines that generate, ingest, or deliver data at high volume
• Background supporting ML research teams with training and evaluation data, especially for multimodal models. (nice to have)
• A track record of building reproducible ML datasets: versioning, lineage, data quality validation, and orchestration (Airflow, Dagster, Databricks Workflows, or similar)
• Production experience with Databricks, Unity Catalog, Delta Lake, and Spark on AWS, including performance tuning, access controls, and operational troubleshooting
• Own infrastructure and deployment automation using Terraform and CI/CD, including AWS storage, IAM, compute, Databricks resources, and distributed processing environments
• Experience working with large unstructured media such as video, audio, or images, including the realities of large files, decoding and transcoding, and metadata at scale
• Experience in security, fraud, trust & safety, or another adversarial domain, or at a Series A or B startup where scope regularly outpaces headcount. (nice to have)
• Strong Python and SQL, and the judgment to own ambiguous platform problems end-to-end on a lean team

Pay

USD 210,000 - 260,000