Member of Technical Staff, Video Data & Model Evaluation
Cantina
- Singapore
- Hybrid
- Lead/Staff
- Posted
Skills
Required
- data strategy / dataset development / annotation / model evaluation
- generative image / video / multimodal models
- supervised fine-tuning
- post-training
- image and video generation
- multimodal generation
- annotation guidelines
- thresholds
- evaluation frameworks
Preferred
- vision-language models
- automated captioning
- synthetic-data generation
- prompt design
- Python
- SQL
- data and workflow automation tools
- model iteration
- data lifecycle—from
About Cantina
- Cantina Labs is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism.
- We bring characters to life, transforming how people tell stories, connect, and create.
- We build and power ecosystems.
- Cantina, our flagship social AI platform, is just the beginning.
Responsibilities
- You will own the full data lifecycle—from defining data requirements and annotation standards to building high-quality datasets, evaluating model outputs, and translating findings into actionable recommendations for model improvement.
- You will work closely with research scientists, machine learning engineers, product teams, and external data partners to strengthen the data–model–evaluation feedback loop.
- Define quality standards for video generation across aesthetics, realism, composition, motion, temporal consistency, instruction following, subject consistency, editing preservation, and text rendering.
- Evaluate model iterations, analyze failure patterns, and translate findings into actionable data, training, and prompting strategies.
- Own end-to-end training data projects, including sourcing, construction, annotation, captioning, quality control, and delivery.
- Develop labeling systems that support dataset segmentation, coverage analysis, quality filtering, training-data mixtures, and model performance attribution.
- Design annotation and captioning standards for video generation, editing, and multi-reference tasks.
- Create and evaluate synthetic-data workflows based on their stability, usability, quality, and training value.
- Develop automated annotation and filtering workflows using vision-language models, system prompts, and quality-scoring models.
- Build structured model evaluation processes covering test-set design, evaluation criteria, failure attribution, and reporting.
- Establish feedback loops connecting model evaluation and negative-case analysis with data strategy and model iteration.
- Train and calibrate annotation teams and external data partners, conduct quality audits, and improve delivery standards.
- Partner with research, engineering, product, and creative teams to execute data initiatives and identify future model capabilities.
Requirements
- Hands-on experience in data strategy, dataset development, annotation, or model evaluation for generative image, video, or multimodal models.
- Experience supporting model development across continued training, supervised fine-tuning, post-training, or evaluation.
- Strong understanding of generative media tasks, including image and video generation, multimodal generation, and instruction-based editing.
- Experience developing labeling taxonomies, annotation guidelines, captioning standards, quality thresholds, and evaluation frameworks.
- Strong visual judgment and the ability to translate subjective quality expectations into measurable data and evaluation standards.
- Experience managing large-scale data-production or annotation projects and collaborating with research, engineering, creative, and external partner teams.
Preferred
- Familiarity with automated captioning, synthetic-data generation, vision-language models, prompt design, Python, SQL, or other data and workflow automation tools is a plus.
Pay
- Competitive salary and generous company equity
- Monthly spending stipend: $500 (~S$635)
Benefits
- Competitive salary and generous company equity
- Personal time off and paid holidays
- Health insurance
- Global travel insurance: Covers you when traveling internationally
- Monthly spending stipend: $500 (~S$635)
- Equipment: All equipment needed for your home office
