AI Research Engineer

Job Purpose As an AI Research Engineer in the TrainAI research team, you will bridge applied research and engineering execution. Your core focus will be designing, building, and scaling continuous LLM evaluation pipelines, experiment orchestration tools, and automated pre-annotation workflows.

About RWS

Life at RWS Life at RWS - If you like the idea of working with smart people who are passionate about growing the value of ideas, data and content by making sure organizations are understood, then you’ll love life at RWS.

Our purpose is to unlock global understanding.

This means our work fundamentally recognizes the value of every language and culture.

So, we celebrate difference, we are inclusive and believe that diversity makes us strong.

We want every employee to grow as an individual and excel in their career.

In return, we expect all our people to live by the values that unite us: to partner , putting clients fist and winning together , to pioneer , innovating fearlessly and leading with vision and courage, to progress , aiming high and growing through actions and to deliver , owning the outcome and building trust with our colleagues and clients.

Responsibilities

• Working directly with the Head of Innovation and data engineering teams, you will turn cutting-edge multilingual evaluation methodologies into reliable, automated research and production tools.
• Key Responsibilities Evaluation Pipelines & Benchmarking: Design, build, and maintain automated execution pipelines for continuous LLM benchmarking (such as TrainAI Bench) across diverse languages, tasks, and models.
• Develop and roll out LLM-assisted pre-annotation and quality assurance pipelines to automate production data annotation, lowering costs and improving data consistency.
• Implement and refine automated evaluation methodologies, including LLM-as-a-judge pipelines, custom evaluation metrics, and scoring algorithms.
• Build experiment infrastructure, datasets, and execution pipelines to support flagship multilingual LLM research studies and thought leadership reports.
• Translate applied NLP research into scalable prototypes and work alongside the core data engineering team to integrate research tools into production operations.
• Collaborate with the research team on writing up research findings, benchmark reports, and thought leadership pieces for internal and public release.
• Prototype novel capabilities in emerging areas such as agentic AI evaluation, synthetic data quality certification, and domain-specific LLM assessment.
• Maintain comprehensive documentation, code bases, and technical guides for TrainAI evaluation frameworks and tools.

Requirements

• Master’s degree (MSc or equivalent) in Computer Science, Artificial Intelligence, Computational Linguistics, or a related technical field, combined with 3–5+ years of hands-on experience in NLP or ML engineering.
• Advanced proficiency in Python and deep learning/NLP frameworks (PyTorch or TensorFlow, Hugging Face transformers).
• Proven experience with LLM orchestration frameworks (e.g., LangChain, LlamaIndex) and automated evaluation techniques (e.g., LLM-as-a-judge, benchmarking harnesses).
• Expertise in data manipulation, structured evaluation data processing, and analysis using pandas, NumPy, or related tools.
• Practical experience deploying and managing ML workflows on cloud platforms (Azure, AWS, or Vertex AI).
• Experience building scalable data or experiment orchestration pipelines.
• Familiarity with multilingual NLP, machine translation, or computational linguistics (e.g., cross-linguistic variations, syntax, morphology).

Preferred

• Knowledge of localization workflows, annotation platforms, or human-in-the-loop data pipelines.
• Track record of contributing to open-source NLP projects, evaluation frameworks, or published technical benchmarks.

RWS embraces DEI and promotes equal opportunity, we are an Equal Opportunity Employer and prohibit discrimination and harassment of any kind. RWS is committed to the principle of equal employment opportunity for all employees and to providing employees with a work environment free of discrimination and harassment. All employment decisions at RWS are based on business needs, job requirements and individual qualifications, without regard to race, religion, nationality, ethnicity, sex, age, disability, or sexual orientation. RWS will not tolerate discrimination based on any of these characteristics. RWS Values Get the 3Ps right – Partner, Pioneer, Progress – and we´ll Deliver together as RWS.

Recruitment Agencies

RWS Holdings PLC does not accept agency resumes. Please do not forward any unsolicited resumes to any RWS employees. Any unsolicited resume received will be treated as the property of RWS and Terms & Conditions associated with the use of such resume will be considered null and void.