Machine Learning Engineer / SME (LLM Eval & RAG Testing) - SME | TS/SCI
Falls Technology
- McLean, United States
- On-site
- Mid Level
- Posted
Skills
Preferred
- RAG testing
- Text classification
- scikit-learn / deep learning
- AI
- Python
- PyTorch / TensorFlow / Keras
- SQL
- GitHub
- Jenkins
- Flask
Prove which LLMs actually work for the mission. You'll design RAG and LLM evaluation experiments, tune models on custom data, and run GPU-accelerated benchmarks so the team ships AI systems they can trust.
About Falls Technology
We're a small business that's been delivering since 2019, so your work is visible and your voice counts.
Responsibilities
- What you'll do Test, evaluate and validate AI systems with a focus on LLM-based systems (including RAG testing)
- Design and run experiments that benchmark LLMs against defined measures
- Tune neural networks / LLMs on custom datasets and apply results to concrete use cases
- AI/ML models and ensembles in Python or R (library selection, preprocessing, model assessment)
- Use deep learning frameworks such as PyTorch, TensorFlow or Keras Leverage GPUs for accelerated training and evaluation
- Write solid SQL (CTEs, set operations, aggregates, nested subqueries) against analytical datasets
- Work in GitHub and Jenkins-style CI for reproducible experiments
- Communicate methodological choices and model results to technical and non-technical stakeholders
- What you bring (required) Active TS/SCI Hands-on RAG testing experience (must-have) Tuning neural nets / LLMs on custom datasets for real use cases
- Building AI/ML models and ensembles in Python or R PyTorch, TensorFlow or Keras (key) GPU-accelerated computing (key) Testing, evaluating or validating LLM-based AI systems (key) Experiment design to benchmark LLMs (key) SQL including CTEs, set ops, aggregates and nested subqueries GitHub and Jenkins (or equivalent) version control / CI Nice to have Hugging Face Transformers library and hub
- Building benchmarks that measure LLM performance
- Cloud computing development and architecture
- Flask or similar front-end/web frameworks
Requirements
- Active TS/SCI required.
- U.S. citizenship required.
Preferred
- Text classification and topic modeling (scikit-learn or deep learning)
Location
On-site in McLean, VA (Full-Time).
Semantic search applications Tableau dashboards Verification/validation test benches, Explainable AI (XAI), ONNX Why Falls A 9% retirement contribution, 100% of family medical premiums paid, and 4 weeks of PTO.
Falls Technology is an Equal Opportunity Employer. All qualified applicants will receive consideration without regard to race, color, religion, sex, national origin, disability, or protected veteran status.
