Sr. AI Engineer
BuzzClan LLC
- New York, United States
- On-site
- Senior Level
- Posted
Skills
Required
- Python
- AI agents
- memory
- evaluation
- observability
- LangGraph
- agent memory
- software engineering
- APIs
- LLMs
- retrieval
- prompt engineering
- SQL
Preferred
- credit risk / market risk / financial-services risk workflows
- Model Risk Management / model development/validation lifecycles / model documentation / governance / model monitoring
- broader risk management
- Snowflake / Databricks / lakehouse architectures / enterprise data pipelines / governed knowledge integration
- Model Context Protocol MCP
- cloud deployment
- containers
- enterprise identity/access controls
- AI coding
- GitHub
AI Engineer (Agentic AI, Python & Risk Management) Location: New York City, NY (Need local resource who can go for In-person) Contract Team: Firm Risk Management (FRM) Technology Additional Points : Strong Python and hands-on LangGraph experience, including context/prompt engineering, memory, evaluation, and observability using Galileo or similar tools.
Responsibilities
- Design and build modular, maintainable Python applications, APIs, and services that integrate LLMs, enterprise data, and business tools.
- Develop LangGraph-based agent workflows with explicit state, conditional routing, tool execution, checkpointing, and human-in-the-loop approval or escalation where appropriate.
- Implement appropriate short- and long-term memory strategies, including persistence, retrieval, summarization, retention, and isolation between users and sessions.
- Build evaluation frameworks with representative datasets, repeatable regression tests, and appropriate automated and human evaluation.
- Assess answer quality, grounding, tool-call correctness, task completion, and failure behavior.
- Implement LLM and agent observability using Galileo or comparable platforms, including execution traces, quality signals, latency, token usage, cost, and failure analysis.
- Apply guardrails for prompt injection, unsafe tool use, sensitive-data exposure, and unsupported outputs; incorporate access controls and auditable execution paths.
- Collaborate with risk specialists, product owners, data engineers, and platform teams to translate requirements into tested applications and support deployment and troubleshooting.
- Contribute technical design reviews, code reviews, documentation, and engineering guidance while remaining actively involved in coding.
Requirements
- Risk management knowledge is highly desirable.
- Python, SQL, and data engineering skills, with practical understanding of AI agents, context/prompt engineering, memory, evaluation, and observability.
- About the role We are seeking a hands-on Senior AI Engineer to design, develop, and operationalize agentic AI applications for Firm Risk Management.
- Potential use cases include cross-risk variance analysis, credit risk lifecycle workflows, continuous news monitoring, and model risk lifecycle processes.
- This is a software engineering role requiring strong Python design and development skills and practical experience building AI agents with LangGraph.
- The successful candidate will own technical design and implementation, with particular focus on context quality, agent memory, evaluation, observability, and reliable operation in a governed enterprise environment.
- select and assemble relevant instructions, retrieved information, tool outputs, and conversation history within token and access-control constraints.
- Design, version, and test prompts, including structured outputs, grounding instructions, and safe handling of incomplete or conflicting information.
- Strong hands-on Python software engineering, including application design, asynchronous programming, APIs, automated testing, error handling, and maintainable code.
- Demonstrated experience implementing AI agents with LangGraph, including stateful orchestration and integration with external tools or services—not only chatbots, framework tutorials, or conceptual designs.
- Practical understanding of LLMs, embeddings, retrieval-augmented generation (RAG), vector search, and the tradeoffs involved in retrieval and context assembly.
- Hands-on experience with context engineering, prompt engineering, and agent memory management.
- Experience designing and implementing evaluation frameworks for LLM or agent applications, including test-set construction and interpretation of evaluation results.
- Experience with Galileo or a comparable LLM/agent evaluation and observability platform, with the ability to trace failures and use findings to improve applications.
- Experience delivering and supporting deployed applications, with sound understanding of security, reliability, version control, CI/CD, and production troubleshooting.
- Ability to explain technical decisions, distinguish personal contributions from team outcomes, and collaborate effectively with business and engineering stakeholders.
Preferred
- Knowledge of Model Risk Management, model development/validation lifecycles, model documentation, governance, or model monitoring.
- Experience in broader risk management, particularly credit risk, market risk, or financial-services risk workflows.
- Experience with Snowflake, Databricks, lakehouse architectures, enterprise data pipelines, or governed knowledge integration.
- Familiarity with Model Context Protocol (MCP), cloud deployment, containers, and enterprise identity/access controls.
- Prior experience using AI coding assistants such as GitHub Copilot, Claude Code, or Amp, with disciplined review, testing, and secure handling of code and data.
- All your information will be kept confidential according to EEO guidelines.
