jobapplier

AI Architect

Insight Softmax Consulting LLC

Skills

Required

  • AI / ML
  • infrastructure / platform / ML engineering
  • LLM
  • vLLM
  • TensorRT-LLM
  • llm-d
  • Triton
  • tensor and pipeline parallelism
  • batching
  • caching
  • performance tuning
  • AI gateway / API management
  • LiteLLM / Traefik / Envoy / Tetrate based designs / Kubernetes
  • Linux
  • AWS / GCP / Azure

Preferred

  • RAG
  • agentic systems
  • model evaluation
  • fine-tuning pipelines
  • Air-gapped
  • FIPS
  • regulated deployment
  • Spectro Cloud
  • Python / Go / Rust

We are dedicated to providing innovative solutions in computing technology to solve complex problems in various sectors including research, science, manufacturing, finance, and data analytics.

We are hiring an AI Architect for our engineering team. You will be the senior technical voice for customers who are building or operating AI infrastructure: GPU clusters, inference platforms, AI gateways, and the services that sit on top of them.

You will also support pre-sales scoping and estimates. This is a hands-on architect role. You are expected to prototype, validate assumptions in a lab, and review the implementation, not only produce slides.

About Insight Softmax Consulting LLC

Insight Softmax is a leading organization in the fields of machine learning, data science, and high-performance computing (HPC).

Our customers range from chip vendors and neoclouds running thousands of GPUs, to enterprises standing up their first production LLM services, to platform vendors who have us build parts of their product.

They ask us how the leading AI labs and AI-native companies run their platforms, and they expect answers grounded in experience.

Our team requires individuals with focus on delivery and results, hunger to learn, ability to adapt, and can thrive in a self-directed environment where you’ll figure out what’s needed to deliver rather than wait for detailed instructions.

Responsibilities

  • You will lead discovery and architecture phases of engagements, produce reference architectures and recommendations, and hand clear designs to our engineers for implementation.
  • Lead technical discovery with customers and translate business goals into AI platform architectures, phased roadmaps and effort estimates.
  • Design inference serving platforms on Kubernetes and bare metal: model serving, GPU scheduling and partitioning, autoscaling, multi-tenancy, and capacity planning.
  • tenant-aware routing, model pools, budgets and quotas, rate limiting, observability, and multi-site failover.
  • Define reference architectures for multi-tenant GPU clouds, sovereign and air-gapped deployments, and hybrid cloud and on-prem estates.
  • cost visibility, token accounting, audit, data handling, and security posture for LLM services.
  • Evaluate and recommend build, buy or compose decisions across open source and commercial AI infrastructure components.
  • Review implementations delivered by our AI and platform engineers and own the technical quality of the outcome.
  • Represent Insight Softmax in customer architecture reviews, partner enablement sessions and executive readouts.
  • Reserve time each week for pre-sales support, internal platform work and mentoring.
  • NVIDIA and AMD accelerators, drivers and runtimes, GPU scheduling on Kubernetes, high-speed interconnects and storage for AI workloads.
  • Track record of leading architecture engagements with enterprise customers, including writing proposals, estimates and architecture documents that engineers can execute against.
  • Clear written and spoken communication with both engineers and executives.
  • traces, token and cost metrics, quality monitoring.

Requirements

  • At least 8 years in infrastructure, platform or ML engineering, with at least 3 years designing and operating production AI or ML platforms at scale.
  • Hands-on experience deploying LLM inference in production (for example vLLM, TensorRT-LLM, llm-d, Triton) including tensor and pipeline parallelism, batching, caching and performance tuning.
  • Experience designing AI gateway or API management layers (for example LiteLLM, Traefik, Envoy or Tetrate based designs, Kubernetes Gateway API inference extensions).
  • Strong Kubernetes and Linux skills.
  • Able to read and debug the stack from the scheduler down to the node.
  • Deep knowledge of at least one of AWS, GCP, or Azure.
  • Preferably AWS.
  • Experience at an AI lab, AI-native company, or neocloud, or building platform products for them.
  • Multi-tenant GPU cloud design, including namespace and cluster-level isolation, quota, billing and chargeback.
  • Slurm or other batch schedulers alongside Kubernetes for mixed HPC and AI estates.

Preferred

  • RAG, agentic systems, model evaluation and fine-tuning pipelines.
  • Air-gapped, FIPS or other regulated deployment experience.
  • Spectro Cloud Palette and PaletteAI, Run:ai, or similar GPU platform products.
  • Python, Go or Rust.
  • Software engineering background beyond infrastructure.
  • Published talks, blog posts or open source contributions in AI infrastructure.
  • Using linux as your primary desktop workstation environment.

Attributes that we value

Transparency Grit Effectiveness Willingness to learn Location and Hours: Full time position Remote work environment 80% of your work schedule will be during the business time zones for Latin America and Canada. Some customer meetings will occur each week at non-standard / nighttime hours, as we currently support a global team across 3-4 continents. Occasional travel to customer sites in the US for architecture workshops, load tests and go-live reviews.

References

3 references will be required.

Notes

AI systems may be used with the recruitment and screening processes for this job requisition.

Easy Apply