AI Infrastructure & LLM Inference Engineer

AI Infrastructure & LLM Inference Engineer at Nexera

Nexera AI Infrastructure & LLM Inference Engineer Remote

• Full time Company website Apply for AI Infrastructure & LLM Inference Engineer

As an AI Infrastructure & LLM Inference Engineer, you will build and operate self-hosted AI platforms for clients who want to run their own inference. As capable open-weight models and accessible GPU hardware keep arriving, more organizations are choosing to keep inference local or hybrid rather than sending everything to a hosted API. You will be the engineer who makes that practical: secure, fast, observable, and supportable. The work spans the full stack beneath the application. You will validate and configure GPU environments, tune the operating system and NVIDIA software stack, select and operate inference runtimes, stand up gateways and identity, build observability, and write the runbooks that let client engineers run the platform confidently after handoff. Deployments range from fully on-premises and air-gapped to hybrid architectures that pair local inference with cloud capacity. This is production inference and platform operations, not foundation-model research. You will deploy and optimize open-weight language and multimodal models, establish performance baselines that hold up under real load, and keep systems reliable under real usage. You will work shoulder to shoulder with other applied AI engineers who build the retrieval, chat, research, and developer experiences on top of what you establish, so the quality of your work shows up directly in what users experience. We care about what you have built and what you have run in production operations, not necessarily how long you have been running it. Many strong candidates will have roughly three to five years in infrastructure, platform engineering, SRE, HPC, distributed systems, or production ML, with hands-on LLM serving, local inference, or GPU-backed AI work. Years of experience are guidelines. We always look at actual work experience as the key indicator of experience and fit.

About Nexera

Nexera is an AI-native consulting firm that builds intelligent, custom solutions for organizations navigating the age of AI. We don't just talk about AI. We build with it, ship with it, and embed it into everything we deliver. Our engineers use the latest agentic development tools to move fast, solve hard problems, and create real business value for our customers. We are a fast-paced startup that is 100% focused on customer success. We invest in our people with cutting-edge tooling, continuous learning, and a culture that rewards pushing limits and driving innovation. If you want to work at the frontier of how software gets built and deployed, not just read about it, Nexera is the place.

Description

What You'll Do

Design and Deploy AI Infrastructure

Build and configure secure, production-ready platforms for running large language models and other AI workloads across on-premises, private-cloud, hybrid, and restricted-network architectures.

Operate Model-Serving Platforms

Deploy, configure, qualify, and maintain models, inference runtimes, gateways, and supporting services throughout their operational lifecycle.

Optimize Performance and Capacity

Measure and improve latency, throughput, concurrency, resource utilization, and model placement using repeatable benchmarks and real workload evidence.

Build for Reliability and Security

Implement monitoring, alerting, access controls, auditability, incident response, recovery procedures, and secure software and model lifecycle practices.

Automate Platform Delivery

Use infrastructure-as-code, configuration management, testing, and release automation to make deployments, upgrades, scaling, and rollback repeatable and boring.

Partner and Transfer Knowledge

Work directly with client and application teams, communicate technical tradeoffs, document decisions and operating procedures, and leave engineers genuinely able to run and extend the platform.

Innovate How We Work

Experiment with and define the agentic workflows, automation, and tooling that improve how Nexera builds and operates AI platforms. Your patterns become part of our operating model.

What We're Looking For

The bullets below are the profile we hire against; the Tools and Technologies section that follows is preferred, not required. If you meet most of what is here and can show real production work, we want to hear from you.

• Systems experience that shows. Infrastructure, Linux systems, SRE, platform engineering, HPC, distributed systems, MLOps, or production ML engineering. Roughly three to five years, or equivalent capability you can demonstrate.
• Hands-on LLM and local-inference work. You have deployed or operated LLMs, inference runtimes, or GPU-backed workloads in production.
• Strong Linux fundamentals.

You are comfortable troubleshooting below the application layer

services, processes, networking, storage, permissions, resource limits, containers, drivers, and OS behavior.

• Practical GPU knowledge. You understand GPU memory, model placement, multi-GPU communication, utilization, thermal and power behavior, and how model architecture, precision, context length, and batching translate into capacity.
• Production ownership. You have run systems other people depend on and know what monitoring, alerts, incident response, upgrades, backups, rollback, and handoff actually require.
• Performance discipline. You measure before you optimize, you state the workload behind your numbers, and you can walk someone through how you found a bottleneck across application, runtime, memory, communication, or hardware.
• Security-minded engineering. Least privilege, network isolation, artifact integrity, identity propagation, auditability, vulnerability management, and restricted-network operations are familiar territory.
• Daily AI tool user. You already use AI tools (Claude, Claude Code, Copilot, Cursor, or similar) in how you code, research, and solve problems, and you can clearly show and demonstrate how you apply them to day-to-day engineering work.
• Relentless curiosity. You follow new model and runtime releases because you find it interesting, not because someone asked you to. You have opinions about where local inference is headed and you want to prove them out.
• Strong communication skills. You can explain a GPU bottleneck to an engineer, summarize risk and options for a program lead, work constructively with security and data-center teams, and document decisions clearly.
• Consulting mindset. You can work in a client’s environment, navigate ambiguity, surface blockers early, make pragmatic recommendations, and deliver against agreed outcomes.
• U.S.

citizenship and residency

Required for this role due to the customer environments and information the work may support.

Military or cleared experience preferred

Prior U.S. military service, experience supporting defense or national-security organizations, and/or a current or previously held U.S. government security clearance are preferred but not required.

No clearance today is not a barrier

Nexera will sponsor clearance processing when assigned work requires it.

Tools and Technologies (Preferred)

You do not need every item below. Depth with one production inference stack, strong systems fundamentals, and evidence that you learn adjacent technologies fast are what we will weigh and evaluate.

Operating systems and hardening

RHEL, Rocky Linux, AlmaLinux, SELinux, FIPS mode, DISA STIG, OpenSCAP, systemd

NVIDIA GPU stack

H100/A100/HGX-class systems, CUDA, NCCL, NVLink/NVSwitch, Fabric Manager, NVML, DCGM/DCGM Exporter, Nsight Systems, Nsight Compute

Inference and ML runtimes

vLLM, SGLang, TensorRT-LLM, NVIDIA Triton or Dynamo, Hugging Face Transformers/TGI, PyTorch, Ray Serve

Inference concepts

tensor and pipeline parallelism, BF16/FP8/INT4, quantization, continuous batching, KV and prefix caching, prefill/decode behavior, speculative decoding, long-context serving, structured output, tool calling

Containers and orchestration

Podman, Docker, containerd, Kubernetes, OpenShift, NVIDIA GPU Operator, Slurm, Helm

Gateway and identity

NGINX, LiteLLM or comparable gateways, OpenAI-compatible APIs, Keycloak, FreeIPA, OIDC, JWT, service accounts, TLS

Observability and operations

Prometheus, Grafana, OpenTelemetry, Elastic, Wazuh, Splunk, Loki, alerting and SLO tooling

Automation and supply chain

Python, Bash, Ansible, Terraform or equivalent infrastructure-as-code, Git, private RPM/Python/container repositories, SBOMs, cosign, artifact signing and offline promotion

AI CLI and agentic tools

Claude Code, Gemini CLI, OpenAI Codex, GitHub Copilot

Additional differentiators

C++, CUDA or Triton kernel work, inference-runtime contributions, InfiniBand/RDMA/RoCE, regulated or classified environments, and on-premises or air-gapped deployments

Position Details

Structure

Full-time or contract. We are flexible and will work out the right arrangement with the right person.

Location

Remote-first (US-based), with scheduled on-site work at client sites and data centers

Travel

Expect approximately 25% travel, concentrated in bursts around platform bring-up, qualification, deployment, and major troubleshooting events, with lighter travel between deployments. On-site work is scheduled in advance wherever possible.

Eligibility

U.S. citizenship and current U.S. residency are required

Access Requirements

Must be willing and able to complete applicable client background investigations, satisfy site-access and data-handling requirements, and obtain and maintain a U.S. government security clearance if required for assigned work

Compensation

Full-time base salary range

$175,000–$225,000, based on demonstrated experience, technical depth, clearance status, and scope of responsibility, plus performance bonus.

Contract engagements

approximately $95–$150/hour (W-2) or $120–$175/hour (corp-to-corp), commensurate with experience, specialization, and clearance.

Benefits (Full-Time)

Time off

Flexible PTO plus federal holidays

Learning

Annual learning and certification budget, plus access to premium AI tooling (Claude Code and the frontier tools you will use daily)

Home office

Home-office setup

Travel comfort

Sensible travel policy — you book what makes the trip productive, and travel time is respected as work time

Our Hiring Process

We keep it fast and substantive — typically two to three weeks end to end:

Intro conversation (30 min)

your background, what you have run in production, and mutual fit

Technical deep-dive (60–90 min)

a working session on systems you have built and operated — GPU environments, inference runtimes, incidents you have handled — plus how you use AI tools day to day

Scenario round (take away task)

walk through a realistic client deployment or troubleshooting scenario, including how you would communicate tradeoffs to a client

Final conversation with leadership, then a written offer

Salary

$175,000 - $225,000 per year

Apply for AI Infrastructure & LLM Inference Engineer

Your Privacy Choices Read our Privacy Policy

By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts.

Cookies settings Reject all Accept all

Privacy Preference Center

When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.

Learn more information about your privacy

Allow all

Manage consent preferences

Strictly Necessary Cookies

Always active

These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work. These cookies do not store any personally identifiable information.

Functional Cookies

Functional Cookies

These cookies enable the website to provide enhanced functionality and personalisation. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.

Performance Cookies

Performance Cookies

These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.

Targeting Cookies

Targeting Cookies

These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Cookie List

Clear checkbox label label

Apply Cancel

Consent Leg.Interest

checkbox label label

checkbox label label

checkbox label label

Reject all Confirm my choices