Software Engineer, ML Systems

About this role

Job Description:
Software Engineer, ML Systems

Location:
San Jose, California, United States (On-site, 5 days per week; relocation support available)

Employment Type:
Full-time

Experience:
1–15 years

Position Overview

We are seeking an
ML Systems Engineer
to optimize the performance and efficiency of large language model (LLM) inference powering an agentic AI platform. This is a hands-on, low-level systems optimization role focused on improving LLM throughput, latency, and cost efficiency in production environments.

You will design and optimize inference systems across multi-node clusters, build benchmarking and evaluation frameworks, and work closely with research and engineering teams to bring advanced optimizations into production. Your work will directly impact the responsiveness and cost-efficiency of AI systems used by leading semiconductor companies.

Key Responsibilities

• Design, deploy, and optimize LLM inference systems across multi-node clusters to maximize throughput and minimize latency for production workloads.
• Implement, benchmark, and validate concrete inference optimizations.
• Profile and analyze inference bottlenecks at the systems level, including GPU kernel execution and memory bandwidth constraints.
• Build robust evaluation harnesses and benchmarking frameworks to measure accuracy, throughput, latency, and resource utilization across parallelism strategies.
• Collaborate with research scientists to integrate new model architectures and optimizations into production inference infrastructure.
• Evaluate and apply relevant techniques from research papers and open-source projects to continuously improve inference performance.
• Work directly with customers to clarify technical requirements and represent the organization professionally.

Required Qualifications

• 1–15 years of experience as a Software Engineer or ML Systems Engineer.
• Experience optimizing performance for high-performance computing (HPC) ...