Inference Software Engineer

About this role

About Etched
Etched is building hardware for frontier intelligence. We co-design chips, racks, software, and manufacturing to deliver best-in-class throughput and latency across both prefill and decode workloads. Our first products are heavily focused on
inference
. Backed by hundreds of millions from top-tier investors and staffed by leading engineers, Etched is redefining the infrastructure layer for the fastest growing industry in history.


Key Responsibilities

• Support porting state-of-the-art models to our architecture. Help build programming abstractions and testing capabilities to rapidly iterate on model porting.
• Build, enhance, and scale our runtime, including multi-node inference, intra-node execution, state management, and robust error handling.
• Optimize routing and communication layers using our collectives.
• Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues.

You may be a good fit if you have

• Proficiency in C++ or Rust.
• Understanding of performance-sensitive or complex distributed software systems like Linux internals, accelerator architectures (e.g. GPUs, TPUs), Compilers, or high-speed interconnects (e.g. NVLink, InfiniBand).
• Familiarity with PyTorch or JAX.
• Ported applications to non-standard accelerator hardware or hardware platforms.

Strong Candidates May Also Have Experience With (Nice-to-have Qualifications)

• Developed low-latency, high-performance applications using both kernel-level and user-space networking stacks.
• Deep understanding of distributed systems concepts, algorithms, and challenges, including consensus protocols, consistency models, and communication patterns.
• Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE).
• Built applications with extensive SIMD (Single Instruction, Multiple Data) optimizations for performance-critical paths.

Benefits

• Medical, dental, and vision packages with generous premium cove...