Rust Systems Engineer - Inference
Together AI
Software Engineering
San Francisco, CA, USA
USD 160k-240k / year + Equity
Posted on Apr 24, 2025
About the Role
Together AI is seeking a Rust Systems Engineer to join our Inference Engine team, focusing on optimizing and enhancing the performance of our AI inference systems. If you are passionate about developing high-performance systems, we want to hear from you. This position offers the chance to collaborate closely with AI researchers and engineers to create cutting-edge AI solutions. Join us in shaping the future at Together AI!
Required Qualifications
- Demonstrated proficiency in Rust programming language and ecosystem
- Strong experience with Rust frameworks namely
axum,tokio,jemalloc,serde,rayon, and/orcrossbeam - Deep understanding of concurrent programming and multiprocessing patterns
- Experience developing and optimizing orchestration and scheduling algorithms
- Knowledge of distributed systems principles and design
- Experience interfacing Rust services with RPC components
- Ability to optimize systems for performance, reliability, and resource efficiency
- Solid understanding of memory management and performance profiling.
Preferred Qualifications
- Experience with one or more ML inference frameworks (HuggingFace TGI, NVIDIA Dynamo, Candle, Pixi, uv, llguidance)
- Background in designing and implementing high-throughput, low-latency systems
- Knowledge of disaggregated serving architectures and continuous batching algorithms for ML workloads
- Familiarity with ML/AI inference systems and performance optimization techniques
- Experience with system benchmarking and performance analysis
Together AI is a research-driven artificial intelligence company. We believe open and transparent AI systems will drive innovation and create the best outcomes for society. Together, we are on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets to advance the frontier of AI. Our team has been behind technological advancements such as FlashAttention, Hyena, FlexGen, and RedPajama. We invite you to join a passionate group of researchers and engineers in our journey to build the next-generation AI infrastructure.
CompensationWe offer competitive compensation, startup equity, health insurance, and other competitive benefits. The US base salary range for this full-time position is $160,000 - $240,000 + equity + benefits. Our salary ranges are determined by location, level, and role. Individual compensation will be determined by experience, skills, and job-related knowledge.
Equal OpportunityTogether AI is an Equal Opportunity Employer and is proud to offer equal employment opportunities to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
Please see our privacy policy at https://www.together.ai/privacy