AI Inference Engineer (San Francisco)
Role Overview
This is a mid-to-senior AI Inference Engineer role at Perplexity, focusing on deploying and optimizing machine learning models for real-time inference. Day-to-day responsibilities include developing APIs for AI inference, benchmarking and addressing bottlenecks in the inference stack, and improving system reliability and observability. The hire will work on a growing team to implement LLM inference optimizations and respond to system outages, impacting large-scale deployment for both internal and external customers.
Perks & Benefits
This role is fully remote, offering flexibility in work location, though time zone expectations may align with San Francisco for collaboration. It provides opportunities for career growth through hands-on work with cutting-edge technologies like PyTorch, Rust, and CUDA, and involvement in novel research. The culture likely emphasizes innovation and technical excellence, with a focus on improving system reliability and exploring new optimization techniques in a fast-paced environment.
Full Job Description
We are looking for an AI Inference engineer to join our growing team. Our current stack is Python, Rust, C++, PyTorch, Triton, CUDA, Kubernetes. You will have the opportunity to work on large-scale deployment of machine learning models for real-time inference.
Responsibilities
Develop APIs for AI inference that will be used by both internal and external customers
Benchmark and address bottlenecks throughout our inference stack
Improve the reliability and observability of our systems and respond to system outages
Explore novel research and implement LLM inference optimizations
Qualifications
Experience with ML systems and deep learning frameworks (e.g. PyTorch, TensorFlow, ONNX)
Familiarity with common LLM architectures and inference optimization techniques (e.g. continuous batching, quantization, etc.)
Understanding of GPU architectures or experience with GPU kernel programming using CUDA
Similar jobs
Found 6 similar jobs