Senior Software Engineer - API Gateway
Role Overview
This mid-level Software Engineer role focuses on developing and maintaining the API gateway for Featherless AI's inference platform, handling authentication, subscription management, and API surface for AI models. As part of the Platform Team, you'll work on feature development, bug fixes, reliability improvements, and incident response to support clients and onboard new models. The role involves debugging complex issues, building instrumentation, and collaborating with a skilled team to enhance the platform's performance and user experience.
Perks & Benefits
This is a fully remote position with a preference for candidates in Toronto, Canada, and the team operates on Eastern Time, offering flexibility within that time zone. The company emphasizes a collaborative culture with values like bias to action, responsiveness to users, and iterative work, fostering career growth through hands-on involvement in a cutting-edge AI infrastructure. Benefits likely include typical tech perks such as health insurance and professional development opportunities, though not explicitly stated.
Full Job Description
About the Role
Featherless.ai is building the world’s most reliable and comprehensive open-model inference platform — the infrastructure powering the next generation of AI creators, startups, and enterprises. Our serverless approach to inference unlocks the best GPU utilization in AI infrastructure.
We’re hiring Senior Software Engineers to support and evolve the API gateway to our inference cloud, which is responsible for
authentication and inference to all models
subscription management and subscription entitlement (e.g. context-length, concurrency limits)
and providing the necessary API surface for applications and builders
API Gateway is constantly evolving in response to the unending stream of new models, modalities, clients and inference load.
What you'll do
The API gateway is managed by the Platform Team, who aim to make Featherless the best place to find and use models. As a member of the platform team, you will
undertake feature development and bug fixes to keep up with clients, resolve user issues, and onboard new models
improve the reliability of the existing API (increasing instrumentation and monitoring, right-sizing infrastructure)
respond to availability incidents
triage and resolve issues of inference quality and reliability
manage the infrastructure on which our gateway runs
What you'll bring
first-hand experience of the user’s we’re building for (familiarity with popular open LLMs, common clients, and experience building with LLM)
experience with the technologies and paradigms of the web (REST, websockets, DNS, networking, opentelemetry)
experience with significant components of our stack (k8s, node, mikro-orm, fastify, redis, mongodb, python, elastic cloud, cloudflare, sentry, otel)
ability to debug complex issues across a wide stack and build instrumentation as necessary
desire to work collaboratively as part of a skilled team
Alignment with team and company values, including
bias to action
responsiveness to users (bug-fixes over features)
instinct to iterate
subscribing to that done means proven by usage data
Other
This team operates on Eastern Time. We are remote, but with a preference to hire in Toronto, Canada.
Similar jobs
Found 6 similar jobs
