As a Staff/Principal Software Engineer, you would be a significant part of one or more of these areas. The key challenges are:

• Shipping quickly. AI is evolving weekly, so there's a ton of opportunity to be had. We want to move fast to capture those opportunities while they are still fresh and full of potential.
• Zero to one. The platform is not a simple copycat. We have a vision for a deep platform/suite of capabilities that make it dramatically simpler for developers to scale and evolve their AI.
• Realtime, online. As consumer applications become more capable of listening and talking, performance will matter, and AI has to adapt in realtime as well. These are bold but exciting challenges.
• Multi-provider complexity at scale. Inworld Router must intelligently route across hundreds of models and providers while handling failover, sticky sessions, cost optimization, and conditional logic, all with minimal latency overhead.

Finally, almost everything here is a collaboration with our sibling ML teams, since ML and AI are critical to providing the learning and adaptability central to this vision.

Please note: This is an IC-focused role. We are looking for someone who loves direct technical contribution alongside very capable peers.

WHAT YOU’LL DO:

• Establish significant scope: Collaborate with the PMs, engineers and leads to determine the biggest product needs to focus on now.
• Operate with technical autonomy: You have considerable leeway to suggest how to address a given focus area, including bringing in new technical dependencies or standards where it's the best choice.
• Collaborate, execute, deliver: This is the core of the building loop. We aim to optimize for both speed and quality, despite it being decidedly non-obvious how to manage that tradeoff exceptionally well.
• Reflect and drive improvements: Especially as a Staff Engineer, advocate for and realize system improvements, both related to and independent of key features.

EXPECTED EXPERIENCE:

Must Haves

• Excellent programming skills and experience in a statically typed backend programming language, preferably Go, Python, C++ or Rust
• Experience developing and deploying cloud-based services to at least hundreds of qps (preferably more)
• Experience with relational databases (PostgreSQL or MySQL)
• Hands-on experience with caching (Redis or Memcached), pubsub/queues, data pipelines (Flink, Beam), and Cloud storage
• Excellent verbal and written communication skills, can collaborate and coordinate with other roles and engineer with ease, trusted and well-regarded teammate

Bonus Qualifications

• Experience building API gateways, routing/proxy layers, or multi-provider orchestration systems
• Experience with analytics or timeseries databases (ClickHouse, Timescale, InfluxDB)
• Experience with OpenTelemetry
• Experience with C++

Candidates must be based in the SF Bay Area or willing to relocate (you will be working on-site in our South Bay office a few days a week).

The US base salary range for this full-time position is $280,000 - $350,000. In addition to base pay, total compensation includes equity and benefits.