tl;dr: research engineer + research scientist, non-profit, full-time or short-term for NeurIPS submission, $60k-250k, remote

Once upon a time, a neuroscientist, a biologist, and a legal scholar were tasked with writing about AI safety — and it turned out to be a perfect fit for them 🎩

📘 A neuroscientist discovered that a model's internal activations reveal when it's lying — ICML 2024. 🪐 A biologist, working with theoretical physicists, proved that transformers build something akin to a world model to predict the next token — NeurIPS 2024. ✏️ A law professor outlined how existing laws can be used to hold AI developers accountable for catastrophic damage. This was covered by Vox 🎩

As you may have guessed, this is not a humorous anecdote. It's a non-profit called Principles of Intelligence (PIBBSS). They award scholarships to researchers from various fields, who in return write papers on safety. Researchers love it — so much so that they often change their profile afterward and go to work at Anthropic or DeepMind. The organization is sponsored by Open Philanthropy, and their grant has increased 7-fold in two years.

With the NeurIPS deadline approaching, PIBBSS aims to submit several new papers to it from their mech interpretability division. Therefore, you can sign up for a short-term contract until May 11th, or opt for a long-term full-time position. The topics include reverse-engineering models, interpretability during training, and inherently interpretable architectures. They are looking for two profiles: ** Research Engineer:** 🔸 Strong SWE background; 🔹 ML experience; 🔵 Ability to run LLM evaluations. 🔸 Plus: GPU programming (CUDA/Triton), distributed computing, AI Safety courses like ARENA. ** Research Scientist:** 🔸 PhD or several years of research experience in CS, mathematics, physics, or economics; ▫️ Publications in ML or related fields. Ideally in interpretability, but PIBBSS is interdisciplinary and adept at working with various directions. ▪️ Plus: experience leading ML research projects.

As mentioned, you can commit to a couple of months with the possibility of transitioning to full-time later — or decide on full-time right away. They pay $60k-$180k per year for engineers, and $80k-$250k for researchers, depending on your location and experience.

Contact Dasha: @d_shteinardt 💌