About this episode
In this episode, I sit down with Michelle Pokrass, who leads a research team at OpenAI within post-training focused on improving models for power users: developers using OpenAI models in the API and power users in ChatGPT. We unpack how OpenAI prioritized instruction-following and long context, why evals have a 3-month shelf life, what separates successful AI startups, and how the best teams are fine-tuning to push past the current frontier.If you’ve ever wondered how OpenAI really decides what to build, and how it affects what you should build, this one’s for you. (0:00) Intro(1:03) Deep Dive into GPT-4.1 Development(2:23) User Feedback and Model Evaluation(4:01) Challenges and Improvements in Model Training(5:54) Advancements in AI Coding Capabilities(9:11) Future of AI Models and Fine-Tuning(20:44) Multimodal Capabilities(22:59) Deep Tech Applications and Data Efficiency(24:14) Preference Fine Tuning vs. RFT(26:29) Choosing the Right Model for Your Needs(28:18) Prompting Techniques and Model Improvements(32:10) Future Research and Model Enhancements(39:14) Power Users and Personalization(40:22) Personal Journey and Organizational Growth(43:37) Quickfire With your co-hosts: @jacobeffron - Partner at Redpoint, Former PM Flatiron Health @patrickachase - Partner at Redpoint, Former ML Engineer LinkedIn @ericabrescia - Former COO Github, Founder Bitnami (acq’d by VMWare) @jordan_segall - Partner at Redpoint
