About this episode
"AI has been the biggest driver of change in experimentation I've seen in my career."That's Frederic De Todaro, Chief Product Officer at Kameleoon (profitable SaaS with 2K+ customers).Fred has been at Kameleoon for 12+ years. In that role, he's helped thousands of teams use AI to experiment faster and smarter.In today’s episode he’s breaking down:How AI changes experimentationHow to experiment with AI featuresLast week, I covered how one aspect of this: vibe experimentation. Today’s video is the A to Z AI impact. If you experiment at work, this episode is for you.----Check out the conversation on Apple, Spotify and YouTube.* Mobbin: Discover real-world design inspiration* Jira Product Discovery: Build the right thing, reliably* AI Product Strategy Certificate for Leaders: Get $550 off----Timestamps:00:00 How AI Changed Experimentation Overview01:54 The 4 Steps of Experimentation Framework14:12 ADS16:00 How AI has Changed Experimentation21:08 User Behaviour Models26:56 Multi-Armed Bandit vs Contextual Bandit30:05 ADS31:55 AI Content Genration35:13 How Vibe Coding Changes Experimentation41:35 Live Demo From Idea to Running Experiment in 2 Minutes43:36 Two-Minute Build Achievement51:49 How to Measure AI Features Properly54:17 Measuring RAG Systems 3 Key Metrics01:07:18 Best Experimentation Company Booking.com01:10:10 Biggest PM Mistakes in Experimentation01:13:52 Ending----Key Takeaways1. Build is the bottleneck. Most teams can't A/B test because developers are busy. AI removes this constraint anyone can now create experiments in minutes using plain English.2. 80% of experiments fail. But with AI opportunity detection, you can drill into failed experiments to find hidden wins, like features that work great on mobile but fail on desktop.3. Vibe coding meets experim
