- play_arrow Try prototype You'll need an invite code
- code Git repo
How might AI make product brainstorming more fun?
Think back to the last time you and your team worked on a new product or feature. You probably threw together a PRD, some Figma or Stitch design sketches, maybe a couple prototypes. You probably used a bunch of different tools and worked in silos, at least some of the time.
One of the reasons collaborative whiteboarding tools like FigJam are so popular with teams during brainstorming is they foster co-creation, levelling the playing field and enabling anyone to contribute, no design degree necessary. Anyone can claim their own little corner of the infinite canvas and push around a couple rectangles to put an idea on digital paper.
But these tools are typically locked in low-fidelity, and the raw materials you’re working with are static: stickies, shapes, and words. What if your team had a richer canvas?
An infinite canvas with superpowers
In this experiment, we brought a couple new superpowers to the canvas:
- An assistive voice-first agent powered by the Live API
- 🍌 Nano Banana for generating design sketches
- A simplified coding agent for generating on-the-fly in-browser prototypes, along with an in-browser runtime powered by
esbuild-wasm - A rich, Wiki-style project knowledge graph for specifying personas, user goals, tech stack alternatives, etc.
- Video calls with unobtrusive, flingable participant video bubbles
A first look
One of the things we focused on in the prototype was the voice agent. We originally thought about including a simple text chat panel, but we quickly realized chat panels force you to keep your keyboard and mouse focused on managing the agent. Voice-based assistance lets you stay in flow, keeping your hands focused on the canvas.
Figure 1. A basic voice assistant on the infinite canvas.
Figure 2. The agent collaborating with you on your canvas. You can see from the recording some of the… hiccups in Gemini’s live models at the time 😛
🍌 Nano Banana for sketches
Typically, if you’re writing a PRD, you’d bounce into design or sketching software, flesh out your ideas, and paste them back into the PRD—your “primary” artifact.
Image generation models are getting much, much better, and especially for quickly sketching out ideas, it’s often faster to have an image (or design) generation model go off and try N variations and drop them back on your canvas.
Figure 3. The agent can generate “napkin sketches” for you. Note: Sped-up video
Throwing prototypes on the canvas
If a picture is worth a thousand words, a prototype is worth… you get the point. One of Stitch’s biggest launches was on-canvas prototypes.
An ideal brainstorming canvas should enable you to explore and remix interactive prototypes. We built an in-browser prototyping runtime powered by esbuild-wasm that lets you live-preview rich prototypes generated by the voice agent. These rich prototypes are React+Tailwind mini-apps or components, and can even pull in 3P dependencies from npm in the browser using import maps.
Figure 4. Working prototypes on the canvas enable you to feel out your concepts. Note: Sped-up video
Figure 5. Try out multiple concepts on one canvas. Note: Sped-up video
Making it multiplayer
Adding realtime, multiplayer collaboration makes things far more interesting, creating a true shared space for the entire team to explore together. Some highlights:
- Realtime video – It quickly became clear that video (or at least audio) was essential to a productive collaboration session. Making video bubbles flingable with matter.js dialed in the “fun” factor. Video feeds powered by WebRTC and (Firebase for session handshakes) in our prototype.
- Multiple cursors – table stakes these days! Powered by Firebase Realtime Database in our prototype.
- Gemini is a peer – You’ll notice in the video below that when a canvas is in its “meeting” mode, Gemini becomes a 3rd bubble, listening to everyone and contributing to the conversation. While this was very prototype-y in our experiment, it felt quite natural and thus shows promise for future collaboration mechanics, especially as live models improve.
Figure 6. David East, Roman Nurik and Gemini working on some ideas together, over video. Everyone is able to focus on the product canvas, with camera video feeds and assistance
What’s next
Want to try it and share your feedback? Check out the repo or reach out to @julesagent, David or Roman for some invite codes for your team to try the hosted prototype.