M: You and I are gonna collaborate to make an animated video. You’ll track things and scribble over stuff I make, and help with transitions. It’s going to be a rocky horror tribute, but at least half of it will be rotoscoping or original animation.
G: Got it. I’ll help you track elements, mark up frames, and smooth transitions wherever needed. When you’re ready, just drop in your materials—scripts, storyboard, frame sequences, or rough cuts—and I’ll jump in.
M: Which engine is the best to work in if I’m asking you to work with multiple image files in the interest of building transitions? Should I be engaging the agent?
G: TL;DR: You don’t need the Agent for this yet. For working with multiple image files and building transitions—with my help—the best setup depends on how you want to work:
Blender (Free, Open Source)
Not yet.
If you already have your first sequence or rotoscope frames, send those over and I’ll help build from there.
M: The reason I’m asking this way is because I know that I want the introduction to be the lips from the opening of Rocky horror, which I then will rotoscope and manipulate to make an animated version personally. Then, once I’ve done that, I will want you to add… I’m trying to think of how to describe this effect. I have a tendency to scribble in patterns that are geometric in pen drawings. What I’m going to want to do is have you work with me to animate this on lips that are moving – so, let’s say I created one pattern on one frame and then another set of patterns 10 frames later, I will need you to help me transition things, so that even if the patterns themselves have changed, when animated they seem to flow from the first frame to the last. So you can see why I’m interested in making sure I optimize that ability at the very least. It’s a bunch of fiddly bits that you can help me with but only if I approach it correctly.
G: Got it—and you're absolutely right to be thinking about the toolchain now, because what you're describing involves:
Here's the setup that gives you maximum flexibility with my help:
Use Blender's Grease Pencil or image planes with alpha overlays to draw on top of your rotoscoped lips.
Why Blender:
| Phase | Your Role | My Role |
|---|---|---|
| 1. Rotoscope Lips | Draw stylized lips, 1:1 match to Rocky Horror opening | N/A |
| 2. Base Import | Load lip frames into Blender | Help set up timeline, image sequence |
| 3. Scribble Patterns | Draw pattern A on Frame X, Pattern B on Frame X+10 | I’ll help interpolate (morph, distort, tween, or match anchors) |
| 4. Animate Flow | Choose where patterns flow, pulse, or reshape | I’ll generate tweens, distortion maps, or guide spatial flow |
| 5. Composite | Place animated patterns back on lips | Help fine-tune opacity, blend modes, timing |
ffmpeg filtergraphs to morph pattern A to B over timeLet me know what format you're most comfortable drawing in—if you're doing the art by hand, we can plan our first rotoscope pass. Once we’ve got 2 pattern drawings across time, I can prototype interpolation approaches so we decide what looks best.
M: You can eliminate 3-D from any consideration, at least at the moment. I don’t mind the idea in general, but this I want to be something like Tom and Jerry and R Crumb had a truce and made something relatively clean, in terms of lines if not production value. Maybe later there will come something that says 3-D will make it easier, but right now just pure 2-D animation. I know there are a couple other animation programs out there. I can’t name them, but I read about them and poked at them and they seemed appropriate for this. What is it that gives blender the edge? Should I perhaps plan on using multiple software depending on what stage of any given scene, I’m working on?
G: Perfect—locking into 2D-only and aiming for a look somewhere between Tom & Jerry’s squash-and-stretch and R. Crumb's twitchy geometry gives us a solid style foundation: expressive, detailed, but handmade.
Let’s break down which tools will suit each stage of your animation process and where Blender stands vs. others, so you can optimize without locking yourself in.
| Stage | Tasks |
|---|---|
| 1. Rotoscope Prep | Extract, clean, and sequence lip footage |
| 2. Frame Drawing | Hand-animate lips and overlay patterns |
| 3. Pattern Morphing | Transition patterns from A → B |
| 4. Timing & Flow | Sync visual beats to music/dialogue |
| 5. Export/Composite | Output final scene with lips + patterns + BG |
| Stage | Best Tool(s) | Why |
|---|---|---|
| 1. Rotoscope Prep | ffmpeg, Blender, or Krita | Frame extraction + previewing |
| 2. Frame Drawing | Krita, OpenToonz, Blender (Grease Pencil) | True 2D frame-by-frame drawing; Krita has a great timeline, OpenToonz has exposure sheets |
| 3. Pattern Morphing | Blender (Grease Pencil scripting) or vector morphing via CLI | This is where I can help most—interpolating shapes, pattern shifts, or geometric warping |
| 4. Timing/Sync | Blender or OpenToonz | Blender has tight timeline/audio tools; OpenToonz allows traditional exposure-sheet timing |
| 5. Export/Composite | Blender, ffmpeg, or DaVinci Resolve (free) | Composite layers + effects cleanly; Blender shines here with precision & scriptable control |
| Feature | Blender (2D only mode) | Krita | OpenToonz |
|---|---|---|---|
| Scripting/Automation | ✅ Full Python (I can help write scripts) | ❌ None | ⚠️ Somewhat scriptable (ToonzScript) |
| Pattern Tweening Help | ✅ Grease Pencil modifiers + scripting | ❌ Manual only | ⚠️ Manual + some FX tools |
| Onion Skin + Layers | ✅ Excellent | ✅ Excellent | ✅ Standard |
| Audio Sync / Timing | ✅ Full NLE / keyframe sync | ⚠️ Basic | ✅ Traditional X-sheet |
| Rotoscoping Ease | ✅ Very good | ✅ Excellent | ⚠️ A bit clunky |
| Output Quality | ✅ Full pipeline / render engine | ⚠️ Basic | ✅ Broadcast-ready |
| Learning Curve | ⚠️ High | ✅ Medium | ⚠️ Steep + Weird UI |
Draw in Krita or OpenToonz
Switch to Blender when
Back-and-forth is OK
Let me know which tool you want to begin sketching in—once I see even 2–3 frames with your lips and scribble patterns, I can start planning how to morph them smoothly in between.
M: As a note, in terms of production, unless I explicitly say so the plan is to try and make this purely black and white, and only worry about coloring afterwards. Can you create a quick drawing of the Rocky horror lips in the style I described? Like, take the lips, give a cartoonish rotoscoping, then shade it with geometric lines and circles.
G: Sure — but to get the style right, I need to generate it based on your input. Since you want:
I'll now generate an image based on that exact description. One moment.
G: [Empty message]
M: The song will be aqua, “cartoon heroes“. To start building a timeline, the lips will only be used from the opening of the song through the first time The singer says “what we do is what you just can’t do” (or whatever the line is specifically at that point). since it will be rotoscope, I hope that the lips will be a little bit more exact, but you have the ink lines correct in terms of bolt style. But your version of the geometric shapes is completely off. I’ll do a drawing to demonstrate, but that will have to be later – I’m just seeing if I can describe it to you right now. What I’m talking about is using patterns smaller than that, more like crosshatching than what you’ve going on there; it flows in the areas I want darkened, sometimes contiguous and sometimes not. Now that I’m trying to describe it I realize exactly how… I don’t want to say it’s complex, but the rubric is very flexible so any repetition usually ends up being on purpose. Very simple shapes and rules, but with a high variability and how those rules are applied.
G: That makes perfect sense—and your description is very clear, even without the visual yet. Here's how I’m interpreting your intent and how we can structure your timeline and pattern style logic:
What you described sounds like this:
| Feature | Your Version | Previous Version (AI-generated) |
|---|---|---|
| Scale | Very small, like ink-level shading | Large, bold decorative elements |
| Style | Flexible rule-driven crosshatch (lines, dots, arcs), intentionally variable | Repeating circles, grids, triangles |
| Flow | Local to the lip shape — follows muscle structure and shadow zones | Global, fills entire region evenly |
| Intentional Irregularity | Yes — repetition implies emphasis | No — repetition was generic |
| Vibe | Pen-doodled shading with internal rhythm, like contour shading mixed with abstraction | Tattoo-like fill |
You’re basically using hand-drawn contour shading rules, but stylized into abstract geometry. Think:
Whenever you're ready, I’ll help build from your actual frame logic — but we’re already on a strong foundation.
M: [Non-text content: image_asset_pointer; content_type=image_asset_pointer; size_bytes=472133]
Here - this is an example of the type of scribbling I’m talking about. It wouldn’t be omnipresent in the lips – it would be there to giveit effect, to shape the lips and raise interest
G: Perfect. This is exactly the missing piece.
Your drawing shows that the geometric shading is:
In your animated lips, this style will:
Here’s how we can approach this in animation logic:
You define areas for detail (e.g. lower lip shadow, edge curls)
Draw the geometric fill in Frame A and Frame B — even rough
I’ll write a custom tween/interpolation logic:
Final output: black lips with embedded, animated geometric shading
| Task | Tool | Notes |
|---|---|---|
| Rotoscope lips | Krita or OpenToonz | Clean B&W frames |
| Draw patterns | Krita | Use your drawing style directly |
| Morph help | Blender + Python or Custom script | I can help animate the texture transitions between your drawings |
| Final composite | Blender or ffmpeg | Overlay geometry onto lips cleanly |
Just let me know when you’ve got 2 pattern passes ready on a rotoscoped frame. Even if they’re rough, I can build the logic to transition between them. You're nailing the style—this'll be stunning.
M: OK. Although I will argue with one thing you said – it is largely algorithmically generated. My usual method involves Kriaten from the outer edge of whatever I’m filling, creating a single layer, and applying these fluid rules. I’m arguing because I think it might make a difference in your application – the rule is there, but it’s almost like there’s two separate sets of rules: one says where to draw, and the second says what to draw in the area the first pattern described.
G: That clarification absolutely makes a difference—and thank you for making it.
What you’re describing now is a two-pass logic system that’s rule-driven, but executed by hand. That gives me a way better model for interpolation and assistance.
Think of it as defining the “field” to be seeded with marks.
These marks have rules (spacing, repetition), but the application is purposefully not uniform—like generative cellular automata with a human touch.
Here’s how I can now assist more precisely:
| Task | Action |
|---|---|
| Between two keyframes | I’ll interpolate both fill areas and internal patterns, treating them as independent layers |
| Fill zone morphing | Use vector field or mesh deformation logic to reshape zones smoothly |
| Pattern morphing | Match primitives across frames and simulate: |
You're not free-forming chaos. You're executing an algorithm by hand—like a procedural generator you happen to control with a pen. That means:
When you have your first two frames built (pattern zones + fills), send them in whatever form is easiest. I’ll break them down and begin building the transition logic.