Exploring Descript's Agentic Editing

Why Descript’s Agentic Editing is Amazing

I can see the vision for this tool, and it is remarkable.

Here’s Andrew Mason explaining what it is all about if you’ve not yet heard about this:

Here’s the short version. Agentic editing using AI to do the busy work of editing so you can sit in the director’s chair and tell the AI how to edit your video or audio.

Automating the busywork of podcast editing is awesome. It feels great to be in the “director’s chair” instead of the editing chair.

This new Agentic Editing on Descript was announced this week, and I’m stoked I got to dig into already!

Keep in mind, I’m playing with this on the first week out! It’s still very beta and I get that.

Let me tell you what it rocks at, from the very beginning:

  • It did multicam for me just by me telling it to
  • It got titles, screens and everything set up correctly on the first try.
  • It used a layout pack that Descript has automatically loaded, and it looks great.
  • It made transitions that look nice and pretty.
  • It knows itself.

There were a few times where the agent didn’t do what I asked it to, and it knew it didn’t, and it gave me clear steps of what to do next to do it manually.

It even pulled my guest’s name and title from the transcript and added it to her introduction screen.

shenita_title.png

I learned some things that I didn’t know before, which is always good.

Biggest Takeaway for this First Version:

Know what you want to do before you start because it’s not going to be great at changing what it already started.

Where Descript’s Agent is Still Growing:

It’s not yet great at additional directions.

Multiple times I got a “There was a technical issue. Would you like me to give you step-by-step directions to do that manually?” as I mentioned above, but the fact that it knew it had failed and told me how to do it myself was really great.

Also, there were a couple issues of hallucination, but in this case, it didn’t make something up, it said that things were done that weren’t.

There are a few things it doesn’t do, yet, and I told it I thought it should. It responded with:

Currently, my direct editing capabilities are limited to text, scenes, layouts, and some visual properties, but not to advanced video color correction (like exposure, contrast, or temperature) on individual video layers. That’s why I provided the step-by-step instructions for you to apply those adjustments in Descript’s interface.

I was going to paste the whole conversation below, but it looks like there’s some info in there that probably shouldn’t be exposed. Also, it got unreasonably long! 13,000 words! So, I’ll just paste the first little bit below:



## User: OK, first, add markers to this for each question that speaker Jethro asks. Then add automatic multicam so the speaker is the focus. You can add smooth transition, and titles with the markers you created that will stay on the screen for a few seconds
Message id: `b70dec92-9c5e-4d24-9b1b-41f6a90b01ba`

Internal context
{"sceneId":"da4c574d-8881-4830-a46a-8994eae4881e","textSelection":{"loc":2,"delta":0},"playheadTime":0}


## Assistant (9.0s)
Message id: `d5b326d0-8aef-48bd-973c-bc593d664e51`
> Here’s my plan to accomplish your request:
> 
> > 1. Identify all questions asked by the speaker Jethro in the script.
> > 2. Add chapter markers at each of Jethro’s questions.
> > 3. Apply automatic multicam so the active speaker is the focus throughout.
> > 4. Add smooth transitions between scenes.
> > 5. Add titles at each marker, using the marker text, and have them display for a few seconds.
> 
> Let me start by:
> - Locating all of Jethro’s questions and adding markers.
> - Then I’ll proceed with multicam, transitions, and titles.
> 
> Let’s get started by finding all of Jethro’s questions and adding markers.

### Tool Calls

Tool Call #1: `query_script_and_scenes`
id: `call_rjxwJMoP49sY4omMf63by6LY`
```json
{
  "include_timestamps": false
}

There are a bunch of these Tool Calls all throughout. If it doesn’t know what to do, it searches help and gives an answer based on that.

It also provides links to help pages to make it easier to do what I need to.

Now, this podcast isn’t completely edited, and certainly it took me longer to write this up than it did to make the edits here.

Questions Remain!

  • Does the length of the video matter in how it makes the edits and content? i.e. token limits?
  • What’s the plan for allowing more advanced features that don’t work just yet?
  • What are the limitations currently?

Overall

I’ve got to say, this is incredible. With complex software, sometimes you just don’t know where the control is for a specific thing you are trying to do. This is a challenge because there is the mechanistic skill of learning the tool itself, where the buttons and functions are, and the organic skill knowing when to make certain decisions. I want to spend my time on the organic skills, not the mechanistic skills.

That is what Descript’s Agentic editing will allow. I see the vision of the future and I can’t wait to continue testing and playing with this.

Here’s the video I edited briefly, in it’s unfinished state.

Everything AI

Notes mentioning this note


Here are all the notes in this site, along with their links, conveniently visualized as a graph.

1 tip i would give an educator to start using ai...Paperless Principal BookSimplicityCommunication CardsHome VisitsSchoolx is here. get your copy todayHow we told families school doesn't matter and...NowAPWorkshopPresentations and SpeakingOnboardingSetting Healthy Tech Boundaries for KidsAn 84% Pay Cut? Would You Take it?ToolsOrganic Skills are the New Soft SkillsOn Lasting ForeverSites I want to model this site afterBest Year EverInstructional Coaches MastermindCUES 2014District MastermindsFamily GoalsFETC22FOCUSIdeal WeekImpactMastermindNational Principal's ConferenceObserveOnline Observation FormPhotographersPlanning PeriodASCD PlayRewire19Scales and Goals 2014SchoolXSeminarSponsorshipsSurveyTraumaUCET 2014Loaves and Fishes, not the 5,000SchoolX Book Study - Transform your School with...SchoolX Book Study - Transform your School with...VERDADSonderSave the webNot all screen time is created equalTrying to Export Substack2 Keys to Building CommunityOSPI Graduation Equity Webinar3 Rules for Academic Writing with AI3 things i'm interested in right now5 best rti episodes on transformative principal5 ways our edu system would be different if...50 Questions to Ask to Solve Problems5rs5sigma20A decade never to be forgottenNobody cares about our life as much as we doWhat I've learned from doing 6000 burpees in 40...A summary of my ship30 writingsAdlerian psychology in schoolsHow We Use AI In SchoolsSchedules are a big puzzle, and I like puzzlesAccelerated learningA less annoying cookie bannerAdd Fathom Analytics to your Obsidian Publish siteA Live Demo Site for a SaaS productAdntbf roadmapAdding a Little Complexity to Make Things More...AI As a Fundamental ExpectationAI Can do Anything, But Not Something SpecificAI Easy and HardAI Produces and Your Brain DevelopsAi leader office hours june 2025Ai for innovationAdvanced teacher's prompts (54)All parents want their kids to be successfulExploring Descript's Agentic EditingAlaska administrative assistantsAn overview of prompt engineering for educatorsAnyone interested in fundraising should listen to...Argentinian Food - Fast Sunday Family Feast for...Big impact mixerBook fair controversy at Prairie Hill sparks...Three reasons you should have a podcast to go with...Based on what you know about meBIM January 25 - Breakout Problem SolvingBe a creator, not a consumerBIM February 2025 - Experiments[Big Impact Mixer] - Are My Social Posts Any Good?BIM May - What are your Gifts?Apprenticeships for the 21st CenturyAssessment (58)Assessment and evaluation (50)[Big Impact Mixer] - Vision, Category Design, 10...[Big Impact Mixer] - MiraclesAssumptionsBim april creating a personalized newsletterBrooks duncan on handwritten notesBuilding classroom community (50)Building parent and community partnerships (50)Compass Among usJanuary - MolcajeteFast Sunday Family Feast - A Jones Family...February - Borscht and PelmeniHow i got my wife to stop nagging me[Big Impact Mixer] - Experiments FollowupFrameworksEdit an episode with me (full podcast production...Don't call it a Substack. - Anil DashHoliness to the Lord in Everyday LifeFit looks like gritFollowing up on my post ai counselor botsDelusional dreams of excess freedomJethro's Appearances and Media ElsewhereIrish Food - March 2025Why the Ebenezer is so Important in Come Thou...Guatemalan Food for April 2025Building an App I will use DailyEducatorsCan we trust the ai?Candy teacher referralCaroline classroom observationTransforming Old Content into Engaging Case Study...FSFF May 2025 - Korean FoodFSFF June 2025 - Good Ol' American FoodWhat certifications are necessary?Chatgpt prompts for educatorsClassroom creativity(55)Classroom differentiation (50)Classroom discipline (50)Classroom environment (50)Classroom gamification (50)Classroom management (52)Classroom technology integration (50)ClippingsThere’s no such thing as learning loss or learning...Collaboration (54)Convertkit essential workflowsCulturally responsive teaching (50)Dan koe's writer's bootcampData dashboardDay 1 ship30Day 17 the 1 app i can’t live without for podcast...December - Brazilian Feijoada“Your Decision, My Decision, Our Decision”: A...Differentiated instruction (100)Do People Call You for Advice?Effective feedback (54)Election Day, but no TrustEthan Mollick on Knowing When to Use AIEverything AIExplicit teaching (54)How i got started in leadershipHow to Be a Transformative Principal By Jethro...How This Site is BuiltHow I lost 70 Pounds in a Sustainable WayHow i would start my online presence todayHow I write a dissertation extremely fastHow writing helped me present betterHow to use mint mobile's minternational passIf I were a ProfessorShould I use AI for to Install Omarchy on a...Join the Compassionate Classroom Global Card...Just subscribe to these youtube channelsI was quoted in a story from Parents MagazineLeadership vs managementLearning is like a springLearning is a lifelong processLesson planning (50)NewslettersLong sentencesMiraclesMove from blogtalkradio to transistorMaking “Social” Social AgainListen LaterLucky MeatDoes Magic Mind really give you clean and calm...New Disease: NovolibrochitatNotes to staciObsidianOur System is Designed to Enable CheatingOctober Fast Sunday Family Feast - Australian FoodOn not using turnitinPaperless principal submittedPaperless principalParent Teacher communication (50)PatiencePedagogical approaches (50)Pedagogy and instructional strategies (100)Podcast gearPolson middle schoolPresentations on aiPraisePresentation resourcesProblem in edu is that we don’t have outcome Based...Professional growth and development (50)Project ManagementPromoting ethical decision Making and responsible...Recipes on the Web are the WORST!Read ai summary from the vaesp lunch and learnRssAsk QuestionsQuoting Cory DoctorowReflections on spell Jif retreatEthan Mollick on Real AI Agents and Real WorkReport on an Analog/Social Media Free MonthCurriculumHomeStudents Who ThriveEnrollmentHow it worksResources for the SPELL JIF RetreatSstSessions Health - Mental Health Practice...Self EsteemShould i learn to codeStop using generative AI as a search engineSponsored Podcast EpisodeWhen the Stone turns to BreadSplitting Rodecaster Pro WAV filesStrivingStudent engagement and motivation (100)Student Centered learning (50)Students Need Human Relationships to Thrive. Why...Summer pinedaSupporting students transition and college and...This Is Mind Blowing.SynergyThe Power of No ExpectationsTailgate - February 2025The leaders who have influenced me mostThe lord keeps things hiddenThe man who saved the world with human intuition,...The pass method how i've taught ai to be an...The power of personality tests in the hiring...The best book i've read on business financeThe best piece of leadership advice i ever...TherapyThree simple tips for recording yourself more...To Radiate Life A Prayer by Elbert HubbardTop 3 podcasts in 2019Knowing how to use Transcripts in Podcasts in 2025Turning learning upside downMy Thoughts on LearningI made some big updates to the site todayUse of data to inform practice (50)Using Vectorize to build an unreasonably good...VAESP Presentation ResourcesVal meditation reminder appProject iSTAR: An Integrated Strategic,...What is Yours? And What Will You Keep After You...Why Do People Like 37signals?Why everyone should read the courage to be...Why do AI company logos look like buttholes?Why young people worldwide want and need...You Can Just LearnYou choose to not changeYou made up your angerAplusPresentation Notes at ALASBOPresentation Notes at Alaska Municipal LeagueSpiritual ArchitectureBe Podcast NetworkCommunication CardsEliminate Stress and Improve Relationships in Just...Best AI ToolsBUHSD Day of LearningBishop Brady High School AI PresentationBlogBooksDesigners for EducationYou Should be Able to Take Your Data With YouHard thingsTeachers Don't Actually Want Feedback. Here's What...ExperimentsHigh Rep WorkoutsFacebook adsHow to get unstuck when you're stuckHow Many Teachers Do I Need To Hire For Social...The Ultimate Inspirational Videos for Teachers and...HumilityiA Presenter Now Allows Sharing on the Webmedia kitLifestyle is your choiceThe Little, Narrow Prison of Language: The...Morning RoutinePaper trackingPbis As MedicineAll Learning Is SpiritualPeople Ruin the InternetPersonal board of advisorsPodcast AppearancesProjectsPsychological SafetyReopening Schools WorkshopResultsSdlHow to Combat Chronic AbsenteeismHow we teach our kids does not happen how we thinkSecret newsletterSubscribeSure Go AheadTake Your Daughter To Work DayThanksThe KindleTrauma Podcast MapUcet2013Your Values are Only Worth What You’re Willing to...UCET 2023Supporting Students with Webs of Support - Podcast...Where To Startand An Action ItemWhy I Do What I DoWhy Most Reopening Plans Are Doomed To FailWorld Down Syndrome Awareness DayYoutube For Schools“Put Your Shoulder to the Wheel”
Follow
Follow
Follow