Speaker: Avindra Fernando

See live blog table of contents for more posts

---

Survey

• Most have ai agent writing code
• Most jave markfown file agent reads
• About half have written a kskill
• A handful have a playbook/automated workflow for skill
• A few have tested whether skill triggers

Reusable intelligence in repo

• agents.md, claude.md

Skills

• Prompt tells an agent what to do once
• A skill tells it how to work here, every time
• Create skill when see action being repeated over and over

Flow

• Context
• Skills
• Automate skills, make playbooks
• Prove did work
• Maintain codebases

When people say “prompt file” it could mean

• Context file (always gets loaded). Not helpful when file gets too long. Overwhelms the agent
• Skill – loaded on demand
• Subagent – roles – ex: planner, tester, reviewer
• Command – playbook combining context/skill/agent
• Hook – systematic gates so can restrict access

Learnings

• Agent will figure out the how. The why is important
• Want agent to stick with rules
• Train like a new human teammate, would give them the why
• Specify hooks – things absolutely don’t want it to do

Open Skills

• Standard format
• Name, description, body

Other notes

• Grill me – takes a vague idea and interviews you to refine it. need to know when to end
• Found a place it didn’t trigger because nothing run

Agent types

Can choose whether to check in with human after each step.

• Planner agent – turns story into a plan. Does not do any coding. Good place to use latest models.
• Implementer/coder agent – takes plan and implements it. No creativity. If problem with plan, that’s on you. Can use less expensive agent.
• Tester agent – run tests
• Reviewer agent – code review. Do not fix; just identify issues.
• Also 3rd party agents can reuse

Playbooks

• Agents for planning, implementing, testing and reviewing
• See what generating – give format want plan output in

Failures

• Capture why
• Refine skill, figure out if wrong skill files, etc
• If agent makes mistake, add to agent’s md file so doesn’t make same mistake
• Surprising changes in cost

Future of reviews

• Different tools, not looking at code
• New people will be used to a higher level of abstraction.

War stories

• The undocumented deployment – wiped out config. Lesson is to create the instructions when inherit an undocumented codbase
• The queue that woke up – mail queue had dormant key for years. When restarted, all the emails went out, lucky not on prod.
• When the reviewer goes wrong – memoized when not right answer and caused infinite re-render

Loop

• Prompt engineering optimizes a message
• Loop engineering optimizes the pipeline

My take

Avidra has been linking tweets about my live blog so cool he gets a post in it! I missed a few bits of this as I had a moment of panic (I booked the wrong flight home and couldn’t put it out of my mind to fix it later). Overall very good. I learned a bunch and am inspired to make skills/playbooks. Particularly enjoyed the demo.

Speaker: Alexandria Hamilton

See live blog table of contents for more posts

---

Context

• One person company
• 32 agents
• Bottlenecks with human in the loop. Repeated a number of times that human is the bottleneck. Balance control and speed
• If fail, because of human
• “we” – agents part of team

Agents

• Each agent has own purpose
• Help with code, calendar, marketing, come up with lyrics, governance, cataloging, checking news
• Agents doing well so getting promoted to cloud and not human in the loop
• Org chart of agents; have leads
• Each agent has a specific thing to do/purpose
• Skills – ex: firecall, opensco, buffer

When don’t need agents

• Agents are expensive
• guardrail: 30 day waiting period before add another agent

Names

• Some people give them human names and treat as human teammates
• But want to be able to destroy when don’t need so don’t get too attached

Log

• Keep log of all workflow
• Log schedule, handoff, interactions, failures
• Stored in Obsidian
• “If it isn’t written down, the AI is improvising it”
• Helpful for finding old conversations/backup of human memory

Judge

• The LLM is never the judge
• Background agents help as judge

Claude

• Filter ideas with Claude
• Refine ideas
• Package and give to agents
• Using Max subscription. Want to space out token use

Consistency

• “Consistency is a byproduct of structure, not of intent, memory, or discipline”
• Slow and steady wins the race
• Ok if things break. Don’t want agents out in the wild doing whatever
• Ok with human being the bottleneck
• Some days failures because human not available
• Pimon looks for anomalies

Operating Loop

• Each agent runs certain days/times on a schedule (try to take Sunday off)
• Want to keep token usage low. Spreading out which days each runs helps
• Some talk to each other
• Give each specific things to do

Agent HQ

• Includes file describing each agent – persona, what does, example invocations
• Calendar showing when runs
• Output of Claude code

Health care

• Marketers need to show accurate
• Still need to go over data

Relationship with AI

• Not replacing us, argue with it every day.
• Partner, not intern
• Ask Claude how to make improvements: ex to agent schedule

Takeways

• Agents are good
• Human is the bottleneck. This is fine. See firsthand everything that went wrong

My take

The session I attended in this room also started 8 minutes late due to video issues. Not the speaker’s fault, but would be nice if someone announced that working on it. Her speaking style is great; high energy and clear which I really needed this morning! I feel inspired. Showing Agent HQ at the end was great!