Speaker: Fernando Cargnelutti

See live blog table of contents for more posts

---

“Faster“

• “AI will make us 4x faster” – claim comes from people selling the tools. Independent measurement shows gap between that and reality
• Moved QA engineers to separate team and less of them
• Also removed Solution Architects, they now do development
• No more ScrumMasters. Do Kanban but without work in progress
• Fired 1 developer per team

Quotes between teammates

• “I wouldn’t create a PR for such a change”
• “That PR is 3 weeks old; bad for metrics”
• “We don’t merge MRs just because they are interesting”
• “I merge changes when results in better quality not because of metrics”
• “I don’t think this needs more discussion; it’s already been merge”

Laws of system

• A building architect gets gravity for free.
• In software, we write the gravity.
• We craft the laws of the system

Legacy

• Program is the theory held by the team
• The code is a byproduct
• Legacy = code whose theory died even if no line of code has changed
• Process of writing code develops knowledge of what doing
• If tests inherit the wrong theory, you have local coherence but global failure

Bugs

• Most expensive bug is the one in the requirements
• Now we write the requirements

Vs Compilers

• 99% of our code is written by AI but 100% of their code is written by compilers
• A compiler is physics; A LLM is a guess
• Compilers automate below the theory line; AI writes where the theory lives
• Source code is a design document
• Compiled code is disposable. Can generate it again.
• Prompt isn’t always persisted. But incomplete even if it is saved because non-deterministic translator
• Commit still attributed to you.

Code gen

• In past, we used UML to generate the plumbing
• Had to fill in middle
• Didn’t work that well; abandoned

Writing GitHub automation script

• Created code, documentation, repos
• To do by hand, read doc, understand API/auth, consider a lot of tradeoffs
• If AI writes code, you don’t feel the tradeoffs
• Struggle manufacturers understanding
• Seniors doubt AI output because they walked the path. They know what costs. Juniors accept fast; they never traveled it.

Mitre 2025 study

• Did study with using chat (and pasting in code)
• Predicted 20% faster than non-AI group
• Said 20% faster
• When measured were 19% slower

Repeated study in 2026

• Now speedup of 18% with same developers
• New developers had speedup of 4%
• Realized can’t trust numbers because nobody wanted to write code without AI.
• All agree write code faster with AI. 18% is not 4x
• Now depending on a tool that takes away judgment

Skill formation

• On quiz of material on tasks just finished. 67% with handcoding, 50% with AI
• Predict bigger difference when measure in agentic environment.

Experiment

• Randomly took 3 developers and an old PR with almost new AI and a new one with more than 60% AI
• Predict won’t be able to explain new PR as well as old one
• Found remember the 8 month old PR, can explain it better, remember file structure, trust it more.
• Walked through code at a glance for handwritten code. Minutes of reading to explain 40% of the new AI PR
• Trust 6/10 on PR that couldn’t explain half of.
• One developer remembered the 3 week PR way better. The structure was biult by AI. She dealt with a bug and spent time on it. She did not remember the code structure as AI did that part.

Other stats

• 81% more duplication
• Went from 22% to 3.8% on refactoring
• 441% incrase in PR review time
• 243% incrase incidents per PR
• 31% PRs merged with zero review
• Time saved in creation was re-allocated to auditing.
• “We’re accumulating code faster than we are accumulating trust”

Urgency and AI

• 4x belief
• roles/buffers removed
• urgency lands on devs
• AI becomes only way to keep face
• struggle removed
• understanding evaporates
• defects and report create more urgency

Future analysis

• Look at feasibility of a feature
• Engineer asked AI vs understanding
• Does it affect design? What could go wrong? How long will it take?

Distance

• False belief of knowing scales with delegation.
• Dev who worked on
• Dev who merged it
• Dev who asked to analyze
• Management

What it costs

• AI answering human conversations
• Production incident
• Can’t explain code form 3 weeks ago
• Having AI analyze instead of humans

What can do

• Choose friction where appropriate
• One minute rule – if can’t explain in under a minute and defend why approach is acceptable, don’t merge
• Architecture audits – review the theory not the syntax
• Manual design first – write the conceptual how by hand
• Real WIP limits
• Harness is team infrastructure. version/review/maintain like code
• “Frictionless teams produced understanding free teams”

My take

This session started 8 minutes late due to issues with the projector. Nobody said anything though. Would have been nice to either give an intro or at least say waiting. I did appreciate a few references to confirming we would be on time to lunch. The content itself was good. I like the perspective and it had good depth. I really liked the difference based on experiences in understanding. And the stats.