Tag Archives: ci/cd

[kcdc 2026] would you trust an intern with production

Posted on by
Reply

Speaker: Bob Walker (_BobJWalker_)

See live blog table of contents for more posts

Relearning the same lesson

  • AI horror stores – clickbait/sensationalism. But outages real
  • AWS – agent decided to rebuild prod
  • AI ant deletes company’s database
  • Not same exact mistake – rhyme with each other
  • Sharing Cook county voter database
  • Problem: People making direct changes to production

Non deterministic users

  • AI and People are both non deterministic users.
  • Both do things that are so baffling that you can’t explain it..
  • Why did you do that?
  • I decided it was the best approach I could take.
  • Both can make same mistakes and burn down production

Why AI is like an intern

  • Book smart
  • Doesn’t understand consequences of production
  • Like tellling an intern not to speed

AI philosophy

  • Terminator: when told not to kill someone, shot in knee instead
  • AI is an amplifier. If you don’t care about something, it won’t and vice versa.

AI Usage types

  • Cyrborgs – AI is pairing partner
  • Centaurs – AI is research assistant
  • Self automators – AI is an Oracle

Causing Prod Problems

  • Did show of hands for who caused a prod incident. Almost everyone. Commented if didn’t raise hand, lying or hasn’t happened yet.
  • “Recently, I was asked if I was going to fire an employee who made a mistake that cost the company $600,000. No, I replied, I just spent $600,000 training him. Why would I want somebody to hire his experience?” – IBM

Humans

  • “Humans will be involved every step of the way”. But humans caused problems too
  • AI PRs can be too big for a human to review.
  • Spent time trying to automate to get people out of the deploy to prod workflow. Solution isn’t to go back.
  • OSS – more/bigger PRs

AI is getting better

  • It is, but not at same rate as it did at beginning
  • Models now tuned to benchmarks. AI companies helped right benchmarks
  • When same problems happen over and over again, does good job. When something novel, tends to fall apart

Where AI fits in software delivery (LLMs and AI agents)

  • Want deterministic process (ex: build) between non deterministic users and production
  • Includes CI/CD pipelines, automated runbooks for common production incidents, creating/deploying infrastructure, logging/alerting
  • AI and people contribute to the pipeline – ex: writing, approving part

Permissions

  • AI should have different permissions than people
  • ex: branch protection

Process Changes

  • The process doesn’t change every day.
  • Monthly at most

AI capabiilities

  • AI can invoke deterministic processes via MCP/CLI/API via token
  • Automate common decisions so non deterministic users aren’t making them.

Sliding scales of trust

  • “I don’t trust my developers” – shouldn’t trust AI. ex: spotify. Inconvenience if don’t but not a material impact on lives
  • “I fully trust my developers” – should trust AI. Have enough automation and guardrails in place to prevent making the wrong choice. Often not working for bank/insurance companies.
  • Every company is different and changes over time. Also depends on company politics and policies

Takeways

  • Non deterministic users shouldn’t make production changes
  • More human involvement isn’t a panacea
  • Automate deterministic decision making for non-deterministic users
  • Non deterministic users use and contirbute to deterministic processes

Action Items

  • Create AI skills or context boundaries as first line of defense. This is like a chain link fence to protect Fort Knox
  • Table top work case scenarios. – ex: maliciuos insider
  • Establish clear boundaries, create deterministic processes/runbooks, create deterministic approvals for common decisions, build contributions guidelines for AI or people
  • AI on behalf of a person uses different API keys/tokens (and different permissions – ex: can deploy but not make changes)

My take

Bob said a few times that purposely vague/vendor agnostic. This session was a great start to the morning. AI course but focused on a perspective. I really like the idea of humans and AI being non-deterministic users as a commonality. I also like trust being situational. Also excellent summary!