[2020 devnexus] open development when you’re not in charge

Bob Paulin @bobpaulin

For more, see table of contents

Open Development

  • Code, processes of contribution available to whole organization.
  • Each project might have some extra governance
  • Developers can see, contribute and work with any code.


  • Communities not product teams
  • Communication is transparent
  • Code/doc/issues accessible to everyone
  • Code not usually externally available (outside company)

Apache Way

  • Community over code – can’t do something that helps one person, but is bad for the community
  • Consensus decision making
  • Open communications – message lists for all projects. “If it’s not on the list, it didn’t happen”
  • Earned authority – earned/voted on
  • Community of pears
  • Responsible oversight

More on Apache

  • Hard to kill an Apache project. Struts is still cutting releases. People are still maintaining it because important to someone.
  • When community goes away, project goes to attic

Open Development at work

  • Consultants – there to advise
  • “You can make buffalo go anywhere, just so long as they want to go there”. “You can keep buffalo out of anywhere, just so long as they don’t want to go there” – from “Secrets of Consulting” – recommends book for getting things gone
  • People will go to great lengths to avoid pain. They need to understand how open development reduces their pain


  • Tribal knowledge – once a month man comes down from the mountain and explains how things work to people
    • localized knowledge
    • can go with benevolent dictator on project
    • passed down mouth to mouth
    • only know if have relationship or proximity to person
    • document for different zip codes – needs to be clear to everyone. only use acronyms can google
    • document for dummies – should be clear to entry level developers
    • curse of knowledge – you’ve forgotten what is hard
  • Man behind the mirror decision making – architect went to talk to everyone. All felt good after meeting. Months later, realized things weren’t working together. Contradictions, misunderstandings
    • Set decision making to INFO – anyone can read decisions
    • People get upset/feel excluded when planning happens via direct message
    • People make decisions in meetings without realizing it was a decision
    • Appoint a decision crier – someone raise hands and says sounds like a decision and notes should write it down. Also helpful to have a secretary who should write things down.
  • Micro manager – can’t expect others to step up if still doing everything
    • Encouraged measured risk taking – let people take risks so can feel comfortable. Ex: can commit, but say something. Still don’t want it to go to production. Focus on learning over veto
    • Self driving car approach – still need to pay attention to wheel. Find bugs by finding mistakes people are continually making
    • Build reputation of new leaders – others start become accountable and show they can do it
    • Have to let make mistakes. Ok to commit something not perfect.
  • Bike shed to oblivion – people get passionate when arguing about something important (to them). What color should the bike shed be.
    • Enabling constructive dissent – community, consensus, earned authority
    • Elephant rules – acknowledge the undercurrents. What is actual reason for dissent. Knowing whether comes from gut feeling. Acknowledge what is problem and what is relevant
    • Build failure hedge fund – sometimes things go wrong. Determine if can learn from them or will ruin company? If the later, veto. Normal to fail from time to time

My take

Good end of the day. Some “obvious” and some to experiment with. Nice inspirational talk. I like the mapping of “general” principles to Apache principles.

[2020 devnexus] Developers Need to Take Notice – Malicious Attacks On Open Source Are Getting Worse

Speaker: Derek Weeks @weekstweets

Deck: https://speakerdeck.com/derekeweeks/devnexus-presentation-derek-weeks

For more, see table of contents


  • XKCD – reinvent the wheel – “We don’t want to reinvent the wheel, so every day we google image search “wheel” and whatever object comes up, that’s what we attach to our vehicles. Sure external dependencies carry risks, but so far, they’ve all been pretty good wheels”


  • If just moving fast, have a problem
  • DevOps teams use more open source
  • 70% deploy at least once a week
  • Challenge: be faster than evil
  • In past 5 years, breaches increased 70%
  • Can’t predict when vulnerability will come up. Have to use without knowing what will happen.


  • Equifax is old news by now. Had opportunity to patch
  • Adversaries can also contribute to open source. ex: npm event-stream attack on CoPay
  • 2019 – Gems bootstrap-saas – added backdoor
  • Typo squatting
  • Backdoors
  • Happening to Docker, Python, Ruby, NPM, etc

Supply Chain

  • 2019 Software Supply Chain Report
    • suppliers (open source)
    • warehouses, (component repos)
    • manufacturers (softare dev teams)
    • finished goods (software applications)
  • Maven Central had over 200B downloads in 2019 alone. Almost 10% had known vulnerabilities they day they were downloaded.
  • JavaScript hit 10B package downloads in 2019 alone. Just over 51% had known vulnerabilities they day they were downloaded.
  • 85% of app is sourced from external suppliers

Enterprise vs Open Source

  • Multiple deploys per day vs versioned releases
  • Consistent Dev team vs fluid group of developers
  • Predictable/well resources vs variable resources
  • Deploymen tvs release frequency
  • Organizational performance vs popularity
  • Mean time to restore vs time to remediate vulnerabilities

Is it true?

  • TRUE: Projects that release frequently have better outcomes – more popular, attack more developers and higher level of support from foundations. Also, avoids problem of having to wait for all the transitive dependencies to be in a version we are using.
  • TRUE: Projects that update dependencies more frequently are general more secure
  • FALSE: Projects with fewer dependencies will stay more up to date. Interestingly, smaller teams tend to have less dependencies. Not clear if correlation or causationl
  • FALSE: More popular projects will be better about staying up to date

Behavioral clusters

  • Small teams; excellent time to update
  • Large teams; excellent time to update, often foundation supported, popular
  • Laggards – release slowly, more likely to be commercially supported
  • Features first – release frequently, but poor time to update
  • Caution – good time to update, but seldom completely up to date


  • 38% schedule dependency updates
  • 46% strive to use latest version
  • 50% have process to add new dependency
  • 30% have process to proactively remove problematic or unused dependency
  • 37% have automated tool to track, managed and/or ensure policy compliance of dependencies

If only do one thing

  • If stay on latest version, by default more secure and less security issues.

My take

Good talk. Especially once the projector issues were fixed. I lik the graphs and data behind the main points. I’ve seen similar presentations, but the newer parts/stats were still good to hear.

[2020 devnexus] afternoon keynotes

For more, see table of contents

Decoding the weather – Kait Parker


  • IBM owns the weather company and app
  • IBM GRAF –  Global High-Resolution Atmospheric Forecasting System 
  • Weather is big business – lost productivity, spikes in sales, inciting new innovation, damage impact
  • Can lost half a billion dollars in a single snowstorm in Boston
  • Whereas a city like Atlanta shuts down for snow
  • Naual disasters increasing – both climate change and not building smart
  • Weather is data
  • Weather company has largest collection of weather data in the world
  • APIs for current data and forecasts
  • US had good technology for weather. Whole world does not.
  • GRAF – more precise, faster refresh, uses 400 GPUs on 90 IBM servers
  • Future: add sensors on planes
  • #CallForCode – international challenge for developers to mitigate impact of natural disasters
  • Project Owl won in 2018. For addressing communication when cell phone towers go down. Creates mesh network based on wifi to gather info of who needs help. Smallest nodes are duck links. They are linked to mama duck. A drone connects to papa duck which connects to first responders.
  • Team Promoteo won in 2019. Monitors first responders. Wear a device. Tracks current conditions and trends.
  • 2020 competition starts February 26

Dehuman Future – Ixchel Ruis

  • EM-DATE – international disaster database

(I had to leave before this one ended as I’m on the podcast the starts after the keynotes. Too bad. Ixchel is a great speaker)