[2021 kcdc] Enterprise Cloud Patterns for Java Devs

This post is my live blog from KCDC. For more, see the 2021 KCDC live blog TOC

Speaker: Pratik Patel (Azul)

Twitter @prpatel

————————-

Philosophy

  • Monoliths are not elastic. Cloud is
  • Doesn’t waste money
  • Must design for elasticity
  • We learn to avoid premature optimization/keep it simple stupid. In the cloud, we have to do some optimization
  • Need to design for horizontal scaling (adding more servers/virtual servers)
  • Need entire architecture to be elastic. Else have created a bottleneck
  • Think about cosT, adapabiity/agil/security, time to market

Cloud native review

  • Containerized
  • Dynamically orchestrated (actively schedule containers to optimize resource utiilization)
  • Microservices oriented (try to avoid dependencies between microservices – not possible to avoid, but minimize)

Types of Patterns

  • Design and implementation – covered today
  • Performance and scalabity
  • Data management
  • Availability and reliabliliy

Ambassador Pattern

  • For transitioning legacy apps – could be monolith or SOAP or Java EE or Spring Boot or whatever to microservices
  • The Ambassador is a proxy. It adds monitoring, retry and a protocol adapter (hide SOAP/RMI from clients) and delegates to the legacy app.
  • This lets the callers all the Ambassador and get these cloud capabilities.
  • Implemented as a sidecar (does extra stuff)
  • Use Spring RestTemplate (now called WebClient) or Microprofile Rest client 2.0
  • For end user/callers

Anti-corruption pattern

  • Also for transitioning legacy apps
  • Callers is another service (microservice, cloud function, event driven api)
  • Does more than ambassador.
  • Facade/adapter pattern
  • Firewalls legacy apps
  • Useful for migrating large glegacy app
  • Protocol adapers
  • Legacy unchangeable APIs
  • Compartmentalize legacy app to protect requests coming in and rest of system in case something gos wrong
  • Spring Reg Template or Microprofile Rest client 2.0
  • Must define specific endpoints

Gateway aggregation pattern

  • For when app needs to call multiple services (ex: auth, media)
  • Calls multiple services and returns the results of multiple services to caller in one shot
  • Groups related services into one API call
  • Avoids death star anti-pattern when everything cals everything else
  • Client calls gateway not services directly
  • MUst think about how to handle failures in downstream services
  • Use SprIng AsyncRestTemplate (or web client) or GraphQL (Sprin, Microproile or SmallRye) – note that graphql introduces dependencies. Good to return a bunch of related data.

Back end for front end (BFF)

  • For flexibiity and performance
  • Adapter by client type (ex: mobile, web mobile, web desktop)
  • Returns data in way most practical and efficient for that client
  • Calls microservice and/or legacy app
  • Can be an implementation of a gateway aggregation pattern
  • Make sure not premature optimzation
  • Data can be shaped and size for specific needs
  • Improved performance
  • Could introduce code duplication
  • Are front and back end devs on same team.
  • Sprint AsyncRet Tmplate or Microprofile CompetableFuture or RestClientBuilder
  • Can be different URLs by service type – ex: mobile url (or header sniffing in API gateway)

Gateway Routing

  • Commonly used/misused
  • Calls proper service based on URL
  • Helps minimize change to clients
  • Reducing coupling directly to services
  • Tradeoffs: latency, bottleneck
  • Spring RestController or Microprofile JAXRS. Or cloud pattforms APAI gateway

URLs

https://docs.microsoft.com/en-us/azure/architecture/patterns/

https://aws.amazon.com/architecture/

My take

This was very fast moving, but I found it understandable. I learned a lot. I liked Pratik’s hand drawings of the patterns. It was also interesting that most people were using AWS at a primarily .NET conference

[2021 kcdc] black holes and revelations

This post is my live blog from KCDC. For more, see the 2021 KCDC live blog TOC

Speaker: Sarah Harper

Twitter @sarita1119

————————-

Why work gets stuck

  • Work moves through the system until it gets stuck in a phase
  • We like to think of things as a process problem. However, often a behavioral issue (not like major HR leve problems)
  • Reasons: someone out sick, not enough people know how to do something, blame testing
  • Agile black holes!
  • Detrimenta to process and team morale
  • Not obvious what problem is
  • examples: deployment, business requirements, code review, qa
  • Biggest black hole is the backlog

Back hole theory

  • Any work status which acts as a “holding pen” for work items will eventuallly turn into a black hole
  • Once enough are blocked for same reason, all future items will becme blocked
  • It becomes acceptable for things to get blocked there
  • No work appears to be done to outside observer – unclear who working on or what it is up to

Symptoms

  • Queue busts – something wrong and swarm around it. Teams decides to focus on something like a customer service queue.
  • Eceeed WIP limits
  • Blocked column
  • Too much in “done” or “ready” columns
  • Long staging cycle time

Learned Helplessness

  • Exercise: everyone got a piece of paper with three anagrams and people raised their hand as solved
  • One one side, first two not soleable. One other side, first two easy. Then same third one.
  • Taught first group learned helplessness.
  • Trained to give up.
  • I tried X times to change this

Locus of control

  • Exercise: how much control do you have over weather, food you eat, comute, home, projects at wor, team member participation. (Scale of none to all)
  • Think about what under control
  • Don’t blame others for things can control
  • Be alert when someone says can’t/imposssible

Self handicapping

  • Don’t try so can’t fail
  • Fear of looking bad
  • ”give to X because X can do fastest” – setting team up for failure if that person can’t do it.
  • That person becomes busy and a black hole
  • ”You miss 100% o the shots you don’t take”

Somebody else’s problem and the bystander effect

  • Brain edits it out because someone ese’s problem
  • Everyone ignores
  • May or may not actually be someone else’s responsibity
  • Bystander effect – assume someone else will take charge

How spot black holes

  • If have to scroll in a kanan board, have a black hole
  • Not obvious

Visual principles

  • Excercise: think about your kanban board For each of these principles
  • Figure ground – foreground vs background. What see vs ignore. Showed optical illusion of face/vase. Usually digital boards have a white background so white cards disappear into background. Can make a different color to stand out. Also, if something has a date contraint or a high priority, add contrast.
  • Common region – when obects located in same region, we perceive them as being grouped together. If everything looks the same, can’t tell. Change blocked cards to a different color
  • Similarity – thinks that look the same are assumed to be the same.
  • Proximity – things close together appear to be more related than things spaced farther apart. This is more of an effect than similiarity
  • Continuity – Elements on a ine are considered more related. Lines follow the smoothest path.
  • Focal point – watever stands out visually will attract and hold our attention
  • Closure – we try to find recognizable patterns
  • Just noticeabe difference – hard to see Jira dots for how long in state
  • Idea: have blocked work in a different color, in a lane on top. So you can see status blocked in.
  • Avoid too much color

Game: https://getkanban.com/pages/free-version

Escaping back holes

  • Have team talk about problem
  • Visualize future state and take action
  • Use a physical board or customize digital board as much as can
  • FInd an influencer on team that will give good feedback and help team accept idea
  • Queue bust (swarm around problem)
  • Root cause everything

My take

I learned a lot at this session. I like how it was different than a lot of agile talks. I have a bunch of things to take back

[2021 kcdc] lessons learned from enterprise cloud securty programs

This post is my live blog from KCDC. For more, see the 2021 KCDC live blog TOC

Speaker:Karl Ots

Twitter @karlgots

Book: Azure Security Handbook

————————-

Cloud Security

  • Need cloud native security – specific to cloud provider using
  • ”Cloud can be as safe or unsafe as what you did before”

Shared responsiblity

  • Customer responsiblity: data, endpoints
  • Shared responsiblity: identity, application, os/middleware, network
  • Cloud provider responsiblity: physical

Security Program

  • Need to balance speed/“shadow cloud” (someone signing up for their own cloud account) with existing security requirements/eisting EA, lack of security awareness
  • Phases – cloud straegy, governance model, security, guidelines (ex: implementation guidelines and refernee architecture)
  • Cloud security framework defines architecture, policies and controls to secure cloud envrionment
  • Don’t be a generic list of controls. Just be tailored. Doesn’t make sense to apply anything. Cloud security alliance has a list of about 200 sample controls. Don’t just go thru a list of controls in Excel.
  • Terraform (or other tools) typically built by subject matter experts. Ex: database expert writes terraform module for database security
  • Certified products/platform concept – if you use products/components/tools that are pre-approved, can get through security faster. Vetted already

Identity and Access Management (IAM)

  • Integrate with existing IAM processes
  • DevSecOps CI/CD deloys using Azure AD ideneity to application resources. Other identities to actuall run (but not deploy and change things).
  • Need to be able to provide the CI/CD credentials not available/used by others
  • Create a vending machine type system so have to request things. ex: give me a X. Makes automated to request things

Detection and Monitoring

  • Need to enforce logging across landing zone and anything deployed
  • Centralize logs.
  • Ok to have temporary copy as well to focus on new info. Also some alerts verbose and ony want to monior key ones
  • Can build custom alerts, but doesn’t scale. If 10K Azure resources all with own logs, can be unmanagable.
  • Integrate with your SIEM an SOC
  • Separate resource logs (ex: who accessed X) and application logs (what developers log)

Network security

  • Cross subscription, cross region and cross cloud
  • Traffic from platform and infrastructure as a service + app level
  • Can grant access through
    • RBAC in the subnet – fast to do, but dev has to do it
    • Pre-provisioned NICs – medium, thru centralized cloud operations
    • Outside Azure/cloud – slow, thru centralized ops

Supply chain

  • New attack with pre-provisioned agent https://www.wiz.io/blog/omigod-critical-vulnerabilities-in-omi-azure

My take

I’ve learned/used AWS. The speaker is an Azure expert. He tried to make the presentation as cloud agnostic as possible. It was realy good for me to see how much is common across clould providers. It was goo to understand how things I’m doing fit into the bigger picture and something I wish we did differently