Category Archives: Conferences

[2024 dev2next] Architecture.Next: 4 trends for architecture

Posted on by
Reply

Speakers: Mark Richards (markrichardssa) & Neal Ford (neal4d)

For more see the table of contents

Architectural Nexûs

  • Need to recognize intersections (I didn’t type “intersection” repeatedly, but they used the word many times)
  • How many architectures to screw in a lightbulb? None; it’s an implementation detail
  • Implementation needs to be fault tolerant; not just architecture
  • Engineering practices need to be agile
  • Team topology makes hard to implement certain types of architectures
  • Often ignore data topologies and system architecture
  • “The enterprise” – processes, standards, frameworks from dept, division, enterprise enforcing for many reasons; usually good
  • Business environment – ex: cost cutting mode vs aggressively expanding, rate of change in business or marketplace. Software must be flexible enough to change as business changes to achieve goals. Cannot be bottleneck
  • Generative AI – can apply governance, find inefficiencies in architectures, etc. This is the fourth trend in this session
  • Architecture can’t live along which is why often system just don’t work
  • Mechanical sympathy – use tool so works at it’s best. ex: bytecode so small, avoid context switching. On prem storage is expensive and CPU is cheap so use CPU to break up data. The opposite is true in the cloud. CPU is more expensive. Ex: we don’t question normalizing data, but new topologies don’t always follow them.

Automated Governance and Fitness Functions

  • We write tests every day. And if you don’t watch out for Venkat 🙂
  • We do a good job testing functionality
  • How do you test structural integrity of architecture? Elasticity? Maintainability?
  • Fitness function – objectively evaluates an architectural characteristics
  • Operational fitness functions – availability, scalability, etc. Scalability problems manifest as responsiveness problems
  • Structural fitness functions – bridge is fine as cars drive over it. Until it’s not.
  • ArchUnit in Java, ArchUnit and NeArchTest for .NET, PyTestArch for Python, TSArch for JavaScript/TypeScript. Get AI does a good job generating the tests
  • ArchUnit example for structural integrity were the package dependency ones [these seem like the easiest ones to write]
  • Data fitness functions – ex: foreign key constraint across databases, checksum to ensure data consistency
  • Process fitness functions – ex: testability measured by error rates
  • https://blog.hello2morrow.com/2018/12/a-promising-new-metric-to-track-maintainability/
  • Architecture as code

Aspect oriented architecture

  • Spring uses AOP as output
  • Hexagonal architecture – loose coupling to separate plumbing from domain stuff. Alistair Cockburn drew a hexagon when talking about it, but too late, it stuck. Almost got it right. Can’t treat database as a separate thing. Most people use a shorthand for separating domain and plumbing.
  • Don’t need hexagonal architecture anymore. [yet new book on it: Hexagonal Architecture Explained ]
  • Need data to be in context. Microservices preserve this boundary
  • Service mesh
  • Data mesh – operational vs analytical data, However, can’t build an analytics sidecar. Instead build cooperative quantum
  • Sidecar/mesh pattern – can build aspect oriented architecture.
  • Governance mesh. Ad hoc governance/fitness function all over. ex: logging, monitoring, circuit breaker
  • join point – governance mesh
  • pointcut – holistic capabilities like observability
  • advice – fitness functions

AI ∩ Architecture

  • Updating thoughtworks tech radar.
  • AI came up a lot, but only one is in adopt which requires maturity
  • Categories – AI assistant software dev, local inference, fine tuning, inference, cloud services, evals and guard rails, structures outputs, prompts, information retrieval , observability for LLM, building agendas
  • Easy to make a talking dog. Hard to get it to talk right in prod. Also hard to get the talking dog to call an API
  • Vector database used for LLMs
  • Guard rails – how prevent from doing something shouldn’t
  • Eval – how well doing
  • More expensive LLM can validate results of cheaper LLM

How four trends related

  • Can use fitness functions to validate code generated by LLMs
  • Architecture as code and fitness functions describe intersections. Need executable, not diagrams
  • Once critical mass of fitness functions, have governance mesh
  • Systems are too large; can’t manually validate
  • When Log4J, people asked architects what in product and didn’t know [solution to that is a tool. don’t need governance mesh. I agree with the point, but not a fan of the example]

My take

I like that they covered a variety of topics while also getting into code for ArchUnit

[2024 dev2next] table of contents

Posted on by
Reply

Opening

  • Venkat tried to start in 2020. Had .5 beta conference online.
  • This is version 1.0
  • Next year is 2.0 in Colorardo Springs Sept 29-Oct 2
  • Random quote “Money doesn’t buy happiness but does grant leverage”

General notes

  • This is the first time I’ve stayed in a hotel that a conference was at. (vs going for the day or staying at a neighboring hotel). I went for a swim between arriving and conference start. And then went down in my pool slippers (forgot to put on my sneakers). In California, people wear these outside, so good enough!
  • Attention to detail was great and I like that the conference started with a request for feedback.
  • First conference the Java Champion community used Linked In as a channel that I’ve been at. Haven’t used Linked In so frequently in my life! Once I found my password, it was fine.
  • It was nice not having the sessions recorded. People were more open and you heard the talks differently. That said, someone missed the world premiere of my time management talk and asked where I’ve given it to see a recording. Alas, it’s not recorded anywhere

Monday

I flew in after the workshops but attended dinner and the evening sessions:

Tuesday

Wednesday

Thursday

AWS Summit 2024

Posted on by
Reply

I went to AWS Summit New York today, a free one day conference. It’s the first time I’ve gone. I didn’t live blog but am writing a summary post of my day after the fact.

Overview

AWS spent a ton of money on this event. They rented out all or most of the Javits Center in NYC (this is where NYC Comic Con is held). They gave coffee/soft drinks and even free lunch. They also spent a lot of money on security. For cause. There were protesters right outside the front door.

I tried to experience the major parts of the event.

Expo

The exhibit hall was large on the third floor with lots of vendors related to cloud. There were also some fun activities like a drone and toy car racing. Lots of space for sitting/networking.

There were also some stages in the expo for shorter (15-30 minute talks). They had headphones for people who couldn’t filter out the background noise of the expo. It was nice because you could flit by and see if you were interested. I listened to some pieces of cert/education talks and a full one from Elastic on LLMs and summarizing security incidents

Breakouts

There were lots of one hour breakout sessions on the first floor. I went to two customer success stories (Venmo and Fannie Mae). It was dark in the breakout rooms. Like most places have for keynotes

Learning highlights for Venmo

Key strategies

  • Distribute load to maximize processing throughput
  • Use event based systems for anything not in critical path

Other notes

  • Django app. Used Celery for async work,  reader db instanes for queries that can use
  • Then added DynamoDB, MongoDB, OpenSeach Service, data lake, microservics, Cassandra (for microservices), Kafka
  • Split Mysql into Auroa MySQL comatible secondary ySQL and Analytics MySQ databases

Social feed data migration

  • Transactions visible, high traffic because home screen
  • Every transaction geerations a feed story along wih certain profile operations
  • 3.6TB of data, 5.6 bllion entries
  • Since digit lateny on data retrieval
  • 90% of memory usage
  • switched to DynamoDB due to cost (90% less), performance (equivlanet), managed servie, data encrption at rest, integration with other AWS offerings
  • Migrated via backfill followed by dual writes. Let verify performne under pro load and confirm data consistent. Then started ramping reads on new database. Started with 1% reading from new DynaeoDB. Finally cut off writes to ol MongoDB

Offloadng transaction history

  • For each payment put message on Kafka queue and write to Cassandra via microservice.  Implemented as best effort write Needed to guarantee 100% of data so could move over use caes taht required full fidelity data
  • Switch to write ahead log – write log essage saying intend to peror action and store in DynaeoDB Then proess transaction/pblish essage. FInally, delete inteded action message ow that completed. Background process looks for pendin messages 
  • Asyc payment processin using Kinesis
  • Problem batches huge and inconsistent for credit car sage, delays, outage costly, can’t send 500 error/need to reconcile, not a way to replay transactions internally
  • Added Kineis Data Stream via think wrapper to put mesage on strea and ackowledge success to upstrea. From KInsis, have consumers/lambda procss. Also usig Auora, DocumentDB, ElastiCache, DynamoDB and SQS

Key learnings for Fannie Mae

data science research 

  • compared research vs deveopment – ex: research has poc, live prod data, latest tools/patterns
  • pilars of platform:
  • data access – prod data, data usage contracts
  •  governance  – control by business, not tech, autoamted integration with governace
  •  operationalization – testing, validation, Ci/CD
  • data science controls
  • register research activities in CMDB so can provision/tag resources. Automated provisioning, strealined architect review process
  • Data access.sharing  contracts, perissions, ingress/egress rules, sensitive data protection rules
  • Cde deployment and change managment  CI/CD, scanning
  • Data science platform architecture
  • code/image repo
  • pblic data endpoints
  • code/package library
  • read only access to enterprise data lake
  • research envs –
  •  collaboration – just in time access – read only access to prod enterprise data lake. results an’t be shared; considered dev
  • validation  – testing/shakeot – still read only
  • operaiton – headless execution/- now can write to prod, create reports and share exterally
  • data access JIT (just in time). Fannie Mae has a patent on this
  • request access to data. could be from many data sources
  • JIT access engine checks against coarse grained contracts
  • Then goes to policy manager to check fine graine access controls. Use UI to create rules. creates new role dynamically so can use token to access

Building a generative AI use case

  • Used Anthropic’s  Claude 3 Sonnet via Amazon Bedrock and Aazon Neptune (graph db)
  • A lot of analysis of unstructured documents, average of 5 hours per doc and 8K dos per year
  • Deep Insight for LLM driven knowledge extraction. Uses ontology (schema( an LLM t generate knowledge graphs. Human in the loop to validate Then knowledge utiilization step to use natural language via a chatbot
  • taxonomy – linear top down hierarchy. Ontoogy – interconnected network representation
  • Disambigution important to avoid duplication
  • graph database  
  • reduces risk of hallucinations because more context
  • two types – 
  • Property Graph (Apache Tinkerpop) . Query with Gremlin or Cypher
  • RDF Graph (from W3C). query with SPARQL
  • extraction uses Bedrock, fargate, lambda, neptune, s3
  • utilization uses – bedrock, fargate, neptune and a chatbot
  •  also uses LangChain – Neptune Open Cypher QA chain (converts natural langague queries into Cyper so can do query(  and Amazon OpenSearch
  • challenges
  • pick onthology framework – Chose Turtle (Terse RF Triple Language for reeasability/ease of reading
  • find best way to chunk. Chose at sections so handle complex tables btter
  • Picking graph type. Chose property graph due to better OSS framework support
  • Amazon Kendra (enterprise search( did not integrate with Amazon Neptune. Used LangChain’s NeptuneOpenCypher QA Chain instea

Chalk Talks

Chalk talks were also on the first floor. They were also an hour but had less prepared content. The one I went to had 20 minutes of talking/demos. Most of the time was Q&A or discussion. They had a whiteboard with a camera to show what was on it so the speakers could write/draw real time. This meant one projected screen was the computer and one was the physical whiteboard.

Learning highlights

  • gen customers what to know what model to use, how to move quickly and how keep data secure/private
  • Bedrock provides foundational models via single API, customize model, RAG (Retrieval Augmented Generation), agents for multi step tasks, security/privacy/safety
  • Models include – amazon’s models, anthorpic,, cohere, meta, etc. ANd lots of variants/versions of each.
  • Two use cases: observability of generative AI itself, using gen AI to help with observability
  • gather metrics – ex: number tokens used for input/output
  • collected metadata/requests/responses so understand how customers use
  • governance/controls/guardrails
  • Cloudwatch – analyze inovcation logs, protect sensitve date, real time metrics and alarms (Ex: more latency on different version of claude), single pane of glass/dashboard
  • recorded demo #1 (while video was recorded, he narrated live. also paused periodically to say more
  • can send model invocation logs to either s3 (if using other loggiing system) or cloudwatch

Builder Sessions

Also on the first floor, these were small group labs. I went to one on Amazon Q. They had 4 areas on the room with 10 chairs each. An instructor from AWS was allocated to each group. After a short intro, the instructor helped anyone stuck and answered questions. This was great.

The lab had an access code good for three hours so you continue a little longer if you wanted. In theory, there was separate wifi for the lab but it didn’t work. The main conference wifi was fine though.

Learning highlights

  • Amazon Q Developer has a free and paid version.
  • The paid version promises not to learn from your data, It’s licensed per person but only billed if the developer uses in a month.
  • IDE integration for VS Code and IntelliJ.
  • Chat bar. Often gives sources/links. From 2023 for public internet. RAG for Amazon so more recent
  • Can explain code, refactor code, fix code and migrate to later version of Java. Can also write a plan for writing code and write code (with some errors)
  • Code Whisperer was folded into Q
  • It was slow, but I was on a conference network

Main dev activities

  • planning – docs, examples, deisgn
    • creating = generate cpde,amage omfra
    • test amd secure – test cases, scan for security vulnerabiliteies
    • operate – identify and mitigate code issues, monitor performance and efficiencey
    • maintenance and modernization – modernize and update old code languages and dependencies

Amazon Q Developer tries to help with all phases

  • plan – explain code with conversational coding (chatbot)
  • create – inline code complete, conversational coding
  • test/secure – unit test generation, OWASP top 10 security scanning
  • operate – debug/optimize code with conversational coding
  • maintenance and modernatization  update code with agent from legacy

Keynote

The keynote was in a big room that wouldn’t fit everyone. They also used all the breakout rooms as overflow and streamed to the stages in the expo. I like that as it was easy to eat and listen. Or talk to the vendors and listen to parts. Or not.