I went to AWS Summit New York today, a free one day conference. It’s the first time I’ve gone. I didn’t live blog but am writing a summary post of my day after the fact.
Overview
AWS spent a ton of money on this event. They rented out all or most of the Javits Center in NYC (this is where NYC Comic Con is held). They gave coffee/soft drinks and even free lunch. They also spent a lot of money on security. For cause. There were protesters right outside the front door.
I tried to experience the major parts of the event.
Expo
The exhibit hall was large on the third floor with lots of vendors related to cloud. There were also some fun activities like a drone and toy car racing. Lots of space for sitting/networking.
There were also some stages in the expo for shorter (15-30 minute talks). They had headphones for people who couldn’t filter out the background noise of the expo. It was nice because you could flit by and see if you were interested. I listened to some pieces of cert/education talks and a full one from Elastic on LLMs and summarizing security incidents
Breakouts
There were lots of one hour breakout sessions on the first floor. I went to two customer success stories (Venmo and Fannie Mae). It was dark in the breakout rooms. Like most places have for keynotes
Learning highlights for Venmo
Key strategies
- Distribute load to maximize processing throughput
- Use event based systems for anything not in critical path
Other notes
- Django app. Used Celery for async work, reader db instanes for queries that can use
- Then added DynamoDB, MongoDB, OpenSeach Service, data lake, microservics, Cassandra (for microservices), Kafka
- Split Mysql into Auroa MySQL comatible secondary ySQL and Analytics MySQ databases
Social feed data migration
- Transactions visible, high traffic because home screen
- Every transaction geerations a feed story along wih certain profile operations
- 3.6TB of data, 5.6 bllion entries
- Since digit lateny on data retrieval
- 90% of memory usage
- switched to DynamoDB due to cost (90% less), performance (equivlanet), managed servie, data encrption at rest, integration with other AWS offerings
- Migrated via backfill followed by dual writes. Let verify performne under pro load and confirm data consistent. Then started ramping reads on new database. Started with 1% reading from new DynaeoDB. Finally cut off writes to ol MongoDB
Offloadng transaction history
- For each payment put message on Kafka queue and write to Cassandra via microservice. Implemented as best effort write Needed to guarantee 100% of data so could move over use caes taht required full fidelity data
- Switch to write ahead log – write log essage saying intend to peror action and store in DynaeoDB Then proess transaction/pblish essage. FInally, delete inteded action message ow that completed. Background process looks for pendin messages
- Asyc payment processin using Kinesis
- Problem batches huge and inconsistent for credit car sage, delays, outage costly, can’t send 500 error/need to reconcile, not a way to replay transactions internally
- Added Kineis Data Stream via think wrapper to put mesage on strea and ackowledge success to upstrea. From KInsis, have consumers/lambda procss. Also usig Auora, DocumentDB, ElastiCache, DynamoDB and SQS
Key learnings for Fannie Mae
data science research
- compared research vs deveopment – ex: research has poc, live prod data, latest tools/patterns
- pilars of platform:
- data access – prod data, data usage contracts
- governance – control by business, not tech, autoamted integration with governace
- operationalization – testing, validation, Ci/CD
- data science controls
- register research activities in CMDB so can provision/tag resources. Automated provisioning, strealined architect review process
- Data access.sharing contracts, perissions, ingress/egress rules, sensitive data protection rules
- Cde deployment and change managment CI/CD, scanning
- Data science platform architecture
- code/image repo
- pblic data endpoints
- code/package library
- read only access to enterprise data lake
- research envs –
- collaboration – just in time access – read only access to prod enterprise data lake. results an’t be shared; considered dev
- validation – testing/shakeot – still read only
- operaiton – headless execution/- now can write to prod, create reports and share exterally
- data access JIT (just in time). Fannie Mae has a patent on this
- request access to data. could be from many data sources
- JIT access engine checks against coarse grained contracts
- Then goes to policy manager to check fine graine access controls. Use UI to create rules. creates new role dynamically so can use token to access
Building a generative AI use case
- Used Anthropic’s Claude 3 Sonnet via Amazon Bedrock and Aazon Neptune (graph db)
- A lot of analysis of unstructured documents, average of 5 hours per doc and 8K dos per year
- Deep Insight for LLM driven knowledge extraction. Uses ontology (schema( an LLM t generate knowledge graphs. Human in the loop to validate Then knowledge utiilization step to use natural language via a chatbot
- taxonomy – linear top down hierarchy. Ontoogy – interconnected network representation
- Disambigution important to avoid duplication
- graph database
- reduces risk of hallucinations because more context
- two types –
- Property Graph (Apache Tinkerpop) . Query with Gremlin or Cypher
- RDF Graph (from W3C). query with SPARQL
- extraction uses Bedrock, fargate, lambda, neptune, s3
- utilization uses – bedrock, fargate, neptune and a chatbot
- also uses LangChain – Neptune Open Cypher QA chain (converts natural langague queries into Cyper so can do query( and Amazon OpenSearch
- challenges
- pick onthology framework – Chose Turtle (Terse RF Triple Language for reeasability/ease of reading
- find best way to chunk. Chose at sections so handle complex tables btter
- Picking graph type. Chose property graph due to better OSS framework support
- Amazon Kendra (enterprise search( did not integrate with Amazon Neptune. Used LangChain’s NeptuneOpenCypher QA Chain instea
Chalk Talks
Chalk talks were also on the first floor. They were also an hour but had less prepared content. The one I went to had 20 minutes of talking/demos. Most of the time was Q&A or discussion. They had a whiteboard with a camera to show what was on it so the speakers could write/draw real time. This meant one projected screen was the computer and one was the physical whiteboard.
Learning highlights
- gen customers what to know what model to use, how to move quickly and how keep data secure/private
- Bedrock provides foundational models via single API, customize model, RAG (Retrieval Augmented Generation), agents for multi step tasks, security/privacy/safety
- Models include – amazon’s models, anthorpic,, cohere, meta, etc. ANd lots of variants/versions of each.
- Two use cases: observability of generative AI itself, using gen AI to help with observability
- gather metrics – ex: number tokens used for input/output
- collected metadata/requests/responses so understand how customers use
- governance/controls/guardrails
- Cloudwatch – analyze inovcation logs, protect sensitve date, real time metrics and alarms (Ex: more latency on different version of claude), single pane of glass/dashboard
- recorded demo #1 (while video was recorded, he narrated live. also paused periodically to say more
- can send model invocation logs to either s3 (if using other loggiing system) or cloudwatch
Builder Sessions
Also on the first floor, these were small group labs. I went to one on Amazon Q. They had 4 areas on the room with 10 chairs each. An instructor from AWS was allocated to each group. After a short intro, the instructor helped anyone stuck and answered questions. This was great.
The lab had an access code good for three hours so you continue a little longer if you wanted. In theory, there was separate wifi for the lab but it didn’t work. The main conference wifi was fine though.
Learning highlights
- Amazon Q Developer has a free and paid version.
- The paid version promises not to learn from your data, It’s licensed per person but only billed if the developer uses in a month.
- IDE integration for VS Code and IntelliJ.
- Chat bar. Often gives sources/links. From 2023 for public internet. RAG for Amazon so more recent
- Can explain code, refactor code, fix code and migrate to later version of Java. Can also write a plan for writing code and write code (with some errors)
- Code Whisperer was folded into Q
- It was slow, but I was on a conference network
Main dev activities
- planning – docs, examples, deisgn
- creating = generate cpde,amage omfra
- test amd secure – test cases, scan for security vulnerabiliteies
- operate – identify and mitigate code issues, monitor performance and efficiencey
- maintenance and modernization – modernize and update old code languages and dependencies
Amazon Q Developer tries to help with all phases
- plan – explain code with conversational coding (chatbot)
- create – inline code complete, conversational coding
- test/secure – unit test generation, OWASP top 10 security scanning
- operate – debug/optimize code with conversational coding
- maintenance and modernatization update code with agent from legacy
Keynote
The keynote was in a big room that wouldn’t fit everyone. They also used all the breakout rooms as overflow and streamed to the stages in the expo. I like that as it was easy to eat and listen. Or talk to the vendors and listen to parts. Or not.