performance engineer’s guide to hotspot JIT compilation – monica beckwith – qcon

For more QCon posts, see my live blog table of contents. This presentation is about the compiler and also the runtime.

Major pieces

  • Execution engine
    • Heap management/garbage collection
    • JIT compilation
  • Runtime
    • VM Class loading
    • Interpretter
    • Byte code verification,etc

Runtime goal – convert from bytecode to native code and do optimizations along the way

Compilation Techniques and Notes

  • Pre-compiled/ahead of time
  • profile guided – based on critical hotspots
  • Adaptive optimization (Java uses Profile guided and Adaptive optimization)
  • Identify root of compilation
  • replace method or on stack – depends on number of times through loop
  • Server compiler has a higher threshold than client compiler for the threshold at which you need optimizations
  • Tiered compilation – tier 1 is client compiler with no profiling info, tier 2 and 3 are client compiler with profiling info. Then comes server compiler
  • CodeCache order of magnitude larger when tiered compilation is enabled. If need more can use -XX:ReservedCodeCacheSize
  • Inlining – many different parameters when figuring out when to inline
  • Vectorization – SIMD (SIngle Instruction Multiple Data). Can generate stubs and benefit from caching size chunks. For SuperWord Level Parallelism, you need to unroll the loop, do analysis/pre-optimization, etc. Still in infancy with Hotspot.
  • Escape analysis – Want to see if object only is used in a compile method. Need entire graph to confirm not in a static field/returned from method/passed as parameter/etc. If really local, can optimize by storing in registers.
  • Objects are 8 byte aligned by default. Fields are aligned by type.
  • OOP (ordinary object pointer) is a managed pointer. The size can be changed to optimize.
  • Compressed Class Pointers – part of the Metaspace. Class data is outside of heap.

Deoptimization

  1. dependency issues
  2. class unloading/redefinition
  3. uncommon path
  4. profiled info isn’t useful for path [like with databases when db assumes something different than you want]

If curious about details

To get information about what compiler thinks/did:

  • PrintCompilation – ex: what level instructions were compiled at
  • PrintInlining – use -XX:+UnlockDiagonsticVMOptions

how containers have panned out – adrian trenaman – qcon

For more QCon posts, see my live blog table of contents. Adrian is from Gilt.

History

  • No off the shelf software to run a flash sale business. Therefore Gilt has to do something custom.
  • Started with Ruby on Rails in 2007. Didn’t scale well enough
  • Moved to Java in 2011
  • Moved to microservices in 2015
  • In a 30 day period, moved bulk of Gilt to Amazon

Problems

  • Isolation problem – nobody should be able to take down someone else’s work
  • A noon outage in 2013 – what happened
  • Impedance mismatch problems. “Developers often think of machines as something that’s all theirs, magically provided by the hardware fairy.”

Machines for Gilt Japan

  • Run 20-40 containers per machine.
  • Load balancer between two racks of three boxes each.
  • Separate machines for the database and email.
  • From developer’s point of view, a machine is a machine.

What did Gilt Japan learn

  • Scalable by time of day
  • Solves impedance mismatch – developers see “a machine”
  • Limits damage one person can do
  • Infra/Devops engineer embedded into engineering team
  • Outstanding potential problems
    • Static infrastructure
    • Resource hogging

Docker topology

  • Dark canary – only for internal use
  • Canary – First prod install. Let it run for a while (ex through a noon cycle for Gilt)
  • Release – Once happy with canary, roll it out to other nodes
  • Gilt has a lot of read only traffic which limits damage you can do and reduces need for staging environment.
  • Gilt has one container per host/EC2 instance
  • Want to have as few moving parts/risk points in deployment process
  • “We could solve this now, or just wait six months and Amazon wil provide a solution”

Projects

  • ION Roller
    • Immutable deployment – Destroy original cluster when done with this process for Docker upgrades.
    • Slow to setup/tear down environments.
    • Can be expensive for continuous deployment
    • Open source, but in house.
  • Nova
    • Uses yaml to deploy
    • No Docker registry. Base images are on Docker. Releases aren’t needed on there so go straight to Amazon
    • Less boilerplate
    • Immutable deployment on mutable infrastructure. Docker container is immutable.
  • Fighting bit rot, chaos-monkey style
    • Don’t want things to run forever in Prod.
    • What if there is a security vulnerability
    • Every day, kill oldest AMI randomly. This forces latest AMI with fixes and fail early.
    • Doesn’t solve vulnerability in Docker container. Would need new release with new base image for that. Hasn’t happened to Gilt yet.
  • Sundial
    • For running batch jobs
    • Automatically reschedules if fail
    • Define a process – group of tasks with dependencies between them

EC2

  • Less configuration
  • Automatic rollout
  • Integrations
  • IAM roles are at instance level, not container level

Using Docker as a local build platform

  • Different projects use different versions of build tools
  • Docker can be used as a versioned build container.
  • A year from now, will still have everything need to run code

Lessons

  • Containers let separate what deploy from how.where deploy it
  • Still the wild west on how containers are deployed
  • Seek immutability in the container, not in the stack
  • The competitive advantage for Gilt is to be able to deploy quickly/frequently/safely to production and therefore can innovate faster. Gilt lets engineers deploy whenever they want without asking permission.

reactive programming for java developers – rossen stoyanchev – qcon

For more QCon posts, see my live blog table of contents.

General notes

  • Spring 5 has reactive features.
  • Maybe a third of the audience is doing something with reactive, a little more have read about it (that’s me) and a lot haven’t heard of it. Which is good given the title of this talk!
  • Moore’s law slowing down. No more free lunch from the hardware
  • Now apps more distributed/independent/cloudbased compared to 10 years ago
  • Now we expect latency
  • Thread now has to do more async things. Hit limits of scale. imperative logic more complicated.

Other approach

  • Async/non-blocking
  • Use very few threads. node.js has a single thread. Must be careful because blocking those few threads blocks the whole app
  • Reactive programming – think async. Not everything is a callback

Comparing styles

  • Imperative
    • call a method, get a result, do something with that result
  • Async
    • return a Future instead of a value. Don’t throw a checked exception because my happen in a different thread. They are asynchronous in that you can get a result later. But when you eventually call feature.get(), it throws different checked exceptions.
    • In Java 8, can return a CompletableFuture and call future.whenComplete((user, throwable) -> {})
    • If doing a findAll(), the future/completable future approach doesn’t give you a callback/return anything until all of them are ready. You can’t stream or could run out of memory.
    • Async results as stream – get one notification per data item and another for completion or error
  • Declarative
    • Focus on what, not how
    • Declare what should happen rather than focusing on callbacks
    • Java 8 stream API uses this style. It is meant for collections and pull based/usable once
    • Hot streams – latency sensitive streams, data need to be pushed for you
    • Cold streams – pull based

Reactive libraries

  • Stream like API
  • Can be used for hot/cold streams
  • Project Reactor – Similar to ReactiveX, easy to bridge to Java 8 streams. (ReactiveX is like XUnit – commonality for different languages)

Abstractions

  • Flux – sequence of 0 to n – equivalent of Java 8 stream – can convert to/from Java 8
  • Mono – sequence of 0 or 1 – can convert to/from CompletableFuture

Reactive streams spec

  • Back pressure – producers must not overwhelm consumers. Communicates if downstream isn’t ready for more items. Ex: a slow client like a mobile device isnt able to handle more data and server isn’t blocking
  • Small API – only 4 interfaces
  • Rules
  • Interoperability across reactive components so can compose chain

Java 9

  • Reactive streams included (the four interfaces)
    • Publisher
    • Subscriber
    • Subscription
    • Processor
  • Use a reactive library. Don’t implement the four interfaces yourself
  • subscribe() triggers the flow of data
  • “push” was missing until now. Want in JDK to have foundation
  • Classes
    • java.util.concurrent.Flow – the four interfaces
    • SubmissionPublisher – bridge to reactie streams
    • Tie-ins to CompletableFuture and Stream

Reactor

  • GA release scheduled for July
  • Currently called 2.5. Might changed to 3.0

Reactive Spring MVC

  • Apps annotate controllers even now.
  • Return a Flux type.
  • Spring MVC itself needs to change a lot – called Spring Web Reactive
  • The servlet API assumes blocking. There are async workarounds. Servlet 4.0 might support reactive spring integration, but probably just the basics. Using own bridge to Servlet 3.1 in meantime.
  • Can still use Jetty/Tomcat. They are non-blocking behind the scenes.
  • Don’t need servlet container. Can use Netty.
  • HandlerMapping – change to return a Mono so can be non-blocking
  • Focusing on REST/microservices scenarios

Other reactive efforts

  • MongoDB – reactive driver
  • Couchbase – reactive driver
  • Thymeleaf – split templates into chunks and throttle to respect backpressure