DevNexus 2018 – Garbage Collection in Java 9

Title: Garbage Collection in Java 9
Speakers: Chandra Guntur

For more blog posts, see the DevNexus 2018 live blogging table of contents

Areas of memory

  • By thread
  • Method area
  • Runtime constant pool
  • Heap


  • Types of collectors – serial (one thread; stopping all app threads), parallel (multiple threads, but still stops all app threads), concurrent (doesn’t stop app)
  • Can mark objects precisely or conservatively
  • Can run all at once or incrementally
  • Can move reachable objects or just release unreachable objects


  • reference counting
    • simplest algorithm
    • keeps a counter for each object
    • obsolete for commercial JVMs
  • mark/sweep/compact
    • change marker type if used
    • remove if not used
    • compact – like Windows defragment
  • copying
    • similar to mark/sweep/compact except copies object when used instead of marking
    • the copying operating compacts/defragments since copied to adjacent memories
  • Java 5 introduced generational GC
    • start young
    • most objects go away rather than getting promoted to older generation
    • generations – eden, survivor, tenured
  • Java 5 – 8 had Concurrent Mark-Sweep (CMS)
    • serial collection and tenured generation
    • most popular GC in Java 5-8
    • deprecated in Java 9


  • inconsistent pause type
  • poor performance for large memory heaps
  • focused more on live objects than garbage

Java 9 – G1GC (Garbage First Garbage Collector)

  • new default in Java 9
  • region – contiguous unit of memory
  • heap divided into 2048+ regions of equal size – may be 1-32MB each
  • each region has a generation (free, eden, survivor, tenured) or be part of a “humongous” allocation
  • humongous objects
    • use complete (or multiple contiguous) regions
    • must be the size 50% of a region or more.
  • goal of  consistent pause time
  • life of a region
    • objects get created and stored in “young” half
    • at some point, does initial mark where concurrent marking process determines which objects are alive. young collection continues in parallel.
    • then does re-mark to confirm still alive. young collection still continues
    • decides if should reclaim space now or wait
    • then does space reclamation
  • young collection includes survivor promotion and tenured promotion
  • java -XX:+PrintFlagsInitial -version > output – print the initial defaults to a file since it is a long list – always same at startup by OS
  • java -XX:+PrintFlagsFinal -version > output – print the initial defaults to a file since it is a long list – changes based on actual running
  • there were lots of involved details. hard to have that as the last session of the day :).

My take

I’m not a fan of “ignore the text; read it later” presentations. That is what the appendix or notes view is for. I’d rather have the pictures and key points be bigger. Chandra even zoomed in on the diagram. It cold have had a whole slide to itself. That said, the presentation was clear and easy to follow.

DevNexus 2018 – Modules in Action

Title: Modules in Action
Speakers: Mark Reinhold

For more blog posts, see the DevNexus 2018 live blogging table of contents

Demo used Java 10 RC.

Using JShell to look at modules

  • “foo”.getClass().getModule() – module java.base
  • “foo”.getClass().getModule().getClass() – class java.lang.Module
  • new java.sql.Timestamp(0).getModule() – module java.sql


  • java –list-modules – ex: java.sql@10
  • There are 75 modules in Open JDK 10


  • The Java 9 JavaDoc shows modules on the home page instead of all the packages.
  • Clicking a module shows you the packages in the module.
  • Java 9 JavaDoc has search box

Random facts

  • tree – unix commands that list what is in directory and children (not installed by default)
  • Showed that can use * in classpath to get all files in lib directory. (since Java 6)

Creating a bare bones module

  • Create file in root of src folder
  • Simplest module file is one line: module moduleName { }
  • If run javap against the class file, you can see “requires java.base;” was added. This is like a constructor calling super(). It is mandatory so you don’t need to type it.
  • jar –file x.jar –describe-module – shows name of file, requires and the package contained
  • java –module-path lib -m moduleName/className – run the program
  • java -classpath jar className – can still run a modular jar on the classpath
  • jar –create –file x.jar –main-class class -C classes – create a jar that can run without specifying the class to run
  • java –modulepath lib -m moduleName – run the program without specifying it by name

Refactoring the module to split into two

  • module module1 { requires packageName; }
  • module module2 { exports packageName; }


  • If remove module2 and try to run, get java.lang.module.FindException on startup since can’t find module. Error inlcudes “Error occurred during initialization of boot layer”
  • If forgot to type “requires”, and compile get “package x is not visible”
  • If two modules require each other and compile, get “cyclic dependence involving package x”
  • If forget to type “exports” and compile, get “IllegalAccessError exception” along with long message about exporting


  • Optional Java linker
  • New directory jmods in JAVA_HOME.
  • Each module in the JDK has a file in jmods
  • JDK got a lot bigger with Java 9 because shipping two copies of each module – the main one and the jmods directory
  • jlink –module-path $JAVA_HOME.jmods –output jre –add-modules java.base – creates a JRE that is an order of magnitude smaller than the full one (44M vs 329B)
  • Can include your custom app into the image in addition to JDK
  • jlink –module-path $JAVA_HOME.jmods:lib –output jrePlusApp –add-modules java.base myCustomModule
  • jlink has a –compress flag to make file smaller
  • Not platform independent. Can cross link to create file for other operating system


  • Want to make it so private really means private and final really means final. Vs reflection today.
  • Illegal reflective access – first time, get warning. The warning tells you how to enable more warnings. There probably are, but don’t want to blast you with them. Eventually, warning will turn into a hard error.
  • Not planning to disallow illegal reflective access until after Java 11.
  • Supplementing “exports”, is “open. It allows extra reflection

Inside a jmod

  • jmods are like a zip file with extra structure.
  • classes directory with all of the classes
  • conf directory for files in runtime image
  • include directory for jni header files
  • legal directory with licenses if any
  • bin directory for launchers
  • lib directory for native code

My take

I like the live demo approach along with the emphasis on both concepts and common problems. I also like that Mark left a lot of time for Q&A.


The Amazon AWS Java SQS Client is *Not* Thread-safe

I recently added long polling to an Amazon SQS project that processes thousands of messages a minute. The idea was simple:

  • Spawn N number of threads (let’s say 60) that repeatedly check an SQS queue using long polling
  • Each thread waits for at most one message for maximum concurrency, restarting if no message is found
  • Each time a message is found, the thread processes it and ACK’s via deleteMessage() (failure to do so causes the message to go back on the queue after the visibility timer is reached)

For convenience, I used the Java Concurrency API ScheduledExecutorService.scheduleWithFixedDelay() method, setting each thread with 1 millisecond delay, although I could have accomplished the same thing using the Thread class and an infinite while() loop. With short polling, this kind of structure would tend thrash, but with long polling, each thread is just waiting when there are no messages available. Note: For whatever reason, Java does not allow a 0 millisecond delay for this method, so 1 millisecond it is!

Noticing the Problem
When I started testing my new version based on long polling, I noticed something quite odd. While the messages all seem to be processed quickly (1-10 milliseconds) and there were no errors in the logs, the AWS Console showed 50+ messages in-flight. Based on the number of messages being processed a second and the time it was taking to process them, the in-flight counter should have been only 3-4 messages at any given time but it consistently stayed high.

Isolating the Issue
I knew it had something to do with long polling, since previously with short polling I never saw that many messages consistently in flight, but it took a long time to isolate the bug. I discovered that in certain circumstances the Amazon AWS Java SQS Client is not thread-safe. Apparently, the deleteMessage() call can block if too many other threads are performing long polling. For example, if you set the long polling to 10 seconds, the deleteMessage() can block for 10 seconds. If you set long polling to 20 seconds, the deleteMessage() can block for 20 seconds, and so on. Below is a sample class which reproduces the issue. You may have to run it multiple times and/or increase the number of polling threads, but you should see intermittent delays in deleting messages between Lines 25 and 27.


import java.util.concurrent.*;
import com.amazonaws.regions.Regions;

public class SQSThreadSafeIssue {
	private final String queueName;
	private final AmazonSQS sqsClient;
	private final int numberOfThreads;
	public SQSThreadSafeIssue(Regions region, String queueName, int numberOfThreads) {
		this.queueName = queueName;
		this.sqsClient = AmazonSQSClientBuilder.standard().withRegion(region).build(); // Relies on locally available AWS creds
		this.numberOfThreads = numberOfThreads;
	private void readAndProcessMessages(ReceiveMessageRequest receiveMessageRequest) {
		final ReceiveMessageResult result = sqsClient.receiveMessage(receiveMessageRequest);
		if(result!=null && result.getMessages()!=null && result.getMessages().size()>0) {
			result.getMessages().forEach(m -> {
				final long start = System.currentTimeMillis();
				System.out.println("1: Message read from queue");
				sqsClient.deleteMessage(new DeleteMessageRequest(queueName, m.getReceiptHandle()));
				System.out.println("2: Message deleted from queue in "+(System.currentTimeMillis()-start)+" milliseconds");
	private void createMessages(int count) {
		for(int i=0; i<count; i++) {
			sqsClient.sendMessage(queueName, "test "+System.currentTimeMillis());
	public void produceThreadSafeProblem(int numberOfMessagesToAdd) {
		// Start up and add some messages to the queue
		// Create thread executor service
		final ScheduledExecutorService queueManagerService = Executors.newScheduledThreadPool(numberOfThreads);
		// Create reusable request object with 20 second long polling
		final ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest();
		// Schedule some thread processors
		for(int i=0; i<numberOfThreads; i++) {
			queueManagerService.scheduleWithFixedDelay(() -> readAndProcessMessages(receiveMessageRequest),0,1,TimeUnit.MILLISECONDS);
	public static void main(String[] args) {
		final SQSThreadSafeIssue issue = new SQSThreadSafeIssue(Regions.YOUR_REGION_HERE,"YOUR_QUEUE_NAME_HERE",60);

And below is a sample output of this, showing that each message took 20 seconds (the long polling time) to be deleted.

1: Message read from queue
1: Message read from queue
1: Message read from queue
1: Message read from queue
1: Message read from queue
2: Message deleted from queue in 20059 milliseconds
2: Message deleted from queue in 20098 milliseconds
2: Message deleted from queue in 20024 milliseconds
2: Message deleted from queue in 20035 milliseconds
2: Message deleted from queue in 20038 milliseconds

Note: The SQSThreadSafeIssue class requires Java 8 or higher along with the following libraries to compile and run. It uses the latest version of the Amazon AWS Java SDK 1.11.278 available from AWS (although not in yet):

Understanding the Problem
Now that we see messages are taking 20 seconds (the long polling time) to be deleted, the large number of messages in-flight makes total sense. If the messages are taking 20 seconds to be deleted, what we are seeing is the total number of in-flight messages over the last 20 second window waiting to be deleted, which is not a ‘true measure’ of in-flight messages actually being processed. The more threads you add, say 100-200, the more easily the issue becomes to reproduce. What’s especially interesting is that the polling threads don’t seem to be blocking each other. For example, if 50 messages come in at once and there are 100 threads available, then all 50 messages get read immediately, while not a single deleteMessage() is allowed through.

So where does the Problem lie? That’s easy. Despite being advertised as @ThreadSafe in the API documentation, the AmazonSQS client is certainly not thread-safe and appears to have a maximum number of connections available. While I imagine this doesn’t come up often when using the default short-polling, it is not difficult to reproduce this problem when long-polling is enabled in a multi-threaded environment.

Finding a Solution
The solution? Oh, that’s trivial. So trivial, I was tempted to leave as an exercise to the reader! But since I’m hoping AWS developers will read article and fully understand the bug, so they can apply a patch, here goes….

You just need to create two AmazonSQS instances in the constructor of SQSThreadSafeIssue, one for reading (Line 21) and one for deleting (Line 26). Once you have two distinct clients, the deletes all happen within a few milliseconds. Once applied to the original project I was working on, the number of in-flight messages dropped significantly to a number that was far more expected.

Although this work-around fix is easy to apply, it should not be necessary, aka you should be able to reuse the same client. In fact, AWS documentation often encourages you to do so. The fact that the Amazon SQS client is not thread-safe when long polling is enabled is a very serious issue, one I’m hoping AWS will resolve in a timely manner!