[2019 oracle code one] CD with Docker and Java

Continuous Delivery with Docker Containers and Java: The Good, the Bad, and the Ugly

Speaker: Daniel Bryant @danielbryantuk

For more blog posts, see The Oracle Code One table of contents



General

  • “Continuous delivery is achieved when stability and speed can satisfy business demand. Discontinuous delivery occurs when stability and speed are insufficient” – Steve Smith @SteveSmithCD
  • Feedback loop
  • Choices are about tradeoffs

Good

  • Dev environment setup can be Dockerized/Containerized
  • Repeatable builds
  • Legacy technology can be sealed

Bad

  • Why is the container image 1GB for a hello world app
  • Dev/test/deploy/loop too long
  • The app runs slow/freezes on Docker

Impact of container tech on CD

  • Install Docker/container on local machine. Important to understand platform deploying to (mechanical sympathy).
  • Store container image, not jar/war
  • Test in container image
  • Container image is single binary – “Build Binaries Only Once (BBOO)”

Lessons

  • Make dev env like prod as much as possible. Use identical base image with same config.
  • Dockerfile content is super important – OS, ports, volumes, JDK
  • Talk to the sysadmin people. Their operational knowledge is invaluable. Avoids both operational and political problems
  • Don’t want JDK in production. [so what use. JRE no longer exists. Can’t use JLink if need Tomcat to run app.
  • Avoid unused Maven dependencies (so smaller]
  • BuildKit – best effort caching
  • Get app/config drift if have different dev/prod containers
  • Use sidecar containers to bundle other things
  • Toolchain may alter when go to container space.
  • Metadata is valuable. Need to know what is running where. Can store in external registry. ex: Artifactory or Nexus
  • Try to do component testing – a few services together
  • Performance – gatling, jmeter, flood.io
  • Security testing
  • Migrate to Java 11 for speed
  • AOT gives performance in short term and JIT in long term
  • Re test app when change any config
  • Set container memory appropriately

My take

Good session. The advice covered different levels of problems which was nice. The flowchart was unreadable. I think I got the gist, but it’s hard to get a complex flow on the screen.

[2019 oracle code one] collections corner cases

Collections Corner Cases

Speaker: Stuart Marks @StuartMarks #CollectionsCornerCases

For more blog posts, see The Oracle Code One table of contents



General

  • Covers things in collections framework a long time. Not well known
  • Good mix of PowerPoint and JShell

JShelll

  • If don’t assign an expression to a variable goes in $1, $2, etc
  • /var $1 – prints type of variable (yes the slash is intentional, it is a JShell command.)
  • /open abc.jshell – “imports” file

View Collections

  • Don’t contain their own elements. Instead the elements are stored elsewhere
  • ex: Arrays.asList(array) – creates list backed by array. If change list, the array gets changed to and vice versa.
  • map.keySet(), map.values() and map.entrySet(). If change these three collections, underlying map changes too.
  • Map.Entry – has setValue() method. Can call setValue() on on entry set to change underlying map

Performance/Clarity

  • values() is O(n) on a Map
  • Creating a temporary HashSet for the values makes in O(1)
  • retailAll() – set intersection
  • Don’t always need streams. [My logic is if you wouldn’t have needed a loop, you don’t need streams]

Sorted Collections & Comparators

  • Interfaces: SortedSet, SortedMap, NavigableSet, NavigableMap
  • Implementations: TreeSet, TreeMap, ConcurrentSkipListSet, ConcurrentSkipListMap
  • Ordering provided by Comparator. Could be natural order or Comparable. May or may not be consistent with equals.
  • HashSet and Set.of() – iteration order undefined. For SortedSet, iteration order is well defined – sorted order
  • Comparator rules
    • <0, 0, >0 for less than, equal and greater than
    • Must provide a total ordering. Any two values must return result
    • reflexive, transitive, etc
  • Spent a while on set vs list (unique elements) – don’t forget what type you are working with
  • HashSet – doesn’t allow duplicates using equals()
  • TreeSet – doesn’t allow duplicates where compare(a,b) ==0
  • String class has built in comparator: String.CASE_INSENSITIVE_ORDER. Inconsistent with equals(). So using case insensitive order with a TreeSet and adding ‘a’ and “A’, doesn’t add ‘A’ to list. Also tree.contains(“A”) returns true if “a” in set. If have HashSet with ‘A’ and TreeSet with ‘a’, tree.equals(hash) returns true but hash.equals(tree) returns false.

My take

I learned something new under 5 minutes in. (keySet()/values() and entrySet() are backed.). I was also pleasantly surprised to learn new shell commands. The rest was good too :).

Also, I was reminded that I hate Map.of(). I fell for misreading them because I didn’t parse properly as alternating keys and values.

[2019 oracle code one] security

Java application security the hard way – a workshop for the serious developer

Speaker: Brian Vermeer @brianverm & Steve Poole

For more blog posts, see The Oracle Code One table of contents



Scary facts

  • In 2016, cybercrime was estimated to be worth slightly more than the illicit drug trade
  • Cybercrime has less risk than drugs. Rarely get caught. And if do get caught, harder to prosecute.
  • Cybercrime is growing faster than drugs.
  • Cybercrime estimated to quadruple between 2016 and 2019. And to triple again between 2020 and 2021.
  • Illicit drug trade is linear/capped.
  • Cybercrime worth about $600 for every person on the planet through tools you rely on.
  • 2017 – Equifax Struts 2 – Remote Code Injection. Discovered July 29, 2017. But the exploit started months prior. The fix for the vulnerability had been out since March 5. On March 5th, they started probing the system for weakness

General

  • With vulnerabilities can: steal your data, change your data, crash your systems, use your compute power and use your system to get to another
  • Hard to spot vulnerabilities – missing code, off by one errors
  • Exploits are chained vulnerabilities
  • https://cve.mitre.org
  • CVEs can be vague so not providing instructions on how to reverse engineer
  • How do you know you are connected to the wifi you expect?
  • How do you know the USB charger you have is yours and not one that has been modified.
  • How do you know a free power charger is just charging phone and not attacking it.
  • In some countries, hotels are designed so only one place convenient to use laptop and they have a camera angled at it.
  • Fixing is easy. Everything else is not – h=How many people affected? How long? How bad?
  • Every time you add flexibility, you add opportunity

Tools for attackers

  • Google filetype:action to learn which app using Struts
  • Browser developer tools returns information. Response headers including web server (unless change config)
  • https://www.shodan.io – search by IP. Used for IoT devices. Can also type keywords like “java”. Information comes from default response haders.
  • https://exploits.shodan.io/welcome – Can search exploits (pre written attack; just provide IP). And filter by platform.
  • https://www.wappalyzer.com – plugin to learn about website
  • https://www.cvedetails.com – search CVEs and see details. Just like it sounds

Demos

  • Note: https://snyk.io/blog/10-java-security-best-practices/
  • Demo: SQL Injection
    • taking parameter in String query vs using PreparedStatement/bind variables
    • SQL Injection still #1 in OWASP top 10 vunlerabilities
  • Demo: XXE (XML External Entity Processing)
    • <!ENTITY xxe SYSTEM “file:///etc/passed”>]> and <a>&xxe;</x>
    • Each XML parser has different way of turning off external general entities
  • Demo: JSON marshalling
    • Don’t trust everyone who can see the log
      Also, laws against having plain text credit card info
    • Automatically provided toString() that contains sensitive info using Project Lombok to automatically provide toString(). Can annotate fields to exclude using Lombok with @ToString.Exclude
    • Also Jackson writeValueAsString() writes out all fields. Easy t send to front end by accident. Has @JsonIgnore annotation to solve this.
  • Demo: Authentication
    • Don’t do yourself unless have to. Better to use an existing provider
    • Use strong crypto hashes – consume a lot of power/CPU so brute forcing less likely. BcCypt or SCrypt. Provided with Spring Security
    • Password encryption should be fast, but not too fast.

Dependencies

  • You write a small fraction of your app.
  • Much is open source that is well known and written by Pivotal (Spring), Apache, etc. Trusted providers
  • Attackers are targeting open source – one vulnerability, many victims
  • Demo: Struts
    • Inject code in Content-type header. Contains Ogml. Injects code including /bin/bash command
    • The bad header is published so effort is low for attackers
    • Upload file to replace a native jdk file by using ..\..\….
    • We think like good guys. Need to think how to break out of the safe parts.
  • Demo: JavaScript with Regular Expressions
    • Backtracking can use a lot of computing power
    • Node.JS app only has one thread
    • If regex is using up CPU, nothing else happens
    • Temporary denial of service
    • Test with long match and long non-matching string
    • Expensive if in cloud because pay for CPU/scaling
  • On average, takes 2.5 years before vulnerability is found/exposed

DevSecOps

  • NPM has most packages indexed in last year (by a large margin). Maven Central is second.
  • Need to look at: yourself, your code and the code you depend on
  • Individual
    • Encrypt laptop so don’t get passwords/keys if steal
    • Don’t reuse passwords
    • Two factor authentication
    • Use password manager
    • Randomly generate password
  • DevOps
    • you have to maintain/support it as the developer
    • open have elevated privileges
  • Solution: Team culture + process + tooling
  • Use tooling to make right process unobtrusive
  • It doesn’t matter what tooling, it matters that you use it
  • Test dependencies for issues before commit
  • SonarCloud/SonarLint

Types of Security Issues

  • Time and state – unexpected interaction
  • Errors – ex: line number in error message gives insight into library in use
  • API abuse – use not as intended
  • Code quality – if can’t understand, how know how it works
  • Security features – authentication, access control, confidentiality, cryptography, privilege management
  • Encapsulation – lack of encapsulation for critical data
  • Input validation & representation – metacharacters, alternate encodings, numeric representations. Trusting input
  • Environment – everything outside source code

Tips

  • Keep libraries current
  • Learn about secure coding
  • Compartmentalize – data, code, access controls, etc
  • Design for intrusion – review levels of “helpfulness” and flexibility
  • Learn more about security tools and services.
  • Lear about penetration testing
  • Understanding that making your development life easier makes the hacker’s job easier.

Free book/report: https://www.ibm.com/downloads/cas/G3L0EPOL

My take

Great start to the morning. A mix of review and new (to me.) Also, useful to have a “the world is a scary place” reminder. Helps motivate to do the right thing!