cloud native streaming and event driven microservices – live blogging from spring days

Main menu:

Topics

Recent Posts

Feeds

RSS Feed RSS - Posts

June 2017
M T W T F S S
« May    
 1234
567891011
12131415161718
19202122232425
2627282930  

Past Posts

Java/Java EE

JDBC

Other

cloud native streaming and event driven microservices – live blogging from spring days

June 21st, 2017 by Jeanne Boyarsky

Cloud-Native Streaming and Event-Driven Microservices
Speaker: Marius Bogoevici
See list of all blog posts from conference

Streaming

  • Streaming is processing large quantitiesof data as fast as possible -near real time.
  • Collection of eents – in order
  • Good for real time ELT, predictive maintenance, fraud detection, QoS, etc
  • high throughput/low latency

Microservices

  • Cohesive, organizational allignment (Conway’s law)/li>
  • Development agility – optimized for replacement, enable continuous delivery
  • Failure isolation
  • Granlar resource tuning – can scale specificparts of pipeline

Event driven microservices

  • Communicate thrugh message broker vs http
  • Decoupled
  • Publish/subscribe makes it easy to add new elements

Think about opertional complexity – need to be able to provision/montitor. Want this part to be boring. It’s not the business value. Hence cloud-native; the platform should take care of these concerns.
Support Spring WebFlow- Cloud Foundry, Apache Mesos, Kubernetes, Apache YARN

Spring XD – first incarnation. In 2015, it turned into Spring Cloud Stream/Task/Data Flow.

Spring Cloud stream

  • event-driven microservices framework
  • Built on Spring Boot and Spring Integration
  • Binder abstraction – core of framework – connects channels to messaging middleware of your choice
  • Apache Kafka and Rabbit MQ are prod ready support
  • Based on pipes and filters. The logic is in the fiters
  • Spring Cloud Data Flow

    • Orchestration of services
    • Provides Stream DSL
    • Describe deployment
    • Deploys using SPI (service provider interface)
    • Deployment descritpor to specify what want to happen

    Interesting. And we got to read more Flux code. Appreciating Victor’s talk from yesterday even more! I would have liked if “opinionated primitives” was described more. While I understand what the concept means, I’m not that clear on how it relates to the framework. I also had to Google the term SPI. This might have been a presentation for which I lacked pre-req knowledge. The session felt longer than the others. Checking the log for this blog post, it was a little longer; but not a ton so. I think just too much sitting for me!

    advanced spring boot with consul – live blogging from spring days

    June 21st, 2017 by Jeanne Boyarsky

    Advanced Spring Boot with Consul
    Speaker: Michael Gendelman @MikeGendelman
    See list of all blog posts from conference

    “when someone writes something clever and leaves, that’s a problem”

    “every time someone logs onto a box at a Fortune 500 company, it is a big deal”

    Consul

    • for service discovery and configuration management
    • Consul agent – does health check and forwards queries. Runs on machine of all apps
    • Consul server – elects a leader. Want 3-5 per data center. Must have at least 3 so can elect leader
    • Has UI – can see properties/services/etc. Can update config in UI in addition to through yml files [talked about risk of having the real properties be out of sync of the yaml. Good that devops has evolved to the point this is a common worry]
    • Same executable for server and agent

    Monitoring
    “health of the service is more important than the health of a single instance”
    Don’t need a page/alert if enough are still up

    Code>

    • Use spring-cloud-starter-consul-all Maven dependency – made by an independent company
    • Have dev mode so can run in isolation but test all the features
    • annotation to enable discovery
    • Have bootstrap.yml file to set up config – instead of application.yml

    thinks there is plugins/script to link version control to properties. good if need different properties in different environments

    I learned enough about Spring Boot yesterday to be able to follow Mike’s example. Seeing the live demo of Consol was nice.

    reactive programming by example – live blogging from spring days

    June 20th, 2017 by Jeanne Boyarsky

    Reactive Programming by Example: Intro to Pivotal Reactor
    Speaker: Victor Grazi @vgrazi
    https://github.com/vgrazi/reactive-demo
    See list of all bog posts from conference

    What is Reactive

    • Think in streams – but more powerful than Java 8 streams
    • reactive streams – high performance, async stream processing non-blocking back pressure – when ready to stop getting data
    • declarative
    • doesn’t vary much by language

    Maven dependencies

    • reactor-core
    • reactor-test – ex: simulate waiting X seconds without actually waiting

    Everything is a stream of messages – ex: events, query results, exceptions (errors aren’t special; still a message)
    Publisher – Flux emits event messages (Observable in rx-java), immutable
    Subscriber – gets next message and handles events on error/complete/subscribe. doOnNext() is like peek() in Java 8 for debugging

    Marble diagram

    • treat events as marbles
    • Website: http://rxmarbles.com – can see different operations and drag marbles (on a computer; doesn’t work on iPad)

    Creating a flux
    Flux.just(singleValue)
    Flux.fromIterable/fromArray – to transform from collection/array
    Flux.range(startNumber, numberOfElementsInRange)
    Flux.interval(Duration.ofSeconds(n)) – emit a value every X seconds
    Flux.merge – combine streams
    Flux.withLatestFrom – unlike zip, get latest available element rather than getting the “next” one
    Flux.create – use lambda to create a listener with logic for publishing elements

    Transformations
    take
    skip
    distinct
    distinctUntilChanged-only eliminate adjacent duplicates
    filter
    map
    flatMap
    zipWith – combine existing stream with another stream in tuples
    sort

    Differences from Java streams

    • push based,not pull based
    • Attached to real time feeds
    • Conncurrent
    • Back pressure
    • Can only use once
    • Can compose streams (zip/merge)
    • Can fully work with infinite streams

    Cold vs Hot Flux
    Cold streams are finite, known elements. Hot streams are more realistics

    Must call subscribe for anything to happen (like a terminal operator in Java 8 streams)

    I learned a lot. Or reviewed a lot and learned some. Having heard this,I realize I had heard some of it before. Also, while the concepts are different than Java 8, knowing Java 8 streams really well helps retain this. The syntax similarities help. As do learning the key differences vs learning every API and not thinking about streams at all. And the demo at the end was easy to follow. This presentation reminds me of Java 8 streams in that it took me a few presentations by different people to finally grok it well.