java 8 stream performance – maurice naftalin – qcon

Posted on by
Reply

This is part of my live blogging from QCon 2015. See my QCon table of contents for other posts.

See http://www.lambdafaq.org

Background
He started with background on streams. (This is old news by now, but still taking some notes). The goals were to bring a functional style to Java and “explicit but unobtrusive” hardware parallelism. The former is more important than performance.

The intention is to replace loops with aggregate operations. [I like that he picked an example that required three operations and not an oversimplified example]. More concise/readable. Easy to change to parllelize.

Reduction == terminal operation == sink

Performance Notes
Free lunch is over. Chips don’t magically get faster over time. Intead, add core. The goal of parallel streamsisfor the intermediate operations in parallel and then bringing them together in reduction.

What to measure?

  • We want to know how code changes affect system performance in prod. Not feasible though because would need to do a controlled eperiment in prod conditions. Instead, we do a controlled experiment in lab conditions and hope not answering a simplified question.
  • Hard to microbenchmark because of inaccuracy, garbage collection, optimization over time, etc. There are benchmarking libraries – Caliper or JMH. [or better if don’t need to microbenchmark]
  • Don’t optimize code if don’t have a problem. What’s your performance requirement? [and is it the bottleneck]. Similarly don’t optimize the OS or the problem lies somewhere else.

Case study
This was a live demo. First we saw that not using BufferedReader makes a file slow to read. [not about streams]. Then we watched my JMeter didn’t work on the first try. [the danger of a live demo]. Then he showed how messing with the GC size and making it too small is bad for performance as well [still not on streams]. He is trying to shw the process of perofrmance tuning overall. Which is valid info. Just not what I expected this session to be about.

Then [after I didn’t see the stream logic being a problem in th first plae], he showe how to solve subproblems and merge them.[oddly not calling it map reduce]

8 minutes before the end of the talk, we finally see the non-parallel code for the case study. It’s interesting code becauase it uses two terminal operations and two streams. At least reading in the file is done normally. Finally, we see that the combiner is O(n) which prevents speeding it up.

Some rules

  • The workload of the intermedidate operations must be great enough to outweith the overheads. Often quoted as size of data set * processing cost per element
  • sorted() is worse
  • Collectors cost extra. toMap*( merging maps is slow. toList, toSet() is dominated by the accumulator.
  • In the real world, the fork/join pool doesn’t operate in isolation

My impressions: A large amount of this presentation wasn’t stream performance. Then the case study shows that reading without a BufferedReader is slow. [no kidding]. I feel like the example was contrived and we “learned” that poorly written code behaves poorly. I was hopingthe talk would actually be about parallelization. When parallelStream() saves time and when it doesn’t for example. What I learned was for this particular scenario, parallelization wasn’t helpful. And then right at the end, the generic rules. Which felt rushed and thown at us.

working remotely successfully – brad greenlee – qcon

Posted on by
Reply

This is part of my live blogging from QCon 2015. See my QCon table of contents for other posts.

Etsy does remote better than anyplace else he worked. Alot of people in Brooklyn office and other offices and people working from home office. He uses “remotes” as a noun as shorthand for “remote employees”.

Advice for Organizations
Number one factor for success is critical mass. Having one remote on the team doesn’t work. Having enough makes communication happen in a remote friendly format. Using chat/email/video conferencing rather than in person/physical whiteboards.

Communication

  • Chat – like IRC or Slack. They use channels; not just one on one chat like Sametime or Lync. Have #remotes channel. Virtual water cooler.
  • Shorter, more frequent interactions build stronger bonds than longer, less frequent ones.
  • Etsy is a “reply all” email culture. Use ignore/mute feature so not reading all.
  • A/V – 4 full timers work on A/V. Google hangout wasn’t enough. Switched to Vidio. Remotes type in name of room to join video conference. Remotes never late to meetings so a remote showing on screen reminds the previous meeting to end
  • Make it easy. Don’t want resistance from on sites to including remotes

Info sharing

  • Can attend talks remotely or watch them later
  • Too many people for monthly all hands to attending in person

Other tips

  • For larger meetings, have remote advocate in room – make sure speaker repeats qustions, remotes are header, et
  • Weekly one on ones are more important. Might be only (virtual) face to face. And chance to give inside info everyone in the office knows.
  • Daily standups. Video conferencing in hasn’t worked well because clustered around computer in open office. Async check ins worked better because cross time zones. [we have remotes by phone and it has been smooth. We have a room with a door though and don’t try to do video
  • Make visits special. Take time to talk, have lunch, etc. Make sure to have seat/monitor/etc
  • Try to go remote once in a while to see what person is dealing with

Policies

  • Remotes can visit any time they want and company pays for. He visits quarterly. Goal: appreciation
  • Local employees aren’t free. Company pays for desk. Should pay for remote to have good space too.
  • Responsive IT group.
  • Mailed hoodies in advance so everyone got on same day
  • Be mindful of decisions and how they affect remotes. Ex: Friday afternoon beers exclude remotes

Obstacles

  • Open office plans suck – not many quiet spaces to speak to remote
  • Remote collaboration is hard – ex: pair programming. Haven’t gotten enough experiencewith a tool to get past this. Don’t have a good virtual whiteboard tool yet.
  • Fear of remotes/fear of unknown

Advice for remotes

  • Visit at least once a quarter. Socialize when there. Visiting to talk to people; not to sit in corner and code
  • Make sure have proper work environment at home or find a co-working space
  • Some people need a dedicated space for work at home
  • Don’t forget to go “home” at end of work day. If start at 7am to sync with East Coast, don’t feel bad about ending at 3pm.
  • At disadvantage in being heard/seen, so put extra effort into being noticed.
  • Support each other; talk to other remotes
  • Mixer app – created app to randomly pair people and suggest they talk

Q&A

  • With open office, do headsets help with noise? Sometimes. Other times, cut in and out. People often don’t have a quick call to avoid disturbing neighbors
  • How do you evaluate remote people? Same as in office. What do you get done
  • How look for workers and find those good fit for remote culture? Networking. % of remotes depends on job function
  • If all remotes used to be in person and are now far away, don’t have critical mass. How address? People at company a long time to handle being only remote person better. [presumably because already have network]
  • Agile and remotes? There was a presumption this can’t work. [I disagree and commented to that effect.]

Good talk from both points of view (company/team and remotes). I also saw an underlying theme that Etsy supports remote to get the best employees. Not to save money on office space. Good intent.

engineer retention – erran berger – qcon

Posted on by
Reply

This is part of my live blogging from QCon 2015. See my QCon table of contents for other posts.

make talent #1 priority

Why should you care about retention- leaves gap in knowledge, morale and resources to do work, your competitors are making progress while you try to re-hire and re-train.

Everyone is trying to be recruited on linked in. Startups/new company IPO threat in San Francisco (ex: uber)

#1 – on day 1, make team a process
There is a taboo about talking about what happens after they leave. talking about career, doesn’t put thoughts in their head. If you know what they want, you can form a partnership and find internal opportunites as they come up.

Do you know if person wants to be a manager or individual contributor. And know if/when changes.

They recognize that everyone is going to leave linked in one day. Discussing goals makes journey ore transformative. More learning. More growth. More throwing self at these oppportunties. The idea is to try to make it so wil grown more by staying than leaving.

This gets people to talk about leaving before already decided so other opportunities n come up.Sometimes still leave,but at least a conversation.

#2 – maximize employee engagement
Competitors are doing the same thing as you. How keep engaged when having problems – financial, support time, etc.

    1. Inspire – knowing big picture makes it so tasks don’t feel repetitative
    2. Challenge
    3. Inspiration

#3 – talk about recognition

    1. People need to know they are seen and valued – don’t want to feel invisible
    2. People stay where seen/appreciated
    3. Receiving feedback is difficult for most people – even if they say “no big deal” still means something
    4. Needs to be specific, timely and personal. Not just “good job guys”

be prepared – by time person comes to management about leaving is too late.

    1. Observe – behavior change
    2. Assess – personal? work related? intrest. Reasons engineeers become unhappy include- FUD, friends leaving, lack of opportunity, nobody is listening (never solve problems), lack of recognition
    3. Connect – try to discuss/openup. Leave the office; go for a walk. People more comfortable opening up when elsewhere. The goal is to find out root cause. They might not know or be able to articulate it. Don’t argue about how they are wrong about wha perceving. Be emotionally supportative. Try to buy time to solve actual problem.
    4. Act with urgeny – if not highest priority, is it worth retaining person

Q&A

  1. What if eeryone can’t be lead? People get caught up in title. See why want? Titles are flexible. Look at impact.
  2. How deal with public feedback? Better to be individual than public. Some people prefer one-on-one communiction to public. Give example to show paying attention
  3. Some people like to move around. If understand objectives, know like to move around. And why? Is it about new languages?If just about change, they should leave. Three months isn’t enough time to learn something.
  4. What can non-managerial senior engineers do? Mentoring. Safe erson to talk to
  5. What about people who change jobs once a year? Average attention span for a new thing is 12-24 months. It’s a bell curve though [Luckily tech changes often so the “same thing” changes overtime]
  6. What happens if not ready to do what want? Coache. Help get there. Create “apprentice manager” role to give it a shot.

Interesting and a lot applies to communication in general. Including mentoring/tech leadership.Not just about retention. That said, it was mostly about management stuff.