Teaching old Streams new tricks By Viktor Klang
Вставка
- Опубліковано 4 жов 2023
- Have you ever wanted to perform an operation on a java.util.stream.Stream only to find that the existing set of operations didn't provide what you needed-forcing you to break out early from the Stream and perform the logic outside of it?
As a matter of fact, java.util.stream was the first JDK API designed with lambdas in mind and was initially released in JDK 1.8, and ever since then it has allowed Java developers to ergonomically manipulate potentially unbounded streams of data.
While the Streams API offers a rich set of operations out-of-the-box, it was always envisioned to offer the capability for user-defined intermediate operations, yet at that time the right way to expose that hadn't been discovered.
Until now.
In this session we'll demonstrate Gatherers, a potential future API to create and reuse intermediate operations for java.util.stream.Stream, enabling you to perform custom transformations as natural steps in your day-to-day stream processing.
Together, let us teach our old streams some new tricks!
VIKTOR KLANG
Viktor is a software architect in the Java Platform Group at Oracle.
In his professional career, spanning over more than two decades, he has focused on enabling developers to solve problems with maximal productivity without sacrificing maintainability.
Specializing in developer ergonomics in the realm of concurrent, parallel, and distributed programming; he has contributed to the standard libraries of several major programming languages, been involved in more than a dozen Open Source projects, and has spoken at numerous conferences and universities. - Наука та технологія
Love Viktor's presentation style and content. Learnt loads from this presentation. It's great news to see so much progress in the JDK!
I really appreciate that, @dazraf-thank you!
I am only 12 minutes into this presentation and I've learned a lot. Great work Viktor
Thank you so much!
Love the inclusion of an andThen api!
Amazing work! Showed me operations that I didn't even think I needed but I do now.
That's awesome to hear!
Amazing work. Looks a lot inspired by Reactive frameworks. But, this is much simpler and sounds like this would be a great addition to Java.
Great presentation!
Thank you! :)
Cool stuff, is there any way to play with it, other that implementing it myself?
I hope this will come as an incubator api soon.
Amazing presentation.
Thank you!
I see this gather is somewhat similar to spark functions data frame api's expr functions. Happy that, we are taking good things...
Nice and detail explain
Thank you, @hepin1989!
I expected the Gatherers to include all the methods from the java-util-stream-Stream as well (perhaps in a different class like `DefaultGatherers` or `CoreGatherers`). This way, I could also use these methods to create connecting Gatherers with the `.andThen` function.
Will Collectors do something about short#-circuiting as well, or will we have to short circuit in gatherers and then reduce? I.e. implementing allMatch.
I personally don't think that's possible to do as the evaluation strategy for Collector is already established and no current evaluation implementation checks for short-circuting. As such, I suspect that it would be hard to implement it in a backwards-compatible way.
When will we have these things in Java to play with?
If Viktor Klang talks about this, I'd like to see the comparison with Scala.
Follow the progress of JEP-461: Stream Gatherers for more info
Love this as I've run into many cases where I've wanted to extend the streams API. I even ended up building something allowing arbitrary operators based on mapMulti since it was the first official operator to provide access to the downstream sink but it had many shortcomings and didnt parallelize, etc. I kept expecting him to point to the library that has this but then disappointed this is not available yet 😢. Looking forward to this. Some feedback though... gather and gatherer don't seem like the right names, why not operate and operator? Also, where does this fit with reactive streams API? Java streams are great but I've found gaps like it being pull-based (and not push-pull with backpressure like reactive streams) and doesn't inherently support asynchronous operations that have made me use reactive streams APIs instead. It would be good to incorporate these concepts into Java streams. That with virtual threads would truly kill reactive streams.
Java Streams are push-based (in general) over a Spliterator. As for naming it is always a challenge-as you can imagine, a lot of different names have been suggested (would be a long conversation to fully elaborate all the concerns to address). What gather+gatherer had going for it is its relationship with collect+collector :)
5:10 Ngl, i am a little bit conused.. reorder() exists as sort(), duplicateConsecutive() exists with distinct(), groupdUsing() exists through collect() and of course mapConcurrent() exists trhough map(). Yes.. you have to do some extra stuff instead of just putting a your lambda operation as argument but... it exists. All of that would be just another method name or syntax.
My thinking is that reorder() can emit elements as they are determined to be next in order (using sequence numbers or otherwise), whereas sort can only start to emit once the entire upstream has been consumed-this will never terminate for unbounded streams. deduplicateConsecutive() is definitely not the same as distinct() as you only want to eliminate adjacent duplicates, and collect() is a terminal operation and not an intermediate one. And, unfortunately, mapConcurrent() cannot be implemented using map() since you need to flush during finish, and the contract of map() is also that the supplied function is stateless and non-interfering.
I hope that helps :)
Looks like most of this featurers Reactor already supports
So... Gatherer is in Java 21 yet?
No, it is not in Java 21
Maybe preview in 22
I would anticipate having both `Gatherer::endThen` and an alias `Gatherer::gather = Gatherer::endThen`.
Why, you ask? This would enable me to effortlessly extract a group of Gatherers from an existing Stream pipeline. Otherwise, I would have to go through the process of renaming them step by step, changing from `.gather()` to `.endThen`, while retaining the same functionality.
I decided against doing so because I found that it made code reviews more difficult to understand (am I looking at Stream composition or Gatherer composition here?).
Reusing the name for composition used by things like Function made more sense to me.