what if there is no data coming for any of the groups and wartermark doesnt progress . how will events get timedout in that case if we use eventtimetimeout ?
Time 5:53 sec , speaker says maintaining the state thru checkpointing allows fault tolerance in both stateful and stateless streaming. As far as I understand we don't maintain the state in stateless streaming. How come stateless streaming become fault tolerant ?
I think he might have meant that you'll still keep your offset saved through the checkpoint (assuming the source has some type of "offset") so in case of failure you'll continue from the last checkpoint. If you also have some state saved through aggregations, etc. Then as far as I understand, the "checkpoint" will save both the offset and the state (using WAL)
We performed Streaming joins using Kafka Streams but faced a barrier with handling late data. Now, eagerly waiting for Spark 2.3 ;)
14:21 with regards to deduplication, why not just use delta merge/upsert ?
what if there is no data coming for any of the groups and wartermark doesnt progress . how will events get timedout in that case if we use eventtimetimeout ?
Explained pretty well
great talk
Time 5:53 sec , speaker says maintaining the state thru checkpointing allows fault tolerance in both stateful and stateless streaming. As far as I understand we don't maintain the state in stateless streaming. How come stateless streaming become fault tolerant ?
I think he might have meant that you'll still keep your offset saved through the checkpoint (assuming the source has some type of "offset") so in case of failure you'll continue from the last checkpoint.
If you also have some state saved through aggregations, etc. Then as far as I understand, the "checkpoint" will save both the offset and the state (using WAL)
What a weird way to make your speaker stop presenting at the end
guys any tutorial on how i can stream crypto data from different sources?
The guy at the end was NOT funny …