If one using low-level one way to avoid to disable auto-commit feature in topic, and once write to debit balance topic is done we manually increase the offset. This way we ensure of app crashed, we didn't commit log, so we can restart. Nevertheless this feature of having transaction in stream is really cool.
Thanks for the explanation. I would like to point out a few points where it sounded vague and unclear to me. 1. The arrows at 3:09 are intertwined and are hard to comprehend easily. 2. 8:12, it sounds like the P0 P1 P7 messages are enqueued into the coordinator queue all at once. But I wonder if they are progressively enqueued as the transaction process steps forward. 3. 10:21, "the broker also returns a little bit metadata (for message #57) so the consumer can ignore the aborted record". But the Abort message (#61) has not been read yet. It's kind of unclear to me how Kafka includes the info "message #57 has been aborted" in the metadata. Maybe I'm just being ignorant as I'm not experienced with Kafka. Any illuminations would be appreciated.
New "revolutionary application" using standards mechanisms, that we have saw in Database and MQ for more than 40 years ... but not 2pc Surely Kafka will implement that in future ...
Abort marker, fence off, honestly so much jargans to make understand things, kafka teams should come with simple explainations
Clear introduction, thank you !
If one using low-level one way to avoid to disable auto-commit feature in topic, and once write to debit balance topic is done we manually increase the offset. This way we ensure of app crashed, we didn't commit log, so we can restart. Nevertheless this feature of having transaction in stream is really cool.
Thanks for the explanation.
I would like to point out a few points where it sounded vague and unclear to me.
1. The arrows at 3:09 are intertwined and are hard to comprehend easily.
2. 8:12, it sounds like the P0 P1 P7 messages are enqueued into the coordinator queue all at once. But I wonder if they are progressively enqueued as the transaction process steps forward.
3. 10:21, "the broker also returns a little bit metadata (for message #57) so the consumer can ignore the aborted record". But the Abort message (#61) has not been read yet. It's kind of unclear to me how Kafka includes the info "message #57 has been aborted" in the metadata.
Maybe I'm just being ignorant as I'm not experienced with Kafka.
Any illuminations would be appreciated.
how can it guarantee that the committed marker is pushed on coordinator and on all destination partitions? it doesn't seem much atomic as an operation
Lol, alot of words about "modern mq", but kafka doesn't support xa-protocol - you can't use kafka transactions with database transactions.
I would say a good start at transactions....but the explanation could have been a bit crisper.
New "revolutionary application" using standards mechanisms, that we have saw in Database and MQ for more than 40 years ... but not 2pc Surely Kafka will implement that in future ...