Replace Kafka, RabbitMQ, Redis and more w/NATS JetStream | Rethink Connectivity Ep 10

In this episode, Jeremy presents a high level overview on how NATS JetStream works, how it differs from Core NATS, and walks through building a JetStream publisher and consumer application in Go.
NATS is a connective technology powering modern distributed systems, unifying Cloud, On-Premise, Edge, and IoT.
Join the NATS Community on Slack: slack.nats.io
Learn More about NATS at docs.nats.io/

[@ch08532]

Excellent presentation skills. Presenting the material in a simple way. Thank you!

[@JohnMcclaned]

Would love to see a video about the upper limits of NATS and jetstream, at what point does it fall apart (if that even exists)? What limits can you push it to? Where are the red lines? It sounds too good to be true?

[@nchomey]

Great timing for this series as I'm just starting to look into using NATS Jetstream for an application that I'm developing! I look forward to the upcoming videos in the series to learn more about it all!

[@maciejgebarski7850]

Great video. I wait for the next videos describing the possibilities in more depth from a golang perspective

[@SynadiaCommunications]

You got it!

[@maciejgebarski7850]

​@@SynadiaCommunications I seen :-) Great job. I can't wait for next content.

[@aniruddhakulkarni3663]

Thanks for help 🙏

[@jaysistar2711]

Thanks! I'm hoping domains and how they act under disconnection and reconnection is covered!

[@suikast420]

Great talk.
Is there a way to publish batch of events in atomic way like in kafka?

[@zxc15zxc]

Thanks, this is great. what is the latest version for NATS docker image with new jetstream feature ?

[@SynadiaCommunications]

In terms of the new JetStream API. All those new updates exist in the NATS client libraries

[@holgerwinkelmann6219]

Thanks Jeremy, have you the comparison to redis? IMHO redis is a bit more flexible in query for data?

[@SynadiaCommunications]

Yup. Redis has a bit more value based semantics. An upcoming version of NATS server will actually begin to add more functionality in this area

[@peterlarsson4875]

Soo, either i can lazy create my streams and consumers in the code or create them upfront through admin? The overhead of doing it lazy is small?

[@SynadiaCommunications]

Yup. Creating streams and consumers on the fly is really quick

[@hamzash3660]

what's the name of the tool ur using for drawing ?

[@maciejgebarski7850]

Hi,
I have a question.
I create stream in this way:
stream, err := js.CreateStream(ctx, jetstream.StreamConfig{
Name: "et",
Subjects: []string{"et"},
})
if err != nil {
panic(err)
}
and ran the application in two different terminals. It works, but I was expecting the "js.CreateStream" function to return an "JSErrCodeStreamNameInUse" error. However, it didn't, and I'm not sure why. Could you explain this to me? Thanks.

[@SynadiaCommunications]

Best bet would to ask in the NATS Slack community so we can help debug more easily

[@ihsanpro9406]

Does that mean redis can be replaced with jetstream ? We using nats core and redis for somewhat data persistency

[@SynadiaCommunications]

Yes. Redis can be replace by using JetStream KV. Stay tuned for the next video, we'll go in-depth on KV

[@EmmanuelOga]

Are the messages deletes once a consumer ACKs the message? I assume this is configurable when you created the stream?

[@SolveJohnsen]

You can configure the stream for how long messages (0 to infinity) last and what kind of deletion policy you want when the stream (disk) is full, newest or oldest message.
You can when you've read a message after the ACK tell your consumer(durable) to rewind and get a re-delivery (replay) of all messages if you'd like at any time.

[@SynadiaCommunications]

Yes. This is all configurable when creating a stream. In a follow up video I’ll go through all the stream configuration options

[@trashmail8]

@@SynadiaCommunications Awesome! This and that is exactly the kind of information I'd love to learn! There's so much potential in this technology, but it isn't fully covered by the documentation at current time. Same for other questions raised here: what are the limits? How does it store data on disk for K/V and objects, etc. etc.
Great video, looking forward to future episodes because I ❤ NATS! 😃

[@Lola1899]

Deleting messages upon customer ack is just one of the operating mode (in this case one of the 'retention policies') when you want to use the stream as a 'working queue' (i.e. like a distributed queue). There even an 'interest' retention policy where the message can be consumed by more than one consumer (group of client applications) and the message is deleted only when all of those consumer (groups) have ack'd the message.

[@BryanChance]

I'm a bit confused about consumer and subscriber. Is consumer like a "subscriber group"?

[@SynadiaCommunications]

A consumer is related to JetStream and represents a stateful view, or window, into a Stream.
A subscriber is a concept in Core NATS that is subscribed to any subject.
NATS + JetStream gives you the ability to have both persistent and fire and forget messaging, hence the existence of both terms

[@hansschenker]

Is Jetstream not a climate thing?

[@miloszkrajewski8240]

You might think this is Erlang, but no, this is lets-handle-this-err-lang ;-)

[@SynadiaCommunications]

lol. Love this take