Kafka for .NET Developers - Ian Cooper - NDC Sydney 2024

This talk was recorded at NDC Sydney in Sydney, Australia. #ndcsydney #ndcconferences #developer #softwaredeveloper
Attend the next NDC conference near you:
ndcconferences...
ndcsydney.com/
Subscribe to our UA-cam channel and learn every day:
/ ‪@NDC‬
Follow our Social Media!
/ ndcconferences
/ ndc_conferences
/ ndc_conferences
#dotnet #architecture #microservices #kafka
Kafka is a low-latency streaming solution with a rich ecosystem of tools such as Kafka Connect and Flink, but is less well-known to .NET developers. In this session we will introduce Kafka, explaining the mysteries of records, offsets, SerDes and schema registries, in-sync replicas, partitions and tools like Connect and Flink. There will be code, as we work with examples in .NET of using as a messaging solution in your applications. By the end of this session you should feel comfortable with the concepts required to use Kafka as a .NET developer.

[@streambased]

Nice talk, I find the produce/consume pattern hard to with and limiting at times when I just want to grab some data from Kafka. Let's say I just want to grab all the messages for a particular filter criteria and time range, it ends up being a read from beginning to end... We built Streambased to provide an ODBC layer on top of a Kafka cluster so that you can use regular SQL to extract streaming data into your C# apps. After all, most streaming problems can be "downcast" to batch and become a lot easier to work with.