StreamNative
StreamNative
  • 394
  • 231 936

Відео

Data Streaming Summit 2024 Highlights
Переглядів 4214 днів тому
Check out key highlights of Data Streaming Summit 2024! You can watch full sessions at ua-cam.com/play/PLqRma1oIkcWgN9agdJ0DQhX2gPf8K2ynk.html
Data Streaming Summit Morning Keynote
Переглядів 4014 днів тому
Data Streaming: Past, Present, and Future Ben Gamble, Field CTO, Ververica Dustin Nest, Technical Trainer, StreamNative Hugo Smitter, Principal Platform Architect, FICO Kundan Vyas, Staff Product Manager, StreamNative Matteo Merli, Pulsar PMC Chair and CTO, StreamNative Naveen Punjabi, Director of Partnership, Google Cloud Sijie Guo, Co-founder and CEO, StreamNative
Measuring Physical Operations in Real Time
Переглядів 2514 днів тому
Leveraging existing cameras infrastructure, Safari AI uses computer vision to detect, track and measure KPIs like guest occupancy, staff engagement, parking utilization, queue wait time and more for our Enterprise customers. We use Pulsar as our main data storage and streaming backbone to collect ML data from edge devices and process it with Flink to timely serve and alert our customers with me...
Scaling Kafka Replication at Uber's Monumental Scale
Переглядів 11814 днів тому
Uber employs one of the largest Apache Kafka clusters in the world, acting as the pivotal hub connecting the entire Uber ecosystem. We aggregate system metrics, application logs, database changelogs, and event data from rider/driver/eats apps. This intricate process guarantees the seamless downstream availability of critical data through the Kafka platform. In 2016, we pioneered our own Kafka r...
Truly Scalable Operational Data Layers for Data Pipelines
Переглядів 2514 днів тому
As streaming systems scale to match the ever-increasing volumes of data in applications, how should data engineers think about the scale properties of the sources and destinations of streaming data? In this session, we’ll discuss scaling from the perspective of an operational data layer (both a destination and a source), or - more tangibly - the global source of truth for data aggregated from a...
From Terabytes to Petabytes: Scaling Strategies with Apache BookKeeper
Переглядів 3414 днів тому
The demand for scalable data storage is relentless as businesses grapple with exponentially growing data volumes. Salesforce's innovation with Apache Bookkeeper has emerged as a game-changer, pushing scalability boundaries. The traditional way Bookkeeper uses its metadata store - Apache ZooKeeper, struggled to keep pace as storage scaled beyond the Petabyte mark. Salesforce's innovation on the ...
Uncorking Real-Time Analytics with Pulsar, Pinot and Flink
Переглядів 4114 днів тому
Imagine a world where analytics is not just for the boardroom but for everyone, everywhere, every moment. Pinot is not your average OLAP database; it’s a turbocharged engine designed to power features that users interact with, delivering insights faster than you can say “real-time.” We’re talking about lightning-fast queries, sky-high concurrency, and data fresher than your morning coffee. In t...
From Swamps to Lakes: Building a Cleaner Data Ecosystem
Переглядів 2414 днів тому
As companies scale their data streaming operations, many find themselves stuck in a data swamp-an unmanageable mess of low-quality, disorganised data. In this session, we’ll explore how to transform these swamps into clean, high-value data lakes by focusing on early-stage data quality, controls, and automation. Using real-world examples, we’ll show how you can reduce risk, improve operational e...
Harmonious Integration of Pulsar with ClickHouse & StarRocks
Переглядів 8914 днів тому
When building traditional real-time data analytics systems, we primarily integrated Kafka and Flink through tightly coupled, predefined static aggregations. This approach had several limitations, making it difficult to respond flexibly to surges in social events or recovery scenarios, and scaling was highly constrained. To address these issues, we adopted Pulsar, which provided messaging and li...
Powering Billion Scale Vector Search at Milvus with Apache Pulsar
Переглядів 5214 днів тому
Milvus vector database is the data infrastructure behind many GenAI applications including RAG chatbots, image/video search and recommender systems. This talk will share how Milvus, the open-source vector database, leverages Apache Pulsar in its distributed architecture to deliver high-performance vector search at billion vector scale. The pub/sub design pattern decouples the data ingestion and...
How to Process Streaming Data: A Quick Guide
Переглядів 6314 днів тому
In an era where data flows in real-time, organizations face the challenge of efficiently processing streaming data to derive timely insights. This talk, “How to Process Streaming Data: An Unbiased Guide,” provides a comprehensive and practical overview of the leading stream processing tools to handle high-velocity data streams. We will explore key concepts, including event-driven architectures,...
The Power of Apache Pulsar. Harnessing Dapr to build high scale messaging at FICO
Переглядів 6414 днів тому
Learn about FICO’s experience migrating from Apache Kafka to Apache Pulsar with the help of Dapr to build high scale messaging services into our platform. We cover how we harnessed Dapr for seamless, efficient, and scalable Pulsar integration. We’ll explore: • An overview of the FICO® Platform and strategic business objectives. • Key contributions to the Dapr community, enhancing its support fo...
MotherDuck & the AI Lakehouse
Переглядів 3514 днів тому
Follow along as I talk about what we mean by "Big Data is Dead" - and what it means for your data warehouse. The world has changed and we cannot continue to use old patterns and expect success! By leveraging DuckDB with MotherDuck's powerful serverless compute, we can extend the data warehouse into the data, reading not only parquet, csv, and json but also Iceberg & Delta. Furthermore, since we...
Take control of your business monitoring with Apache Pulsar, Apache Flink and RisingWave
Переглядів 4714 днів тому
Vehicles in the automotive industry generate an impressive amount of data during their lifetime. Aircraft engine maintenance is a long, complex process that generates data that allow tracking progress and identify issues. Despite their differences, these use cases share the same need for efficient processing of the produced data, in order to monitor the processes and ensure the expected quality...
VERA: The Engine Revolutionizing Apache Flink
Переглядів 4414 днів тому
VERA: The Engine Revolutionizing Apache Flink
Iterable's Data Pipeline with Pulsar
Переглядів 58514 днів тому
Iterable's Data Pipeline with Pulsar
Achieving Data Interoperability with Real-time Stream Processing
Переглядів 3614 днів тому
Achieving Data Interoperability with Real-time Stream Processing
Lessons from building streaming first Data Infra on a tight budget at CityStorageSystems
Переглядів 3614 днів тому
Lessons from building streaming first Data Infra on a tight budget at CityStorageSystems
Process Realtime State Efficiently with Apache Pulsar and Apache Spark transform WithState Operator
Переглядів 2714 днів тому
Process Realtime State Efficiently with Apache Pulsar and Apache Spark transform WithState Operator
Learnings from running high-volume data streaming with Kafka & Flink
Переглядів 7114 днів тому
Learnings from running high-volume data streaming with Kafka & Flink
Making Kafka Connectors Dance with Apache Pulsar
Переглядів 9314 днів тому
Making Kafka Connectors Dance with Apache Pulsar
Kafka Mirror Maker vs Pulsar Geo-Replication: Best Practices for Disaster Recovery
Переглядів 2414 днів тому
Kafka Mirror Maker vs Pulsar Geo-Replication: Best Practices for Disaster Recovery
Unlocking Real-Time Insights: StreamNative & Timeplus Integration for Instant Analytics
Переглядів 4828 днів тому
Unlocking Real-Time Insights: StreamNative & Timeplus Integration for Instant Analytics
Three Challenges of Data Streaming in the Cloud and How to Overcome Them
Переглядів 62Місяць тому
Three Challenges of Data Streaming in the Cloud and How to Overcome Them
Data Streaming Summit 2024 Morning Keynote
Переглядів 2242 місяці тому
Data Streaming Summit 2024 Morning Keynote
Cost Effective Kafka: Strategies for Cost Reduction and Increased Efficiency
Переглядів 642 місяці тому
Cost Effective Kafka: Strategies for Cost Reduction and Increased Efficiency
SweetStreams Episode 4: Real-Time AI with Zilliz and StreamNative, guest Timothy Spann of Zilliz
Переглядів 772 місяці тому
SweetStreams Episode 4: Real-Time AI with Zilliz and StreamNative, guest Timothy Spann of Zilliz
StreamNative Q32024 Roadmap Webinar
Переглядів 1393 місяці тому
StreamNative Q32024 Roadmap Webinar
StreamNative Q42024 Product Roadmap
Переглядів 683 місяці тому
StreamNative Q42024 Product Roadmap

КОМЕНТАРІ

  • @uwontlikeit
    @uwontlikeit 19 днів тому

    thanks for the video. is there any good UI for Pulsar? pulsar-manager is barely working

  • @FLaNK-Stack
    @FLaNK-Stack 2 місяці тому

    This is awesome

  • @kirylvalkovich
    @kirylvalkovich 2 місяці тому

    Timecodes, please!

  • @liquidmetal718
    @liquidmetal718 2 місяці тому

    Nice

  • @kirylvalkovich
    @kirylvalkovich 2 місяці тому

    Thanks for the episode. Timothy is always fun to listen to and watch :)

  • @khemendrabhardwaj2552
    @khemendrabhardwaj2552 4 місяці тому

    Hey, I want to adjust broker configuration (bundle splitting algo ) , but those attributes not present in pulsar helm chart , what should I do ?

  • @deltagamma1442
    @deltagamma1442 4 місяці тому

    You said there are pulsar functions using node js too? I don't see it anywhere. Could you please clarify.

  • @faranahmadk7401
    @faranahmadk7401 5 місяців тому

    Thanks for the demo. Can you share the producer app?

  • @user-rc6bh4oe2p
    @user-rc6bh4oe2p 5 місяців тому

    Thank you for the video. The explanation is good. i am looking for more information apart from running few commands. Can someone please let me know any other resources from stream native for pulsar? As i am a beginner, i would like to explore more.

  • @canonicalizer
    @canonicalizer 6 місяців тому

    Thank you for this short introduction video for Apache Pulsar.

  • @bhaskardabhi
    @bhaskardabhi 6 місяців тому

    Can this functions be used to process data like fetching data from salesforce and store it into database or storing data from CSV to database?

  • @saeedafzal
    @saeedafzal 7 місяців тому

    Very useful information. Thanks, Julien. Would be helping if you can create a video on schema based transactions instead of simple text messages.

  • @gjanardh
    @gjanardh 8 місяців тому

    This is a good resource to understand Pulsar Broker and Bookie GC. Is there a English version of this presentation or video?

  • @שלומיטובול
    @שלומיטובול 10 місяців тому

    great video Asaf !

  • @100xbiz
    @100xbiz 10 місяців тому

    anyone using this ?

  • @kevinnguyen163
    @kevinnguyen163 11 місяців тому

    Great walkthrough, thank you.

  • @jamessimon4804
    @jamessimon4804 Рік тому

    Great video

  • @sharofiddinpardayev5555
    @sharofiddinpardayev5555 Рік тому

    Great! Can you upload source code to public version control?

  • @arulprakasan1697
    @arulprakasan1697 Рік тому

    Why There is no Transaction Support for C++ Client, As this Transaction support is the biggest problem due to which many existing products not using Pulsar !! please work on it !

  • @qiwei7361
    @qiwei7361 Рік тому

    great!pulsar

  • @kirylvalkovich
    @kirylvalkovich Рік тому

    Good problem and the solution plan. I hope that in 2024 we'll be able to try it in action. Having the mentioned links in the UA-cam video would be very useful. It applies to every video. Also having presentations on SlideShare would be just wonderful! > Pulsar Function authors will get OTel interface, and use any type they want. Nice! Also a bit off-topic. Is there any work going on to support WASM in Functions?

  • @kirylvalkovich
    @kirylvalkovich Рік тому

    It would be great if the video description included time codes.

    • @streamnative
      @streamnative Рік тому

      Thank you for your comments! We added the time codes on the description.

  • @r0shav
    @r0shav Рік тому

    1. Stop on error - needs alert system and manual invervention 2. Ignore error - may cause data loss. 3. Retry - out of order processing, potential infinite message redelivery 4. DLQ - out of order processing again, finite number of redeliveries. 5. Retry queue

  • @anantababa
    @anantababa Рік тому

    nice ne , can you please these sample code

  • @anantababa
    @anantababa Рік тому

    please share the sample project repo

  • @SathiskumarPerumal-d1n
    @SathiskumarPerumal-d1n Рік тому

    Hi, I have followed these steps and completed the Geo replication setup. When i try to produce message from US-WEST cluster , I am not able to consume from US-EAST. Can you please help me to fix the issue

  • @kirylvalkovich
    @kirylvalkovich Рік тому

    Thank you, Julien. We need more materials with such production quality on Pulsar!

    • @julienjakubowski6272
      @julienjakubowski6272 Рік тому

      Thank you so much for the feedback! I'm glad you enjoyed the video! This is not the last video: stay tuned 🙂

  • @dovuddev5938
    @dovuddev5938 Рік тому

    Hello. Please, Send this project here for a sample

  • @thomasechen
    @thomasechen Рік тому

    This stuff is really nice

  • @karizadk3875
    @karizadk3875 Рік тому

    Nice talk Tarun!

  • @tianzicai8443
    @tianzicai8443 Рік тому

    Nice work! The first few synchronous "pulls" from the newly created Pub/Sub subscription did not work because the first "hello" was published before the subscription was created. Generally, synchronous "pull" is guaranteed to return messages even if you set how many messages you want to receive.

  •  Рік тому

    Amazing introduction. Thank you

  • @kirylvalkovich
    @kirylvalkovich Рік тому

    Such videos are very useful. Thank you.

  • @kirylvalkovich
    @kirylvalkovich Рік тому

    Thanks to Sijie and Matteo for the presentation!

  • @kellyfj
    @kellyfj 2 роки тому

    Very cool - blue/green deployments should have better support within Pulsar.

  • @Alex-ph1th
    @Alex-ph1th 2 роки тому

    🤔 Promo SM!!!

  • @kirylvalkovich
    @kirylvalkovich 2 роки тому

    Thank you for a great talk!

  • @kellyfj
    @kellyfj 2 роки тому

    Yeah Sijie and Addison!

  • @priklyucheniya-elektronika
    @priklyucheniya-elektronika 2 роки тому

    How about commenting at least something you're doing on the screen? And reduce speed a little? Material itself is interesting. Demo is far from the good one

  • @iamrmin
    @iamrmin 2 роки тому

    With this receive approach, you are processing messages one by one. What if you get 10k messages all at once. it takes a lot of time to process them. how can we process them in batch? like a Semaphore of 20 and it call receive only when any semaphore becomes available.

  • @JCArtuso
    @JCArtuso 2 роки тому

    How to find documentation (ebook) and examples about Apache InLong? It seems amazing but I couldn't get good documents about it.

  • @mojo8821
    @mojo8821 2 роки тому

    👀

  • @mustafataskir7262
    @mustafataskir7262 2 роки тому

    Hi Ioannis, If we wanted to do a load test, how could we do it? It is said in the document that we can do it with pulsar-perf.

  • @somratdutta
    @somratdutta 2 роки тому

    please provide timestamps

  • @somratdutta
    @somratdutta 2 роки тому

    nice

  • @jamessimon4804
    @jamessimon4804 2 роки тому

    Java is so unnecessarily overly complicated

  • @tratkotratkov126
    @tratkotratkov126 2 роки тому

    Thanks for sharing ! This is gold !

  • @idealumesh
    @idealumesh 2 роки тому

    no audio for few key minutes :(