- 562
- 778 710
Flink Forward
Приєднався 26 бер 2015
Apache Flink® is a powerful and easy-to-use open-source system for data stream processing with a very active developer and user community, one of the most active big data projects in the Apache Software Foundation. Visit the Apache Flink website for more information: flink.apache.org/
Flink Forward is the conference for Apache Flink and the stream processing communities. Join core Flink committers, new and experienced users, and thought leaders to share experiences and best practices in stream processing, real-time analytics, event-driven applications, and managing mission-critical Flink deployments in production.
Be sure to check out www.flink-forward.org
Flink Forward is the conference for Apache Flink and the stream processing communities. Join core Flink committers, new and experienced users, and thought leaders to share experiences and best practices in stream processing, real-time analytics, event-driven applications, and managing mission-critical Flink deployments in production.
Be sure to check out www.flink-forward.org
Flink Forward Berlin 2024: Zero Interference and Resource Congestion in Flink Clusters with Kafka
As Flink workloads scale, resource congestion becomes a major challenge. While managing Flink cluster congestion is well-documented, Kafka congestion is less obvious but equally critical because it is shared with many other applications, leading to noisy neighbors that degrade its performance. The two main solutions are horizontal scaling of Kafka, which is complex and doesn’t truly isolate Flink workloads, and setting up a mirrored Kafka cluster for Flink, which increases latency, data duplication, and costs.
Переглядів: 114
Відео
Flink Forward Berlin 2024: Flink for Energy and Commodities trading business
Переглядів 11821 день тому
Flink for Energy and Commodities trading business. Oleksii Pominovskyi, Information Technology Project Manager & Sudha Ramraj, Senior Technical Consultant, Uniper, Uniper is an international energy company with operations across over 40 countries. During this presentation, Uniper's Sales and Trading IT team discuss their implementation of Apache Flink, highlighting the technology's importance i...
Apache Flink 'Bootcamp' and 'Masterclass' Sessions at Flink Forward Berlin 2024
Переглядів 10221 день тому
Hear attendees' thoughts from our Apache Flink Bootcamp and Masterclass sessions, held during Flink Forward Berlin 2024! Relive all the action from Flink Forward Berlin 2024! Watch all conference recordings online now at: www.ververica.academy/c/6e539aa939548c2a1fa14eb4dc3d5b5624390762 Discover more about Flink Forward: www.flink-forward.org/
Flink Forward Berlin 2024: Intesa Sanpaolo’s Journey to Cloud - is Batch processing still relevant?
Переглядів 10921 день тому
Intesa Sanpaolo’s Journey to Cloud - is Batch processing still relevant?. Fulvio Pascotto, Data Architect Lead & Raffaele Saggino, Senior IT Infrastructure Architect, Intesa Sanpaolo, This session covers Intesa Sanpaolo's implementation of a new cloud-based Apache Flink solution as part of their migration to off-premises systems, showcased through the release of "IsyBank." Dive into the design ...
Flink Forward Berlin 2024: Leveraging Flink to Prevent Ship Groundings in Rivers and Ports
Переглядів 9821 день тому
Streaming Up the River: Leveraging Flink to Prevent Ship Groundings in Rivers and Ports. Roee Hasson, Engineering Lead, DockTech, In this presentation, Roee showcases how DockTech uses Apache Flink to enhance maritime safety by preventing groundings and improving navigation around ports and rivers. Learn how real-time data from multiple vessels is collected, cleaned, and processed to alert capt...
Flink Forward Berlin 2024: Scaling Flink: Insights from Running Flink for five Years at Stripe
Переглядів 21321 день тому
Scaling Flink in the Real World: Insights from Running Flink for five years at Stripe. Ben Augarten, Software Engineer & Pratyush Sharma, Software Engineer, Stripe, In this session, explore the challenges of running Apache Flink in production at Stripe, where it's become the backbone for hundreds of applications performing tasks like CDC, data ingestion, ML feature computation, and real-time le...
Flink Forward Berlin 2024: Streamlining Real-Time Data Processing at Uber with Protobuf Integration
Переглядів 32121 день тому
Streamlining Real-Time Data Processing at Uber with Protobuf Integration. Yang Yang, Senior Staff Engineer & Sai Sharath Dandi, Senior Software Engineer, Uber, At Uber, Apache Flink powers real-time analytics and decision-making, processing over 4 trillion messages daily across 2000 Flink SQL jobs. Traditionally, Flink used Avro-formatted Kafka topics, while online applications used Protobuf, l...
Flink Forward Berlin 2024: Leveraging Apache Pulsar & Flink for Real-Time Vehicle Profile Management
Переглядів 13321 день тому
Driving the Future: Leveraging Apache Pulsar and Flink for Real-Time Vehicle Profile Management. Christos Anagnostakis, Enterprise Architect, Toyota Motor - Europe & Sijie Guo, Co-founder & CEO, StreamNative - Silver Sponsor, This talk explores the integration of Apache Pulsar and Apache Flink for real-time vehicle data processing in the automotive industry. Attendees will learn how this archit...
Flink Forward Berlin: Flink Is All You Need (for Digital Bank)
Переглядів 14921 день тому
Flink Is All You Need (for a Digital Bank). Maciej Bryński, Data Architect, Cledar, This presentation showcases how to build a comprehensive streaming architecture for a Digital Bank using Apache Flink exclusively. It highlights the use of Flink CDC for real-time data synchronization, Flink SQL for complex financial data analysis, and demonstrates the design and implementation of a scalable, re...
Flink Forward Berlin 2024 - Conference Highlights
Переглядів 10828 днів тому
Relive all the action from Flink Forward Berlin 2024! Watch all conference recordings online now at: www.ververica.academy/c/6e539aa939548c2a1fa14eb4dc3d5b5624390762 Discover more about Flink Forward: www.flink-forward.org/
Flink Forward Berlin 2024 Highlights
Переглядів 10928 днів тому
Relive all the action from Flink Forward Berlin 2024! Watch all conference recordings online now at: www.ververica.academy/c/6e539aa939548c2a1fa14eb4dc3d5b5624390762 Discover more about Flink Forward: www.flink-forward.org/
Flink Forward Berlin 2024 Opening Session | Live Stream
Переглядів 637Місяць тому
Join us for the highly anticipated 'Opening Session: Past, Present and Future' of Flink Forward Berlin 2024! Kicking off an incredible lineup, this session sets the stage for two action-packed conference days dedicated to Apache Flink and everything streaming data. Hear from industry experts, the masterminds behind Ververica, and inspiring voices across the data ecosystem as they share groundbr...
Flink Forward Berlin 2024 Closing Session | Reflections & Future Directions
Переглядів 126Місяць тому
This closing session brings together highlights from the Flink Forward Berlin 2024 conference, featuring key takeaways and inspiring insights from the Flink community. 00:00:00 Introduction Join the closing ceremonies of Flink Forward Berlin 2024. Hear from Alex Walden, Ververica CEO, as he shares final thoughts on the future of Apache Flink. Hear as Field CTO Ben Gamble presents a short closin...
Join Ververica at Flink Forward Berlin 2024!
Переглядів 1944 місяці тому
Ververica is proud to be the organizers of global Flink Forward conferences, including the upcoming Flink Forward Berlin 2024! Alex Walden, @ververica CEO, shares his thoughts about the 10 year anniversary of Apache Flink and what's in store for the streaming data community at Flink Forward this year. Are you ready for expert Flink training, 2 days of speaker sessions, panels led by thought lea...
Interview with Gunnar Morling, Flink Forward Berlin Program Committee Member
Переглядів 1496 місяців тому
Join Kaye and Gunnar Morling, Senior Staff Software Engineer at Decodable and one of this year’s Program Committee Members for Flink Forward Berlin 2024! Call for Papers is open until 30 May Tune in to this insightful discussion to get to know Gunnar! Understand from one of the Call for Presentations (CFP) reviewers themselves the tips and tricks for submitting a great talk! Do you have a data ...
Interview with Giannis Polyzos, Flink Forward Berlin Program Committee Member
Переглядів 1787 місяців тому
Interview with Giannis Polyzos, Flink Forward Berlin Program Committee Member
Interview with Xintong Song, Flink Forward Berlin Program Committee Member
Переглядів 2177 місяців тому
Interview with Xintong Song, Flink Forward Berlin Program Committee Member
Use Cases Breakout Track Sessions - Flink Forward 2023
Переглядів 1798 місяців тому
Use Cases Breakout Track Sessions - Flink Forward 2023
Technology Breakout Track Sessions - Flink Forward 2023
Переглядів 1168 місяців тому
Technology Breakout Track Sessions - Flink Forward 2023
Operations Breakout Track Sessions - Flink Forward 2023
Переглядів 818 місяців тому
Operations Breakout Track Sessions - Flink Forward 2023
Industry Breakout Track Sessions - Flink Forward 2023
Переглядів 918 місяців тому
Industry Breakout Track Sessions - Flink Forward 2023
Ecosystem Breakout Track Sessions - Flink Forward 2023
Переглядів 668 місяців тому
Ecosystem Breakout Track Sessions - Flink Forward 2023
Highlights of Flink Forward Seattle 2023
Переглядів 58410 місяців тому
Highlights of Flink Forward Seattle 2023
Keynote: Quo Vadis Stream Processing? Dude, where is this streaming to?
Переглядів 45010 місяців тому
Keynote: Quo Vadis Stream Processing? Dude, where is this streaming to?
Keynote: Flink Unveiled - Journeying from DataStream to Streamhouse
Переглядів 1 тис.10 місяців тому
Keynote: Flink Unveiled - Journeying from DataStream to Streamhouse
Keynote: The Vision and Practice of Stream and Batch Unification - LinkedIn's Story
Переглядів 81710 місяців тому
Keynote: The Vision and Practice of Stream and Batch Unification - LinkedIn's Story
Keynote: Riding the Streams: Powering Stream Processing at Cash App with Ververica
Переглядів 55810 місяців тому
Keynote: Riding the Streams: Powering Stream Processing at Cash App with Ververica
Sponsor Session: Stream Processing in AWS by Example
Переглядів 17510 місяців тому
Sponsor Session: Stream Processing in AWS by Example
Sponsor Session: Real-Time Marketing Segmentation and Product Journey Computation at Business Scale
Переглядів 11910 місяців тому
Sponsor Session: Real-Time Marketing Segmentation and Product Journey Computation at Business Scale
Sponsor Session: Building an Integrated Data Streaming Platform with Pulsar and Flink Operations
Переглядів 11110 місяців тому
Sponsor Session: Building an Integrated Data Streaming Platform with Pulsar and Flink Operations
When are we going to get a paid affordable training for bootcamp ?
Just want to ask, according to fluss cluster architecture on 1:28:08, Does Fluss containes paimon or iceberg within its source code? (Like Flink CDC uses Debezium) or those two table formats just pluggable elements, working independently with fluss itself? Also, If Fluss uses arrow, Is C++ a main language for fluss? (I guess Rust maybe...)
Currently, Fluss has a Paimon integration (Iceberg is in progress), it's included in the source code, and it is pluggable. Fluss provides Java and SQL based interfaces at the moment. Fluss plans to have Rust sdk in the future, so that it can be easily integrated with Rust/C++ query engines.
It's super hard to follow. Such a pity because it's a great explanation of pyflink sdk.
Holographic representation rocks!
robot talks on Flink
Nesito yo estoy en mexico no puedo salir demexico canselaron los biajes 29 ya estoy aya sanfrasico undiscupa
Buenas noche cuando kieres aser bisnes los utimo deste mes pliz cuando
Great insights
the accent!!!!!!!!!!!!
I am confused. How is this different from Kafka Streams and KTable?
If you have multiple fraud-counter http servers or multiple merchant-scorer http servers running (for high-availability), does the storage synced across those servers? If so, how does the syncing works?
The question about partial deserializing wasn't bad. Interesting.
where can I find the link of demo at the description? @8:38
Very prescient talk
I'll build on this
Good talk but the audio is not that great.
no wonder 781 people watched this in last 3 years....
Very useful introduction of Apache Flink. I was able to test Flink features using Google Colab.
Hi, is there a way to improve the subtitles ? It's very hard to understand the audio and subtitles don't help at all.
What's the disadvantage of solving this problem with Kafka stream?
운전점수 확인 정말 유용하게 사용 중인데 엄청난 기술력이 있었네요.
cool
Please an update of pyflink processing kafka topics
Hi, that's really interesting! I have 2 questions: How do you handle race conditions? In the example, if the counter and merchant functons respond at the same time, they could both think they're the first to answer and we end up with both in cache but not sent to the model, or does the context run one message at a time guarantied, like in Akka? How do you handle resilience? THe 30 days delay on a message is a great feature made simple here, but let say the app dies, and is rebooted, is it going to be lost? Is there a sync to something like S3 for the state?
I guess from 30:00 onwards Flink have addressed this with HybridSource? Havent used it yet but seen a few talks on it
The second guy is brilliant
Noice.
Thank you for sharing the video. However, it seems that Flink is not performing well in terms of application upgrade. For instance, if we have a lambda that consumes Kinesis stream, a lambda update won't result in any downtime or duplicate records and everything will be handled smoothly. On the other hand, Flink application requires us to handle several aspects such as save point, checkpoint, blue-green deployment, dual stack, and duplicate data on our own. Even Amazon Managed Flink and KDA may experience downtime during application upgrade. I am optimistic that Flink can do better than current one
This is amazing, we have a similar usecase in our company and i was confused between feast, sagemaker. I was also worried about explaining the infrastructure requirements to PMs, managers if we chose Feast. This has simplified the whole setup and ideal usecases for small startups.
Hello, thank you for the video
Whether there is need to install RocksDB in kubernates cluster or just adding state.backend: rocksdb ... these line in conf/flink_conf.yaml is enough?
Hi, thanks for course. I have a question about first demo, there is no output when "table.limit(10).execute().print()" code runned. Have you got any iade about this problem ?
Great Presentation!! Neatly sequenced with how the problems were conquered and the way forward.
What kind of join do you propose between 2 streams if there are more than 1 key possible. For the example you explained suppose one needs to join 2 streams, order and sale, either on sale_id or userid.
Great video! Thanks
Can you please create a sample project on this..
How does Flink CEP Manages it state? We take patterns in real time.... but flink does not removes pattern from state whose window time is over?.. is there any solution
0.75x speed should be a lot better
This is just a brilliant piece of information!
How does unordered input cause checkpoint failure. Can you please elaborate on the use case which caused this failure. My understanding is that even if you add allowed lateness to consider unordered event, even then checkpoint will keep happening to store current state.
does anyone know where I can find the state management processes that TableAPI uses, I understand that is under the hood and is being done automatically, but was hoping to understand further how it does so, such that I could use it in my own project.
How does the flink application in blue environment catch-up with the lag introduced due to re-deployment ?
Thank you for the video. A quick honest feedback. Presentation is disorienting. I would like to hear it like: * What Flink is * Who uses it? why? (not generic, specific) * What you are introducing in this video, (story, expectation) * Demo * show requirements * show the solution * show what's implemented * explain what you do the things you do * Conclude * What if you don't use flink (flink vs hard way to do it)
agreed. this is very poor compared to confluent kafka tutorials.
Very useful and there is not such less learned content about Flink implementation sharing. Wandering about the new version of Flink will resolve any of those issues yet ?
only keep last 2 checkpoints . Is is configurable for incremental checkpoint ??
It feels not correct that the optionally scaling apply to any of the "Aggregation" and "Windowed Retention" as the state size is promotional to the number of keys which may varies depending on the input. If we say the group-by keys space are finite and we have an estimate for that, then it can fit in optionally scaling.
I installed flink but can not access the UI.
great talk
Hello, this is excellent video. Very helpful. I have e requirements to insert data into Parent and Child table, can you please suggest or share a reference o how to do this?
Flink Forward is becoming my favourite conference of all. To the point, tech stuff and no jargon’s. Thank you Brent for all the valuable points.