- 249
- 126 299
OpenSearch
Приєднався 3 чер 2022
OpenSearch Project is a community-driven, Apache 2.0-licensed open source search and analytics suite that makes it easy to ingest, search, visualize, and analyze data. Developers build with OpenSearch for use cases such as application search, log analytics, data observability, data ingestion, and more.
OpenSearch Deep Dive - Indexing (part 2)
Continuing the journey started in part 1 (ua-cam.com/video/Nc5Za3KJzYk/v-deo.html), we get into the BulkOperation, which is responsible for dispatching parts of the indexing bulk request to individual shards. Along the way, we look at how aliases get resolved into “concrete indices”. We also dive into document routing and how documents are routed to shards by default or based on a custom value specified in the request. We also talked a little about AbstractRunnable and ActionRunnable.
Переглядів: 91
Відео
OpenSearch Deep Dive - Indexing (part 1)
Переглядів 672День тому
In the first of a series of videos diving into OpenSearch’s expansive codebase, we look at how OpenSearch processes single-document indexing requests. We start from the indexing REST handler and see how that gets mapped to the underlying transport action. All index/delete/bulk actions get processed by the same underlying “bulk” code, eliminating duplication. This first dive finishes partway thr...
Akash Shankaran - Improvements to OpenSearch using Intel Sapphire Rapids and hardware accelerators
Переглядів 125Місяць тому
In this session, we will share performance improvements in vector search scenarios due to the use of Intel Sapphire Rapids on AWS. The value of hardware accelerators such as AVX512 in Lucene and Faiss libraries will be discussed, along with performance results. We will further dive into the topic of compression, and some innovations in that space. This will focus on using Intel QAT (Quick Assis...
Laysa Uchoa & Yuliia Barabash - Search Results with Amazon Personalize and OpenSearch
Переглядів 60Місяць тому
Full title: Search Results with Amazon Personalize and OpenSearch: Making Searches Smarter OpenSearch is a powerful open-source search service, but its traditional ranking system doesn’t take into account user behavior. That’s where Amazon Personalize comes in! By combining OpenSearch with Amazon Personalize, we can re-rank search results based on what users have done in the past and various me...
Lucian Precup & Amine Gani - Leveraging OpenSearch for enhanced e-commerce search experiences
Переглядів 104Місяць тому
Dive into the transformative capabilities of OpenSearch in revolutionizing e-commerce search experiences. This presentation will showcase how OpenSearch’s robust architecture supports advanced features like auto-completion, targeted merchandising, and more. Discover strategies to optimize search functionalities with practical, real-world examples and innovative tools. Speaker: Lucian Precup - o...
Mohit Godwani - Optimizing Storage Performance & Efficient Index Configuration
Переглядів 159Місяць тому
Full title: Optimizing Storage Performance and Efficient Index Configuration with Compression Codecs in OpenSearch In the realm of distributed search and analytics engines, efficient storage management and index configuration are paramount to ensuring optimal performance and scalability. As data volumes continue to grow exponentially, optimizing storage utilization and reducing disk footprint h...
Jonah Kowall - Implementing OpenTelemetry and Jaeger with OpenSearch
Переглядів 186Місяць тому
Observability is top of mind today, we’ll start with the basics and then jump into why OpenSearch should be part of your observability story. Today, there are so many great open source options, but the complexity of running your open source stack can be daunting. There are so many options out there today, in this talk you’ll learn why OpenSearch is the most flexible option to be a key part of y...
Charles Christian & Jay Wilks - Managing the SIGMA detection rule lifecycle
Переглядів 180Місяць тому
Full title: Managing the SIGMA detection rule lifecycle using a python, git repository and the OpenSearch API How to take control over the default SIGMA detection rules in OpenSearch Security Analytics and deploy a custom set of detectors and sigma rules using python, a Git repository, and the OpenSearch API. Speaker: Charles Christian - opensearch.org/community/members/charles-christian.html J...
Samuel Herman - Lucene And Beyond - Core Storage Extension In OpenSearch
Переглядів 161Місяць тому
OpenSearch is tightly bound to the Lucene core APIs that facilitate the following functionalities: - Encoding - Transactions - Merges - Search - And more… In this presentation I will discuss how the OpenSearch storage encoding can be extended to popular formats (e.g. Parquet, Avro) that are readable by public big data systems such as Apache spark. This provides a strategic long term benefit for...
Kassian Wren - OpenSearch Notebooks: Monitor, Report, Learn, Explore and more!
Переглядів 69Місяць тому
OpenSearch notebooks are a powerful tool within the OpenSearch platform. They allow you to combine text, code, and visualizations of your data in one place. In this talk, I’ll walk you through the concept of the OpenSearch notebook, create one using some real-world Open API data, and talk about how this tool can be used for so much more than basic reporting. Speaker: Kassian Wren - opensearch.o...
Laysa Uchoa & Yuliia Barabash - Exploring OpenSearch, Python, and Serverless
Переглядів 32Місяць тому
Full title: Exploring OpenSearch, Python, and Serverless: Crafting Efficient and Modern Search Applications Facing challenges with search capabilities in your production environment? Discover how the combination of OpenSearch, Python, and serverless architecture can be your solution. This talk provides hands-on examples, from building efficient queries to implementing production-ready practices...
Aruna Govindaraju, Eric Pugh & Stavros Macracis - You’ve Deployed User Behavior Insights. Now What?
Переглядів 42Місяць тому
Full title: You’ve Deployed User Behavior Insights to understand your users. Now What? You’re collecting data about what your users are doing based on the search results they are seeing, ideally using OpenSearch’s User Behavior Insights capablity…. That is just the first step. What are you doing with that data? In this talk you’ll learn about the new capablities in OpenSearch that leverage that...
Moutupsi Paul, Xiaotang Wang & Amulya Sharma - Airbnb Embedding Platform
Переглядів 192Місяць тому
In this session, you’ll learn how Airbnb is leveraging OpenSearch as a vector database for its embedding platform. This talk is for everyone, starting with an overview of how Airbnb is using vector embeddings to solve guest, host & marketplace challenges, followed by specific use cases that led Airbnb to adopt OpenSearch as its vector database. We will share lessons learned and provide insights...
Ramesh Kumar Manickam - Scaling Operational Intelligence into Business Intelligence
Переглядів 61Місяць тому
Full title: Scaling Operational Intelligence into Business Intelligence using OpenSearch & GenAI for global rollouts At Transunion, we build products that provide insights and provide “Information for good”. Global roll out of cloud native applications and products require secure and scalable observability that spans Monitoring , Alerting , Logging and Traceability across Microservices and Syst...
Gustavo Sanchez - Future-proof AI applications with OpenSearch as vector database
Переглядів 86Місяць тому
Large Language Models (LLM) have shown immense growth and advancement in the field of AI recently.LLMs can intelligently generate new content and recommendations using vast amounts of data. Famous LLMs such as GPT, BERT, PaLM, and LLaMa are revolutionising the AI industry by imitating humans. The vector database is a new and unique database that is gaining immense popularity in the LLM space. T...
Trey Grainger - Building AI Models with OpenSearch's User Behavior Insights
Переглядів 47Місяць тому
Trey Grainger - Building AI Models with OpenSearch's User Behavior Insights
Migrating from Self managed Elasticsearch to OpenSearch & How Indias Fantasy Sports Giant Leveled Up
Переглядів 244Місяць тому
Migrating from Self managed Elasticsearch to OpenSearch & How Indias Fantasy Sports Giant Leveled Up
Unconference Talk: Fernando Rejon Barrera - How to finetune state of the art vector embeddings
Переглядів 57Місяць тому
Unconference Talk: Fernando Rejon Barrera - How to finetune state of the art vector embeddings
Anandhi Bumstead - OpenSearchCon North America 2024 Opening Remarks
Переглядів 107Місяць тому
Anandhi Bumstead - OpenSearchCon North America 2024 Opening Remarks
Charlie Yang - Toward Semantic Search, what we have now and what it will be in the future
Переглядів 259Місяць тому
Charlie Yang - Toward Semantic Search, what we have now and what it will be in the future
Pallavi Priyadarshini & Samuel Herman - OpenSearch Project Roadmap 2024 and Beyond
Переглядів 294Місяць тому
Pallavi Priyadarshini & Samuel Herman - OpenSearch Project Roadmap 2024 and Beyond
Derek Ho - Building Your First OpenSearch Dashboards Plugin: A Hands-On Demo
Переглядів 298Місяць тому
Derek Ho - Building Your First OpenSearch Dashboards Plugin: A Hands-On Demo
Mehul A. Shah - Luna: LLM-powered Unstructured Analytics (aka “RAG-supercharged”)
Переглядів 307Місяць тому
Mehul A. Shah - Luna: LLM-powered Unstructured Analytics (aka “RAG-supercharged”)
Prabhat Chaturvedi & Shikhar Jain - Attribute based access control (ABAC) & the Future of FGAC
Переглядів 341Місяць тому
Prabhat Chaturvedi & Shikhar Jain - Attribute based access control (ABAC) & the Future of FGAC
Unconference Talk: Infino - Prometheus / OTel integration added natively to Lucene
Переглядів 812 місяці тому
Unconference Talk: Infino - Prometheus / OTel integration added natively to Lucene
Abe Abernethy & Rick Boldt - We love OpenSearch, so you don’t have to!
Переглядів 2052 місяці тому
Abe Abernethy & Rick Boldt - We love OpenSearch, so you don’t have to!
Unconference Talk: Bob Wall - Securing Hybrid Search
Переглядів 442 місяці тому
Unconference Talk: Bob Wall - Securing Hybrid Search
Unconference Talk: Stavros Macrakis - Full analytics!
Переглядів 372 місяці тому
Unconference Talk: Stavros Macrakis - Full analytics!
Unconference Talk: Yuliia Barabash - Reindex like a Pro
Переглядів 502 місяці тому
Unconference Talk: Yuliia Barabash - Reindex like a Pro
There is nothing unethical about measuring employee performance.
nice efforts by amazon
Thank you for this. I tried looking for the first video you mentioned a year ago but could not find it. Any chance you could link it.
Here you go: ua-cam.com/video/9nD0vinQ7Gs/v-deo.html
Иссуси
Hello, I followed the guide and waited some time after getting the link for the dashboard in the console but I still get the red error. I tried both on windows wsl and macos. Please can anyone help?
intersting!!
Kudos!! Great talk Mohit 😃
Great insights. Very informative session 🔥
This is great guys!!
🤩🤩
Hi, Why only Kafka and In Memory Buffer. Can we not have file based buffer like in Fluentd and like Persisted Queue in logstash. Also can we introduce painless like script for advanced processing in processor
Awesome content!
Nice video, thx for uploading.
where can I see references/demo code on the otel-collector, fluentbit, and data-prepper config files?
Great insights on integrating threat intelligence with security analytics! How do you prioritize which threats to focus on with so much data coming in?
Heyho! Why can't i use an index-pattern in the correlation engine? Would i have to work with an index-alias instead? My index would be named like "logs-aws-cloudtrail-yyyy-MM-dd" by the ingestion pipeline. Would it be better to name it "logs-aws-cloudtrail" and do a rollover via ISM policies instead, to have a better usage with e.g. the correlation engine?
Hello! Is it possible to specify the required number of events to generate an alert? Example: I want to detect a brute force scenario followed by a successful login on a Windows computer. To do this, I need there to be at least 3 failed logins and, after these failed logins, a successful login event. Thanks!
Hello! Is it possible to specify the required number of events to generate an alert? Example: I want to detect a brute force scenario followed by a successful login on a Windows computer. To do this, I need there to be at least 3 failed logins and, after these failed logins, a successful login event. Thanks!
Is there any update about Management Console?
Sync issues with the video
बहुत खूब
how to take backup of the data from the DB's persistent volume?
No explanation of how to create docker image. Not present in this video and not present anywhere in the docs as well except the README present in docker folder of opensearch-build repository. It advises to run a script which will use the Dockerfile present in the repository. But the script itself fails, even after passing appropriate parameters to the script.
This person doesnt know where Voskopoja is, and writes history about Voskopoja You can trust this person about databases etc.
the dimensional explanaton of vector space, (3d) is excellent at 12:50
Great presentation
The presentation is disorganized. Just like random stuff being talked about by a bunch of people. The dashboard and visualization terms in opensearch are not obvious. I wish you can present more clear ways on what to accomplish. The sample data is also nothing useful.
Thanks for highlighting the evaluation piece of the puzzle, and the fact that most don't have good ways to validate that your semantic solution is working!
This was fun Charlie! You are much more natural than I!
Fantastic job! The video was beautifully executed and incredibly insightful. Keep up the great work!
Hi , How to convert seconds or millisecond's to HH:MM:SS FORMAT, our data is contact center data, we are working on an real time contact center dashboard
Bears. Beats. Battlestar Galactica.
Threat intelligence feature missing in on priem version
How you created index ?
Sound like these "challenges" and tricks are really the result of poor design. It's poor design to leave everything customizable because you have no idea and no guarantees about system behaves under load. You just offload engineering work to your customers...
ScreenCapture fail starting at around 16:00
Hi everyone, I want all OSD packs because I am having problem downloading them via link "Yarn OSD Boot Strap". I would like to receive packages in "Zip" format from you Note: I am using Windows 11.
Hi everyone, I want all OSD packs because I am having problem downloading them via link "Yarn OSD Boot Strap". I would like to receive packages in "Zip" format from you Note: I am using Windows 11.
I want all osd packages because I have problem downloading them via link "Yarn OSD Boot Strap". I would like the packages in "Zip" format please.
Thanks
Thank so much bro
Good to see your presentation, David!
Great to see this Mehul and Jon. Thanks for going into this level of details.
Great presentation!
could you please open source the code, react components.
Hello. You should be able to find the code you are looking for here: github.com/opensearch-project or under Aryn.ai's org: github.com/Aryn-AI Hope this helps.
Jon is best
Amazing ;-)
Thanks for the presentation. Like mike, I'm also wondering how can I enhance our keyword search to semantic search without major changes. Please provide a reference or code samples to use SPLADE function to get the expanded text data.
Continue the discussion with us: www.meetup.com/opensearch/events/297176339/
Great explanation DB and Dan👏👏