![George Bridgeman](/img/default-banner.jpg)
- 23
- 174 406
George Bridgeman
United Kingdom
Приєднався 18 лип 2021
I publish tutorials on The Elastic Stack; Elasticsearch, Logstash, Kibana, and Beats.
Working with dates in Elasticsearch and OpenSearch
I don't think I've ever met a Developer who loves working with dates, times, and timezones. Fortunately, however, Elasticsearch and OpenSearch have great support and tooling for with temporal data.
In this tutorial, I show how to store and query date values. There's more detail to the topic than first meets the eye!
In this tutorial, I show how to store and query date values. There's more detail to the topic than first meets the eye!
Переглядів: 2 392
Відео
Bool queries in Elasticsearch and OpenSearch
Переглядів 2,7 тис.Рік тому
Individual queries are quite limiting. We need the ability to combine multiple queries with boolean logic, and that's exactly what the bool query is for! I show how to combine multiple queries in AND, OR, and NOT-type operations. I see a lot of people get caught out by the default behaviour of the `should` clauses, which are often interpreted as an `OR` operator, yet don't work that way by defa...
Range queries in Elasticsearch and OpenSearch
Переглядів 3,1 тис.Рік тому
Range queries allow you to find documents where field values are greater than, greater than or equal, less than, and less than or equal to certain values. They work over numeric, IP, and even range field types. In this tutorial, I show some common uses of the range query type.
Term queries in Elasticsearch and OpenSearch
Переглядів 3 тис.Рік тому
Term queries are the more simple type of query in Elasticsearch and OpenSearch. They can be used for matching text, numerics, IPs, and other field types in a precise way. In this tutorial, I explain some of the common term query types.
Introduction to Elasticsearch and OpenSearch queries
Переглядів 6 тис.Рік тому
Let's write our first query! We'll start with a match_all query to return all documents from one or more indices. This first lesson introduces the main concepts we need to know before looking at some more advanced queries.
Updating documents in Elasticsearch/OpenSearch
Переглядів 3,9 тис.Рік тому
Updating documents works differently to how you'd likely expect. There's a special API for it, and updating a document is not a single operation. I show how to update single documents to change the value of existing fields. More complex operations can be done using scripted updates, which we'll cover in a later lesson.
Reading documents in Elasticsearch/OpenSearch
Переглядів 4,8 тис.Рік тому
We have created some documents in an index, so let's now have a look at how to read those documents back from the index.
Creating documents in Elasticsearch/OpenSearch
Переглядів 7 тис.Рік тому
Creating documents in Elasticsearch and OpenSearch can be done in different ways depending on your needs. I demonstrate different ways of creating and replacing documents in this video, showing how document IDs can be generated for you, and how to use your own.
Introduction to Elasticsearch and OpenSearch documents and CRUD operations
Переглядів 13 тис.Рік тому
I explain what a document is, and how we can construct one from CSV data. I then introduce the different operation types that allow you to perform create, read, update, and delete operations through Elasticsearch/OpenSearch APIs.
Elasticsearch and OpenSearch field types
Переглядів 3,7 тис.Рік тому
There is a huge range (pun intended) of different field types available in Elasticsearch and OpenSearch. In this lesson, I introduce some of the more commonly used ones.
Explicit index mappings in Elasticsearch and OpenSearch
Переглядів 3 тис.Рік тому
Dynamic mappings are great for getting started with a new index, but you'll often want your mapping to be more rigid and not allow new fields to be added automatically. Explicit mappings allow you to define the mapping up-front and deal with clients trying to add new fields to the index. I show how to define an explicit mapping and set options for how unmapped fields should be dealt with.
Dynamic templates in Elasticsearch and OpenSearch
Переглядів 2,9 тис.Рік тому
How to use dynamic templates in your Elasticsearch or OpenSearch mappings. This allows you to specify custom mappings for new dynamic fields based on field names or types.
Dynamic index mappings in Elasticsearch and OpenSearch
Переглядів 3,8 тис.Рік тому
I introduce Elasticsearch/OpenSearch dynamic mappings, show how they work, explain some of the drawbacks of using them, and demonstrate type detection.
Elasticsearch and OpenSearch index creation
Переглядів 9 тис.Рік тому
How to create an Elasticsearch or OpenSearch index, specifying settings, and defining a mapping.
How I fixed my Elasticsearch index mapping
Переглядів 4,6 тис.Рік тому
I created an Elasticsearch/OpenSearch index mapping that couldn't answer questions I asked it, so I built two alternative mappings for the same data. One is a very wide, dynamic, and update-heavy mapping that's easy to query. The other is strict and tidy, but uses nested documents. There are pros and cons to each, and I discuss those. Three costumes for the same data!
Elasticsearch anti-patterns and bad practices to be aware of
Переглядів 27 тис.Рік тому
Elasticsearch anti-patterns and bad practices to be aware of
Elasticsearch text analysis and full text search - a quick introduction
Переглядів 19 тис.2 роки тому
Elasticsearch text analysis and full text search - a quick introduction
Visualising an Elasticsearch cluster
Переглядів 1,6 тис.2 роки тому
Visualising an Elasticsearch cluster
Elasticsearch lab exercises Part 2 - exam practice and learning
Переглядів 1,5 тис.2 роки тому
Elasticsearch lab exercises Part 2 - exam practice and learning
Elasticsearch lab exercises - exam practice and learning
Переглядів 6 тис.2 роки тому
Elasticsearch lab exercises - exam practice and learning
Installing and configuring Elasticsearch using Ansible
Переглядів 4,6 тис.3 роки тому
Installing and configuring Elasticsearch using Ansible
Technical certifications : What are they? Are they worth it? Who can they help?
Переглядів 1,2 тис.3 роки тому
Technical certifications : What are they? Are they worth it? Who can they help?
What are Elasticsearch shards? Why do they matter? Elasticsearch cluster architecture explained.
Переглядів 40 тис.3 роки тому
What are Elasticsearch shards? Why do they matter? Elasticsearch cluster architecture explained.
Very clearly explained. Thanks
Awesome explanation, I love your narrative style, it really underlines the why and how of the current ecosystem!
This is really one of the most useful videos that introduced ES to me.
Thanks a lot!! Impeccable content!
I just have say that you are a great presenter! I've only done minor stuff with ElasticSearch but there were lots of things to consider if our company decides to expand the use cases that are implemented with help of ES.
Hi thanks for a video. For example we have: "unassigned_shards" : 40, When we run: GET _cluster/allocation/explain?filter_path=index,node_allocation_decisions.node_name,node_allocation_decisions.deciders.* { "index": "elastalert_past", "shard": 0, "primary": false } We reiceve next answer: "explanation" : "a copy of this shard is already allocated to this node [[elastalert_past][0], node[JaLzrdasdajQ], [P], s[STARTED], a[id=OmY9kwpHTlybJfSrWvdsadada6g]]" We have only one node and what we can do in this situation ? Also we have "number_of_replicas" : "0", "auto_expand_replicas" : "false", what we can do in this situation ? GET /.kibana/_settings { ".kibana_2" : { "settings" : { "index" : { "number_of_shards" : "1", "auto_expand_replicas" : "false", "provided_name" : ".kibana_2", "creation_date" : "1601664093", "number_of_replicas" : "0", "uuid" : "WKdIpzLFSP-ydObLw", "version" : { "created" : "7090299" } } } } }
Nice job. Very clear
I have just set up singlenode ELK Stack in our environment. It aggregates logs from network application stack via Filebeat from multiple servers and are then futher parsed via Logstash. It's main use should be for diagnostics. Basicaly vision is that L3 can quickly look at the logs and create visualization for single session via REST call from some Web server frontend. They can then easily focus/lens on problematic packets or diagnose where is the problem without need to use Oracle DB and writing queries. Everything is sort of streamlined and easy to use. I can imagine that similar solutions are used for example for security logs. Like i have said I dont have that much knowledge/experience with ELK Stack, but I think that it depends on your application. My opinion is that i wouldn't necessary use Elastic for metrics because there are much better tool for that out there. Also i would like to add that I guess that developers of ELK knows that it's complicated and their documentation is quite nice and understandable, but be careful because you can get lost in the "loop", because there are multiple link to manual that talks about he same thing in different levels of granularity so I would suggest before configuring anything to read through the manula very quickly just to be sure the end result is what you want. And as always less is sometimes more, so keep it simple at the start and as you get more knowledge you can add more features or scale up.
where was thsi video 1 month ago. you should be paid by these software companies... bless your heart
great job
Thank you for creating this it is very helpful indeed.
Many thanks for your work! It's awesome video!
why don't you use the http "PATCH" method?
this is incredible guide! the whole playlist is amazing and very helpful! thank you for this!
I was planning to run a elastic search image for work on docker, thinking it would be a simple process. this video is an eye opener, now I'm not sure if I should go for it or not, our use case is something like yelp with much less data
Thank you so much :)
1000th like 😅
These are great points - within an organisation where elastic ends up in production without really fleshing out the way it should be used and having teams onboarding their data without true understanding of the data modelling that needs to be done and understood upfront is a major pain to fix later down the line. Common fields are essential (i would say critical) to make most of it when trying to get your data out in a useful way.
great video!
good video, but, how to di it with certs, i mean lots of certs, lots of chages we need to create certs and send it to other nodes and there move to anoter paths, and rewrite some code, My queston is how to send it via ansible?
Its two year late but the lesson is extremely value
I just started learning Elastic search and this is the best an clear information on Elastic search architecture. Thanks for sharing!
I can attest to the challenges you described. I’m the only one in my organization using elasticsearch. Everyone else is afraid to touch it. I do not blame them.
How about using Elasticsearch for only term searches and doc id queries? Bad use case?
you are god sent <3
I am trying to install and configure Elasticsearch , Logstash , Kibana and Filebeat to collet some logs but struggling to get this thing going.
Hi, can you publish certain lab exercises for Elastic certified analyst? Thanks
Thanks for the exercises, for ex. 9 I used a bool with a filter and for ex. 10 I used grok
Well that's just put me off completely
this is great content. thank you.
Thanks for this outstanding series. I only wish more tutorials were this clear and concise.
This is amazing. Lot of topics covered in one short video. Kudo's and keep up the good work.
am currently building a project that requires semantic search, i wanted to pay someone soo quickly to get this done (i can do it myself but i don't want to spend the time), i thought ELK would be such a breeze, thanks for saving me the time and money :)
If elasticsearch distributes the data between the shards of an index such that each lucene store roughly holds the same number of documents, when you run a search query, elasticsearch, despite the inter-node communication, only knows which shards hold that index and not which particular shard will have that document? So it has to run the query against all the shards and merge results, it cannot just search the one shard that contains that document? It does not know beforehand based on how documents are distributed among shards.
This video changed my life. No exaggeration.
I didn't know about minimum_should_match, that explains a few things!
These are great, nothing like this on the web keep them coming!
I think that shard in ES has the same concept with partition in Kafka when they have all partition replicas in different nodes
You can also use runtime fields to pull and query that field from the _source. The pro is that in large data sets, you save on storage space with fields that are not indexed. The con is that it uses resources to run the runtime field's request. The nice thing is, that field will exist when running the query so you can then search on it even though it was not indexed.
Great point! You're absolutely right. There will be a whole video on runtime fields; they're very useful and are on the Elastic Certified Engineer curriculum now.
The worst thing is when you meet Solution Architect who want you to increase index mapping total_fields limit just to fit badly design mapping. Good point! Good sound quality. wish to have same one day
There's a time and a place for increasing it. It can sometimes get you out of a bind but it's not a good long term solution and you're better off fixing the mapping as soon as practical.
Great work sir
Awesome explanation
Hi @GeorgeBridgemanData - can Elsatic search be used for types of files that it may never have seen before and "shown" how to parse them?
Great work sir ❤
Great work
Hi George, how can I contact you in person? We would like to invite you as a guest to a function of ours to talk about Elasticsearch.
Hi there. Feel free to contact me via LinkedIn.
Certainly can attest to point #1 - We went into our project (hundreds of millions of records, for full document text search) thinking we would have completely flattened data. Later on in the project we saw a serious need for some form of hierarchy/relationship - I see now why ES HIGHLY recommends flattened data approaches - the hasParent/hasChild queries can be complicated and super expensive, especially for aggregations.
Did you try to use dynamic templates in OpenSearch? It seems OpenSearch doesn't support it.
I've tested all these commands against OpenSearch (2.2.1 in my case) and they work fine.