Don't Make These 6 Prometheus Monitoring Mistakes | Prometheus Best Practices & Pitfalls
Вставка
- Опубліковано 12 чер 2024
- In this video, I go through some of the biggest pitfalls you can run into when you're new to Prometheus, and I also explain how to avoid them.
Check out my Prometheus training courses if you want to learn Prometheus in a structured way from the ground up:
training.promlabs.com/
Related PromLabs blog post about these pitfalls:
promlabs.com/blog/2022/12/11/...
Chapters:
00:00 Introduction
00:22 Mistake 1: Cardinality Bombs
02:28 Mistake 2: Aggregating Away Too Many Labels
03:48 Mistake 3: Unscoped Metric Selectors
04:49 Mistake 4: Missing "for" Durations in Alerting Rules
06:52 Mistake 5: Too Short Rate Windows
08:36 Mistake 6: Using Functions With Incorrect Metric Types
10:20 Outro & PromLabs Trainings
---------------------------------------------------------------------------
CREDITS: "Subscribe Button" by MrNumber112 • Free Download: Subscri... - Наука та технологія
I am really hyped to find your channel, been working with prometheus for a few years now, it was my primary reason to begin learning golang also
instantly subscribed, I'll be lurking here on my weekends! ♥
YAYYY!!! It's awesome to see how this series is progressing. Another winner 😎
Thank you for the video! :) Very good content!
Great content, thank you for recording it!
Thanks Julius for all your videos. They are very useful for me. 👍
Loved the video. Looking forward to more.
This would definitely make a great series!
Great video! Thanks Julius
Thank you for great videos!
Thanks for the video! Very informative
Your videos are so good
Ahhh!! All mistakes I’ve encountered! (including ones I made …). Great summary. I’m sure I’ll refer back to this one.
Ohhh that's good to hear that it's relevant to pitfalls you encountered as well, thank you! :)
Awesome channel
Nice video and channel.
Let me know what other kinds of Prometheus pitfalls and best practices you'd like to learn about! These were some technical ones that I've mentioned a lot in talks, but there are surely many others.
Hey Julius, thank you for your video. Which metric type would you propose to avoid cardinality bombs in case I want to record labels that can have lots of different values? Is there a way to change the grouping mechanism in a way that prometheus will not create a new time series for every label combination?
A time series is identified by its metric name and unique set of labels, so any addition, removal, or change of a label will automatically mean that it's a different / new time series. So there's no way to not create a new series for a new label combination. Other than that, the only thing that matters is the total number of series generated by a metric, which results from the combinations of label values it can have. The metric type doesn't have any impact on that (and mostly just depends on what you are trying to measure), except that of course e.g. in the case of histograms, there's always an automatic "le" bucket label that gets multiplied into the total cardinality, since every custom label combination also has one series per configured bucket (so one strategy is just to configure fewer buckets if histogram cost is becoming an issue).
Hi, just now started using prometheus. I am trying to set rate function for counter.. so i have a counter metric and the values is 1.. when i use rate for counter rate of first entry a at that time starts with 0..why for the rate of first new single instrumwntation entry starts with 0. How to resolve that..i want rate /changes/ increases should starts with non zero and then it should become 0 when its idle how to achieve this
hey julien, i am pretty new to prometheus. i want to know that can we also track user level behaviour in prometheus or would i need to use a total separate tooling for that. i wanted to know what my users are doing, and watch indidividual user behaviours
If you have more than a handful of users, then a metrics-based monitoring system like Prometheus is probably not the right choice. Imagine you have a million users and you want to track one metric about each. Then you already have 1 million time series just for that. The total budget for a big server is usually a couple of million series, so this will only work if you have very few users or you don't track a lot of info about each one :)
Awesome content, Julius. I find it extremely useful, but it took me 2 months to finde it! I'll do what I can to spread the word through my channels. Insanely popular as it is, most implementations I come across during SRE consultancy engagements, only exploit a small fraction of its value. All the best, @XavierThibaudon