We use Dynatrace today for tracing.. I will investigate openTelemetry and see how it can add value and prevent vendor lock-ing. Thanks for another great vid!.
Thanks for the content, Viktor. Would love to see a trilogy of videos on 1. the otel instrumentation needed to get red metrics, such that one can answer health questions 2. instrument more rich spans around the places where metrics are instrumented and unlock answering questions about change. 3. wrap it up with trying to answer health and change questions in various backends (garafana, datadog, honeycomb).
Huge topic for just one video, a serie for this, digging in the different parts (tracing, monitoring, alerting and logging) and the integration with other tools, grafana, Jaeger, Prometheus etc..., would be terrific 😊. Anyway thanks for your work, keep it up!
Thank you for the video! As a DevOps engineer with little experience in monitoring, I'm really grateful for now knowing what instrumentation is as well as what makes OpenTelemetry stands out as an instrumentation tool.
Thanks for these series! Open tracing and Open Census were good project and with Open Telemetry it also comes with auto-instrumetation. For Java there are Java agents based that propagates the headers, traces, spams, etc.. to upstream services.
Excellent and concise overview. Thank you! Teams must also be mindful of how this data is used and queried. From what you said in this video, I think you'll get into that in more detail in the next video.
Often, I wait to see whether people are interested in a topic I propose and, if they are, I start working on it some time in the future. That often results in it coming weeks or months later. Jaeger is different. Many already asked for it and I'm already on it. It will be released next week.
as you asked, in term of Otel Receiver, you might consider Grafana Tempo as Receiver and Visualization, as you have all three types of Otel in one Visualization, Traces, Logs, metrics, and cold even correlate them (implemented right)
@@DevOpsToolkit BTW, do you know Crossplane (runtime) will get OTEL Tracing one day, I think it's a great fit for such a tool interacting with so. many components (Compositions, Providers) down and upstream...
I must NOT ask my app vendors to do additional coding (plus maintaining it) that will cost for "emit" when that application consumed unless consumer is really needed that if and when. Or stick with git-driven config change enforcement for enabling those information elements that will steer app to ship that that to a remote server via pull or push rather, some stuff we inherit from legacy not necessarily bad but optimized over ages of operational experience and cost for it.
You mentioned several times that we should use OpenTelemetry to be able to switch one observability solution to another. I remember a time, when people had the same argument for ORMs, which allegedly allow to switch database, but give you least common API. So you kinda exchange part of features to possibility of changing database/observability solution. Maybe in some cases a Datadog agent is not that bad, especially if we consider that observability consist of several parts (agents, collectors, storage backend, UI and so on, and one vendor could integrate these parts better, than common solution).
Many other attempts to standardize something failed because of the lack of support. Having a standard that is not implemented by many is not very useful. OpenTelemetry, however, is one of the few "standards" that managed to rally everyone. Almost every vendor or a project in the observability space supports it or is working on the support (including Datadog). Something similar happened to Kubernetes API a bit earlier (almost everyone gathered behind it). So, OpenTelemetry is different than others due to the size of its adoption and contributions coming from vendors and projects. As a counter example, we have OAM (Open Application Model) for which I had high hopes but which failed to convince a significant percentage of the industry to adopt it.
One more thing... I don't care much about agents. They can be OpenTelemetry or anything else since there is almost no work involved in switching from one to the other. What I do care is what we use to instrument our code since that represents invesment (mainly in time).
Datadog supports otel pretty well but it works much better with the agent. Same with New Relic, Instana, splunk and most of the hosted services out there. It's how the softly push you in to vendor lock in to make it less appealing to switch away.
@@davehoffman515 That follows typical stages of adoption of standards. 1. We will not support this. Our stuff is better and we have the competitive edge. 2. We have to support this because others are supporting it and prospects are asking for it, but we will not make it as good as our own stuff. 3. We have to support it well because our customers do not want to use our own API. They are leaving because they do not want to be locked in. Most vendors are in the phase 2 right now, and will be moving towards the phase 3 soon. P.S. Most attempts to standardize something never pass the phase 1 and eventually die. OpenTelemetry managed to get to the phase 2 thus (probably) securing its future.
Robusta is two dudes spamming Linkedin constantly with their vision about "Limits make no sense, dont put limits in your deployments"... After short discussion with "CEO" stopped wasting my time with them.
OpenTelemetry does not replace any specific tool or, to be more precise, does not have to replace anything. It's mostly focused on standardizing the API that other tools might use and instrumenting your code with that API.
@@DevOpsToolkit When I access the webapp (not node.js) which is a java based webapp, can I get the js files and the js functions that gets invoked from the browser side?
I have conducted research on enabling OpenTelemetry for multi-services written in Go, and I have chosen to utilize Signoz as a visualization tool that combines traces, metrics, and logs. Signoz is also an open-source project, and I highly recommend that you check it out.
Do you instrument your code?
Hi Viktor, you type it "Robuta" in description, hope you find this usefull
Thanks for the note. Corrected :)
We use Dynatrace today for tracing.. I will investigate openTelemetry and see how it can add value and prevent vendor lock-ing. Thanks for another great vid!.
@@fpvclub7256 Dynatrace support OpenTelemetry (not sure whether they support all all of or only a subset), so that part should not be a problem.
Thanks for the content, Viktor. Would love to see a trilogy of videos on 1. the otel instrumentation needed to get red metrics, such that one can answer health questions 2. instrument more rich spans around the places where metrics are instrumented and unlock answering questions about change. 3. wrap it up with trying to answer health and change questions in various backends (garafana, datadog, honeycomb).
Huge topic for just one video, a serie for this, digging in the different parts (tracing, monitoring, alerting and logging) and the integration with other tools, grafana, Jaeger, Prometheus etc..., would be terrific 😊. Anyway thanks for your work, keep it up!
Jaeger video was just published and is available at ua-cam.com/video/FK0uh-7nDSg/v-deo.html.
Dear, you have one of the best channels about DevOps. I always enjoy your content. Please keep it on.
Wow, thank you!
This channel is my daily podcast ❤
Thank you for the video! As a DevOps engineer with little experience in monitoring, I'm really grateful for now knowing what instrumentation is as well as what makes OpenTelemetry stands out as an instrumentation tool.
Man, this is a best, most clear and concise answer to the question on what is OT. You are simply the best!
Great video on OpenTelemetry! It's definitely a game-changer in the world of observability. Can't wait for the future videos.
Need to do a full devops training course. You are a very good teacher
My new favorite channel 😃❤ thank you Viktor
Thanks for these series! Open tracing and Open Census were good project and with Open Telemetry it also comes with auto-instrumetation. For Java there are Java agents based that propagates the headers, traces, spams, etc.. to upstream services.
Waiting for next video.... very excited :)
It's coming next Monday :)
I'm really curious to see how the instrumentation work with jaeger tracing components. Looking forward to it!
Already scheduled for next week :)
@@DevOpsToolkit It would be also interesting to see comparison with other tracing tools like Zipkin
It would. I’ll do my best to do it soon.
Done!
ua-cam.com/video/FK0uh-7nDSg/v-deo.html
Excellent and concise overview. Thank you! Teams must also be mindful of how this data is used and queried. From what you said in this video, I think you'll get into that in more detail in the next video.
It's a very big subject so it'll take more than one more video. The next one related to OTel is coming next Monday.
I love your overviews so much! Thank you and good luck!
Great insight! Great video as always!
Video with Jaeger would be nice
Often, I wait to see whether people are interested in a topic I propose and, if they are, I start working on it some time in the future. That often results in it coming weeks or months later. Jaeger is different. Many already asked for it and I'm already on it. It will be released next week.
Done!
ua-cam.com/video/FK0uh-7nDSg/v-deo.html
as you asked, in term of Otel Receiver, you might consider Grafana Tempo as Receiver and Visualization, as you have all three types of Otel in one Visualization, Traces, Logs, metrics, and cold even correlate them (implemented right)
Tempo is amazing
@@DevOpsToolkit BTW, do you know Crossplane (runtime) will get OTEL Tracing one day, I think it's a great fit for such a tool interacting with so. many components (Compositions, Providers) down and upstream...
@holgerwinkelmann6219 it's on the roadmap.
Yeah awesome video! I need now to build in the tracing library!
A video about Jaeger and Tempo would be nice, Viktor!
One of those two requests is coming next Monday :)
Jaeger was just published (ua-cam.com/video/FK0uh-7nDSg/v-deo.html).
Looking forward to the tracing video!!
Thank you for explaining it.
Just found your videos, amazing thanks
I must NOT ask my app vendors to do additional coding (plus maintaining it) that will cost for "emit" when that application consumed unless consumer is really needed that if and when. Or stick with git-driven config change enforcement for enabling those information elements that will steer app to ship that that to a remote server via pull or push rather, some stuff we inherit from legacy not necessarily bad but optimized over ages of operational experience and cost for it.
Title: MUST!!!!
Actual video: maybe should 🥺👉👈
You mentioned several times that we should use OpenTelemetry to be able to switch one observability solution to another. I remember a time, when people had the same argument for ORMs, which allegedly allow to switch database, but give you least common API. So you kinda exchange part of features to possibility of changing database/observability solution.
Maybe in some cases a Datadog agent is not that bad, especially if we consider that observability consist of several parts (agents, collectors, storage backend, UI and so on, and one vendor could integrate these parts better, than common solution).
Many other attempts to standardize something failed because of the lack of support. Having a standard that is not implemented by many is not very useful. OpenTelemetry, however, is one of the few "standards" that managed to rally everyone. Almost every vendor or a project in the observability space supports it or is working on the support (including Datadog). Something similar happened to Kubernetes API a bit earlier (almost everyone gathered behind it).
So, OpenTelemetry is different than others due to the size of its adoption and contributions coming from vendors and projects.
As a counter example, we have OAM (Open Application Model) for which I had high hopes but which failed to convince a significant percentage of the industry to adopt it.
One more thing... I don't care much about agents. They can be OpenTelemetry or anything else since there is almost no work involved in switching from one to the other. What I do care is what we use to instrument our code since that represents invesment (mainly in time).
Datadog supports otel pretty well but it works much better with the agent. Same with New Relic, Instana, splunk and most of the hosted services out there. It's how the softly push you in to vendor lock in to make it less appealing to switch away.
@@davehoffman515 That follows typical stages of adoption of standards.
1. We will not support this. Our stuff is better and we have the competitive edge.
2. We have to support this because others are supporting it and prospects are asking for it, but we will not make it as good as our own stuff.
3. We have to support it well because our customers do not want to use our own API. They are leaving because they do not want to be locked in.
Most vendors are in the phase 2 right now, and will be moving towards the phase 3 soon.
P.S. Most attempts to standardize something never pass the phase 1 and eventually die. OpenTelemetry managed to get to the phase 2 thus (probably) securing its future.
Excellent content!
You should do a video on OpenFeature!
Adding it to my to-do list... 🙂
And here I thought Robusta is a coffee species.
Robusta is two dudes spamming Linkedin constantly with their vision about "Limits make no sense, dont put limits in your deployments"... After short discussion with "CEO" stopped wasting my time with them.
Excellent fwd to the twitter
Is the logging part intended to replace promtail + loki or does it do something differently?
OpenTelemetry does not replace any specific tool or, to be more precise, does not have to replace anything. It's mostly focused on standardizing the API that other tools might use and instrumenting your code with that API.
Any thoughts on Grafana Tempo vs Jaeger? Does it make sense to stick to Grafana tools if you already use them for most other things?
Grafana tools are awesome and they are my default recommendation. Tempo works with OpenTelemetry so no problems on that side.
Hi, have you seen any way how to document Otel metrics based on some kind of Schema, basically like OpenAPI or MIB for Cloud Native Metrics?
I haven’t seen something like that :(
Hi, Can we get the javascript files and method name details in tracing in a non node.js app?
I'm not sure I understood the question. Are you referring to traces in other languages or tracing with JavaScript without nodejs?
@@DevOpsToolkit When I access the webapp (not node.js) which is a java based webapp, can I get the js files and the js functions that gets invoked from the browser side?
@cristineees not that I know. I believe it is server side only. I don't work much with frontend so I might be wrong though.
Otel collector + tempo pls 😆
Tempo is already on my to-do list :)
OT logging is still to this day inexistent or experimental.
Yes. Logging is behind other types of telemetry.
I have conducted research on enabling OpenTelemetry for multi-services written in Go, and I have chosen to utilize Signoz as a visualization tool that combines traces, metrics, and logs. Signoz is also an open-source project, and I highly recommend that you check it out.
Adding it to my TODO list... :)