- 48
- 12 178
Eclipse Trace Compass
Приєднався 20 січ 2022
Eclipse Trace Compass is an open source application to solve performance and reliability issues by reading and analyzing traces and logs of a system. Its goal is to provide views, graphs, metrics, and more to help extract useful information from traces, in a way that is more user-friendly and informative than huge text dumps.
More info at eclipse.dev/tracecompass/
More info at eclipse.dev/tracecompass/
Setting Up and Using the TSP Python Client with the Trace Server | Part 2
In this video, Kaveh Shahedi, a Machine Learning Intern will show you how to use the TSP-Python-Client using pure Python code. Building on Part 1, which explored using CLI for interacting with TSP-Python-Client, now now want to know how to programmatically interact with trace data, enabling you to build automated analysis pipelines without directly accessing the Trace Compass interface.
What You'll Learn:
- Writing Python scripts to communicate with the Trace Server
- Building automated analysis pipelines for trace data processing
- Creating custom tools for trace analysis and visualization
- Integrating trace analysis into your existing Python workflows
Technical Implementation:
- Using the TSP Python Client API directly in your code
- Managing trace and experiment lifecycles programmatically
- Retrieving and processing trace data outputs (e.g., CPU usage, memory analysis, etc.) using Python data structures
Prerequisites:
- Basic Python programming knowledge
- Installed Trace Server and JDK 17
- Cloned TSP-Python-Client
- Familiarity with trace analysis concepts
Links:
- Part 1 of this series: [Link to Part 1](ua-cam.com/video/C8RSJZ8oh_A/v-deo.html)
- TSP Documentation: [Documentation Link](github.com/eclipse-cdt-cloud/trace-server-protocol)
- TSP Python Client [Repository](github.com/eclipse-cdt-cloud/tsp-python-client)
- Trace Compass documentation: eclipse.dev/tracecompass/
Connect with us:
If you have questions about implementing your own trace analysis solutions, reach out to the Trace Compass team! Don't forget to like, share, and subscribe for more advanced tutorials on machine learning, software tools, and trace analysis.
#TraceCompass #PythonDevelopment #DataAnalysis #MachineLearning #Ericsson #KavehShahedi #AutomatedAnalysis #PythonAutomation
What You'll Learn:
- Writing Python scripts to communicate with the Trace Server
- Building automated analysis pipelines for trace data processing
- Creating custom tools for trace analysis and visualization
- Integrating trace analysis into your existing Python workflows
Technical Implementation:
- Using the TSP Python Client API directly in your code
- Managing trace and experiment lifecycles programmatically
- Retrieving and processing trace data outputs (e.g., CPU usage, memory analysis, etc.) using Python data structures
Prerequisites:
- Basic Python programming knowledge
- Installed Trace Server and JDK 17
- Cloned TSP-Python-Client
- Familiarity with trace analysis concepts
Links:
- Part 1 of this series: [Link to Part 1](ua-cam.com/video/C8RSJZ8oh_A/v-deo.html)
- TSP Documentation: [Documentation Link](github.com/eclipse-cdt-cloud/trace-server-protocol)
- TSP Python Client [Repository](github.com/eclipse-cdt-cloud/tsp-python-client)
- Trace Compass documentation: eclipse.dev/tracecompass/
Connect with us:
If you have questions about implementing your own trace analysis solutions, reach out to the Trace Compass team! Don't forget to like, share, and subscribe for more advanced tutorials on machine learning, software tools, and trace analysis.
#TraceCompass #PythonDevelopment #DataAnalysis #MachineLearning #Ericsson #KavehShahedi #AutomatedAnalysis #PythonAutomation
Переглядів: 32
Відео
Eclipse Trace Compass: TSP Python Client - Unleash your CLI
Переглядів 26121 день тому
In this video, I’m Kaveh Shahedi, a Machine Learning Intern at Ericsson, and I’ll introduce you to the TSP Python Client for Trace Compass. The TSP Python Client is a powerful Python-based wrapper that lets you communicate with the Trace Server to open traces, create experiments, and retrieve data-all through the command line or Python code. What You'll Learn: Setting Up the Trace Server and TS...
Installing Trace Viewer for VSCode - Trace Compass cloud edition
Переглядів 194Місяць тому
Description: In this video, Bernd from the Trace Compass development team walks you through the steps to install and use the Trace Viewer for VS Code. Available on both the VS Code Marketplace and Open-VSX, the Trace Viewer extension allows you to analyze traces directly within your IDE. You'll learn: * How to install the Trace Viewer extension. * The role of the Trace Server in trace analysis....
Eclipse Trace Compass: Execution Comparison!
Переглядів 1852 місяці тому
The feature allows a user to run a program, trace it, patch it, trace it again and observe the differences. This is a great feature that took years to bring to you. We are excited to show off the initial results. This allows differential flame graphs to be shown comparing two executions. A flame graph is a visualization tool used to analyze performance and identify bottlenecks in software syste...
AI Powered: Trace Compass Extended Assistant
Переглядів 1402 місяці тому
In this video we show how to use the Trace Compass Extended Assistant project. It will soon be part of the Eclipse foundation. It allows creation of Data Driven XML based custom analyses in minutes rather than hours. Shout out to Sai Sree for taking on this initiative and delivering so quickly! Download the TCXA here: (LINK TBD)
Trace Compass: Summarize text traces! Event Count Analysis
Переглядів 453 місяці тому
In this video, we will demonstrate a simple way to summarize any trace, structured or unstructured by aggregating fields. This is a very quick and useful way to explore textual logs as well as user space traces containing data trace compass doesn't have a built-in way to visualize. Shout out to Vlad for this great presentation! Get Eclipse Trace Compass here: eclipse.dev/tracecompass
Follow-up on previous video: Making the benchmark scale better
Переглядів 713 місяці тому
In this video we follow-up on an oddity from the previous video: the lack of scalability of the sum-of-squares benchmark, when running with 2, 3 or 4 (or whatever number of physical cores are present on a computer) workers vs one. We use the classic Trace Compass to look at new traces obtained after some minor adjustments, that fix the previously mentioned issue. Benchmark: github.com/MatthewKh...
Eclipse Trace Compass: Showing an unbalanced workload and benchmarking issues through kernel tracing
Переглядів 1403 місяці тому
In this video we use Trace Compass to look at an odd behaviour. A benchmark where a fixed load is split among many threads is slowing down when we add more cores. The results were surprising. The worker threads would finish at different times and therefore we needed to wait for the longest worker. We utilize the Theia based front end for this demo as it allows exploration by CPU usage rather th...
Eclipse Trace Compass: How to record an LTTng Kernel and Userspace trace and read the results
Переглядів 2155 місяців тому
This is a demo of how to trace your application easily and view the results. It is an effective hello world in low level tracing. We use LTTng, lttng-utils, and Eclipse Trace Compass in order to monitor the performance of a given program. The program tests memory access patterns and compares x-then-y vs y-then-x vs memset for performance.
Eclipse Trace Compass ❤️ the uftrace Community!
Переглядів 825 місяців тому
Recently, uftrace was demoed into trace compass. The uftrace community saw the video and took the initiative to make the workflow even better! Congrats and thanks to the uftrace team. A soon release (v0.17?) will support a utc offset using the difference between utc time and the clock source. Fix in question: github.com/namhyung/uftrace/commit/fae85a8a40d2b64c2ed415d13c31bfbc87bf2710 Again, get...
Synchronizing an LTTng and UFTrace together for a full view of an execution using Trace Compass
Переглядів 1275 місяців тому
This is showing a bit more of what an experiment can do. We are recording an #lttng #trace and a #uftrace trace then opening them in #tracecompass to see the merged results. You will see that the total is greater than the sum of the parts. Reminders: * lttng can be obtained from lttng.org * uftrace can be found here : uftrace.github.io/slide/#1 * You can get LTTng-Utils here github.com/dorsal-l...
Applying Time Offset in Eclipse Trace Compass
Переглядів 1026 місяців тому
Trace Compass allows users to manually apply a time offset to one or many trace's event timestamps. It is also possible to calculate and apply the time offset by selecting matching events from different traces. See also: Eclipse Trace Compass Automatic Trace Clock Synchronization for Critical Path Analysis ua-cam.com/video/kilgZ5m1WUI/v-deo.html
XZ-tool backdoor traced using uftrace, Visualized in Trace Compass
Переглядів 4526 місяців тому
This is showing a quick flow of how you can compare two runs. This explores the XZ backdoor. I did not find data as compelling as Andres Freund did. This is an attempt to find a way to visualize the backdoor in a tool rather than an infographic. The major takeaway is also that libraries have a major impact on execution flow too. It is quite visible with this trace. the command used is: echo exi...
Eclipse Trace Compass Automatic Trace Clock Synchronization for Critical Path Analysis
Переглядів 1527 місяців тому
Trace Compass's Critical path analysis can handle traces from multiple nodes, servers, or computers. This brings up the issue, how can we synchronize clocks. NTP is good, but sometimes we can have clock skews of 10 seconds, this is not acceptable for nanosecond precision traces. Different clocks can even drift at different temperatures! We show the simple solution here. If you're interested in ...
Viewing an LTTng and UST trace simultaneously in Eclipse Trace Compass
Переглядів 1617 місяців тому
This is showing a bit more of what an experiment can do. We are recording an #lttng #trace and opening it in #tracecompass to see the merged results. You will see that the total is greater than the sum of the parts. Reminders: * lttng can be obtained from lttng.org * You can get LTTng-Utils here github.com/dorsal-lab/lttng-utils * You can install either by * $ sudo pip3 install upgrade git gith...
Eclipse Trace Compass's Critical Path Analysis
Переглядів 1398 місяців тому
Eclipse Trace Compass's Critical Path Analysis
Explaining "Bottlenecks" using Trace Compass
Переглядів 2269 місяців тому
Explaining "Bottlenecks" using Trace Compass
Using CDT Cloud to try Trace Compass cloud for free online!
Переглядів 1609 місяців тому
Using CDT Cloud to try Trace Compass cloud for free online!
Changing memory settings in trace compass
Переглядів 6110 місяців тому
Changing memory settings in trace compass
Open Telemetry: Opening Jaeger traces in Trace Compass and correlating them to LTTng Traces
Переглядів 17011 місяців тому
Open Telemetry: Opening Jaeger traces in Trace Compass and correlating them to LTTng Traces
Eclipse Trace Compass exporting to Excel
Переглядів 126Рік тому
Eclipse Trace Compass exporting to Excel
Reading Perfetto Traces in Trace Compass
Переглядів 615Рік тому
Reading Perfetto Traces in Trace Compass
Trace Compass Visual Filtering Boot Camp
Переглядів 119Рік тому
Trace Compass Visual Filtering Boot Camp
Eclipse Trace Compass: What's the deal with Experiments?
Переглядів 252Рік тому
Eclipse Trace Compass: What's the deal with Experiments?
Tracing Power Utilization and seeing the implications
Переглядів 166Рік тому
Tracing Power Utilization and seeing the implications
March update, Trace Compass 8.3 and Cloud updates
Переглядів 82Рік тому
March update, Trace Compass 8.3 and Cloud updates
Trace Compass Update December 2022: 8.2 + TraceCompass.cloud!
Переглядів 126Рік тому
Trace Compass Update December 2022: 8.2 TraceCompass.cloud!
Trace Compass 8.1 is out, CDT.Cloud Demo of trace compass
Переглядів 2262 роки тому
Trace Compass 8.1 is out, CDT.Cloud Demo of trace compass
Nice job Vlad!
Truly a great intern!
DTW. Dynamic time warp. Would align automatically
We support DTW, in fact we have the first implementation of LIANA. www.sciencedirect.com/science/article/abs/pii/S1084804514001623 However, this would not help in the context of flame graphs, but flame charts. Here is a video describing the difference. ua-cam.com/video/ZrjuJKjAUns/v-deo.html . Basically a flame chart shows your callstack vs time, a flame graph is a weighted tree of function durations that are aggregated from a callstack over time. Also, we find the naming (Flame Graph/Flame Chart) to be confusing. Callstack over time and flame graph would be clearer. However, we tried to align with industry standard nomenclature. Thanks for the comment!
Nice video! Can you tell me if lttng-record-trace (lttng-utils) is currently maintained somewhere? I added some feature to perform the tracing on a embedded remote target via ssh and I would be happy to share this.
As far as we know, it's here github.com/dorsal-lab/lttng-utils I think we should update the readme to reflect that. LTTng utils is unsupported. It is a good will project, but nobody really supports it. I am certain that dorsal would love a new maintainer. ;)
Cool and Great work
Thank you!
Looks nice, great work!
Thank you so much!
Well done!
Thanks! We're glad to see ways we can help!
Cool!
Thanks for the amazing video. Any idea how Tracecompass builds a critical path analysis for a thread. When I look at the order of kernel events they map to the control flow view and not critical flow view. Would be great to know how Tracecompass deduces it from trace events?
You can look in the source code, it's open source. But basically the linux kernel is well instrumented. We know stuff like sched wakeup, switch etc.. and we can get more data like syscalls for files and network access. I cannot go into too much detail in a youtube comment, but reach out to the community @ tracecompass.org for a better deep dive!
Is it normal to have to manually place Jaeger Tracing in the application to be able to place the body inside Jaeger Web or is there an automatic way to see the body on a Tracing platform like Jaeger Web?
We do! git.eclipse.org/r/c/tracecompass.incubator/org.eclipse.tracecompass.incubator/+/127410 No users expressed the need for this! If there is a need we can accommodate, or if you have a cooler way to do it, please share a patch, we'll review with giant smiles!
What GDB version is used in this demo? GDB tracepoints have been broken for a while now AFAIK.
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2 Sorry, I should have answered faster. I didn't notice the message. I love your nickname btw.
Wow, this looks so awesome! Can't wait to check it out. I've just implemented a (near-)zero-overhead PHP Stacktrace capturer and Flamegraph generator. The information you get out of flamegraphs is just amazing. Would love to capture the information for a whole server like you did. Do you have a tutorial, where I can learn to capture the data on a linux production server without overloading the server?
Hey, sorry for the long delay, I thought I answered it. check out www.tracecompass.org for the tutorials, there's a project called tracevizlab. We hope to see interractions/collabs soon!
Hi Matthew, lets agree to disagree about the relative coolness of GDB tracing vs lttng. It's true that lttng is way more flexible, faster, easier to capture traces for, has less limitations, and more. But is it cooler? Are CDs cooler than vinyls? I think not. Other than that little detail, excellent video! :)
I cannot argue with such wisdom. GDB tracing has some very nice use cases. IMO it's much easier to set up, but you get less information out of it. As an individual who likes getting the full picture, that is limiting. :) The best though, is integrating a GDB Trace with a system trace and a cloud trace to get the full pic of what's going on.
Very slick and a nice explanation for why overclocking is such a dicey proposition.
We just report the facts. If someone wants to lose efficiency for speed and they live in the antarctic and have solar/wind power. Go for it!
Good video!