How not to debug your programs.
Вставка
- Опубліковано 15 жов 2024
- Patreon ➤ / jacobsorber
Courses ➤ jacobsorber.th...
Website ➤ www.jacobsorbe...
---
How not to debug your programs. // print statements are the new programmer's go-to tool, but if you're going that route, you're making things to difficult for yourself.
Related Videos:
Debugging playlist: • Learn GDB in 60 seconds
***
Welcome! I post videos that help you learn to program and become a more confident software developer. I cover beginner-to-advanced systems topics ranging from network programming, threads, processes, operating systems, embedded systems and others. My goal is to help you get under-the-hood and better understand how computers work and how you can use them to become stronger students and more capable professional developers.
About me: I'm a computer scientist, electrical engineer, researcher, and teacher. I specialize in embedded systems, mobile computing, sensor networks, and the Internet of Things. I teach systems and networking courses at Clemson University, where I also lead the PERSIST research lab.
More about me and what I do:
www.jacobsorbe...
people.cs.clem...
persist.cs.clem...
To Support the Channel:
like, subscribe, spread the word
contribute via Patreon --- [ / jacobsorber ]
Source code is also available to Patreon supporters. --- [jsorber-youtub...]
Yep, exactly. Instead of “printf” you can use “assert” from assert.h. It takes an expression - even if expression is equal to 0 it stops the program and shows debug info. And unlike printf you shouldn’t remove all these assertions - you should just define NDEBUG macro before you include assert.h. But I still prefer gdb :)
Thanks for the tip
@@tonyjames9929 I am glad to hear that!
I like your tip but the problem is that asserts are dangerous in a cpp code as they might abort the program and prevent any memory deallocations.
@@ashwinalagiri-rajan1180 Sure, can be dangerous. But I used another trick: I create my own function like that:
void safe_assert(bool expr, size_t adress_cnt, …){
#include
if (expr != 0) return; //true expressions will be ignored
va_list p;
va_start(p,adress_cnt);
for (size_t ca = 0; ca < adress_cnt; ca++)
free(va_arg(p,void*));
va_end(p);
assert(expr); //will abort program after cleaning
}
@@IT_Shkolnik Will you have to pass in all the pointers you've allocated with new/malloc?
I found printf very useful for two things.
1.) Is live visual feedback of the data you're analyzing. I.E. moving the mouse around the screen and seeing how it's position changes.
2.) Is logging what code paths have been hit or not. You can do this in a debugger, but you have to take the time to manually step through the code and you have to remember what has happened where as printf logs all this information in the console.
Git tells me what debug statement I've added that need to be removed.
As much as I agree, that printf is not a debugging-tool, there are cases it might be useful (even though I would prefer using logging-tools) :
a) your release-build crashed with a segfault, but runs fine when compiled with debug-support. (usually this is when I try valgrind first)
b) you need to find a pattern, that may cause issues. Here having a lot of output, may help to see the pattern instead of stopping at every breakpoint
a-case is the worst type of bugs followed by a random hardware malfunctions and compiler errors. Sometimes combined.
It's always comes to "time to use print" vs "time to use debugger". Unfortunately not always debugger is faster.
Thanks for this one. I will definitely recommend it to my students.
One variant of "printf debugging" that I've used in the past is to write out data as a .csv file and then paste it into a spreadsheet for analysis. This is especially handy if there is some sort of calculation error in the code; you can have Excel do the same calculation and narrow down where the problem lies. It's also useful for finding patterns in the incorrect calculations.
Also, in Windows at least, it is surprisingly easy to write your own custom debugger, especially if you're just interested in high-level events like thread creation, DLL loading, and OS-level exceptions. Search the documentation for WaitForDebugEvent to see how.
I've spent most of lifetime writing embedded code on various platforms, some in assembler, most in C. I regularly use printf() to diagnose issues. On some smaller systems the hardware I am developing would take considerable effort to emulate, it is almost always better to make real code, run it on the target and use printf (or testing some functions on other platforms) to debug. On Linux I use printf followed by fflush(stdout), no issues with buffering. Sometimes I use gdb, but this can be more of pain than is worth, i've also seen a number of occasions on Linux where code running under gdb will not fault but the code run from the prompt does. Code with heavy use of fork can be a pain to debug on gdb.
Yup because code runs much slower when being debugged and this changes the program behavior.
I once got some advice that stuck with me. If you needed print or other debugging tools to make some code, then the person who is going to debug or refactor it later will also need it.
So leave those tools in*
*In a way that can be turned on and off, and/or do not disturb the code
I like that. Thanks.
Its good to remind beginners they have other options, and through most of school a debugger is better than print, but in my professional career for difficult bugs I probably have access to a debugger half the time, and even beyond that sometimes using prints is the best most efficient way because the interaction is too complex to step through because the bug happens when 100 things are happening at the same time. Prints & debugging are different tools in your toolbox, the important thing is to learn when to use them.
I know this will sound derogatory, but I feel like advice like this, presented as a "NEVER DO THIS" or "ALWAYS DO THIS" is very academic and not given by someone with a lot of real world experience.
I loved your opening skit and how you used the "Oh no, my program! It's broken" clip twice to show that no progress had been made. :)
If you wrap printf up with #define inside preprocessor conditional, you can control the printf output with compiler option -D[NAME].
The builds would be fine but the code would be dirty.
If an editor has pattern matching feature, like regex in Vim, it may be able to clear that. Given the identifier of function-like macro is DEBUG, vim commnad :%s/\(DEBUG\)(.\+
//g removes every lines starting with 'DEBUG('.
Another option I will use is fprintf(stderr, "debug message"), since stderr is unbuffered by default. In C++ I always use std::cerr instead of std::cout for that reason.
Alternatively, printf("debug message
"), since stdout is line-buffered by default.
A big reason I still often use print debugging is that I work in a lot of different languages these days, and most of them need/are best with different tools.
Print debugging works in pretty much all languages.
Yeah if you do complex stuff in C or similar it might fail because of the buffering and execution time influence, but honestly I don't really need to debug much these days, if something goes wrong I often have an idea already how to solve it.
But yes. In many situations, and if you're not constantly jumping languages, a purpose build debug tool is probably very helpful.
To mitigate the issue with removing printf, just wrap it in your own function, and then you can trivially edit your wrapper (either directly or via a compiler flag) to make it do nothing. Or use a function macro that can be empty under some compiler flag option
I've used a macro before. In one mode it is replaced with printf in the other it is nothing. So you can be sure all your debugging gets disabled and has no performance impact on the release build. That said a debugger is probably better and more powerful especially with watching variables. Sometimes the log nature of printf also helps when it's a logic error that doesn't crash. I think Devs should be able to do it both ways.
One way to get around the output buffering is to force flush the output. This can be done with a newline such as printf("hello
"), or with fflush() imediatly after you printf statement.
btw you don't seem to realize that ptrace is also slowing down a program execution.
So where timing matters that printf makes the bug go away, the same applies to gdb: all the ptrace syscalls also slow down the program.
As a seasoned "printf debugger" user, I always put my debug printf in the extreme left text column. The only other things in that column are the function names. Easy to spot and remove. But I do agree there are other issues like you mentioned with "printf debugging". It does modify the underlying code as well as affecting execution speed which can be an issue in some cases. When I first started embedded programming in 1985, there were no C debuggers I could use with Z80 and 6502 cross compilers! Old bad habits die hard!
Yeah, I still run into situations with microcontrollers where some flavor of printf-style debugging is my only option. If it makes you feel better, I still struggle to pronounce valgrind correctly. 😀
For embedded, I have Trace output to the serial debug port. The default level can be made to print nothing. Then you have error, warning, info, debug, and verbose levels that show more and more information. You leave these print statements in the code and then it is also available in the field. Heavily debugging your own code with a debugger can be a sign that you don't understand your code very well. I see novice programmers do this often, they run the debugger and hack the code until they get something working. The result is poorly written, fragile code.
this actually depends on the case, there are occasions when printing is the only option you have (unfortunately)
Yes, concrete example is always helpful. Now more curious for an example.
Awesome and informative video! Thank you Jacob! :)
I use a set of macros that I define as debug1(...). When the program is ok, debug1() yields a semicolon; When setup, the contents are displayed.
I find that I can leave these debug1() statements in place as they are set to yield the ; //no code to insert //
I can turn these debug statements on and off at different locations in the code. I do that when debugging functions.
I am guessing this video was not meant for VS and C# or chrome dev tools. Both allow watches and calls stack to be examined at certain break points.
There are loads of other debuggers as well, both command line and gui based ones for all languages including assembly. But I agree, using a dedicated debugger is better than prints. There are some cases where debugging is more difficult and debug or trace level logs are quicker and easier to set up. When the process is running remotly and you only access the code through network request calls. Still if you can set up a local server and mimic the call, a debugger is better
I honestly LOVE your content. It is very hard to find high quality information in C on UA-cam. You also make It very entertaining and fun to watch because you just try cool things like what happens if I try to go crazy with malloc and all that kind of stuff. Keep Up the great work, you Will get bit results
Using a logging framework is sometimes the only way to debug concurrent code. You can step through a thread's code in GDB but if it depends on conditions on other threads at the same time it can be hard to juggle all those threads in GDB. Split up your code into components with specific tasks and then you can look through the log and see what did what and when (logs must also have timestamps to be truly valuable).
It can all be useful. Print, I am actually using that in a small Python project right now, appropriate as it is small. For production stuff, I have used a logger, similar to print but a mask (set of flags) deciding to log or not and having ready access to turning the flags on or off at runtime. At development time the built in debuggers are usually OK, some can get over complicated and effect run time behavior or show wrong or unknown values for a variable.
I almost always agree with your opinions, but here I STRONGLY disagree:) I use every possible tool (debugger, print statements, logging to the server, logging to the disk) to find bugs. Sometimes print statements will help you find bugs way quicker. The only issue about them I might have is they are system calls and can change not the code base (you can revert changes), but the code execution. BUT you should use everything you can come up to do the job done quickly.
But how did the programmers that wrote the debugger, debug their code? 🤔🤔
maybe they suffered with the printf statements so that we dont have to.
They used the previous version. Bootstrapping everywhere!
Same way stackoverflow devs created stackoverflow without referring to stackoverflow.
Thank you! Great tips for debugging
Thanks a ton for all your videos.. I would also like to know how to debug a core file, what are the data that i can fetch from the frames, registers etc., Request you to make a video on this topic.
Yeah...okay, I get it conceptually; don't neglect a debugger if you can use one. However, in large real-world applications, there's almost always a logging system. Kernel has it, most binaries have a log of some sort. These are basically the same thing as printfs. Granted, they're not the same as DEBUG printfs, but lots of developers will wrap their debug statements in a macro that they can turn on or off with either arguments or #defines. There's a lot to be said for the usefulness of logging. You might have a core dump to look at, but to know how you got to this point a log is invaluable.
That's what I do: I wrap them in defines. Very handy because complex logging for this or that issue written a year ago is still available by uncommenting one line if an issue pops up months after it was "fixed" the first time.
when i learned data structures in C, my rule for myself was that every function should have an assert statemet, saved me a lot of time.
but now I've started developing on the cloud, and there's no debugger, and there's no local environemnt to run on..
so it's back to debug statements everywhere. only this time it's alot worse, because there's a jenkins job running before each deployment.
and yes, I'd love to see cases where the printf is lying.
Thanks for the info about buffered output. I've done printf debugging before, only to have my program crash before seeing anything. This made no sense to me because the print statement executed first, but now it makes sense.
Pro tip: Even if you can't do a raw write() to the respective file descriptor, the standard error stream stderr is unbuffered by default.
I need to watch your gdb videos. Have meant to do that.
I think that good log could be very helpful. If you have robust multi threaded configurable logging subsystem - then you could leave logging in place for prod deployment and use it for issue analysis. As usual there is no hard rules on what to log - one would need to strike a balance between logging sufficient information and avoiding excessive log generation. Also one should be careful not to log user sensitive information or allowing log subsystem be the root case of issues.
Since I love print statements so much for debugging, I do this instead:
At the top of my cpp file, write this macro:
#define debug
#ifdef debug
#define dbg(x) std::cerr
Great video! Thank you!
Normally I'm right down with your advice but this one's giving me pause. In industry we used bugs as an opportunity to add more comprehensive debug-level logging... so it's not just a printf, but it's basically a printf. We never had to do the cleanup step, which saves work.
Another good choice for printf debugging is to add a macro in code that wraps print output; then you can undefine that macro if it seems likely you want to keep it around or you can easily find and remove it without hitting all the other print statements in your code.
I'm pretty sure I already addressed this somewhere in the comments, but yeah, you're right. My intent here is not to criticize well thought out logging, but the "printf sprinkling" that beginners do.
@@JacobSorber I felt a little foolish when I saw your follow-up video that answered someone else making the same point.
I'm guilty of reinforcing the habit with my students, "show me the printfsf" is my immediate response when a student says their program is doing something weird. I should at least be demonstrating the wrapper macro approach.
That intro... Whose idea was the "oh no, my program, it's broken"? XD
Agree, intro was excellent
i still find them usefull, the best way the inconsistency pitfalls of printf is the run fflush(stdout) after ever print.
Tried to debug using debugger multiple times but it did break the flow and it was annoying and slower. Prefer writing unit test and printing things out. They were some situations where debugger was useful.
Generally I'd agree, use the debugger when you can, but it's not always possible or helpful. I work on a physics engine for a big racing game project which can't run in debug mode because it's too computationally intensive. We use a sort of "debug light" version so we can set breakpoints in larger functions and use the debugger when possible and where it makes sense, but a lot of stuff is optimized away like release builds so it's impossible to set a break point in smaller functions. Annoyingly this is always the case when developing new functions because they start small anyway, so using the debugger on new code is pretty much impossible until the functions get big enough. In cases like these logging is really the only way to debug.
Other examples:
1) We've had cases where the physics engine blows up or fails. Trying to figure that out by setting break points is next to impossible. I remember one time on a PS4 a few months ago we had a problem which turned out to be some obscure code struct written probably 20 years prior that hadn't been initialized which snowballed into NaNs which made physics go boom. Sure, you could set a break point somewhere and see one of zillions of NaNs after 15 minutes of building/compiling rerunning on a PS4, but then you have to repeat that for a gazillion calculations happening before that and are not anywhere close to finding where the NaN snowball started. So we did it with logging big chunks and certain functions and eventually were able to back track it to the cause. Doing that with breakpoints would have taken 1000x longer. In this particular case it would not have even been possible because I was helping another programmer find the issue in another country. He had a PS4 dev kit and I didn't, so there's no way I could have used a debugger there anyway. I had to show him what log stuff to write into the code, he'd do it, build and run it and send the log back. Each time was a 15 minute iteration time just to build and run it. It took us a couple days to find the problem, but we eventually did. It would not have been practical via debugger.
2) For tracking complicated variable changes over a period of time (say how forces and impulses develop in a physics engine), debugger rarely cuts it. You need something more like telemetry which logging does perfectly well.
On the "go back and remove the print statements:" Yes, that's a pain, but when logging really is needed over a debugger, what I do is wrap all these in a define specific to the particular problem I'm investigating. This way if/when something goes wrong in (or on the way to) that code again in the future I can just uncomment the define and the logging comes right back. When doing it this way there's no need to remove any of the logging until enough time has passed that I'm pretty I won't ever need it again. All I have to do is comment out one line and presto, all gone. I have never once had a logging statement itself cause a problem unless I made an error in the statement itself, nor has the removal of a logging statement ever caused one either that I can recall in my 20+ years of working on game engines. So the "it changes the code" argument, while technically correct, doesn't really ring any practical bell in my day to day experience. I don't worry about that at all when logging, and I can't recall a single time where it's ever been a problem. With new programmers maybe that's the case now and then (I don't work with students so may be missing that part of the picture), but I wouldn't want new programmers to get it in their heads that none of the pros use logging. We most certainly do. :)
Another argument was that the print statements come "some time later." While I've often heard that's the case, I can't think of a single instance where that bit me in any sense. Usually when logging I'm tracking something from one part of the code to the next over time. If you print A, then B, then C, they will always show up in that order regardless of when they were actually written out. Yes of course this does mean that if you logged something and your function crashes the log may not show up, but in 99% of the cases where I'm using logging this isn't an issue at all. I almost never debugging an actual crash anyway.
It should be noted that both Unreal and Unity engines both have windows dedicated to logging. It's there for a reason.
Having said that, I'd agree that it's best to use the debugger when you can, and new programmers should take the time to learn how it works and play with it, but they shouldn't hold this "don't use logging" as gospel as though it's something only noobs do. It's not. There is absolutely a place for it, mainly when the debugger isn't of much use which happens fairly frequently in my experience.
The debuggers learning curve is a really good example of how putting the time in upfront can pay dividends later on.
I use #ifdef for my debug code and wrote a debug library which i can switch off in release mode. Yes it changes your code. But debugging with a debugger changes the behaviour aswell when it comes to timings. (i am embedded developer). When i use my debug libary i am always keeping in mind that i have to test my code in release mode too. For me it works like a charm and delivering fast.
for linux I prefer gede, gives me a semi-decent GUI for gdb stuff
My code compiled with no errors but the program is ending early in C, i am working on an algorithm and it got about 100 lines of code long in C, I am ysing print statement to track the progress of that algorithm on my input matrix and I am not being able to find where the problem is, I worked throughout the code by doing the algorithm by hand and it looks like everything is fine, but the code... it's not working as it is supposed to. Any idea on how to debug programs that compiles successfully?
Yeah unfortunately I end up a lot in the situation where breakpoints do not work because the debugger just doesn't want to connect to the build. And if you can get Visual Studio to connect to Unity every time, on Windows AND Linux (VS Code in that case) great! But it's not happening. Print debugging is fine, I have never caused problems by "leaving print statements in", version control will catch it. You should never blindly commit, always look at all the changes. That gives me a chance to remove any I forgot. Also this has never "broken the build", just printed unnecessary statements. And as others have stated, debug mode changes code behaviour as well, so the advice in this video can never be a hard and fast rule.
printf are often faster when you debug not for crashes, but for incorrect execution due to mistakes in algorithm
I used to do this lol, nowadays I just do 3 things: logging, assertion, and spawn a debugger when needed.
I've had something similar happen to me in Java.
I was using a (probably over complicated) multithreaded system, and it was occasionally crashing. When I put a print statement to try and see what was going on, the problem dissapeared. Turns out, printing something in Java synchronizes it, equivalent to putting something in a synchronized code block.
Nowadays, I mostly diagnose crashes with the IDE debugger, and console prints are limited to general information.
You can use Git to remove the print statements.
fflush stout after each printf :)))
But seriously, printf debugging is useful in many situations, and in many situations it's also the fastest way to find bugs. But especially for more automated testing and more generic testing it's not very useful and it also isn't always reliable as you mentioned.
What should be the best approach to debug (for implementation or logical errors) while coding for coding exams with short time limits (3-4 hours)?
Here the code is small, only 1-2 files, so you don't have to worry about removing debug statements,
and there won't be any segmentation faults or anything (most likely)
What should be the best way?
What is your opinion?
Thanks in advance.
Open VS Code, click the "Run and Debug" tab on the left and debug your code using that. No need for asserts or printfs, and if you're using an interpreted language like Python then you don't even need to click the "create a launch.json file" text.
It is criminal to not know how to use gdb (or similar debugger for your platform). If you write C code, knowing how to effectively use a debugger is a must..
I even ask debugging questions in interviews...They judge people better than leetcode style questions..
I think, sometimes, you still need to use the old way "printf" like when your program depends on interrupts . I could be wrong.
@gsdcbill no, not from an interrupt routine. I mean when your code depends on interrupts and you need to debug it in real time. For example, when I have a GTK application running using interrupts reoccurring at every 10ms. How can you debug?
This is especially problematic in embedded. Introducing printf causes major disruption in the code execution timing and size. I've even seen how sometimes code is developed with all the printfs, then once you are done developing and you remove them, the application doesn't work!!
They are a good tool for runtime debugging, but they can cause major problems and bottleneck unless you go extra creative and complicated and use dma and a big ring buffer, which kinda defeats the purpose a little bit.
Thanks
definitely concrete needed
No problems to create my program with debug and assembler mnemonics. Printf never used.
The best debugger that's been around for centuries: pen and paper with pseudocode and mindmaps/activity & sequence diagrams/truth tables.
If your code isn't right at the high level, how would you hope to get it right at the low level?
I find it works the other way around for me. I work with the code at a low level first and make the architecture mistakes and once those mistakes make themselves loud and clear, I fix them and find better ways of doing things. Then when I like the state the code is in, I find ways to make the architecture more powerful by pulling things out into separate functions and grouping data into structures.
This doesn't happen in a step by step process though, like I can start with the data structures first or I can start to immediately break a piece of code into a separate function. But overall I find it easier to make good working code from revisions of a bad architecture than to make good working code by laying everything out from the start.
@@nates9778 Yeah I got myself a non linear process too where there's a key inspiration that jettisons the ideation process; whether it's a small data structure, a visual, or a cool code snippet. Creating SOME idea of what the program is going to do through a simple REPL hones in on an expected target range of behavior, which is super helpful for the lower level stuff that I do. That kind of exploratory spiral of cycling between low level & high level prototyping has been something I've done for over a decade and it's taken the better part of the last few years for me to create a sensible playbook of what needs to be done at what stage, but I still feel the need to "instinctually" find my way around the code with some good frameworks to help me along the way until I find a good solution for the problem.
Sometimes if the buggy code forces you in the weeds, stepping back and seeing the forest with pen/paper observation is fine. Sometimes if you know exactly what changed the behavior to be buggy, going for the debugger or printf introspection is more useful. Thank you for sharing and being a part of the discussion. :)
It's a different kind of debugging. Both are needed in different situations.
You won't find bugs introduced by one-off errors or similar with such high level "tools", and these tend to be the biggest part of my bugs.
But yeah, if you've been at it a while it's a good idea to use these tools to ensure your basic concept is working as intended.
Writing documentation caught so many screw ups in my career, you wouldn't believe it. Formulating what I just did in plain English made me all too often go: Wait, that's not what I just implemented there. 😄
@gsdcbill That never really worked for me. Writing tests is still in context code for me and using gherkin feels strange. But yeah, I'm aware that that's my own little thing so I second your suggestion of giving it a try.
ok I agree and disagree. I agree because a debugger IS an essential tool. Where the video goes wrong I think is that embedding printfs into your code is a good way to debug if you can turn them on or off, ie: a logging mode. IE: you put the debugging statements there and keep them there as a framework where you can flip a switch and see them again.
In my mind, every statement like this is a little historical record. You see common places in the code where things have happened in the past and you can see why people printed out these statements. If bugs cluster like they usually do, you can debug them again. whereas a debugger is ephemeral - all the work you put into it goes up in smoke when you leave.
I always advise people to have an effective feedback loop and an Experiment Turnover Time (ETT) as low as possible. Effective feedback and low ETT give you more ideas and hypothesis to test. It prevents you from becoming AIR (AI Replaceable). 😂😂😂
0:13 An experienced printf debugger will not indent the debug statements, so they're easier to remove later. :p
Idk. If setting up a proper build system like CMake with unit tests wasn't so hard, I wouldn't use print debugging and much rather create meaningful unit tests instead. But after all, it's still C, what am I asking for? Cannot expect that 😂
How can we debug vector crashes? Eg if I load an image into a debugger and it jumps to vector crash and the debugger goes crazy and you can’t see any memory info or local variable etc - what’s the process to debug this?
while (program.broken()) {
furiously.add(print_statements);
}
Dis guy be over here spamming the backspace key to delete his debug statements when he could be using vim key bindings and just dd
0:46 my mind: why he doesn't just comment it
Adding a macro wrapper is a great option
None of the performance issues all of the printf fun. printf debugging is particularly useful than gdb if your program crashes after a long period of time.
#if VERBOSE
DEBUG(X) printf(x)
#else
DEBUG(X)
#endif
Wrapping all prints
DEBUG("HELLO world
");
Capitalize the "NOT" in your title :P
I have a PhD in printf debugging
OH NO... MY PROGRAM... ITS BROKEN
but you dont provide any solution..
We all were there
for the algorithm
I was expecting something more visual than just talking for almost 5mins without visuals. A little bit lazy in my opinion.
printf("yeet");
fflush(stdout);
You are wrong
Think. Then put in a minimal where it might be...
If something changes Hurrah!
If not, take it out.
Think
Goto top
(This is true if you use printf() or your debugger breakpoints)
It is, nevertheless, great fun, especially if you didn't write the code in the first place.
The point is to understand WHY it was not doing what was intended. Won't do that agin....
Good stuff, Jacob. (It is a call to my past, but I like your view on the business of coding)
That is why I ,like a good programmer should , always use fprintf(stdout,
why not printf?