keep in mind that sending core dumps may have privacy implications. For example, if Excel or Word crash and send a full core dump to microsoft, it would be sending the document the user was viewing/editing as a side effect (since it is loaded on memory), it is also true for any program data manipulate user files/data.
I like C. It's pretty low-level, and it occasionally feels like chopping down a tree with a really sharp pocket knife, but from a language-standpoint it's pretty simple. The complications mostly come from the underlying system, which it doesn't hide from you-not the language.
Being a part of organisation as a QA, I find this video very helpful. Explained in a way which is very easy to understand. Thank you and keep sharing the knowledge.
Hi Jacob , this was a great video but I still have a question. When you deploy your code you are usually using release build which is optimized and probably won't have the debug symbols and further more the target might not have a lot of memory. So how can we use this core dump in such scenario?
IAs someone who writes code for embedded systems and uses mainly python for tasks on my PC, this was _really_ interesting, because I didn't ever do a core dump - I mean, normally I don't even have a file system 😅
Thanks. I enjoyed your video. I'm trying to run RLCraft (a modpack for Minecraft) and have fairly rudimentary coding skills and knowledge. Anyway, the crash report that I got when I loaded the launcher said, "Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again." So that's how I found your video. I still don't know what to do to fix the issue but I do now understand what it's talking about. Thanks
I always thought that a core dump was something you had to submit to the CPU manufacturer (Intel/Amd/etc) or to NASA so that technicians in white lab coats could read the machine code that was in the CPU at the moment after the program crashed using an open CPU with lots of wires connected to it and where they looked at the contents of the memory with an electron microscope. But then it turns out that anyone can look at the program code with a simple debugger. Why couldn't they say this 40 years ago when I started as a programmer?
great video!, Since you usually don't release with debug-symbols, I wonder how useful the coredump may be in this case. How can you extract anything without debug-symbols?
Good question (and good idea for a future video). Debugging without debug symbols is definitely more painful. You can still look at the assembly. You can also release with debug symbols (accepting the downsides). Or, you can generate two versions of the binary (one with and one without debug symbols) and then use the debuggable version with the core dump. For release builds, I think I favor the last one.
@@JacobSorber A video on the Linux (POSIX?) LD_PRELOAD environment variable would be interesting. Solving circular includes (such as with include guards) would be a useful video for people writing code. (Assuming you haven't already covered these, haven't yet watched all of the videos. They're quite good and to the point.)
@@JacobSorber Ah, ok. I had been watching the videos from oldest to newest to make sure I watched them in order, and hadn't gotten to them yet. There is inline asm (unsure if clang has it, but GCC does), but that's rather obscure and most people don't use it. Videos about common C patterns, such as common ways of structuring your programs would be interesting. (Such as writing code that uses flags, and why you would do such a thing.) Could go into common ways that C++ library methods are written using C. This bit me when I recently tried using the Windows Direct2D api as the way it's structured to translate to C isn't obvious (the docs are all for C++ as well..) but I don't know if that's a common way of translating C++ to C or not. There is writing a less-featured clone of a linux utility (perhaps ls?) and comment about the design decisions you make as you write it. That would be different from your usual videos though. Since you seem to do embedded fairly often (?) you could talk about writing efficient code, and how the compiler also at times knows better than you. Examining the assembled output of simple C programs to see how there can be differences. (godbolt.org is quite nice for such a thing, but this would also be a good way to also show some linux utilities)
How to deal with coredumps that are generated/dumped in the target which is running on a different architecture? Assume, -> Host is x86 and Target is armhf -> Target doesn't have gdb/memory to debug on Target itself.
So... If I set the limit to a finite number and the core dump gets truncated... how useful is it gonna be than? Can I be sure to at least have the stack in it then?
Yeah, gdb doesn't come installed by default on MacOS. Also, Apple favors clang and lldb, and has done some things that make it tricky to get gdb working well on MacOS.
One of my binary crashed. And the bt is pointing to (static) library part I tried compiling library with debug sumbols enabled. But the bt is pointing to wrong code How to debug this scenario ?
You didn't explain how to compile the program to contain all the debugging information so that the core file could be debugged in situ on a remote system where the source code to step through would not be available, for example at a client's site.
Wait, that means the user finding bug will need to tinker with the machine (assuming can get admin privileges) then try to crash it again to actually get a core dump. Sounds like too many loops to jump to help the developer to fix the app. Then how those apps with crash reports can get it without that?
It's only an assumption. But I guess programs with crash reports run a small runtime that catches segfaults and saves them before the operating system receives and trashes them.
If you set it to like half the size of the full core dump is will that possibly not include the info you want or maybe not let the debugger run properly? I don't understand what information it absolutely needs.
Good question. If the core dump is larger than the limit, it won't save any of it (rather than saving part of the memory). It's all or nothing, because it would be difficult to know what memory is most helpful.
my school won't let me use ulimit -c to anything other than 0 on their machines. So I just used git and used it on my local machine. Hope that helps anyone.
One way is to use a shim to intercept malloc, calloc, realloc, and free calls, and then keep track of what blocks have been allocated and their sizes, and then check to see what blocks aren't freed at the end.
"Now, when this program eventually crashes, as we knew it would, it says that it 'Core dumped'. It means the computer dumped the content of the process' memory when it crashed. Except for the fact that it really didn't. I'm sorry to break it to you, but sometimes computers lie." - Jacob the Sorber, 1984, University of Wakanda
keep in mind that sending core dumps may have privacy implications. For example, if Excel or Word crash and send a full core dump to microsoft, it would be sending the document the user was viewing/editing as a side effect (since it is loaded on memory), it is also true for any program data manipulate user files/data.
Which is the entire point. Else it's useless.
Another great video, and a massive help for all of us who love coding in C but haven't had much of a chance to use it professionally. Thanks Jacob!
I like C. It's pretty low-level, and it occasionally feels like chopping down a tree with a really sharp pocket knife, but from a language-standpoint it's pretty simple. The complications mostly come from the underlying system, which it doesn't hide from you-not the language.
@@JacobSorber True..
"You could fill up your disk pretty quick, especially with how frequently your code crashes."
ROASTED
Mile-a-minute, dense, chocked full of stuff I missed or never knew. Wish I'd had a prof this knowledgeable when I went to college. Great stuff, Jacob!
Being a part of organisation as a QA, I find this video very helpful. Explained in a way which is very easy to understand. Thank you and keep sharing the knowledge.
That explosion break my ear lol
Sorry. I hope it heals. Segfaults are serious. :)
Your comment saved my ears 2 seconds before the Big Bang
Thanks a lot 😅
Same here
Nice video! But of the most helpful things with core dumps is the ability to walk the call stack in gdb.
Hi Jacob , this was a great video but I still have a question. When you deploy your code you are usually using release build which is optimized and probably won't have the debug symbols and further more the target might not have a lot of memory. So how can we use this core dump in such scenario?
Good question. Sad I see no answer:(
Thank you for this! I have a GTK widget that crashes every other leap year, will examine with the core dump.
There isn’t a better video than this in the world!!
@Jacob Sorber... one of the best video on core dump analysis ...Thanks Jacob
My pleasure. Glad you liked it!
Professional grade explanation, much appreciated.
Superb... Gonna watch all of his video... Thanks a lot sir...
You're welcome. All the best.
Loved the analogy and animation
Thanks. We had fun making it.
IAs someone who writes code for embedded systems and uses mainly python for tasks on my PC, this was _really_ interesting, because I didn't ever do a core dump - I mean, normally I don't even have a file system 😅
For a detailed discussion, you could have talked about the "??" problem when debugging with coredump files.
Good idea for a future video. Thanks.
Thanks. I enjoyed your video. I'm trying to run RLCraft (a modpack for Minecraft) and have fairly rudimentary coding skills and knowledge. Anyway, the crash report that I got when I loaded the launcher said, "Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again." So that's how I found your video. I still don't know what to do to fix the issue but I do now understand what it's talking about. Thanks
love the analogy with space travel. coredump are precious things. :)
I always thought that a core dump was something you had to submit to the CPU manufacturer (Intel/Amd/etc) or to NASA so that technicians in white lab coats could read the machine code that was in the CPU at the moment after the program crashed using an open CPU with lots of wires connected to it and where they looked at the contents of the memory with an electron microscope. But then it turns out that anyone can look at the program code with a simple debugger. Why couldn't they say this 40 years ago when I started as a programmer?
thanks for sharing the knowledge, btw we are using the same slack notify sound. lol
Great video... Thank you, Professor
Most of them are sending minidumps, along with other possible values. It would be lovely if you made a video on minidumps as well.
I'll see what I can do.
kill -SEGV 4787
"Oops, wrong pro..." *universe explodes*
great video!, Since you usually don't release with debug-symbols, I wonder how useful the coredump may be in this case. How can you extract anything without debug-symbols?
Good question (and good idea for a future video). Debugging without debug symbols is definitely more painful. You can still look at the assembly. You can also release with debug symbols (accepting the downsides). Or, you can generate two versions of the binary (one with and one without debug symbols) and then use the debuggable version with the core dump. For release builds, I think I favor the last one.
@@JacobSorber Right, But as far as I understand it, if I put in Debug-Symbols I cannot use the optimize-flags.
Thankyou for helping the ‘C’ community
This was very helpful. Thank you!
Thanks! Your videos are very informative, and have helped me a lot.
Glad they're helpful. Let me know if there are other topics you think I should cover.
@@JacobSorber A video on the Linux (POSIX?) LD_PRELOAD environment variable would be interesting.
Solving circular includes (such as with include guards) would be a useful video for people writing code.
(Assuming you haven't already covered these, haven't yet watched all of the videos. They're quite good and to the point.)
@@MinusGix I have covered LD_PRELOAD (see the videos on shims) and include guards (fairly recent video on headers).
@@JacobSorber Ah, ok. I had been watching the videos from oldest to newest to make sure I watched them in order, and hadn't gotten to them yet.
There is inline asm (unsure if clang has it, but GCC does), but that's rather obscure and most people don't use it.
Videos about common C patterns, such as common ways of structuring your programs would be interesting. (Such as writing code that uses flags, and why you would do such a thing.)
Could go into common ways that C++ library methods are written using C. This bit me when I recently tried using the Windows Direct2D api as the way it's structured to translate to C isn't obvious (the docs are all for C++ as well..) but I don't know if that's a common way of translating C++ to C or not.
There is writing a less-featured clone of a linux utility (perhaps ls?) and comment about the design decisions you make as you write it. That would be different from your usual videos though.
Since you seem to do embedded fairly often (?) you could talk about writing efficient code, and how the compiler also at times knows better than you.
Examining the assembled output of simple C programs to see how there can be differences. (godbolt.org is quite nice for such a thing, but this would also be a good way to also show some linux utilities)
How to deal with coredumps that are generated/dumped in the target which is running on a different architecture?
Assume,
-> Host is x86 and Target is armhf
-> Target doesn't have gdb/memory to debug on Target itself.
Great video
very helpful , thank you !
So... If I set the limit to a finite number and the core dump gets truncated... how useful is it gonna be than? Can I be sure to at least have the stack in it then?
I LOVE YOUR VIDEOS!!!!!
great, but an issue, i have a mac, running vscode on its terminal, when i tried gdb but it says "bash gdb command not found"
Yeah, gdb doesn't come installed by default on MacOS. Also, Apple favors clang and lldb, and has done some things that make it tricky to get gdb working well on MacOS.
Jacob Sorber yeah, lldb worked
thanks matthew
Are you sitting on the floor or some small chair? I am looking for ergonomic sitting positions as debugging can take hours or days.
That intro is NOT a movie💀😭
One of my binary crashed. And the bt is pointing to (static) library part
I tried compiling library with debug sumbols enabled. But the bt is pointing to wrong code
How to debug this scenario ?
very Informative, Thanks for the video!
You're welcome!
great video! thank u
You didn't explain how to compile the program to contain all the debugging information so that the core file could be debugged in situ on a remote system where the source code to step through would not be available, for example at a client's site.
Can I unlimit -c for certain processes, but not for all?
My linux system has core dumps to unlimited by default.
I'm using Arch
Wait, that means the user finding bug will need to tinker with the machine (assuming can get admin privileges) then try to crash it again to actually get a core dump. Sounds like too many loops to jump to help the developer to fix the app. Then how those apps with crash reports can get it without that?
It's only an assumption. But I guess programs with crash reports run a small runtime that catches segfaults and saves them before the operating system receives and trashes them.
"core" file is not being generated in my linux. I went through all the same commands, as it is explained in the video
If you set it to like half the size of the full core dump is will that possibly not include the info you want or maybe not let the debugger run properly?
I don't understand what information it absolutely needs.
Good question. If the core dump is larger than the limit, it won't save any of it (rather than saving part of the memory). It's all or nothing, because it would be difficult to know what memory is most helpful.
@@JacobSorber so what is the use of adding a limit if it won't use the limit?
@@XenoTravis maybe it means.. if the size is lesser than the limit then only make core dump, otherwise forget it.
I have some problem in usage of GDB. can you please help me out here?
sad little zero made my day
i have core dump file how can i analyze it and which tool i need to download
My problem under Ubuntu - no core dumped generated. I tried everything already..
is there any video about makefile that you uploaded ?
Yeah, I have a few. ua-cam.com/video/a8mPKBxQ9No/v-deo.html
@@JacobSorber thx jacob, hope you can make series about make more than 60 seconds 💪💪💪
rand()%255 should have been rand()&255 or rand()%256 :)
Not if the purpose was to generate random number from 0 to 254. :-)
what is this std bloatware you speak of I write my HAL in my applications
Are coredumps available in python?
There's hope...
my school won't let me use ulimit -c to anything other than 0 on their machines. So I just used git and used it on my local machine. Hope that helps anyone.
nice
i didn't find core file after dump
Feeling sad for the one who was flying 🥲
Yeah, seg faults are sometimes scary and sad. But, I have it on good authority that he bailed out in time.
@@JacobSorber now I can sleep comfortably. 🙏
Poor Lego guy :(
I know. Tragic.
"Taint analysis" , "core dump".
lol, that was an overly dramatic intro.
Oh boy, core files are certainly big. I have a small program that generates a 71MB core file.
How to write our own memory leak detector?
One way is to use a shim to intercept malloc, calloc, realloc, and free calls, and then keep track of what blocks have been allocated and their sizes, and then check to see what blocks aren't freed at the end.
@@JacobSorber Is it possible with MSVC on windows?
@@kapilverma2668 Yes
@@BetaChri5 How
3:00 The dump is a lie...
"Now, when this program eventually crashes, as we knew it would, it says that it 'Core dumped'. It means the computer dumped the content of the process' memory when it crashed. Except for the fact that it really didn't. I'm sorry to break it to you, but sometimes computers lie." - Jacob the Sorber, 1984, University of Wakanda
A good introductory video, but a bad waste of 1 minute at the start. Cheers!
Starts at 2:00