wow I just completed my assignment a week ago involving concurrency and shared memory... and you just now post this haha I could've used mmap but I preferred using shmget and shmat for my assignment.
Hi Jacob, Good job in making these topics more simplier. I am a computer science instructor just like you and I highly recommend your videos to my students. Keep it up!
Great video Jacob. To continue in the same subject, I'd like to see some example where share memory is useful. Also, are multi process queue implemented with shared memory ?
Is shared memory actually faster than cached disk I/O? Cached I/O, especially if it's memory inside the L1 or L2 cache, seems to be as fast as it gets, right?
Loved the video! But I feel there's so much to explain here that a video might not always be enough, so for those seeking for more resources to understand shared memory (or system v ipc) like me, check the "Linux programming interface" book. Chapter are self contained so there's no need for reading the entire book if you have some prior knowledge.
Hi! Love your videos. I would love a video of how to setup a testing environment for C. Is there a testing library for C and how do I set it up? I've heard that some people use C++'s Boost library to test their C code, but I'm not sure if that's the way to go...
Hi Jacob, For ftok system call, the man page written "Today, proj_id is an int, but still only 8 bits are used. Typical usage has an ASCII character proj_id, that is why the behavior is said to be undefined when proj_id is zero", but in the example you put the 0 there, I tried several alternatives, it seems fine with 0 or nonzero. Do you know why? Thank you
This is definitely very interesting, but how does one prevent memory leaks in case the program crashes? Does the linux kernel know how to free memory if none of the programs that interacted with it are running or does that block of memory stay inaccessible until reboot?
You're right. This can get a bit tricky. The memory needs to be explicitly removed with shmctl, so if the program dies, you will have a leak unless you have another process that can clean things up. Depending on the application, it is often possible for one of the involved processes to detect that a process crashed, and make sure that appropriate cleanup happens.
Vim and I have a complicated relationship. Still trying to figure out exactly how to define the relationship. Let's just say I like both for different reasons.
Hi Jacob, thanks for the Awsome tutorial. I have a question about the file that we associate with shared memory. If this file(name) is one of the process (executable file) itself, will that may cause any issues??
in my operative system course they call this 'message queues' and for me, it's so difficult to grasp. They ask to solve problems with threads synchronization first with semaphores and then do the same but with messsage queues.... using this like blocking access tools! AAAAH
It may sound dumb but if we are not using a common file as a means of sharing memory between processes, why do we need to specify a correct pathname of a file that must exist on disk?
It just doesnt feel like in the spirit of "Shared memory". Like all this in effect is doing is allowing shared access of a file that MUST exist on a disk. Although I rewatched the video and the part where you describe that I must not confuse them with mmaping content of a file but from what I get is shmget() is basically doing the same thing as mmap() but guaranteeing that contents of file are not changed. Shared memory in essence or what I would normally expect is one process being the host of the memory and other just have access to it like it normally would. For example the earlier tutorials about shared memory for parent-child fit the definition of what is expected. But I think in this shmget method we are introducing uncessary mediators like the file that must access on the disk, and then the particiapting process would intearct with it like they would with a file (but without changing its contents). Seems like a very old and inconvineint shared memory api, because in theory, existence of a mediator (like a file) shouldnt be involved at all during shared memory. It should just work the same as they were in the parent-child scenario I would love to hear your thoughts on it and if i said something stupid.
I coded along while watching the video and I just wrote #define SEMAPHOR_PRODUCER "seemaphor producer" And did the same thing for semaphor consumer as well.
Ok , but how to implement that shared memory functions (shmget()) without using existing header files, I mean implement that header files by your own sir
imagine if you could share memory in the kernal and have daul or more internet connections, one std ethernet and this hotspot at the same time and you could direct your packets rx tx, i wonder what would happen if i could power the kernal down and still have windows running and then make a system image
@@JacobSorber Can you also make a video on Program execution sequence , what actually happens when we enter ./a.out or (run any executable file)internally in O.S
I mean that the block of shared memory is associated with a filename. So, if a process wants to find the block of memory, it looks it up using the filename. It's simply the mechanism for finding the block. There is no relationship between the shared memory and the contents of the file.
Hello Jacob, due to Makefiles I think following topics to consider: - %.c : %.o notations and alternatives if any - some defaults rules to write makefile as short as possible with some variables describing comiler, compile/linkage flags - how to mix C and C++ build in makefile - considering CMake to generate makefiles - imvoking one makefile from another if possible
Like if I decode it : I am writing filename different .small GCC -g -wall -c write.c shared_memory.o -o write.o GCC:error: shared_memory.o: no such file or directory Consider write.c as writeshmem.c
very difficult format for compiling as I dont know how to break down your method to compile alternatively - will check out your compile video **make: *** No rule to make target 'writer.elf', needed by 'all'. Stop.**
Meaning, how to detect a particular frequency, additive noise (sine wave added on top of other signal), or how to detect whether the wave file is a single tone (sine wave)?
@@JacobSorber i am trying detect certain pattern . The sound pattern is a sin wave whose length is 0.01 seconds . The sound wave starts with a maximum amplitude but decreases gradually giving it a click sound . When i view the wav file in audacity I can clearly see the two spikes but when i read that file in to a numpy array things get messy . Tried finding the rms valve of 0.01 seconds of audio at a time . I thought I will get a abnormally large rms valve and i get that but i get multiple large rms value in a wav file with just 1 click
I'm loving the new layout and graphic highlighting! This was much easier to follow.
Glad you like it! Always trying make things a bit easier.
I find your videos incredibly well explained. The drawings and explanations in the begining really help before getting to see actual code.
Great video, would really like to understand semaphores in next.
Thanks. I'm not 100% when the semaphore video will come out yet, but it's coming.
Thanks for the awesome content, very well produced.
I'm literally binge watching the entire channel haha
A video on FFT/signal_processing or bluetooth would be cool.
Thanks. I'll see what I can do.
8:16 that shmdt cracked me up.
Prior to the C89 standard, identifiers had a max length of 6. That's probably a big part of the function names.
Good point. I sometimes think I have repressed those memories. :)
@@JacobSorber So that's an example of where shared memory shouldn't be used!
This video saved my soul in a current project.
Thank you very much, keep it up!!
greetings from Austria
That's an awesome explanation! You made is so much easier to understand this :) Thanks a lot Jacob
wow I just completed my assignment a week ago involving concurrency and shared memory... and you just now post this haha I could've used mmap but I preferred using shmget and shmat for my assignment.
Nice. I hope it worked out ok, even though I was slow.
@@JacobSorber No worries it worked out perfectly fine! I actually plan to help my peers by referencing your videos! Keep up the great work!
Thank you for the explaination !!
Pronunciation of “shm” functions is what’s made this video
Also a snapshot of Make file in key moments would be amazing!
Hi jacob .
Great video, would really like to learn about epoll in next video
I just wanna say..thank you..thank you so much..this is so helpful
You're welcome. Glad I could help.
Hi Jacob,
Good job in making these topics more simplier.
I am a computer science instructor just like you and I highly recommend your videos to my students.
Keep it up!
Thanks, Artemio. I appreciate the support. Let me know if there are topics that you think your students would specifically benefit from.
@@JacobSorbermake a full c language course videos from beginning to advance level
Maybe I'm an old UN*X hand, but the promise of unified VM/filesystem is that two programs mmaping the _same_ file effectively shared memory.
In a way, I would call this "how to play with SYSV shared memory"
Just an idea of something I'd love to see: make a video about sharing memory but between Python and C.
That would be fun. It would probably be basically the same mechanism, with some sort of python bindings on it. I'll see what I can do.
@@JacobSorber Thanks for the great videos. Does one already exist that covers the request of @John McKeown?
Mr Sorber,
You are really doing a great job ❕👍🏻
Thanks You 🙏🏻
Great video Jacob.
To continue in the same subject, I'd like to see some example where share memory is useful.
Also, are multi process queue implemented with shared memory ?
Great knowledge sharing. 😄
That was excellent!
Outstanding!
Thanks!
Is shared memory actually faster than cached disk I/O? Cached I/O, especially if it's memory inside the L1 or L2 cache, seems to be as fast as it gets, right?
Very useful. Thanks
You are welcome.
Loved the video! But I feel there's so much to explain here that a video might not always be enough, so for those seeking for more resources to understand shared memory (or system v ipc) like me, check the "Linux programming interface" book. Chapter are self contained so there's no need for reading the entire book if you have some prior knowledge.
way better than my Chinese tutor
This video can still be helpful for Windows users who use WSL.
Thanks for pointing that out. I haven't spent much time with WSL, so I'm often hesitant to make claims about what it does and doesn't support.
Also a posix approach aka with shm_open etc would be cool
Hi! Love your videos. I would love a video of how to setup a testing environment for C. Is there a testing library for C and how do I set it up? I've heard that some people use C++'s Boost library to test their C code, but I'm not sure if that's the way to go...
Hi Jacob,
For ftok system call, the man page written "Today, proj_id is an int, but still only 8 bits are used. Typical
usage has an ASCII character proj_id, that is why the behavior is said to be undefined when proj_id is zero", but in the example you put the 0 there, I tried several alternatives, it seems fine with 0 or nonzero. Do you know why? Thank you
This is definitely very interesting, but how does one prevent memory leaks in case the program crashes? Does the linux kernel know how to free memory if none of the programs that interacted with it are running or does that block of memory stay inaccessible until reboot?
You're right. This can get a bit tricky. The memory needs to be explicitly removed with shmctl, so if the program dies, you will have a leak unless you have another process that can clean things up. Depending on the application, it is often possible for one of the involved processes to detect that a process crashed, and make sure that appropriate cleanup happens.
Hi Jacob. Thanks for the content. Would this same concept work to share memory between docker containers?
You are the best teacher! Do you actually prefer using VS Code instead of vim, or is it just for those lessons?
Vim and I have a complicated relationship. Still trying to figure out exactly how to define the relationship. Let's just say I like both for different reasons.
@@JacobSorber lol
In the future, you may want to use a De-Esser on your audio file.
hmmmm, @7:33 are you comparing result to casted char * of (-1) ?
shouldnt it be: if (*result == IPC_RESULT_ERROR) ? ..
@@hasanalattar9561lol this is 3 years late, but pointers are usually unsigned numbers, so he can’t compare -1 directly.
Hi Jacob, thanks for the Awsome tutorial. I have a question about the file that we associate with shared memory. If this file(name) is one of the process (executable file) itself, will that may cause any issues??
What’s the difference of using those methods versus using fopen(), and writing and reading from a regular file?
great as always.. but instead of saying unix based you might consider the term POSIX :) ..may linux folks might argue otherwise :P
awesome, !!! could you please post a video about how to use semaphores in thread synchronization,,!
That's the plan. Not sure when it will come out. But, it's coming.
What is the logic behind separating the creation of a shared memory segment and getting a pointer to that memory segment?
Can a bool function return NULL in c? 9:08 (line 52)
no
in my operative system course they call this 'message queues' and for me, it's so difficult to grasp. They ask to solve problems with threads synchronization first with semaphores and then do the same but with messsage queues.... using this like blocking access tools! AAAAH
It may sound dumb but if we are not using a common file as a means of sharing memory between processes, why do we need to specify a correct pathname of a file that must exist on disk?
How do you use shared memory on bare-metal applications (e.g. on an ARM Cortex-A15 core)?
thanks a lot.
You're welcome! Glad you enjoyed it.
In destroy memory block function you wrote get_shared_block(filename,0);
Shouldn't it be get_shared_block(filename,BLOCK_SIZE);
Am I wrong?
however this doesn't work from a sandboxed app on mac (shmget gives "operation not permitted").
It just doesnt feel like in the spirit of "Shared memory". Like all this in effect is doing is allowing shared access of a file that MUST exist on a disk. Although I rewatched the video and the part where you describe that I must not confuse them with mmaping content of a file but from what I get is shmget() is basically doing the same thing as mmap() but guaranteeing that contents of file are not changed.
Shared memory in essence or what I would normally expect is one process being the host of the memory and other just have access to it like it normally would. For example the earlier tutorials about shared memory for parent-child fit the definition of what is expected. But I think in this shmget method we are introducing uncessary mediators like the file that must access on the disk, and then the particiapting process would intearct with it like they would with a file (but without changing its contents). Seems like a very old and inconvineint shared memory api, because in theory, existence of a mediator (like a file) shouldnt be involved at all during shared memory. It should just work the same as they were in the parent-child scenario
I would love to hear your thoughts on it and if i said something stupid.
So this is a implementation via the System V standard/mechanisms, right? POSIX way should be thru mmap and shm_open?
Yep. I have a few videos on mmap. Probably need another video on shm_open, though.
Can you make videos for remaking the functions of the standard C libraries. Like making setjmp,longjmp,malloc,free,etc.
Yeah, probably. I'll add that to the list. thanks.
What is the idea behind using a FILENAME and ftok to generate a key for the shmget? Why not hardcode a number that both reader and writer agree on?
I coded along while watching the video and I just wrote
#define SEMAPHOR_PRODUCER "seemaphor producer"
And did the same thing for semaphor consumer as well.
I'm guessing they were supposed to be pronounced "shared get", "shared ctrl" etc
Ok , but how to implement that shared memory functions (shmget()) without using existing header files, I mean implement that header files by your own sir
imagine if you could share memory in the kernal and have daul or more internet connections, one std ethernet and this hotspot at the same time and you could direct your packets rx tx, i wonder what would happen if i could power the kernal down and still have windows running and then make a system image
Can you plz make a video series in tool chain of gcc or one video on elf file format?
What aspects of the toolchain are you most interested in? There are a lot of pieces in there.
tools that help us to find memory faults like Coredump
@@JacobSorber Can you also make a video on Program execution sequence , what actually happens when we enter ./a.out or (run any executable file)internally in O.S
Will this work in android ndk ?
What you mean by associating an address with filename?
I mean that the block of shared memory is associated with a filename. So, if a process wants to find the block of memory, it looks it up using the filename. It's simply the mechanism for finding the block. There is no relationship between the shared memory and the contents of the file.
i came from Java , No unsigned byte , No accessing Shared Memory,
Going to C++.
Thank you.
Hello Jacob - great talk !. Could you make some content about Makefiles ?
Thanks. I already have some. What aspects of makefiles would you like to see more about?
Hello Jacob, due to Makefiles I think following topics to consider:
- %.c : %.o notations and alternatives if any
- some defaults rules to write makefile as short as possible with some variables describing comiler, compile/linkage flags
- how to mix C and C++ build in makefile
- considering CMake to generate makefiles
- imvoking one makefile from another if possible
I can see OBJS=shared_memory.o but you don't have any shared_memory.o .. so this should give error no such file or directory
Like if I decode it : I am writing filename different .small
GCC -g -wall -c write.c shared_memory.o -o write.o
GCC:error: shared_memory.o: no such file or directory
Consider write.c as writeshmem.c
I use /tmp/file as a shared mem
It's always an option.
very difficult format for compiling as I dont know how to break down your method to compile alternatively - will check out your compile video **make: *** No rule to make target 'writer.elf', needed by 'all'. Stop.**
How to detect sin wave in a wav audio file
Meaning, how to detect a particular frequency, additive noise (sine wave added on top of other signal), or how to detect whether the wave file is a single tone (sine wave)?
@@JacobSorber i am trying detect certain pattern . The sound pattern is a sin wave whose length is 0.01 seconds . The sound wave starts with a maximum amplitude but decreases gradually giving it a click sound . When i view the wav file in audacity I can clearly see the two spikes but when i read that file in to a numpy array things get messy . Tried finding the rms valve of 0.01 seconds of audio at a time . I thought I will get a abnormally large rms valve and i get that but i get multiple large rms value in a wav file with just 1 click
@@JacobSorber the sin wave is played by speaker and my mic records that so the sin wave is with some noice lke cpu fan,ceiling fan noise or tv noise
@@ciph3r836 One option would be to compute an FFT of your audio signal, and look for a spike at the expected frequency.
@@JacobSorber sir i just started doing this just minutes earlier. Fft seems to be promising
shmdt, Shared Memory DeTach? duh?
What tfok!!??
you can pronounce it shmem-at, shmem-get, shmem-...