Hey if you are seeing this, let me tell you, this is the best video in UA-cam if you want to understand the concept of virtual memory, and others in simple and sweet. Perfect! ❤ Edit: Coming back after long time. Writing a blog, and really needed an low level and mostly complete explanation of virtual memory. Also, following windows internals part-1 book. This video is still gold.
Fantastic video. I'm taking a OS course at university and we have a virtual memory assignment. The hand out documents left me feeling confused and a bit lost, but then this 20-minute video made all concepts clear as day. Thank you for putting effort into making these videoes, you earned a new subscriber!
Start with the problem to understand the solution !!!!! Again and again I’ve seen explanation that detail the concepts without starting with the why. If you don’t understand the problem you’ll never truly get the solution. If you can’t put your finger on the problem… that’s the reason you’re having a hard time understanding. Don’t try to skip this step. Excellent job! Amazing video.
Thanks a lot for taking the time to comment. It means a lot to hear from people that it was worth it! Exactly. I always like to start with the problem first. It makes everything much easier
This might be the best content I have found on UA-cam this year. You explained these concepts in ways my teachers couldn’t, and you did it for free too ! You have my respect , and subscription haha !
I had a course back when i was in engineering school, the professor did a horrible job explaining memory and virtual memory, I ended up hating the class and i never really understood how it worked. until this day when I watched your video, you did an excellent job, I wish youtube had something like this before. Fast forward today, I work as a sysadmin/infra engineer and i really didn't have an idea how virtual memory works, again until I watched your video, thank you for the effort you put here, I am sure many will watch it and gain valuable informations.
There's a quite good reason 4 KB is used as a standard page size. In most disk partitioning software the minimum data you can read/write to is 4KB, so it can move it in one swoop
Thanks. I didn’t know that. Do you have any material or references that provide motivation for 4KB page size? I read some articles that suggest it was chosen empirically long time ago, but I’d like to learn more about it.
This is simply untrue if you grew up during the 80’s or 90’s (or, shoot, even later). Research something called a “filesystem”. Also while you’re at it, check out an article on “Solid State Drives” or “Flash Memory” or “NVRAM” if you’re wondering why 4K is the block size. 😂😂😂
At 8:12, can someone explain how does "one entry for every word" differ from "one entry for every virtual address" ? Are they the same? What does 2^32 addresses per each byte = 2^30 words mean? Edit on Sun Oct 27 01:22:37 PM +0530 2024: After some thinking and search, self understanding is, What's meant by "2^32 addresses per each byte" is each memory address holds 1 byte of actual data and, a word is 4 bytes (32 bits), So the CPU actually access 4 bytes of data at a time, therefore 4 memory locations at a time. So, 2^32 addresses (each addressing one byte / byte addressable) means 2^30 words (each addressing 4 bytes / word addressable). Therefore 2^30 entries not 2^32 entries because 1 entry per 4 addresses. ((2^32)/4)=2^30.
Found your channel from UA-cam recommendation. Nice work. Dude you have a huge overlap of interests as me. I was a math major and now a SW engineer for over 10 yrs, play chess, plays basketball, plays games (not sure what game you play but I mostly play an old game called StarCraft 2). I recently started to play soccer as well.
@@TechWithNikola nice! I always start the game with a cannon rush. Currently stuck in M2 for two years because of my mediocre apm. Considering how much time I spend on the game, I probably need a real StarCraft coach to break my current bottleneck. My two most watched genre of videos on UA-cam are C++ stuff and StarCraft 2 games. Somehow they just feel comforting. I also feel that StarCraft 2 is like C++ and StarCraft 1 is like C programming. In SC 1 everything is so manual.
@@ShaunYChengvery cool. I got stuck with sc2 when I had to keep up with all the build changes between seasons. I haven’t player in 5-6 years. I watch Rust videos lately, but I’ve used C++ the most throughout mu career :-)
I need this resource for my OS assignment about virtual memory this week. So glad I find this video. This helps me understand the concept of virtual memory in 20 mins more than I have ever understand from the 20 chapters we have been task to read for 4 weeks now. LOL. Thank you for this.
Thank you for this amazing video! It's truly refreshing to encounter such well-scripted and nicely illustrated content. It makes really enjoyable trying to learn such complex concepts.
Wow, I was really surprised! I was just relaxing on UA-cam to fall asleep and came across your video. I thought it might be trash, but it turned out to be very good with excellent detail and explanation. Great job! Best video on youtube so fare 🎉🎉🎉
Im just sitting here fascinated...and it makes sense.. knowing just the basics... nice work...a dumb thing to say but this is relaxing...straight to the point ..ask a question ...then answer it... terrific
Good explanation and yes 4kb is the optimal size to move , its similar like why the size of a page node of a BTree in a database is of 1024 because its optimal for I/O. But I also would like to see the explanation of exactly what does MMU do here.
great video. is it posible that there is a mistake in 11:39 when the animation copies the offset bits from the virtual adress to the physical adress? they arent the same first 12 bits
Great video ! I am preparing for my master degree and found this gold in UA-cam. I believe I can ace my upcoming OS class. I have liked your video and subscribed to your channel. Thanks a lot for the video. Greatly appreciate it.
Thanks a lot for taking the time to comment, as well as like and sub. It means a lot to me. I’m glad that the video was helpful. Good luck with acing your classes!
Such a Beautiful Explanation. Thanks a lot Man. This is the Best explanation, it removed all the confusions I had for the last 6 years on this OS chapter
Correction - At 10:50, page size is 4kb(kilobytes) then last 15 bits(not 12 as mentioned in the video) of the both the Virtual address and the Physical Address should be the same. Otherwise, if you mean to be 12 then in the video you should say kilobits instead of kilobytes.
Finally ... a proper explanation! I say this because countless self-made experts get it wrong. You see, Microsoft seems to have invented the misuse of the term "virtual memory" around the time of Windows 95. They used the term VM to describe what is really demand paging. Microsoft publications at the time did NOT make this mistake. (See Jeff Richter's excellent book describing the internals of Windows NT, published at the time.) But in the settings UI ... I guess "virtual memory" sounded more appealing to Microsoft than "demand paging". So they misused the term ... and misled generations of self-made CS people who subsequently grew up unaware of the misuse. They often claim that VM size can be adjusted. (It cannot be.) They believe that the disk swap/paging space is virtual memory. (It is not.) You see, virtual memory was created as a MAPPING PROCEDURE that solved the long-standing problem of mapping a program binary into the run-time address space of a computer. It was preceded by address space paging (remember the early 80x86 paging registers?). This was present in minicomputers as far back as the late 1960s. Paged memory was, in turn, preceded by run-time relocation ... a tedious run-time editing of a binary to "patch" all of the code addresses and data addresses within a program binary before execution could begin. VM is superior to both of those methods. Meanwhile some systems implemented something called swapping ... a predecessor of demand paging. When swapping, a program's ENTIRE memory consumption is swapped to disk to make way for another program. This mechanism allows the system to get around memory size limitations ... but it performs a lot of disk swapping! Demand paging goes one better by swapping only small blocks of memory. And even better, it loads only portions of a program when needed. No need to load an entire program when most of that program code won't be executed any time soon. It's all quite efficient. Virtual memory is not demand paging, and demand paging is not virtual memory. And you don't have to use VM with demand paging. And you don't have to use demand paging with VM. But the two appeared at a similar time in computing history ... and they work hand in hand to improve performance. So you always see both of them together. This video correctly presents a unified view of demand paged/virtual memory. Nice work!
Thanks a lot for taking to time to write such a good comment. I agree, and I’m also very happy to hear that this video met your expectations. Apologies for the late reply.
Thank you for providing such valuable content. The examples you shared have greatly enhanced my understanding of the working principles of virtual memory.
Thanks! You explained this better than any professor did when I was still in university. Much appreciated, will be sending this video to people who ask me about this 😅
Great video. But I would like an explanation regarding the mapping of the physical translated hex value into binary - the hex to binary conversion doesn’t seem correct, even assuming a left or right 0 padding to fill in the extra 17th and 18th bit. Could you explain this please, thanks
Hi, there is a mistake in animation in one part. I don't know if that's what you are referring to? When the offset is kept, I accidentally used wrong values for the final result.
Great video. So you mention that physical address is the "memory RAM". You also mention the "page tables" are store in the "memory RAM" and how there are difference level of page tables and some can be store in disk. Is virtual memory also store in the "memory RAM"?. I know I have to rewatch this video and keep studying to fully understand and that I have gaps in my knowledge. So in a high level the OS has to do a mapping between virtual memory and page tables. And then between page tables and physical address? Thanks.
Hi, apologies for the late response. So virtual memory on its own doesn’t exist really. It’s just a number, for example from 0 to 2^32-1. As such, it is not stored anywhere. It only becomes interesting when we talk about translating that number to its physical counterpart. This translation is stored in page tables, and page tables are stored in RAM. It you rewatch the video you will see the exact details on how the translation works, but in a nutshell the first part of the address is remapped via page tables, and the last 10 bits are copied (called an offset). The mapping is actually happening in the MMU (memory management unit) which you can think of as a small chip on the CPU. You are right that OS plays an important role, and specifically, OS programs the memory management unit and specifies how to do the mapping. Does that make sense?
I don’t know to laugh, cry or to be mad. I had heard about the book What Makes it Page in this subject written by an unknown software engineer from Italy. Due to one of the cybersecurity researchers’ recommendation, I always wanted to learn this book. So, about more or less 3-4 days ago, I gave it a shot. After a couple of days into it, I can assert that the text could be written more cohesively and simply; however, the journey I started is kind of amazing. The harder it gets, the slower you get reading the text, the more enjoyable it becomes. I’m writing this not to say only that but about a dumb-looking guy who were reading with me. We spent HOURS to learn a chapter and now when I looked at him found out that he didn’t learn a piece of shirt!
This is a very good, concise and clear explanation, please keep the work's up, I really enjoyed it : ) PS: can you make a video about how compression algorithms work?
Glad to hear that, and you thank you for taking the time to comment. :-) Yeah, some compression algorithms are in my queue, but it will take a while before I get some free time to make them. Stay tuned!
Hi @Tech With Nikola, Either your Virtual to Physical address calculations are wrongly animated OR the calculative number is wrong (0x4321678) at 11:48 time stamp with 30 bits. it should be like as shown => (00,0100,0011,0010,0001,0110,0111,1000) , but yours (0000,0100,0011,0010,0001,0110,0111,10), Wondering, if it depends on Endianness of the system, please let me know. Thanks !
Hey, yeah, unfortunately I made a mistake in animation -- it used some old numbers when it moved. They are supposed to remain the same. This doesn't depend on Endianness of the system.
Everything was nice thanks for your effort.But in multi paging,the first page table contains 1024 entries which maps to 1024 second page tables ,The address range for 1024 entries is 0x000 to 0x3FF
I've written some 6502 assembler for the BBC Master, where the most complicated it got was having to write to a memory-mapped hardware register to switch banks of RAM so that you can access 128Kb with a 16bit address bus. It's a bit mind-boggling to think about all this indirection that goes on in modern memory systems. All this back and forth would bring 8bit systems to their knees!
This was great. I'm a bit lost (11:09 in) how virtual has 12 to 31 bits left and we somehow fit this in physical address that is 12 to 29. So we are missing 2 bits. What is happening with the last two bits. What did we do with it or are they ignored and we hit a limit of virtual because the size of the physical
CPU/MMU doesn't do any kind of math to figure out physical address (so we're not cramming 20 bits in 18 somehow), it just does a look up in /page mapping table/ and gets one of: 1) **completely arbitrary** physical address that's points to start of that page 2) some special result to indicate "sorry, it was swapped to the disk", explained right after at 11:54 3) (?) some special result to indicate that page was never allocated by the program in the first place So there's no reason why physical address couldn't be shorter than virtual. If program uses all addressable virtual memory, then most pages in mapping table would contain 2nd case of "it's on disk" and OS would juggle them around as needed transparently to the program.
I don't get the Security problem, isn't one of the most important services of an Operating System is to not allow that scenario to even occur? Like Segmentation Faults(if I understand correctly...?).
Hey, that is a great explanation. I'd like to congratulate you on your work. Keep up the good work. Now I understand how computer memory works. I am trying to understand how the database works on saving the pages. How does the computer with 8GB ram can sort 10GB of data.
Yes, that's technically possible. Virtual page doesn't really say the data is on DISK. Instead, it has a bit that says whether the mapping is valid or not, valid meaning it's somewhere in RAM. When it's invalid, the MMU will raise a page fault. For invalid mappings, the OS can use the remaining bits to store additional information that will help find the data on a device. It's up to the kernel to interpret these bits, which may mean the data is on SSD, USB flash, or somewhere else, and then load the data back into RAM. So we can use any device as a backing storage. Linux has a struct called swap_info which stores information about devices used for paging. Actually, it has an array of such structs, each identifying a device (potentially the same) and some metadata. Linux then uses PTEs to store an index into this array, as well as some other metadata. I'm not too familiar with the internals of how linux does this myself, but this is the general idea.
Good catch. I think that my intent was to say “CPU cannot read it from RAM” (because it’s not there), but you’re also right that CPU doesn’t know how to read it from disk
I don’t think so but I could be wrong. I guess it depends on the implementation details and I can see both being done during the time before virtual memory was a thing. In either case, neither behaviour is ideal. I would argue that crashing is potentially better than wraparound because it highlights a problem whereas wraparound would continue working with a potentially silent error. Have you come across any references that discuss this? I’m interested to learn more
Is Page Table update and TLB update treated as 1 atomic operation? If yes, how do we maintain atomicity? If no, how do we handle scenarios where the TLB has stale entry?
Hey if you are seeing this, let me tell you, this is the best video in UA-cam if you want to understand the concept of virtual memory, and others in simple and sweet. Perfect! ❤
Edit: Coming back after long time. Writing a blog, and really needed an low level and mostly complete explanation of virtual memory. Also, following windows internals part-1 book. This video is still gold.
Thank you! :)
Not really. He forget that OS claims RAM too
@@siegfriedbarfuss9379 3:54
@@siegfriedbarfuss9379 Why is that relevant to this demonstration? You could just generalise an OS as another program? It's also shown at 3:58...
Fantastic video. I'm taking a OS course at university and we have a virtual memory assignment. The hand out documents left me feeling confused and a bit lost, but then this 20-minute video made all concepts clear as day. Thank you for putting effort into making these videoes, you earned a new subscriber!
I’m glad it was helpful. You’re welcome, and thank you for taking the time to comment.
Start with the problem to understand the solution !!!!!
Again and again I’ve seen explanation that detail the concepts without starting with the why. If you don’t understand the problem you’ll never truly get the solution. If you can’t put your finger on the problem… that’s the reason you’re having a hard time understanding. Don’t try to skip this step.
Excellent job! Amazing video.
Thanks a lot for taking the time to comment. It means a lot to hear from people that it was worth it!
Exactly. I always like to start with the problem first. It makes everything much easier
because the people who are teaching you that way themselves have only memorised the concept.
Thank you for the video. The explanation was clear, structured and concise, better than most other explanations on youtube.
Thanks a lot for the kind words and for taking the time to leave the comment. I'm so happy to hear that you've liked the video!
Holy crap. UA-cam algorithm finally came through. This is some amazing content.
Thank you :-)
Diet
It sure did
yooo W disco elysium pfp
This is amazing! More detail than most Operating System courses in 20 minutes. Awestruck!
Thanks. Glad you think so! I’ll try to make more videos on similar topics in the future.
This might be the best content I have found on UA-cam this year. You explained these concepts in ways my teachers couldn’t, and you did it for free too ! You have my respect , and subscription haha !
I’m so glad that you like my content. It means a lot to me when people leave comments and subscribe, so thank you.
Videos like these are the only reason I cannot stop using UA-cam. You are such a great explainer, dude!
Thanks a lot! It means a lot to me when I receive such comments.
Amazing explanation, always wanted to know core knowledge. Thanks, keep up with these videos!
Thank you Dusan. I'm glad you've enjoyed it :)
I had a course back when i was in engineering school, the professor did a horrible job explaining memory and virtual memory, I ended up hating the class and i never really understood how it worked. until this day when I watched your video, you did an excellent job, I wish youtube had something like this before. Fast forward today, I work as a sysadmin/infra engineer and i really didn't have an idea how virtual memory works, again until I watched your video, thank you for the effort you put here, I am sure many will watch it and gain valuable informations.
Thank you for the comment. I’m glad you’ve found it useful!
There's a quite good reason 4 KB is used as a standard page size. In most disk partitioning software the minimum data you can read/write to is 4KB, so it can move it in one swoop
yeah i figured this was the reason and was wondering why he didn't say so
Thanks. I didn’t know that. Do you have any material or references that provide motivation for 4KB page size? I read some articles that suggest it was chosen empirically long time ago, but I’d like to learn more about it.
This is simply untrue if you grew up during the 80’s or 90’s (or, shoot, even later). Research something called a “filesystem”. Also while you’re at it, check out an article on “Solid State Drives” or “Flash Memory” or “NVRAM” if you’re wondering why 4K is the block size. 😂😂😂
@@TechWithNikola They probably choose it because 0x1000 is pretty nice to type. ARM macs use 16kb pages I think.
It has nothing to do with disc sector/cluster size, like at all.
And yeah, 64 bit osses use 32k pages.
At 8:12, can someone explain how does "one entry for every word" differ from "one entry for every virtual address" ? Are they the same?
What does 2^32 addresses per each byte = 2^30 words mean?
Edit on Sun Oct 27 01:22:37 PM +0530 2024: After some thinking and search, self understanding is,
What's meant by "2^32 addresses per each byte" is each memory address holds 1 byte of actual data and, a word is 4 bytes (32 bits), So the CPU actually access 4 bytes of data at a time, therefore 4 memory locations at a time. So, 2^32 addresses (each addressing one byte / byte addressable) means 2^30 words (each addressing 4 bytes / word addressable). Therefore 2^30 entries not 2^32 entries because 1 entry per 4 addresses. ((2^32)/4)=2^30.
Thank you so much, sir. I was also struggling to wrap my head around where 2^30 came from.
That's a very good, high quality production with a top notch explanation! Keep making more.
Thanks :)
This is the single greatest explanation of virtual memory I've ever seen, thank you so much.
I'm amazed at how someone can explain things with such clarity, just like you did.
You just earned a subscriber. Thank you so much.
Found your channel from UA-cam recommendation.
Nice work.
Dude you have a huge overlap of interests as me. I was a math major and now a SW engineer for over 10 yrs, play chess, plays basketball, plays games (not sure what game you play but I mostly play an old game called StarCraft 2).
I recently started to play soccer as well.
Hey dude, nice to hear that. I’ve played SC2 a looot as well 😀 got to diamond 1 but never broke to masters…
@@TechWithNikola nice!
I always start the game with a cannon rush. Currently stuck in M2 for two years because of my mediocre apm. Considering how much time I spend on the game, I probably need a real StarCraft coach to break my current bottleneck.
My two most watched genre of videos on UA-cam are C++ stuff and StarCraft 2 games. Somehow they just feel comforting.
I also feel that StarCraft 2 is like C++ and StarCraft 1 is like C programming. In SC 1 everything is so manual.
@@ShaunYChengvery cool. I got stuck with sc2 when I had to keep up with all the build changes between seasons. I haven’t player in 5-6 years.
I watch Rust videos lately, but I’ve used C++ the most throughout mu career :-)
Brat objasnio u 20 minuta content sa tri predavanja System Level Programiranja na faksu. Svaka cast!
Svaka cast burazeru. Ovo sam na faksu radio, ali si dosta bolje objasnio.
Hvala :) Drago mi je sto cujem da je dobro objasnjeno.
At 10:00 the page 2 in both virtual memory and physical memory is 8kb instead of 4kb. This is the great explanation video right now!
I need this resource for my OS assignment about virtual memory this week. So glad I find this video. This helps me understand the concept of virtual memory in 20 mins more than I have ever understand from the 20 chapters we have been task to read for 4 weeks now. LOL. Thank you for this.
please never stop, really good video
Thank you! I'm glad you've enjoyed it. I'll keep making them for sure :)
Wow! Clear, concise and very neat animations! Subscribing to a channel after a long time!
Thank you for the sub!
the graphics really helped me visualize how this works! youre a lifesaver ^^
I'm glad! Graphics are a great tool for explain IMO :-)
@@TechWithNikola It's not just your opinion. It's a FACT! Graphics for complicated ideas are a must!
And thank you, your video was amazing!
I wanna thank the youtube algorithm for recommending this video. This is absolute gem for explaining the Virtual Memory concept.
Thank you!
An excellent, succint and illustrative explanation :)
Thanks a lot Andrej!
Thank you for this amazing video! It's truly refreshing to encounter such well-scripted and nicely illustrated content. It makes really enjoyable trying to learn such complex concepts.
Great video, brilliantly explaining the concept in details ! Thank you so much!
Best explanation of virtual memory ever seen. Thanks for putting this together, you are awesome.
Thank you very much! I finally understand how virtual memory works. If only university professors could be structured like this...
Ahh…the swap file in Linux!!! I used to allocate a portion of my hard drive for swapping!! Wow!!! Such a refresher !!
This is amazing. I am slowly realizing I am a visual learner. It was always hard to grasp content when it just a text. Worth a Sub... :)
Thank you :)
Brilliant videos with great explanations. You’re a natural when it comes to teaching
11:38 I’m wondering when you copy the last 12bits to the physical memory, why the number changes from 0110 0111 1000 to 010110011110
Oh that’s a good catch! It’s a bug in the animation. Sorry about that. It should remain unchanged.
@@TechWithNikola Thank you very much! Your videos are really helpful!!!
The variety of content provided is much appreciated.
wow!! never felt like a computer science before. learned a lot on this clip. keep them coming
This is awesome, made things so clear to me. Thank you.
You're welcome. Glad it helped!
This is the BEST explanation i have had about this topic. I am a newbie to such concepts and you have explained in such an elegant way. Liked n subbed
the second part is a bit more complex but I love the visuals + explanation nonetheless. thanks for the effort put into making this one!
Wow, I was really surprised! I was just relaxing on UA-cam to fall asleep and came across your video. I thought it might be trash, but it turned out to be very good with excellent detail and explanation. Great job!
Best video on youtube so fare 🎉🎉🎉
Excellent description on how cpu/memory/os are working together! Well done!
Thanks for the content. You really have a talent, keep this going.
Im just sitting here fascinated...and it makes sense.. knowing just the basics... nice work...a dumb thing to say but this is relaxing...straight to the point ..ask a question ...then answer it... terrific
Thanks. I’m glad you like this style.
thank you so much for this video! I'm learning about hugepages right now, and i wanted a recap on how normal pages work.
This video is really good, the way that you simplified Virtual Memory is amazing!
Glad it was helpful!
Wonderful illustration of Virtual Memory. Good Job !
Thanks
Good explanation and yes 4kb is the optimal size to move , its similar like why the size of a page node of a BTree in a database is of 1024 because its optimal for I/O.
But I also would like to see the explanation of exactly what does MMU do here.
great video. is it posible that there is a mistake in 11:39 when the animation copies the offset bits from the virtual adress to the physical adress? they arent the same first 12 bits
Thanks :) Yeah, there is a mistake in the animation. Sorry about that.
Great video ! I am preparing for my master degree and found this gold in UA-cam. I believe I can ace my upcoming OS class.
I have liked your video and subscribed to your channel.
Thanks a lot for the video. Greatly appreciate it.
Thanks a lot for taking the time to comment, as well as like and sub. It means a lot to me.
I’m glad that the video was helpful. Good luck with acing your classes!
Such a Beautiful Explanation. Thanks a lot Man. This is the Best explanation, it removed all the confusions I had for the last 6 years on this OS chapter
hi, what does 0x0010 in "2nd level entry" column mean at 19:28? it doesn't seem to be used in the process of finding the physical address.
I have to words! That video is amazing! You won yourself a new subscriber, my friend! ❤
Welcome aboard! I’m glad you’ve liked it, and thank you for taking the time to comment.
@@TechWithNikola My pleasure! Have a lovely day!
Correction - At 10:50, page size is 4kb(kilobytes) then last 15 bits(not 12 as mentioned in the video) of the both the Virtual address and the Physical Address should be the same. Otherwise, if you mean to be 12 then in the video you should say kilobits instead of kilobytes.
Finally ... a proper explanation!
I say this because countless self-made experts get it wrong. You see, Microsoft seems to have invented the misuse of the term "virtual memory" around the time of Windows 95. They used the term VM to describe what is really demand paging. Microsoft publications at the time did NOT make this mistake. (See Jeff Richter's excellent book describing the internals of Windows NT, published at the time.)
But in the settings UI ... I guess "virtual memory" sounded more appealing to Microsoft than "demand paging". So they misused the term ... and misled generations of self-made CS people who subsequently grew up unaware of the misuse.
They often claim that VM size can be adjusted. (It cannot be.)
They believe that the disk swap/paging space is virtual memory. (It is not.)
You see, virtual memory was created as a MAPPING PROCEDURE that solved the long-standing problem of mapping a program binary into the run-time address space of a computer.
It was preceded by address space paging (remember the early 80x86 paging registers?). This was present in minicomputers as far back as the late 1960s.
Paged memory was, in turn, preceded by run-time relocation ... a tedious run-time editing of a binary to "patch" all of the code addresses and data addresses within a program binary before execution could begin.
VM is superior to both of those methods.
Meanwhile some systems implemented something called swapping ... a predecessor of demand paging. When swapping, a program's ENTIRE memory consumption is swapped to disk to make way for another program. This mechanism allows the system to get around memory size limitations ... but it performs a lot of disk swapping!
Demand paging goes one better by swapping only small blocks of memory. And even better, it loads only portions of a program when needed. No need to load an entire program when most of that program code won't be executed any time soon. It's all quite efficient.
Virtual memory is not demand paging, and demand paging is not virtual memory.
And you don't have to use VM with demand paging. And you don't have to use demand paging with VM.
But the two appeared at a similar time in computing history ... and they work hand in hand to improve performance.
So you always see both of them together.
This video correctly presents a unified view of demand paged/virtual memory. Nice work!
Thanks a lot for taking to time to write such a good comment. I agree, and I’m also very happy to hear that this video met your expectations.
Apologies for the late reply.
Thank you for providing such valuable content. The examples you shared have greatly enhanced my understanding of the working principles of virtual memory.
I’m very happy to hear that. Thank you for taking the time to comment.
What an excellent overview. Nice job. It clarified a few things i was unclear about.
Thanks a lot. Glad it clarified things :-)
Thank you very much for this video, it was very detailed and very well explained!
Thanks! You explained this better than any professor did when I was still in university. Much appreciated, will be sending this video to people who ask me about this 😅
Glad to hear that you’ve found it useful, and thank you for sharing the video :-)
you just earned a subscriber within a few minutes of this video!!
Fantastic explanation expecting more videos from you🎉
Wooow this video is awesome! Great work on the animations and examples 🤝😎
Thank you so much! I'm glad to hear that you've liked it.
Very time I watch it, I learn something new, many thanks!
Great video. But I would like an explanation regarding the mapping of the physical translated hex value into binary - the hex to binary conversion doesn’t seem correct, even assuming a left or right 0 padding to fill in the extra 17th and 18th bit. Could you explain this please, thanks
Hi, there is a mistake in animation in one part. I don't know if that's what you are referring to? When the offset is kept, I accidentally used wrong values for the final result.
Great video. So you mention that physical address is the "memory RAM". You also mention the "page tables" are store in the "memory RAM" and how there are difference level of page tables and some can be store in disk. Is virtual memory also store in the "memory RAM"?. I know I have to rewatch this video and keep studying to fully understand and that I have gaps in my knowledge. So in a high level the OS has to do a mapping between virtual memory and page tables. And then between page tables and physical address? Thanks.
Hi, apologies for the late response.
So virtual memory on its own doesn’t exist really. It’s just a number, for example from 0 to 2^32-1. As such, it is not stored anywhere.
It only becomes interesting when we talk about translating that number to its physical counterpart. This translation is stored in page tables, and page tables are stored in RAM. It you rewatch the video you will see the exact details on how the translation works, but in a nutshell the first part of the address is remapped via page tables, and the last 10 bits are copied (called an offset).
The mapping is actually happening in the MMU (memory management unit) which you can think of as a small chip on the CPU. You are right that OS plays an important role, and specifically, OS programs the memory management unit and specifies how to do the mapping.
Does that make sense?
Yes, thanks for the reply
18:03 would it not be 256 chunks?
(4096 Pages / Chunk) x (256 Chunks) x (4096 Bytes / Page) = 2^32 Bytes ?
absolute banger of a tech channel
thanks, will watch again
-IGN
Thank god I found this gem before my exam!
Dude, you're videos are so helpful, thanks so much!!!
Glad you like them! Thanks for taking the time to comment.
I don’t know to laugh, cry or to be mad. I had heard about the book What Makes it Page in this subject written by an unknown software engineer from Italy. Due to one of the cybersecurity researchers’ recommendation, I always wanted to learn this book. So, about more or less 3-4 days ago, I gave it a shot. After a couple of days into it, I can assert that the text could be written more cohesively and simply; however, the journey I started is kind of amazing. The harder it gets, the slower you get reading the text, the more enjoyable it becomes. I’m writing this not to say only that but about a dumb-looking guy who were reading with me. We spent HOURS to learn a chapter and now when I looked at him found out that he didn’t learn a piece of shirt!
This is a very good, concise and clear explanation, please keep the work's up, I really enjoyed it : )
PS: can you make a video about how compression algorithms work?
Glad to hear that, and you thank you for taking the time to comment. :-)
Yeah, some compression algorithms are in my queue, but it will take a while before I get some free time to make them. Stay tuned!
@@TechWithNikola I'll be waiting 4 sure :3
Thank you for your high quality work. 👏👏👏
Hi @Tech With Nikola, Either your Virtual to Physical address calculations are wrongly animated OR the calculative number is wrong (0x4321678) at 11:48 time stamp with 30 bits. it should be like as shown => (00,0100,0011,0010,0001,0110,0111,1000) , but yours (0000,0100,0011,0010,0001,0110,0111,10), Wondering, if it depends on Endianness of the system, please let me know. Thanks !
Hey, yeah, unfortunately I made a mistake in animation -- it used some old numbers when it moved. They are supposed to remain the same. This doesn't depend on Endianness of the system.
Very clear! It helps a lot, thank you very much!😀
You’re welcome! :-)
Everything was nice thanks for your effort.But in multi paging,the first page table contains 1024 entries which maps to 1024 second page tables ,The address range for 1024 entries is 0x000 to 0x3FF
watched the first video in this channel, subbed!
Thank you for the sub. I'm so glad to hear that you've liked it!
From Aritmetic concepts , the virtual memóry obeys bijective function (injective and surjective ) about Domain and Image .
It is unbelievable !
I've written some 6502 assembler for the BBC Master, where the most complicated it got was having to write to a memory-mapped hardware register to switch banks of RAM so that you can access 128Kb with a 16bit address bus. It's a bit mind-boggling to think about all this indirection that goes on in modern memory systems. All this back and forth would bring 8bit systems to their knees!
This was great. I'm a bit lost (11:09 in) how virtual has 12 to 31 bits left and we somehow fit this in physical address that is 12 to 29. So we are missing 2 bits. What is happening with the last two bits. What did we do with it or are they ignored and we hit a limit of virtual because the size of the physical
CPU/MMU doesn't do any kind of math to figure out physical address (so we're not cramming 20 bits in 18 somehow), it just does a look up in /page mapping table/ and gets one of:
1) **completely arbitrary** physical address that's points to start of that page
2) some special result to indicate "sorry, it was swapped to the disk", explained right after at 11:54
3) (?) some special result to indicate that page was never allocated by the program in the first place
So there's no reason why physical address couldn't be shorter than virtual. If program uses all addressable virtual memory, then most pages in mapping table would contain 2nd case of "it's on disk" and OS would juggle them around as needed transparently to the program.
that music player & video game example is hilarious to think about
I don't get the Security problem, isn't one of the most important services of an Operating System is to not allow that scenario to even occur?
Like Segmentation Faults(if I understand correctly...?).
this is gold, thank you so much
10:39 All pages are 4096 long except Pages 2, is that a correct?
1:15 programs used to crash in 1950s? What programs?
clear as water, thanks!
Great to hear that :) Thank you!
Thanks a lot about this simple explanation
Hey, that is a great explanation. I'd like to congratulate you on your work. Keep up the good work. Now I understand how computer memory works. I am trying to understand how the database works on saving the pages. How does the computer with 8GB ram can sort 10GB of data.
is there a chance that a virtual page does not map to eithjer the disk or the physical page? what happens in that case? 12:45
Yes, that's technically possible. Virtual page doesn't really say the data is on DISK. Instead, it has a bit that says whether the mapping is valid or not, valid meaning it's somewhere in RAM. When it's invalid, the MMU will raise a page fault. For invalid mappings, the OS can use the remaining bits to store additional information that will help find the data on a device. It's up to the kernel to interpret these bits, which may mean the data is on SSD, USB flash, or somewhere else, and then load the data back into RAM. So we can use any device as a backing storage.
Linux has a struct called swap_info which stores information about devices used for paging. Actually, it has an array of such structs, each identifying a device (potentially the same) and some metadata. Linux then uses PTEs to store an index into this array, as well as some other metadata. I'm not too familiar with the internals of how linux does this myself, but this is the general idea.
Your explanations are amazing
Thank you, I hope to see more
Thank you. Yeah, definitely, I’ll try to find some time to make more videos soon
When you are taking the 12 LSBs ( 0x678), and putting those 12 bits of offset, why are you removing bit 0 and 1?
@12:17 He says cpu does not know how to read it from RAM should not it be Disk ?
Good catch. I think that my intent was to say “CPU cannot read it from RAM” (because it’s not there), but you’re also right that CPU doesn’t know how to read it from disk
1:00 Wouldn't the memory address wrap around and not crash immediately?
I don’t think so but I could be wrong. I guess it depends on the implementation details and I can see both being done during the time before virtual memory was a thing. In either case, neither behaviour is ideal.
I would argue that crashing is potentially better than wraparound because it highlights a problem whereas wraparound would continue working with a potentially silent error.
Have you come across any references that discuss this? I’m interested to learn more
@@TechWithNikola Ben Eater's 65c02 based computer has the most significant address bit (A15) disconnected, causing a wraparound behavior.
Got it, thank you for that. I’ll read more about it.
You're a blessing!
bro you are amazing, love your energy
Thanks. I appreciate that!
Thank you for this videos , it is reali amazing❤
Is Page Table update and TLB update treated as 1 atomic operation? If yes, how do we maintain atomicity? If no, how do we handle scenarios where the TLB has stale entry?
Amazing video, wish I saw it while still in college 😊
Wow, good video. Good job and thanks for sharing.
Thank you! You’re welcome.
Can you also call a programm a process with threads etc. or are those two seperat things?
0:35 As far as I understand, 4Gb of memory means less than 4Gb of RAM. CPU thinks that ROM and south bridge devices are memory as well.
excellent explanation!!
one doubt isn't the page table size 16MB as there 2^22 pages and we uses word size for each page.
so 4*2^22 16MB?
32MB considering map between virtual address and physical address
Amazing explanation