If I have a Memory read with the address 'hFFFF_FFF0. How can I decode this? How do I know If I am reading Internal device memory or a memory attached as an External card? If I decode the address I see that I am Accessing bus 255 and Device 32 and function8. What is the memory location I am reading?
How are the devices on PCI-e get enumerated on dual CPU system with separate PCI-e controllers? I noticed that 0x80 bus id is some boundary between sockets but I didn't find any justification beyond that...
Typically the BIOS splits the 256 buses evenly in a dual CPU system, so each CPU gets 128 buses (CPU0 gets 0-x7F, and CPU1 gets 0x80-0xFF). This doesn't have to be case and the BIOS can balance the buses depending on the population of the PCIe end-points. Some modern CPUs support Multi-PCIe segments i.e, each CPU can get it's own 256 bus range.
my question is ...when will the pci config cycle is initiated ? .....after the link is formed or before the link is formed? if it happens after the link is formed then does it happens after the LTSSM ?
PCIe Config cycle is initiated by software when it tries to access the config space of a PCIe device, either by CF8/CFC or by MMCFG. The processor generates config cycles in response to this software request. If the link is not trained at this point, then the config transaction will return an error. If the link is trained, then the cycle will go the device on the other end of the link. We'll talk in more detail when we go deeper into PCIe protocols in future sessions.
Hi Sir,. After OS upgrade, Two PCI devices are not visible in the output of "lspci" command. In dmesg log, BAR memory assignment logs also not visible for those two PCI devices. What could be the reason ?
could you please share some video on how tlp/dlp/physical layer transactions happens in Pcie...which layer comes first in picture when RC & End point communicates
I have six GPUs installed and 1 of them is having driver issues. When I open the category "Display Adapters" in the Device Manager, the faulty GPU shows a yellow exclamation sign in the corner and the properties of this card show that it is connected to PCI bus 3. How can I find out which GPU is physically connected to PCI bus 3? So that I can replace the GPU with another one, rather than removing all of them one by one.
First you need to find the config space for Bus 3. You might be able to do it via the RW Everything utility. Look for the root port that has Bus 3 as the secondary bus #. The tricky part then is mapping that to the physical slot in the system. If your card has LEDs or some other status indication, then initiate a "Secondary Bus Reset" (SBR) by writing to the root port's bridge control register. This will reset just the card on Bus 3. But to physically identify the card, the card should be able to provide some visual indication (LED/Fan etc) that resets in response to SBR.
thanks for the presentation. If I followed you correctly, a device (End Point or Bridge) can have its config space programmed as reaching from only one CPU i.e. on a system with two CPU's it is not possible to reach the same end point device via two separate paths one from cpu0 and another from cpu1. One can reach devices under cpu0 or cpu1 via an inter-cpu link. Please clarify if this is correct ? If I am right then the device tree is static with the only change being the type of end point device that a user might plugin. For example user may plug a NIC into slot A and a GPU into slot B and in another configuration change their slots i.e. NIC into slot B and GPU into slot A.
Not sure if I fully follow your question, but the device config space can be reached from any CPU, immaterial of which CPU the device is connected to, via the CPU-CPU Link.
Is there a book detailing more information on this subject that you learned from? Where did you get this information from? Like the video by the way, very detailed and conveyed in a manner everyone can understand!
This is the best presentation I have found so far on this subject matter to date. Thanks for putting in the time to spread this knowledge!
One of the best videos to understand PCIE enumeration
Thank you so much sir!! This is one of the most clear videos on PCI. Keep up the good work.
simple and lovely explanation - a good refresher before an interview
Clear & Simple explanation. Than you @Sarathy Jaykumar!
Great videos, Most valuable Informations, these are the "MISSING LINKS" of the "BIG PICTURE" . Great work, expecting more videos, THANK YOU SO MUCH.
Awesome & great explanation
clear !! nandri
Thanks for your knowledge sharing session. Very clear explanation, kudos.
Great, clear explanation! Thank you very much.
Best video ever, so clear
If I have a Memory read with the address 'hFFFF_FFF0.
How can I decode this? How do I know If I am reading Internal device memory or a memory attached as an External card?
If I decode the address I see that I am Accessing bus 255 and Device 32 and function8. What is the memory location I am reading?
You are reading the reset vector at 4G and not any device memory.
best explanation .
I got a doubt Enumeration will done in GEN1 L0 or at max data rate if it supports?
In soc level, how different device number assigned to different IP's in intel architecture ? This is also statically assigned at hardware level?
great lecture
Thank you. Very informative video.
How are the devices on PCI-e get enumerated on dual CPU system with separate PCI-e controllers? I noticed that 0x80 bus id is some boundary between sockets but I didn't find any justification beyond that...
Typically the BIOS splits the 256 buses evenly in a dual CPU system, so each CPU gets 128 buses (CPU0 gets 0-x7F, and CPU1 gets 0x80-0xFF). This doesn't have to be case and the BIOS can balance the buses depending on the population of the PCIe end-points. Some modern CPUs support Multi-PCIe segments i.e, each CPU can get it's own 256 bus range.
after so long time ...u make video sir....make it fast series plz
Hi Arshid, trying to get into a weekly cadence, unless I'm travelling or otherwise tied up. Just uploaded the next session.
excellent explanation
my question is ...when will the pci config cycle is initiated ? .....after the link is formed or before the link is formed?
if it happens after the link is formed then does it happens after the LTSSM ?
PCIe Config cycle is initiated by software when it tries to access the config space of a PCIe device, either by CF8/CFC or by MMCFG. The processor generates config cycles in response to this software request. If the link is not trained at this point, then the config transaction will return an error. If the link is trained, then the cycle will go the device on the other end of the link. We'll talk in more detail when we go deeper into PCIe protocols in future sessions.
Hi Sir,. After OS upgrade, Two PCI devices are not visible in the output of "lspci" command. In dmesg log, BAR memory assignment logs also not visible for those two PCI devices.
What could be the reason ?
could you please share some video on how tlp/dlp/physical layer transactions happens in Pcie...which layer comes first in picture when RC & End point communicates
Sure. I can do a overview on the PCIe protocol.
Any source for PCIe code understanding?
I have six GPUs installed and 1 of them is having driver issues. When I open the category "Display Adapters" in the Device Manager, the faulty GPU shows a yellow exclamation sign in the corner and the properties of this card show that it is connected to PCI bus 3.
How can I find out which GPU is physically connected to PCI bus 3? So that I can replace the GPU with another one, rather than removing all of them one by one.
First you need to find the config space for Bus 3. You might be able to do it via the RW Everything utility. Look for the root port that has Bus 3 as the secondary bus #. The tricky part then is mapping that to the physical slot in the system. If your card has LEDs or some other status indication, then initiate a "Secondary Bus Reset" (SBR) by writing to the root port's bridge control register. This will reset just the card on Bus 3. But to physically identify the card, the card should be able to provide some visual indication (LED/Fan etc) that resets in response to SBR.
thx a lot dear sir
can anybody share any link for NVME express tutorial?
Nice video please also touch upon ACPI topics
That's a bigger subject. I'll finish PCIe, Interrupts and then go on to ACPI
thanks for the presentation.
If I followed you correctly, a device (End Point or Bridge) can have its config space programmed as reaching from only one CPU i.e. on a system with two CPU's it is not possible to reach the same end point device via two separate paths one from cpu0 and another from cpu1. One can reach devices under cpu0 or cpu1 via an inter-cpu link. Please clarify if this is correct ? If I am right then the device tree is static with the only change being the type of end point device that a user might plugin. For example user may plug a NIC into slot A and a GPU into slot B and in another configuration change their slots i.e. NIC into slot B and GPU into slot A.
Not sure if I fully follow your question, but the device config space can be reached from any CPU, immaterial of which CPU the device is connected to, via the CPU-CPU Link.
Sir , please explain the interrupt scheme in PCIe
Will do. I have separate sessions planned for Interrupts in general.
Is there a book detailing more information on this subject that you learned from? Where did you get this information from? Like the video by the way, very detailed and conveyed in a manner everyone can understand!
thanks, nice presentation.
Sir a Video on DMA in PCI will be good.
Nice job!
Fantastic!!!
nice, thanks
Thanks a ton !!!!!
Thanks.