mxshift over in The Serial Port Discord made a great observation: "cache engine using a FD:FD:10 MAC address seems wrong. That's an unallocated OUI and marked as multicast. That feels like an autogenerated MAC used when the NVRAM is trashed. That would also explain the 'is a .' as the model designation would also be in NVRAM" Perhaps some corrupt NVRAM? More investigation to do on the Cache Engine for sure!
That's exactly what I'v wanted to tell you. You must fix your NVRAM or CMOS RAM or whatever is in there. Arp has no response because of bad source address and then nothing else could work. But you have to fix that NVRAM issue first since otherwise it will never boot from anything if there is no model number stored there.
Not NVRAM... IDPROM. And that will be "impossible" to fix. This is obvious from it's lack of hardware identification. Most of their things made in that old PIX form factor use an I2C EEPROM (flash) for the hardware details. It's either failed, or something scrambled it.
I'm reminded of the Sun Microsystems issue. Can't even boot the machine until you've run through safe boot processes for an agonizing 2 minutes, then you get to tell the machine what it is and what it's MAC is, before you can finally try to boot. And that's before you discover that the replacement battery costs $100 both new or "gently discharged"
I'm out of work due to chronic pain from an injury years ago. I'm dealing with my depression through the company of my pets and a few UA-camrs whose videos brighten my day whenever they upload. Every time I see one of clabretros videos I think "ah just what I need" no hyperactive editing,clear and thoughtful communication of information and an obvious true love for what he's doing. Adrian's digital basement and the backlogs (very different channel topic) are my comfortable places on the internet, amongst a few others. I wish I was in a financial position to be supportive of the people who provide all of this to us, maybe one day, but I hope our engagement and joy for their content can be enough to keep them doing what they do for a long time to come. Cheers to you all, take care and happy holidays.
Tip: On the NIC (network 'card') you use to capture the mirrored packets with Wireshark (or other similar software) - turn off everything except the packet capture driver (e.g. NPCAP or similar). Under "Properties -> Networking -> This connection uses the following items". Just remember to turn them back on (if needed) later. Wireshark (or rather the packet capture driver) will still be able to capture all the mirrored packets to/from the device under test/inspection (i.e. source). However it will stop any 'pollution' (though this can be filtered) of the capture from the device being used to do the capture (such as at 4:27). I write this without having watched the entire video yet, or having read all the comments, just something I find useful to do when I am doing the same - mirroring and packet capture to allow analysis of the network traffic of a device (usually embedded of some kind).
Thanks that answers my question, Of whether the capture interface needs to have an IP address on the network or not. = Since it's using the capture driver it does not. However this sounds like advice to disable functions within windows and others should note the equivalent for Linux or Mac.
@@imark7777777 Yes sorry... correct... that is Windows centric. I could be wrong (or there might be better ways) but in Linux I believe similar can be achieved by ensuring that interface to be used for capture doesn't a) have a dhcp-client configured to automatically get an IP (and/or do auto-IP or similar); and b) that you don't statically assign an IP; and c) that, if not already, it is in promiscuous mode (which of course it will need to be - otherwise it is just going to ignore anything not for it's MAC).
With these old machines, I always set the time and check for any batteries that may have dried up decades ago. You never know how a machine will behave if its time is wrong.
It's rare that there's content on youtube where I can actually watch someone learn something I know fairly well for the first(?) time - kind of fun actually! Wireshark is a godsend. While "it's the network!1!!" is usually *not* true, it at least helps prove your case when you're trying to get the fingers pointed in the right direction...
I love watching you tinkering with my daily job. I sitting there and shouting at my telly, when youre following a wrong trail to solve the problem. I would love to show you the real cisco kings, like a Callmanager 6,8.5, the ISE, old Wifi Controller and LightAP. Man there are many rabbitholes you could follow :)
"portfast"... I'm pretty sure the cache engine is not going to wait 30s for spanning-tree to complete loop detection. It's a VERY common issue. Watch the front of the switch. The light will flash orange while it's blocked. Until it goes green, no traffic will be forwarded.
Yeah most likely. Portfast and bpdu guard all the client facing ports. It's super annoying and some devices really really hate it. Edit: after finishing watching the video, it's not the port being down because then you wouldn't see any packets with the port span.
Yes! Portfast! . Seeing the ARP requests and no response I thought "the problem is the switch" Might be the agonizingly long STP wait time on the cisco switch. This brings me back to my CCNA days, some guy accidentally whipped a class router and the teacher assigned us to bootstrap it and install it over the network. Long story short, we wasted hours banging our heads trying to figure it out. It was the STP...
@clabretro at 21:08 you can see the LED as orange, which on that era of Cisco switches means that spanning tree hasn't gone fully active yet and is not passing traffic. It then switches to green a few second later. I would definitely want to rule that out. It would explain why the ARP requests not receiving a response. I would set the interface parameters on both the device and monitor port to "switchport mode access" and then "spanning-tree portfast"
Working at a mixed vendor ISP back in ancient times we had quite a few problems when connecting Ciscos to other switch vendors. Mostly this was solved by disabling auto-negotiation on switchports and set them hard to 100/100 FDX or 1000/1000 FDX (newer equipment). Mostly we needed this when adding HP or 3Com switches. Sometimes we needed to do the same for some 3rd party server equipment too. That could be the problem with old Cisco talking to your Ubiquti switch.
Port mirroring is super useful. You can even do this with virtual switches. This issue is rather sketchy though lol You have a link between the switch and the cache device, but then the uplink to the other network goes down ? You get no responses to ARP requests coming from the cache. You can tell that it doesn't work in wireshark because there was no response :P I am unfamiliar with the cache device from cisco although I have used cache devices in the past. There is a chance the 5500 is acting as a switch and your home lan is going 'nope' when it sees another switch being plugged in to a lead switch, this is BPDU guard or root guard I think it was called. That might explain the uplink to the network going down. But this does not always make the link light go off. Open up a telnet to the switch and turn on console logging before turning on the rest to get a sense of what that's doing when the link becomes active. Another thing that might explain this is vlan configuration but I don't think you have those configured ? I would expect to see vlan oriented broadcasts. It's also possible that something created an ACL on the switch ? Like a default config thing ? Then there's you saying the mac address isn't right probably, is it possible the cache device is actually binding to fastethernet1 instead of 0 ? That's the only thing I can think of that would cause that beyond some sort of hard coded thing or there being a boot menu item on the cache device that has been misconfigured. In regards to the invisible cache, the way this used to work is that on the gateway (L3 or NAT) you would use traffic rules to intercept port 80 traffic and redirect it to the cache server which then just acts like a regular proxy server and serves its own content or fetches it from the internet. Port 443 gets a little more tricky due to certificates and 'you're basically doing a man in the middle attack on your own network'. It's been a while since I messed with it but the new L4 and up firewalls might be able to do encrypted traffic. If you want a laugh on this topic, look up 'Upside down ternet' which is basically this concept in a nutshell but then to taunt people stealing your wifi🙂 (edit) And the portfast thing like the other guys said in the comments, I totally forgot about that as it's one of the first things I turn on and then forget about lol
Never heard of port mirroring before, but it seems like it turns part of your switch into a hub of sorts ^^ On a different note, when playing with older gear that didn't behave nicely with modern network gear I'd grab an old laptop and connect it directly for tftp. The laptop itself has Debian 12 on it, but I can chroot into a Debian Etch environment and start a tftp server from there. I really hope for you to be able to get the cache running again, despite the probable NVRAM issue. Wishing you best of luck :)
Some uses for port mirroring is for IDS systems. Intrusion Detection System. Some examples at work are monitoring industrial networks. Actively doing stuff in those networks can be very dangerous. From monetary, lost production, to life threatening, things going sideways in the field.
We used it for IDS mostly, DoD used it for both IDS and full packet capturing of traffic. We also used it during a campus wide network issue, around 150 C2950's hanging off of a pair of C6509's that were fully populated and a C4506, for that one, turned out to be a misconfiguration by the boss that triggered a spanning tree storm base wide. Of course, taking longer to fix than to diagnose.
By the way, I bought 2 Cisco WAVEs.. they have a VGA connector hidden under the disks. They will be 2 excellent machines for me to install VMWARE ESXi (and cheap, here they ask for 80 dollars or less)... As a cache, I have no way to test them, they are unlicensed. Thank goodness sellers haven't discovered that these machines work VERY WELL as a PFSENSE/OPNSENSE firewall or virtualizer, for laboratories.
Try pinging .40 from .200. Run tcpdump on .200 while the cache boots, make sure you see an ARP reply being sent. Check your 2950 config for any vlan config left over - arp is a layer 2 function, which sits "beside" vlans. A "factory reset" may not have cleared them. Also check that portfast is turned on, or that spanning tree is turned off. The standard delay is 30 or 45 seconds from memory.
Cool feature. In lieu of such a thing, you can also simply get a cheap 100mbits rated HUB from 1997, and connect sniffer and Device under Test to it. You will have the same effect.
tftp is really tricky, not sure how many times I've used port mirroring for that. normally you get exactly these kinds of strange error messages. having said that did you try the other ethernet port (1) - I guess the ports should somehow be configured for what side of the caching you have it connected. how does the inside looks like?
Clabretro, I went down the SAME rabbit hole trying to get a VOIP system set up. In order for your TFTP to work you need to have a DHCP server running on the same box as the TFTP. It doesn't matter that you've hard coded an ip address into the Cisco box. Something has to tell the Cisco box where the TFTP server is. In the DHCP config you will need an entry for your TFTP address. With the two working together, your ARPs will be answered and the Cisco box should be able to contact the TFTP box. You *might* have to stuff your DHCP server address into the gateway section on the Cisco box ... only experimenting will tell. GOOD LUCK ... and great content as always!!
This doesn’t make sense. He is specifying the tftp server. Why do you think the cache box will use tftp config from dhcp ? Because your voip phone did? I don’t follow.
@@mattym8 I know it sounds odd, but when I was setting up the VOIP system the only way to get the ARPs responded to was to also set up a DHCP server. Once DHCP was set up and running it responded to the ""who has xx.xx.xx.xx" requests. I scratched my head for days before trying this ... and it worked.
DHCP is NOT necessary for TFTP. You can absolutely TFTP without DHCP. What DHCP gives you (aside from IP config) is extensions to specify stuff like the initial TFTP boot image name. (TFTP is a much older protocol than DHCP).
I think you are thinking about booting from a tftp image automatically, like PXE-ish. Then you need a set of DHCP/BOOTP options set to point the machine to the correct source to find the image, but that is not necessary when you just want to store/get a file from a tftp like in the video.
What a coincidence, just the other day I used a monitor session to prove to a service provider that they setup a new link incorrectly. But it wasn't using another port as the destination. We just got a new 10G interstate link between a new office and our head office, I was seeing packets sent and received, but it couldn't resolve an ARP for the router at the other end of the link, same for the other end. I figured there must be something wrong with the packets they're sending, but what do you do when you've got a 10G link that you have to capture the traffic from, but no device with a 10G port to mirror to and capture? Easy, you set the destination to a file! Obviously you can't do it on these old switches, and it's mostly just supported on more modern Cisco router platforms. However, it is supported on the NCS 540 router that I was using. So I configured a monitor span session with the source set to the port with the new link and the destination set to a file, started the session and pinged from the other end of the link, then stopped the session and downloaded the file. Opening it in wireshark, lo and behold I see the packets have an 802.1q header with a VLAN ID, they were tagging the packets with a VLAN ID, even though it was supposed to be untagged. Once I told them what I saw, eventually they figured out where the extra tagging was coming from and fixed it, and then I could ping from the router at both ends of the link to the one on the other side. And that's why the concept of port mirroring is still an important and useful tool for troubleshooting, even so many years after the gear you're using it on came out.
I have an OS/2 Warp 4 pc that I have been having a hard time getting to connect to my network. I'll have to try mirroring to see what its actually doing!
Love the channel Clab! Noob question, why cant the cache just be directly connected to your workstation and you wire shark it that way, why do you need the switch inbetween?
Can you suggest a decent Cisco router to start doing the same type of things you are doing? I've looked at eBay and there are all sorts of models with all sorts of options at all sorts of prices. Obviously I don't want to pay a huge amount for shipping but I'd like to find the right one to start my journey with. Thanks for all your amazing videos!
There was a list of flags in the boot message that the firmware can be configured to use, did you try setting the flags along the boot message instructions?
Hey uhh… are you sure that the traffic coming from the cache is reaching any other ports on the switch? I’d recommend checking by firing up wireshark on that linux machine that’s acting as a TFTP server as well, see if you have any incoming ARP traffic. If not, the cisco network switch is causing a problem.
@clabretro Duplex. That's 100M. It could be a duplex issue. In the olden golden days, 100M never did autonegociate very well. If you try forcing both sides to 100M full, see if that solves it. A common symptom of duplex mismatch is that you can send, but, you cannot receive.
Hey Clab, when you show the "New FTP Server" at 22:05, is that an old "U3 Enabled" Geeksquad USB drive in the back? If it is, I haven't seen one of those in forever! Were you an Agent back in the day? Also, it LOOKS like its not getting ARP response like you suspected. Is the MAC address of your server FF:FF:FF:FF:FF? If so, its likely causing an issue. Per Google LABS "A Cisco device cannot resolve an ARP request when the MAC address is set to "FF:FF:FF:FF:FF:FF" because this is the broadcast MAC address, meaning it sends the ARP request to every device on the network, and no single device will respond as the intended target, causing the ARP resolution to fail."
I'm pretty sure the default Mac address for arp is the broadcast address of all Fs so it would be weird for Cisco devices to drop them unless they were configured to do that
@ax14pz107 I couldn't tell if it was a broadcast or if it was reaching out to his server using that Mac address. I kind of hoped it was just broadcast but couldn't be bothered to rewind and scour the video LOL
@@deadreaver666 lol I haven't watched the full video yet so I'm not sure either. On the initial arp frame the dst should be broadcast but the src needs to be the device.
Yes, but it won't have to explicitly ask because it will learn the cache's MAC address when it (the cache) does its ARP request. IP stacks are usually opportunistic and will take advantage of any traffic that it can see to learn MAC addresses. There's even such a thing as a "Gratuitous ARP," where a client can transmit an ARP response without anyone having requested it. This is often used when a device knows that its own IP-MAC pair has changed, while other devices on the network will not know that has happened. An example of that is with redundancy for high availability. If you have an active and backup router, and something happens where the active router fails or reboots or is otherwise disabled, the backup router will transition to the active role and send a gratuitous ARP so everyone on the network knows "I'm claiming this IP and my MAC address is ...." The rest of the devices update their MAC address table and immediately start forwarding to that address instead. This prevents having to wait for their stale ARP entries to age out normally, which could easily take tens of seconds or even minutes, depending on the timer settings for each host.
With some of the Cisco systems I have found that I have to rename the file it's looking for to what the device has on it. Try that and see what happens.~bp
Your port Fe12 to your other network shut down probably because it's access by default and received a BPDU from your Ubiquiti gear and shutdown to prevent a loop
Network switches only send packets out ports that need it (they keep a MAC address table of every MAC address they've seen on every switch port, and only send packets addressed to a particular MAC address out the port that it knows that recipient is on), except for broadcast packets. So let's say you have computers plugged into ports 1, 2, and 3 of a switch. If you run Wireshark on computer 1, it will not see the traffic between computers 2 and 3 because the switch knows it doesn't need to send the packets to computer 1. With port mirroring, you are able to tell the switch to send all of those packets to the port computer 1 is plugged into. Back before switches were common, ethernet hubs repeated *all* packets on *all* ports, so you didn't need port mirroring. This was closer to how original Ethernet (where every computer was connected to a single shared wire) worked, where all devices saw all traffic and could collide with each other. Overall network throughput potential was way lower since only one computer could send a packet at a time, and there was all sort of collision detection and backoff and retry stuff to try to coordinate sharing a single collision domain.
In this case, you could just run tcpdump on the target Linux TFTP host (or Wireshark on a Windows/Mac/Linux (GUI) host) to watch the network from its point of view. I suspect the use of a managed switch is its own goal here.
@clabretro Yup, no arp RESPONSE... iT should be 192.168.1.1 IS at (mac addres) tell (ARP src req IP) or just a mac response - I don't remeber exactlu...
😆 port mirroring.... I once wrote automation software to allow port mirror of any port of any device anywhere in the network. 8k locations, 38k switches, 1.8 million ports, 650k connected devices.
I think part of your problem was that ports Fa0/11 and Fa0/12 seemed notably absent on your switch when you ran `show ip int brief` at 2:31 . Even if the local tftp server errored out, that certainly wouldn’t have helped things
He stopped the printout of the sho command before it got to 11 and 12. The port could have been shutdown or he may need a crossover cable. Gigabit can auto-cross but 10/100 ports did (do?) not.
@@nathangreer4685 from my experience, it’s not so much if 10/100 ports support MDIX as it does if the actual switch does. 2960 and 3750 switches definitely do, I haven’t worked with Cisco switches older than that, but I’m gonna hazard a guess and say that a cross over would’ve helped here
So this is how a SHADY government worker performs a man-in-the-middle attacks.. I have been lately having troubles connecting to microsofts azure network, local ISP tells, that its mallware or phishing ... what a shame
mxshift over in The Serial Port Discord made a great observation: "cache engine using a FD:FD:10 MAC address seems wrong. That's an unallocated OUI and marked as multicast. That feels like an autogenerated MAC used when the NVRAM is trashed. That would also explain the 'is a .' as the model designation would also be in NVRAM"
Perhaps some corrupt NVRAM? More investigation to do on the Cache Engine for sure!
That's exactly what I'v wanted to tell you. You must fix your NVRAM or CMOS RAM or whatever is in there. Arp has no response because of bad source address and then nothing else could work. But you have to fix that NVRAM issue first since otherwise it will never boot from anything if there is no model number stored there.
Not NVRAM... IDPROM. And that will be "impossible" to fix. This is obvious from it's lack of hardware identification. Most of their things made in that old PIX form factor use an I2C EEPROM (flash) for the hardware details. It's either failed, or something scrambled it.
@@jfbeam How do you know it's EEPROM and not a battery-backed SRAM, for example?
Most cisco gear used sdcard for flash. Does not look like this cache engine has one. The chip might be cooked.
I'm reminded of the Sun Microsystems issue. Can't even boot the machine until you've run through safe boot processes for an agonizing 2 minutes, then you get to tell the machine what it is and what it's MAC is, before you can finally try to boot. And that's before you discover that the replacement battery costs $100 both new or "gently discharged"
I’ve been unemployed and depressed for 7 months now. Your videos bring me unspeakable joy. Thank you Mr. Retro.
It's incredible how much UA-cam(rs) can uplift you, right?
I'm out of work due to chronic pain from an injury years ago. I'm dealing with my depression through the company of my pets and a few UA-camrs whose videos brighten my day whenever they upload.
Every time I see one of clabretros videos I think "ah just what I need" no hyperactive editing,clear and thoughtful communication of information and an obvious true love for what he's doing. Adrian's digital basement and the backlogs (very different channel topic) are my comfortable places on the internet, amongst a few others.
I wish I was in a financial position to be supportive of the people who provide all of this to us, maybe one day, but I hope our engagement and joy for their content can be enough to keep them doing what they do for a long time to come. Cheers to you all, take care and happy holidays.
Hang in there guys, better days are ahead.
Broke my foot 3 months ago, same brother.
its currently like midnight in germany, and this is exacly what i needed. thank you.
Same here lol, watching this in bed XD
Ye 23:53 In czechia RN and just got home from party. Just the content I need to fill rest of my friday night
A huge terr attack just hapened in germany
@@KabodankiI wouldn't call it huge, still horrible though
@@Kabodanki Where???? How tf do i live in germany and not know any of this????
Tip: On the NIC (network 'card') you use to capture the mirrored packets with Wireshark (or other similar software) - turn off everything except the packet capture driver (e.g. NPCAP or similar). Under "Properties -> Networking -> This connection uses the following items". Just remember to turn them back on (if needed) later.
Wireshark (or rather the packet capture driver) will still be able to capture all the mirrored packets to/from the device under test/inspection (i.e. source). However it will stop any 'pollution' (though this can be filtered) of the capture from the device being used to do the capture (such as at 4:27).
I write this without having watched the entire video yet, or having read all the comments, just something I find useful to do when I am doing the same - mirroring and packet capture to allow analysis of the network traffic of a device (usually embedded of some kind).
very useful, thank you!
Thanks that answers my question, Of whether the capture interface needs to have an IP address on the network or not.
= Since it's using the capture driver it does not.
However this sounds like advice to disable functions within windows and others should note the equivalent for Linux or Mac.
@@imark7777777 Yes sorry... correct... that is Windows centric.
I could be wrong (or there might be better ways) but in Linux I believe similar can be achieved by ensuring that interface to be used for capture doesn't a) have a dhcp-client configured to automatically get an IP (and/or do auto-IP or similar); and b) that you don't statically assign an IP; and c) that, if not already, it is in promiscuous mode (which of course it will need to be - otherwise it is just going to ignore anything not for it's MAC).
Great video! I like to watch you poke around with "old" tech and fideling things out! Very helpful and inspiring for my own homelab!
Thank you! ❤
With these old machines, I always set the time and check for any batteries that may have dried up decades ago. You never know how a machine will behave if its time is wrong.
Been so deep into telephony at work, it was nice to hear a bit of a refresher on wireshark.
It's rare that there's content on youtube where I can actually watch someone learn something I know fairly well for the first(?) time - kind of fun actually! Wireshark is a godsend. While "it's the network!1!!" is usually *not* true, it at least helps prove your case when you're trying to get the fingers pointed in the right direction...
I love watching you tinkering with my daily job. I sitting there and shouting at my telly, when youre following a wrong trail to solve the problem. I would love to show you the real cisco kings, like a Callmanager 6,8.5, the ISE, old Wifi Controller and LightAP. Man there are many rabbitholes you could follow :)
ITS NOT EVEN BEEN 7 WHOLE DAYS MY HEART CANT TAKE THIS MUCH CLABRETRO!!
Oh those big GBIC slots takes me back!!! Thanks for the memories.
I recall buying big blue non Cisco ones for my home Cisco meddling back in 2003 together with sun sparc happy meal cards
Gee, this is what I was thinking about yesterday when I decided to have a closer look at a IPTV client.
Happy New Year!
Its really intersting learning about hardware from when i wasnt even planned, keep up the great work!!!
Love seeing all the older equipment being setup and configured! Great job as always!
"portfast"... I'm pretty sure the cache engine is not going to wait 30s for spanning-tree to complete loop detection. It's a VERY common issue. Watch the front of the switch. The light will flash orange while it's blocked. Until it goes green, no traffic will be forwarded.
very good observation, someone else mentioned this as well! i'm definitely going to play around with that
Yeah most likely. Portfast and bpdu guard all the client facing ports. It's super annoying and some devices really really hate it.
Edit: after finishing watching the video, it's not the port being down because then you wouldn't see any packets with the port span.
Yes! Portfast! . Seeing the ARP requests and no response I thought "the problem is the switch" Might be the agonizingly long STP wait time on the cisco switch.
This brings me back to my CCNA days, some guy accidentally whipped a class router and the teacher assigned us to bootstrap it and install it over the network. Long story short, we wasted hours banging our heads trying to figure it out. It was the STP...
I'm so used to immediately turning on portfast on switches that I managed that I totally forgot about this one lol
@clabretro at 21:08 you can see the LED as orange, which on that era of Cisco switches means that spanning tree hasn't gone fully active yet and is not passing traffic. It then switches to green a few second later. I would definitely want to rule that out. It would explain why the ARP requests not receiving a response. I would set the interface parameters on both the device and monitor port to "switchport mode access" and then "spanning-tree portfast"
I haven't had to do this since college in Cisco cli and have totally forgotten how.
Thank you now i want a whole retro rack of old sun and cisco gear lol
Love this Cisco stuff! Keep it up, I’ll keep watching!
glad to hear it!
Never missed a single clabretro vid since March 🔥🔥
niiiice
Love it!! I can watch your retro videos for hours on end!
Working at a mixed vendor ISP back in ancient times we had quite a few problems when connecting Ciscos to other switch vendors. Mostly this was solved by disabling auto-negotiation on switchports and set them hard to 100/100 FDX or 1000/1000 FDX (newer equipment). Mostly we needed this when adding HP or 3Com switches. Sometimes we needed to do the same for some 3rd party server equipment too. That could be the problem with old Cisco talking to your Ubiquti switch.
Port mirroring is super useful. You can even do this with virtual switches.
This issue is rather sketchy though lol
You have a link between the switch and the cache device, but then the uplink to the other network goes down ?
You get no responses to ARP requests coming from the cache. You can tell that it doesn't work in wireshark because there was no response :P
I am unfamiliar with the cache device from cisco although I have used cache devices in the past.
There is a chance the 5500 is acting as a switch and your home lan is going 'nope' when it sees another switch being plugged in to a lead switch, this is BPDU guard or root guard I think it was called. That might explain the uplink to the network going down. But this does not always make the link light go off.
Open up a telnet to the switch and turn on console logging before turning on the rest to get a sense of what that's doing when the link becomes active.
Another thing that might explain this is vlan configuration but I don't think you have those configured ? I would expect to see vlan oriented broadcasts.
It's also possible that something created an ACL on the switch ? Like a default config thing ?
Then there's you saying the mac address isn't right probably, is it possible the cache device is actually binding to fastethernet1 instead of 0 ? That's the only thing I can think of that would cause that beyond some sort of hard coded thing or there being a boot menu item on the cache device that has been misconfigured.
In regards to the invisible cache, the way this used to work is that on the gateway (L3 or NAT) you would use traffic rules to intercept port 80 traffic and redirect it to the cache server which then just acts like a regular proxy server and serves its own content or fetches it from the internet.
Port 443 gets a little more tricky due to certificates and 'you're basically doing a man in the middle attack on your own network'. It's been a while since I messed with it but the new L4 and up firewalls might be able to do encrypted traffic.
If you want a laugh on this topic, look up 'Upside down ternet' which is basically this concept in a nutshell but then to taunt people stealing your wifi🙂
(edit) And the portfast thing like the other guys said in the comments, I totally forgot about that as it's one of the first things I turn on and then forget about lol
Man dude. Your troubleshooting videos are awesome man! You're so cool! Awesome job again dude!
thank you!
WCCP oh boy, flashbacks to setting up Websense 7.5 with a Catalyst 6509.
Yeah!!! a new clabretro video, watching while coming home from work
"Come to bed Clab!" "Be up soon Honey, I am port mirroring." :)
LOLZ!
Never heard of port mirroring before, but it seems like it turns part of your switch into a hub of sorts ^^
On a different note, when playing with older gear that didn't behave nicely with modern network gear I'd grab an old laptop and connect it directly for tftp. The laptop itself has Debian 12 on it, but I can chroot into a Debian Etch environment and start a tftp server from there. I really hope for you to be able to get the cache running again, despite the probable NVRAM issue. Wishing you best of luck :)
Wow I had no idea those 2950s came in 12 port but full 19” width. So weird seeing that huge blank space between the RJ45s and GBICs
In later Cisco switches, monitoring was/is? called port spanning or span. Not sure what it is today. Just FYI. Used it a lot back in the 2000s
Some uses for port mirroring is for IDS systems. Intrusion Detection System. Some examples at work are monitoring industrial networks. Actively doing stuff in those networks can be very dangerous. From monetary, lost production, to life threatening, things going sideways in the field.
Exactly. Vectra is one of these systems. Absolute amazing to see what happens across a large company network.
We used it for IDS mostly, DoD used it for both IDS and full packet capturing of traffic.
We also used it during a campus wide network issue, around 150 C2950's hanging off of a pair of C6509's that were fully populated and a C4506, for that one, turned out to be a misconfiguration by the boss that triggered a spanning tree storm base wide.
Of course, taking longer to fix than to diagnose.
By the way, I bought 2 Cisco WAVEs.. they have a VGA connector hidden under the disks. They will be 2 excellent machines for me to install VMWARE ESXi (and cheap, here they ask for 80 dollars or less)...
As a cache, I have no way to test them, they are unlicensed. Thank goodness sellers haven't discovered that these machines work VERY WELL as a PFSENSE/OPNSENSE firewall or virtualizer, for laboratories.
very interesting! let me know how they work out
It’s cool to watch the skills sharpen.
🔪
Let’s go!!!! 6am here, but it don’t matter.
Great content as always brother!
thank you!
Try pinging .40 from .200. Run tcpdump on .200 while the cache boots, make sure you see an ARP reply being sent.
Check your 2950 config for any vlan config left over - arp is a layer 2 function, which sits "beside" vlans. A "factory reset" may not have cleared them.
Also check that portfast is turned on, or that spanning tree is turned off. The standard delay is 30 or 45 seconds from memory.
Cool feature. In lieu of such a thing, you can also simply get a cheap 100mbits rated HUB from 1997, and connect sniffer and Device under Test to it. You will have the same effect.
I need to try portmirroring on my cisco gear to. Looks like fun
Ah back in the days before HTTPS was a thing. Good times, good times.
For quick and dirty port mirroring just place a hub inline on the link you want to monitor.
tftp is really tricky, not sure how many times I've used port mirroring for that. normally you get exactly these kinds of strange error messages.
having said that did you try the other ethernet port (1) - I guess the ports should somehow be configured for what side of the caching you have it connected.
how does the inside looks like?
YEEES A NEW CLAB VIDEO FOR MY SATURDAY
You should put the image on internet archive if it was hard to find.
Clabretro, I went down the SAME rabbit hole trying to get a VOIP system set up. In order for your TFTP to work you need to have a DHCP server running on the same box as the TFTP. It doesn't matter that you've hard coded an ip address into the Cisco box. Something has to tell the Cisco box where the TFTP server is. In the DHCP config you will need an entry for your TFTP address. With the two working together, your ARPs will be answered and the Cisco box should be able to contact the TFTP box. You *might* have to stuff your DHCP server address into the gateway section on the Cisco box ... only experimenting will tell. GOOD LUCK ... and great content as always!!
This doesn’t make sense. He is specifying the tftp server. Why do you think the cache box will use tftp config from dhcp ? Because your voip phone did? I don’t follow.
@@mattym8 I know it sounds odd, but when I was setting up the VOIP system the only way to get the ARPs responded to was to also set up a DHCP server. Once DHCP was set up and running it responded to the ""who has xx.xx.xx.xx" requests. I scratched my head for days before trying this ... and it worked.
Interesting! Certainly no harm in trying that out
DHCP is NOT necessary for TFTP. You can absolutely TFTP without DHCP. What DHCP gives you (aside from IP config) is extensions to specify stuff like the initial TFTP boot image name. (TFTP is a much older protocol than DHCP).
I think you are thinking about booting from a tftp image automatically, like PXE-ish. Then you need a set of DHCP/BOOTP options set to point the machine to the correct source to find the image, but that is not necessary when you just want to store/get a file from a tftp like in the video.
Enable spanning-tree portfast on the port that's going to the cache server. You can see it's not forwarding traffic yet when the port LED is orange.
What a coincidence, just the other day I used a monitor session to prove to a service provider that they setup a new link incorrectly. But it wasn't using another port as the destination.
We just got a new 10G interstate link between a new office and our head office, I was seeing packets sent and received, but it couldn't resolve an ARP for the router at the other end of the link, same for the other end. I figured there must be something wrong with the packets they're sending, but what do you do when you've got a 10G link that you have to capture the traffic from, but no device with a 10G port to mirror to and capture? Easy, you set the destination to a file! Obviously you can't do it on these old switches, and it's mostly just supported on more modern Cisco router platforms. However, it is supported on the NCS 540 router that I was using.
So I configured a monitor span session with the source set to the port with the new link and the destination set to a file, started the session and pinged from the other end of the link, then stopped the session and downloaded the file. Opening it in wireshark, lo and behold I see the packets have an 802.1q header with a VLAN ID, they were tagging the packets with a VLAN ID, even though it was supposed to be untagged. Once I told them what I saw, eventually they figured out where the extra tagging was coming from and fixed it, and then I could ping from the router at both ends of the link to the one on the other side.
And that's why the concept of port mirroring is still an important and useful tool for troubleshooting, even so many years after the gear you're using it on came out.
@4:39
It is called : apipa 😊
Isnt the eth0 the "wan" port and eth1 the "lan" port. Why would it try to fetch tftp stuff from the "wan" port
I have an OS/2 Warp 4 pc that I have been having a hard time getting to connect to my network. I'll have to try mirroring to see what its actually doing!
Love the channel Clab! Noob question, why cant the cache just be directly connected to your workstation and you wire shark it that way, why do you need the switch inbetween?
it's more fun this way 😂
😂😂😂 love it @clabretro
Can you suggest a decent Cisco router to start doing the same type of things you are doing? I've looked at eBay and there are all sorts of models with all sorts of options at all sorts of prices. Obviously I don't want to pay a huge amount for shipping but I'd like to find the right one to start my journey with. Thanks for all your amazing videos!
I'd recommend a Cisco 2600
Even if I'm not super into the topic I enjoy your reverse-valley-girl cadence if that makes sense
I take that as a compliment
@@clabretro it was!
There was a list of flags in the boot message that the firmware can be configured to use, did you try setting the flags along the boot message instructions?
Quite needed to get the cables working.
Hey uhh… are you sure that the traffic coming from the cache is reaching any other ports on the switch? I’d recommend checking by firing up wireshark on that linux machine that’s acting as a TFTP server as well, see if you have any incoming ARP traffic. If not, the cisco network switch is causing a problem.
just a dumb thought, Try a / in front of of the file name when attempting to boot from net.
have you tried turning it off and on again? that normally fixes most stuff
@clabretro Duplex. That's 100M. It could be a duplex issue. In the olden golden days, 100M never did autonegociate very well. If you try forcing both sides to 100M full, see if that solves it. A common symptom of duplex mismatch is that you can send, but, you cannot receive.
good idea!
Those old 2950's did not have auto mdx - You need a cross over ethernet cable between that switch and your modern network equipment.
169.x.x.x is APIPA (4:50)
Have you tried a crossover cable between the switch and cache?
Not since like 1994
Sanity check: are the ports you're attached to adminstratively down?
What do you have against maximising windows ?
Two things I always have on my work laptop: serial terminal software and Wireshark
Indispensable, both of them!
Is your stack of linksys devices growing from video to video?
occasionally lol
Hey Clab, when you show the "New FTP Server" at 22:05, is that an old "U3 Enabled" Geeksquad USB drive in the back? If it is, I haven't seen one of those in forever! Were you an Agent back in the day?
Also, it LOOKS like its not getting ARP response like you suspected. Is the MAC address of your server FF:FF:FF:FF:FF? If so, its likely causing an issue.
Per Google LABS
"A Cisco device cannot resolve an ARP request when the MAC address is set to "FF:FF:FF:FF:FF:FF" because this is the broadcast MAC address, meaning it sends the ARP request to every device on the network, and no single device will respond as the intended target, causing the ARP resolution to fail."
I'm pretty sure the default Mac address for arp is the broadcast address of all Fs so it would be weird for Cisco devices to drop them unless they were configured to do that
@ax14pz107 I couldn't tell if it was a broadcast or if it was reaching out to his server using that Mac address. I kind of hoped it was just broadcast but couldn't be bothered to rewind and scour the video LOL
@@deadreaver666 lol I haven't watched the full video yet so I'm not sure either. On the initial arp frame the dst should be broadcast but the src needs to be the device.
I feel so young bc I have NEVER seen this Cisco logo before
Doesn’t the TFTP server need to ARP to find the MAC address for the cache engine to?
Yes, but it won't have to explicitly ask because it will learn the cache's MAC address when it (the cache) does its ARP request. IP stacks are usually opportunistic and will take advantage of any traffic that it can see to learn MAC addresses.
There's even such a thing as a "Gratuitous ARP," where a client can transmit an ARP response without anyone having requested it. This is often used when a device knows that its own IP-MAC pair has changed, while other devices on the network will not know that has happened. An example of that is with redundancy for high availability. If you have an active and backup router, and something happens where the active router fails or reboots or is otherwise disabled, the backup router will transition to the active role and send a gratuitous ARP so everyone on the network knows "I'm claiming this IP and my MAC address is ...." The rest of the devices update their MAC address table and immediately start forwarding to that address instead. This prevents having to wait for their stale ARP entries to age out normally, which could easily take tens of seconds or even minutes, depending on the timer settings for each host.
booting from ata0 indicates it's trying to reach an IDE device, maybe a CF card inside?
With some of the Cisco systems I have found that I have to rename the file it's looking for to what the device has on it. Try that and see what happens.~bp
Your port Fe12 to your other network shut down probably because it's access by default and received a BPDU from your Ubiquiti gear and shutdown to prevent a loop
Wouldn't you want to use Cisco-CE BIOS Version 1.0 or higher rather than Version 0.9B ?
Does that thing require RARP (reverse ARP)?
i wonder.... if that cache worked would it cache HTTPS or only HTTP??
I'd imagine HTTP
Whats the difference with just running wireshark on the network?
Network switches only send packets out ports that need it (they keep a MAC address table of every MAC address they've seen on every switch port, and only send packets addressed to a particular MAC address out the port that it knows that recipient is on), except for broadcast packets. So let's say you have computers plugged into ports 1, 2, and 3 of a switch. If you run Wireshark on computer 1, it will not see the traffic between computers 2 and 3 because the switch knows it doesn't need to send the packets to computer 1. With port mirroring, you are able to tell the switch to send all of those packets to the port computer 1 is plugged into.
Back before switches were common, ethernet hubs repeated *all* packets on *all* ports, so you didn't need port mirroring. This was closer to how original Ethernet (where every computer was connected to a single shared wire) worked, where all devices saw all traffic and could collide with each other. Overall network throughput potential was way lower since only one computer could send a packet at a time, and there was all sort of collision detection and backoff and retry stuff to try to coordinate sharing a single collision domain.
@ Thank you for the clear answer!
One problem was already found, but otherwise my hunch would be to port-mirror the device on the other end too.
Port mirroring is cool, but I'm surprised you didn't go even more old school and just use a hub.
In this case, you could just run tcpdump on the target Linux TFTP host (or Wireshark on a Windows/Mac/Linux (GUI) host) to watch the network from its point of view. I suspect the use of a managed switch is its own goal here.
U could use any hub and you would get same result without any config
Also u could setup bridge on 2network interfaces on Linux and would get same 😊result without
@clabretro Yup, no arp RESPONSE... iT should be 192.168.1.1 IS at (mac addres) tell (ARP src req IP) or just a mac response - I don't remeber exactlu...
😆 port mirroring.... I once wrote automation software to allow port mirror of any port of any device anywhere in the network. 8k locations, 38k switches, 1.8 million ports, 650k connected devices.
isn't that a hub with extra steps?
yup lol
except you get to choose exactly which ports to monitor.
Return of the Spudger….
Cache Cache Money
These days caching isn't much use when everything uses TLS.
I think part of your problem was that ports Fa0/11 and Fa0/12 seemed notably absent on your switch when you ran `show ip int brief` at 2:31 . Even if the local tftp server errored out, that certainly wouldn’t have helped things
He stopped the printout of the sho command before it got to 11 and 12. The port could have been shutdown or he may need a crossover cable. Gigabit can auto-cross but 10/100 ports did (do?) not.
@@nathangreer4685I think it depends on the switch. Older ones don't but newer ones I think have auto mdix.
@@nathangreer4685 from my experience, it’s not so much if 10/100 ports support MDIX as it does if the actual switch does. 2960 and 3750 switches definitely do, I haven’t worked with Cisco switches older than that, but I’m gonna hazard a guess and say that a cross over would’ve helped here
If you listen he broke out of it
So this is how a SHADY government worker performs a man-in-the-middle attacks.. I have been lately having troubles connecting to microsofts azure network, local ISP tells, that its mallware or phishing ... what a shame
me, having just finished a comptia network+ and a+ class: "hey, i've heard of this!"
I'm just starting mine at 36, wish me luck 🫡
Ciekawe czemu zabrakło rekordzisty na 1 miejscu: Bernie Madoff ?
well everyone knows how to setup port mirroring on simple cisco like that.
12:09 errno 🤣🤣🤣 cisco your drunk my guy...
"errno" is a common shortening for "error number". Not strictly a Cisco-ism and you can see this term used widely in the software world.
Connect cache engine directly to computer, boom done. This is why I don't make youtube videos
Just use a hub lol.
Finally Linux again. Cant stand that crappy Windows machine! ;-) Reminds me of the old videos.
What? 25 mins and no solution 😭