I got hired by a startup when I was an amateur. First week of browsing through their repos - 3 0-click RCE and remote shell exploits patched. I had to educate their "senior" on how much they f'd up. I don't think hiring amateurs is that expensive (in some cases), it's much like with doctors. Experienced doctors will naturally go for the most obvious explanations, while med students will still remember their training enough to consider the "rare cases". That's why you would generally want an opinion of an experienced doctor and a newbie to extrapolate what may actually be wrong.
eh... I think its even less than that. Honestly, amateurs are often MORE likely to try and stick to standards because they know fewer tricks on how to get arround them. The problem is when your leadership demands you finish the project NOW!!! because they think yelling is an effective motivator... so the development team slaps something together meeting the bare minimum of requirements (which of course, didn't include security provisions) and call it a day. Those on the team with the most experience, usually have experience in exactly this... how to get code out the door that meets those requirements quickly, and which standards they know the company isn't going to bother checking for.
Why? This was messed up by a professional. Professionals are just people who do it for money with the least amount of work necessary and caring more about unions and pay raise than security.
We got porn ban in India, and it used to work before I left for my studies to the US 9 months back. Back again for summer break and I see that the blacklist only works intermittently, guessing they deployed more hardware to handle the traffic and just didnt enable/implement the firewall correctly on them. So all I do is hit the website several times to get through the blacklist lol.
It kinda sends shivers down my spine that ISP's can be on such a level of incompetence. RW access to millions of devices, even FBI customers by just running a HTTP request multiple times. Jesus christ i dont know if i can ever trust anything software ever again.
@@dustee2680 There is a joke that exists that goes along the lines of "The tech enthusiast has everything in their house linked together. The tech worker has a twenty year old printer and a gun to shoot it if it makes a noise he doesn't recognise."
I work for a Large ISP, we can push settings, pull settings, get train rates, see connected devices, i can bridge ports to WAN so I could expose anything plugged directly into our gateway directly to the public network, if you changed the passwords on the device I can force reset them, or even factory reset the whole device. If you care about your privacy and safety use your own router
Some ISPs that offer gigabit fibre have the fiber termination point and WiFi router in the same package, I had no other option than to get the UniFi Dream Machine SE and disable the WiFi on the modem and plug a local network to the wan port on the udm se, then all traffic goes through the udm and the UniFi WiFi gateways, but this vulnerability and how they reacted to it has me concerned, the modem is huge and gives off a lot of heat and is quite capable, all that power, complexity and control is breeding ground for breaches and malware, I just hope I can set up pf sense to work well
But you have to admit, from a tech support perspective the reason to have that much control is because the vast majority of customers calling in can't figure out how to set up a wifi password and just want the ISP to do it for them so they can get on with their day.
You can accidentally find something purposefully placed. Like that time I accidentally noticed a glass pane with my head while running. It was purposefully placed there to be a sound dampener.
Cox once put corporate domains from an email whitelist (to allow them to send emails en masse) onto a blacklist, ensuring hundreds of firms cannot send emails to Cox users, then pretended those corporations are at fault. Those corporations are blocked to this day
@BillAnt the problem is that those corporations that were now blocked all asked Cox to revert it, and Cox just either ignored them or blamed them for it. Source: it included my former employer, who is actually a household name in my country in western Europe.
This reminds me of another story I read about a German ISP years ago, must've been at least 5 years ago. The ISP in question (I believe it was Vodafone Germany, Unity Media or Kabel Deutschland; definetly one of these three) wasn't providing customers with the credentials necessary to use any router on their network (at the time they didn't had any legal obligation to do so, so they forced customers to use the rental routers). So one of their customers didn't want that rental unit and instead wanted to use their own router, so they started digging in the rental router from their ISP to get it to spit out the network credentials. While doing so they found out that bypassing the rental router and getting direct access to the ISPs network also gave them direct access to a similar maintanance API, completely without any authentication requirements (after all, you wouldn't normally have completely free access to this part of the network). So it was fairly trivial for them to change any router from that ISPs network, just by using his own router. Not only did the ISP fix that vulnerability quickly, it also sparked a big legal debate on whether or not it was legal for that ISP to demand usage of rental routers. The outcome was no and now everyone can use any router, even on the cable networks (which is now all Vodafone Germany)
@@Chickenbreadlp Can you not just buy a separate modem? So long as it's compatible (DOCSYS version, etc.) you should just be able to put your own router behind it...
@@PhysicsGamer to use a modem you still need the network credentials from the ISP, which they weren't giving out at the time. They weren't even offering the option to just get the modem. It was rental router or nothing. Iirc only fiber ISPs provide a dedicated modem, but even that's a controversial topic, because it's another device drawing its own power, when a modem+router combo is power efficient...
Just a heads up, towards the end of the video (27:19) you said you "think Cox fixed the issue within a month or two." Using the day/month/year calendar, that makes sense based on what this post has written, but I think this post had the date in a month/day/year format, suggesting the bug was hot fixed within a day of it being reported, not a month. I can only imagine how many heart attacks this researcher nearly had from beginning to end of this story.
I have to deal with the date differences all the time at work, because we have people in the US, Germany, Romania, Greece, etc. I'm at the point that I write dates in a format like today is 11Jun2024 instead of 5-11-24 or 11-5-24 or 24-5-11
@@TheJunky228 I am in finance in the U.S. but my corporate clients are global. I spell the month out also, otherwise it gets too confusing. I generally follow the day month year rule but even then there are times you have to question it so I also went to always spelling the month out.
The scary thing is that all the described stuff is really easy and very basic stuff. Its not something highly sophisticated or really obscure, the exploit just uses the most basic building blocks anyone who ever did anything in networking is familiar with.
That's really the crazy part. I'm not exactly the hacker type, I don't have low-level hardware knowledge, but I can fool around with APIs, craft http requests, log responses, etc. This is all so basic I can't even believe it, no special knowledge required.
@@thecakeredux I do bug bounty and this happens alllll the time you'd be surprised. Almost all the bugs i've found have been a similar story, random api endpoint that has no business being exposed to the public
To be fair, it still requires both an understanding of network protocols, and experience in exploiting systems, because you will absolutely not come up with "hey does this work" prompts as the researcher did, unless you've already done similar plenty of times. It's easy to forget that the majority of the planet's population would struggle to even explain what a network protocol is, much less do what was done above.
@@Alblaka As a teenager I could hack sites easily (not illegally!). Because you dont know stuff and you are curious to understand every details, you begin to tinker a lot. Kids come up with all sorts of unfiltered hypothesis.
kudos to Cox for eventually opening a conversation with Sam. the poor office worker must have been very confused. "this doomsday apocalypse guy claims to have found a way to hack all of us"
Biggest problem with pretty much any ISP in the world is that every possible interaction interface you have to them is through these kind of workers and there's direct way to talk to someone that has any actual knowledge about their tech and those guys won't put you through because a) company policy doesn't allow it and b) they couldn't care any less anyway because they're underpaid and overworked
As someone who has worked at an ISP I laughed so hard imagining what was going through that worker's head. 🤣 If I had a dollar for every time a crazy customer told me they were hacked I'd have been retired by now.
He has definitely earned the right to have that original modem back to tinker with. It's likely been collecting dust in some warehouse for years, I'm sure they could find it with a little effort.
@@Hitman12. Whoa whoa whoa, you mean to tell me we can put a little effort in and do something good, or do nothing at all? Well hot damn, nothing at all it is, "Sorry, sir, we cannot give you back the original modem"
That article reads a bit... like someone focused on the wrong thing: "I wonder how they hacked my modem"... [ 5 minutes later ] ... "Darn, this is the inside of a national bank vault, I do not want to be here" ... [ 5 minutes later ] ... "hacked the addresses of 5 FBI offices... also not quite what I was after" ... "nuclear launch codes" ... mmmh... "Ah here... this is it! Mr Router-company, please fix your software. I do not want my router to be hacked again, this is just such a nuisance!".
When I bought my own router, my ISP insisted on configuring it to be remotely accessed, I allowed them to configure it just because I was already pissed and disabled it right after, no no no, no ramdom person access here
The way many managed services work, they reach out to the control server periodically. While it may not be "remotely accessed", it can still be remotely managed. (and if it does check in, the backend systems mark it offline.) Considering 99.9999999999999999999999999999999999999999999999999% of customers are morons, this is the system we all have to live with.
Have them setup your network and tell them that you're modem will be plugged directly into a PC, and that you won't have a router. Then, once your internet is functional, setup your router, and tell your ISP to suck it! NEVER allow your ISP access beyond the modem. And NEVER rent hardware.
One more reason I like separate equipment...because the way DOCSIS works you can't actually override the ISP's control of the device that is part of the modem.
yeah if you thought there was some crafty zero day exploit in your router and you are a exploit researcher first thing you would do would be to pull a firmware dump off the device, probably even directly via the chip itself incase its some NSA level shit that has hijacked the normal device operation and if a firmware dump is initiated either feeds back unmodified data or quickly deletes itself to avoid the code being exposed, handing the device back to some minimum wage worker who will just tag it as damaged/faulty on the system and chuck it in a bin was a pretty silly move, you would either just pay the extra and say you lost it or dump everything you might find useful before giving it back, Cox said they checked and hadn't seen any evidence that anyone else had used this exploit, now that's not to say they hadn't but cleared up after themselves to hide it, its also possible that whatever hacking group had infected his router possibly just paid a Cox support engineer to access his account directly and push an infected firmware out to his device via official channels rather than via this exploit, that's the risk with any of these engineer protocols, it only takes 1 low level employee with the right level of access to completely compromise the entire system for a few hundred bucks per hit, seen similar stories with people working for hacking groups actually going out and getting jobs with places like Verizon so they can clone SIM cards etc to compromise giant accounts via 2fa exploits to push crypto scams etc
Thats why I'd never ever use any hardware provided by an ISP and obviously have all remote managing protocols disabled on the router I bought myself. Just the idea of anyone having access to my hardward would drive me crazy.
Among the many reasons I use strong authentication, encryption, and firewall rules even within my own LAN, I don't trust my ISP provided modem. Every network is potentially hostile.
@@par5ek Generally (in the USA anyways) you can buy compatible cable modems assuming they're on the ISP support list, but I'm uncertain on how much access they still retain as an ISP. By definition there has to be some sort of ISP side configuration or else you couldn't connect to their network.
Luckily my ISP provided router (Eero, yes the amazon one) was so Atrocious i just bought a new one back in 2021, then i started to learn about this stuff and am glad i did. Unfortunately still have ISP provided modem but im pretty sure its just a fibre to ethernet converter so i dont mind too much.
I worked tech support for a budget ISP a while back. We could see your wifi password. If you had email with us, we could even read your emails. No I didn't need anything to verify you gave us permission to access the account - just clicked a "yes I've identified the customer" button. Never use an ISP that doesn't let you BYOD. Always use your own device, turn off remote access protocols you don't own, don't use the ISPs own email service.
"they fixed the issue in a matter of a month or two, which is really really solid" Was listening to the vid on the side when I heard this and stopped what I was doing to do a double-take. This vid wasn't scripted so I might be a bit pedantic here but I did want to make one comment. It seem that it was fully *resolved* in a month or two, but the vulnerability itself was shut down less than 24h after it was reported. That's honestly really cool they had that fast of a turnaround given how large Cox is. I've worked at enterprise companies in the past that brush security issues under the rug, so this is refreshing to see.
I mean, that's why there's triage and priority management. There are many kind of bugs and vulnerabilities that show up that might need to sit on the back burner while compensatory controls are put in places. This is definitely not one of those issues lmao
I wouldn't say it's "fixed". But they definitely put a band-aid over the hole in the fence. (changed the settings on the front-end server to Just Say No(tm). Whatever backend didn't have the correct settings very likely hasn't been touched.) It's common for places to address the symptom without giving any thought to the actual disease.
This is the kind of issue that makes everyone and their mother shit a whole ass brick. You bet they fixed it in 24 hours. Either that, or the FBI comes in and does it for them lol
I hate Cox, having them as my ISP. I dont use their hardware because they charge rental fees for it. They also removed all my port forwards after I told them it stopped working and then told me "Your plan doesn't allow for port forwarding."
@@devnol That should only be able to touch your modem, though - did you buy a combined modem and router for some reason? I'm not sure I've ever seen those on store shelves, since they're pretty pointless for consumers.
@@PhysicsGamer well the isp here hands out a single device to customers (it's leased with the contract) and it's a modem/router/access point/switch/voip gateway combo.
@@soanvig I think it would reduce the number of potential attackers. Having API docs that can be found by web scraping makes you an obvious target to someone just poking around for misconfigured websites.
@@Zer0ji Be on your toes the whole time when it comes to authorization. That's all. Having open Swagger just reminds you about it. Don't give yourself any sense of "lowering chance of potential attack" by obfuscation.
@@devrim-oguz although I think that would be hilarious, since he was reporting the venerability to the ISP, and then publishing it after it was fixed, it probably would have been too easy for the ISP to see that he used it himself maliciously, or just a bad idea in general, which might have opened himself up to having broken some kind of terms of service or legal action against him, which is why he probably limited it only to reading information about them that he then redacted and didn’t spread since he though it wouldn’t work anyway (but did) and only made changes to his own equipment/network which he already had legitimate authority to make changes to, just not normally via these exploits.
Small ISPs are in a better position to fix things. First off, their "higher ups" are likely a bunch of retired IT guys who already know how to handle this kind of thing. Secondly, the corporate environment isn't overbearing on them because its not a very big corporation. They can put orders through with little delay, emergency work can be started immediately. There is no pass the buck culture of excuses for failure in a smaller ISP. I love my local ISP, the big guys have outages all the same and schedule a fix that can take hours maybe days to get techs dispatched. But the locals are here within minutes...once they lose access to their hardware they already know and just fire the trucks up immediately and get to work. Even if its 3AM.
I have a very small ISP (epb) and they’re light years ahead of my old isp (Comcast) in software, hardware, and security. It’s the large bloated megacorps that are very slow and sloppy with everything.
Certainly some. But I'd trust some small ISPs over the megacorps. I've worked for a small ISP and then for a company that made software for ISPs and a lot of them don't want tools like remote execution on the customer's modem specifically because of the risk that can open you up to. A lot of shadier ISP stuff like tracking everything you've ever done or injecting notices into your browser that you're almost at your bandwidth cap isn't a high priority at some 100 employee ISP. Overall I think they usually were pro net-neutrality as well. Though I'm certain there's also plenty of them that don't prioritize security sufficiently.
Everyone has these sorts of problems at one point or other. In this case, one of the backend servers doesn't have the same config as the rest, allowing unauthenticated requests. As for the nonsense that started this whole crap, pretty much every router ever made has some type of flaw. Since he gave it back, we'll never know what was going on with it.
I did a gig with a ISP for about a year. I've raised enough alarms about all the security holes in tr69 implementation that they had, and ultimately it got me a sac. Sad part is that this ISP still distributes their binaries with hardcoded admin passwords for TR69 implementation that can be reached from anywhere in the world ... yeah. Needless to say I run my own router and everything past the wall is isolated from ISP.
@@cutecats532 and they had circa 240 milion customers when I did a gig with them :/ unfortunately people are stupid enough to chose complete crap to save 50p a month
TR-069 is a loaded gun pre-pointed at every customers foot. All it needs is an ISP fucking up a little. But there is no way publicly traded ISPs will be cheap about software and security, right?
@@sas408 none of my routers have for the past 20+ years. seems kinda crazy that people who are into computer security would let ISP's boxes into their home network
Why are you downplaying such a big issue? It's not even funny, it's reet harded. Who cares about leeching on your wifi when you can create a botnet of millions of devices?
I hate it when ISP manages the network Gateway. The equipment they give always feels slow, cheap and unsecured to me. Plus the UI for a lot of the settings "if your lucky enough to have access to them" sucks, like 90% of the time.
Yeah, if there's not a bridge mode, I say "heck it" and run double-NAT. It adds about .2ms to everything but in return, I get control over my network again.
@@xor128 The ISP can detect that when it doesn't react to some of their manual test requests, and then throttle your connection to force you to reinstall their proprietary equipment.
I'm a programming beginner currently studying API development and testing and this just made my heart rate explode the more this story developed. Thank you for sharing!
5:48 that's why you always put another firewall in the front of your ISP crap device and never let it have direct access to the private network, so if it gets malware, their problem, as long as it doesn't make the internet slow
It could log the websites you visit, unless you're using a VPN or encrypted DNS. Your temp IP could end up on a blacklist. The internet activity from the remote adversary would at first be attributed to you. "We found a server at this IP address, traced to this residential address...". That's exactly how they have found people running CSAM servers (CSAM=Child Sexual Abuse Material)
@@collectorguy3919 "It could log the websites you visit, unless you're using a VPN or encrypted DNS." You *could* but is there any particular guarantee that your VPN provider hasn't been cracked? Seems to me it's just a question of 'is faceless corp A or faceless corp B more trustworthy'. Is the answer to that question even really knowable?
The hack was a poetic setup. Server from "Digital Ocean" + every request played back ... just like waves crushing on the sand at the beach and then slowly falling back into the sea *sighs*. Yep, true poetry.
I know it's very difficult to have a system where services like ISPs don't become monopolies, But I really think that one of the root causes of this is them being monopolies. They don't really have the incentive to make things secure, because customers can't leave anyway.
This is why Title II has the "right of non-discriminatory access ... [at] the same rates as competitors" clause. So that ISPs don't each need to run their own lines to set up their own networks.
@@Jason9637 Why are you lying? Starlink isn't competing with piss. I would assume atleast 30% of internet users in the US are gamers or have a gamer in their area. Good luck getting 17 ping on Starlink.
@10:10 Why is the ISP special (that they can remotely affect your modem)? Answer: The ISP must provision the modem. To provide Services the Internet Service Provider must configure the cable modem through a process known as provisioning. Provisioning is done remotely. The cable company, Cox, like every ISP depends on being able to remotely provision your modem and thus configure your service without you having to physically bring in the modem for changes to your service.
He states at the end of the article that it couldn't be this service because this service went live in 2023 and he was originally hacked 3 years prior. (In response to him still not knowing the original http stuff)
@@gorak9000 yeah you could be right. My ISP has had a function where they can reset my router for at least 4.5 years, now I'm gonna have to investigate how to firewall off the fiber router without them throwing a shit fit.
@@gorak9000 it's really an "it depends" type of question. Generally speaking APIs don't have graphical interface so I would assume it's newer or they left developer permissions enabled on prod. (Seen this alot personally). Unless the actual protocol had an issue this is just bad API setup/migration
During my time at Spectrum Cable, if I was the one working behind the counter, I would have just notated the account to say "Customer states modem stolen, new modem issued, replacement fee applied" and let them buy the modem to reverse engineer. That said, I was actively working threat intel undercover when I was working at Spectrum and honestly would have loved to have let someone find some interesting malware.
Spring Boot defaults beans to application scope. Most Spring Boot apps use a servlet thread model. I've seen a lot of people miss the implications of those two statements who should know better. They're clearing using custom security based on the encryptedValue parameter. I'd guess "authenticated" is a class variable and you can trigger a race condition and effectively steal the authentication of a prior request if your request comes in close enough. I've actually seen a very similar bug.
Yeah, this is a very good point. A huge subtlety that is easy to miss is all Spring beans are singletons by default; it's so easy to just throw annotations on classes without actually knowing what is going on in the background. If you're using fields as state variables inside singleton objects, you're going to see issues. Things like Tomcat use a thread pool to process incoming requests - all of which will share the exact same reference of your Spring beans.
TBF to the guys in the ISP's shop. They've probably heard someone absolute headcases say stuff about their modems being hacked before. Years ago i worked in a supermarket, and a customer lost their shit about some product display watching people. It had an infrared sensor that triggered a noise when people walked by....
The crazy part is that some vending machines theses days do actually monitor people with cameras and in the fine print on the digital displays for them almost all of the ones with cameras reserve right to send that data back to home base for whatever purposes they want
Seeing these tools again is bringing me back. I was actually working as a tier 1 support agent for the business sector at Cox. Like the author explains we had a lot of power to remotely control the devices. However, residential often was running the short end of the stick so usually they had to follow a predefined tree script. So business customers you can say have preferential treatment and usually higher skilled agents. Outside of what I know at this time. But interesting to see this in my feed.
TLDR: Man got his box infected. Showed his Cox to unimpressed store staff. Friends confirmed the source of infection was someone who put it about but remained anonymous.
"API dev forgot to turn or swagger" or you know, the api is designed for external consumption. You'd imagine business with alot of accounts would want something like that. Disabling swagger is security through obscurity, which isn't security. the fact that their API wasn't validating tokens on the other hand.......
If it was obscure, the API URIs might've not been located and abused, which is a big plus for security, but if authentication/authorization security worked how it should've then it shouldn't matter. Both are important points though
Maybe, but doubtful. It's not covered in much detail in the video, but a naive load of the swagger API page just resulted in a redirect loop and shouldn't have been actually usable for external users. He had to do some work to find a way to get it to load any actual resources. In the blog post this is the heading "Loading Static Resources from Reverse Proxy API".
I mean, maybe? I get the point but if you could have the complete information about every system then you can even predict the random numbers that are generated to provide cryptography features. I know that in this case there was a bigger vulnerability hidding but also I can see how the amount of information you provide to attackers closely match the level of security you have.
@mideno7619 not if they're using hardware entropy generators for key generation which they should be. Hiding api documentation that can be inferred by reading the Javascript is not security in any sense. Thinking it is security cause people to ignore or be overly confident about real systemic issues.
No, security through obscurity is when you're _relying_ on the obscuring to protect you. They had """auth""" mechanisms in place, so that's clearly not the case. Disabling swagger is good practice because it exhibits security _in depth_.
They could use the API to set up port forwarding and start snooping on your network. Security cameras that don't have good security or known vulnerabilities, unpatched workstations, cheap streaming devices with known back doors. The scary part is not them reading your network traffic, but the access they have to all your connected devices.
I don't know anything about security. I just like these videos cause they're like interesting story time. So it's crazy to me how much of this was just "I wonder if I can do this thing? ...oh, it worked. So then can I do this other thing too?" And so on.
I'm only a rookie, but yeah that's generally what the process is like. It's like any puzzle, you try different stuff until something works out. Of course, you build intuition along the way
@@keylanoslokj1806 basically malware (a virus or malicious software) Is installed into your internet modem box to watch your Internet traffic. They can inject ads. Or other dangerous applications into your network as well and once they control the modem it can be used to attack other machines inside your home or workplace.
There are a lot of stories regarding ISP managed devices, back in the day we use to get higher bandwidth than contracted just by hardcoding the external mac address of a router to a specific prefix.
Your content is really great for advanced technical users that aren't exactly security experts. The explanations you offer are easy to follow and make a lot of sense. Thank you for breaking down and going through these articles with us!
I found an issue a week ago with my ISP that let me get any customer’s information with just an account number. Then I found out how to get an account number from either an address or phone number. Thankfully it looks like it’s fixed already from what I could tell.
Intentionally, it’s a documented backdoor that only a few were supposed to know but they had to make it open to later claim that it’s a mistake and not intentional.
That's why been years not using any ISP modem. I've seen this exact thing happening (let 1 of the backend servers with auth unconfigured) before. 2 times to be exact. The first is what happens when you have draconian security rules for a service, and operators alone on weekends with boureoucracy problems to get the access. One day, some guy, uses one incident to open one of the 10 servers and take note of the IP, so he can ensure the support. The other time was due to some testing. A node was removed from the pool, and disabled auth to test some changes. It was put back to the pool without re-enable.
On DOCSIS, even if you use your own modem, the ISP updates the firmware on it, so you're still potentially vulnerable regardless of if you bought the modem, or if you rent it from the ISP - they're all running the same code anyway.
The entire video is reading an article and explaining sometimes either clear implied things everyone would have got, or things nobody without technical background would understand more than when just reading the article...
Well that's how you royally f*Ck up and lose customer trust. Btw wouldn't they have thought of the consequences of using such a loose security system when they built it in the first place?.
Definitely, anyone working on anything like that understands the necessity of security. A functional prototype would have been built and demonstrated; then management would have forced the Devs to move on to other projects before doing the invisible but essential tasks. Tale as old as time.
@@CentreMetre yeah, Cox is American. At least these days, Canada requires that the large ISPs must sell wholesale access to small carriers, so there can be some competition here. (Since 2016)
Working at an ISP decades ago with DSL modems, we started to see the DNS entries (which would be populated out to systems of the network as they received an IP address through DHCP) being poisoned; It was beautiful hack, because if they thought their URL redirection to impersonated web pages was noticed, they simply turned off the machine they had set for the first DNS server entry (the secondary DNS entry would be untouched, so requests would then be passed to it). We set ACLs after that, but would sometimes see the modem reprogrammed from a system inside the network. Most modem manufacturers thankfully start doing a multi-tiered security privilege level to prevent critical settings being changed.
The scary thing is that if this API backdoor was used to initially hack his modem, it means the exploit had been open and known to hackers for at least 3 years
With how many business accounts tied to it? If his router was hacked and someone was listening in, who and what else was being observed? Business emails, online shopping records, video calls? WFH logins with potential for getting further into business networks?
If I remember correctly from the video, one of the API calls was to get browser traffic from the modem (maybe the modem collects this for customer service troubleshooting). Is it possible that someone was making an api call to retrieve that for his original MAC address, and then making the same requests? The delay between the requests weren’t constant. Instead of actual malware installed on the modem, it could’ve been someone else exploiting the API.
this reminds me of a news story several years back, when someone discovered that basically the same http replay was happening on any/every? Telstra (Australia’s largest ISP) connection (discoverer had just run up a new server which no one knew about, but logs showed a repeat of his own http request a few seconds later from an IP in Canada IIRC). at the time it was reported to be Telstra doing some kind of customer traffic analysis outsourced to a foreign 3rd-party.
I absolutely love this article, thanks for breaking it down to something the rest of us can understand! Hilarious that Cox just had a server that had no authentication whatsoever. More videos like this please!
@@martinzihlmann822bro users can't even find the power button on their monitor and PCs sometimes and you want them to find an obscure button on a device they've probably never touched otherwise?
Some tips for the internet, guarantees in fact: * DNS is unencrypted (except Dot/DoH). If you let your ISP be your DNS provider, they can see what websites You're visiting. * HTTP is unencrypted (duh) * ISP owned modems are like asking for a man in the middle attack. Own as much as your own network infrastructure you can. * Traffic over predefined ports can easily be suspected to identifying it's purpose and potentially being modified, unless there's integrity measures for the protocol in use. * Even if you encrypt all traffic going out of your router and to the Internet, the source and destination IP address will always be visible in plain text effectively. This is required for IP to work. If you want to avoid even this from being spied on, use a local VPN or multiple gateway VPNs for even more anonymity. Effectively form a Tor network.
I've found tons of exposed API docs, and if that's not available and I have a valid login I can still observe my traffic and make out as much of the schema as I have access to. At the end of the day that's fine, as long as authentication and authorization is working correctly.
Nothing wrong with swagger doc in production. Not having your API properly protected with authentication is the issue, and not having swagger won't protect you against that.
Cox intentionally put this backdoor in their routers and then moved the management access to the routers from the intra to the clear. I called them and told them I refuse to use their insecure router update but they refused to roll it back so I left them. They then tried to get childish with me about getting back the modem which I own and to this day are trying to bill me for it as it sits on my shelf gathering dust. I may see them in court over it but its not on the top of my todo list. ;)
I’ve reverse engineered isp routers before and they have terrible security around tr69 in particular. Disclosed the issue to them and they said “it is not an issue they were concerned with” even though you could remotely compromise the router…
Now a serious question: ISPs worldwide have similar problems when it comes to security. How can we protect our traffic? One approach might be to avoid using the modem/router provided by the ISP, although I know some ISPs do not allow this. Another approach could be to use a second router connected to the ISP's modem/router, and from there, connect to a VPN deployed on a major cloud provider, assuming they offer a safer way to pass your traffic. Could you verify if this is technically sound and correct any grammatical errors in my question?
As someone that worked at a cable ISP in engineering I have to say this shouldn't be a surprise. Fistt off no isp worker will allow a random customer to reverse engineer their equipment. Even if someone knowledgeable was involved they would at the most contact internal engineering/security folks on the back end. Employeees are not authorized to give equipment up for reverse engineering. In addition whether it is your modem or the ISPs make sure you have a REAL firewall between the modem and your network. This could be a server running ips and firewall code or a dedicated appliance but it gives extra protection from ISP snooping although they still have things they can try and snoop with. Obviously IPv6 is trickier if you use it because all individuals will have their own public v6 address and even v4 theu can analyze ports to estimate how many systems you are running even with PAT in play. If you care about privacy don't trust ANYONE. Also use ur own dns over tls if possible. They love reading ur dns queries.
This could explain a lot of what I have seen on my network. A few years back it appeared that someone was using our modem as a gateway for their nefarious whatever and causing network saturation and slowness of traffic. Once the modem was replaced with our own it all stopped.
Why would you think that, given that they have an established responsible disclosure program? I hate ISPs as much as the next guy but that's just silly
The TR-069 protocol is why I absolutely refuse to use ISP supply equipment beyond it acting as a 'modem'. Everything beyond that is mine, under my control, and under no control of the ISP. It had been common knowledge for a while that this was a capability and that given that they can access one part of the device they can probably access all of the device and not just see what is attached but it can be a gateway to those devices.
@@keylanoslokj1806 If you had/have the following choices with your ISP, which would you choose? A. traffic snooping + vulnerability opening your network to intrusion B. traffic snooping only I would choose B every time.
@@SirMartinMuriithi Critic?. You mean Critique? He doesn't though. He just reads the webpage and provides very little analysis or (Critic). He is leveraging off someone else's work for views.
When you stumble onto a poorly implemented NSA backdoor lol this is exactly what I'd expect from Cox - the ISP that uses Yahoo for their webmail. Their software engineering talent pool has less depth than a Michael Bay film
jesus christ I dont trust any isp that didnt go through 20 staging servers and unit testing on **everything** before deploying. Especially anything API needs brutal and constant testing
write better code than Cox. learn to code at lowlevel.academy and get 20% off lifetime access with code THREADS20 PogChamp
Only if I can learn how to do this to mine. 7:03
Cox didn't write a single line of that *ROUTER* firmware. (it's not a g.. d... MODEM. hacking modems would be a serious issue.)
hacker grease mmmmmmmm delicious ....
Nice i3wm default window styling. It really confused me at first.
I'm interested in this but I click on 'Pricing' and I'm asked to set up a account, I want to see the prices before I hand over any details.
"If you think professionals are expensive, try hiring amateurs."
Most people don't take into account opportunity cost. Especially, amateurs.
I got hired by a startup when I was an amateur.
First week of browsing through their repos - 3 0-click RCE and remote shell exploits patched. I had to educate their "senior" on how much they f'd up.
I don't think hiring amateurs is that expensive (in some cases), it's much like with doctors. Experienced doctors will naturally go for the most obvious explanations, while med students will still remember their training enough to consider the "rare cases". That's why you would generally want an opinion of an experienced doctor and a newbie to extrapolate what may actually be wrong.
eh... I think its even less than that. Honestly, amateurs are often MORE likely to try and stick to standards because they know fewer tricks on how to get arround them. The problem is when your leadership demands you finish the project NOW!!! because they think yelling is an effective motivator... so the development team slaps something together meeting the bare minimum of requirements (which of course, didn't include security provisions) and call it a day. Those on the team with the most experience, usually have experience in exactly this... how to get code out the door that meets those requirements quickly, and which standards they know the company isn't going to bother checking for.
Why? This was messed up by a professional. Professionals are just people who do it for money with the least amount of work necessary and caring more about unions and pay raise than security.
@@cherubin7th We don't have unions here.
'How did you bypass auth'
'Simple, do the request again'
If at first you don't succeed, try, try again! LOL
This made me laugh tbh
FR lmao
Its like a child begging for candy till he gets it
We got porn ban in India, and it used to work before I left for my studies to the US 9 months back. Back again for summer break and I see that the blacklist only works intermittently, guessing they deployed more hardware to handle the traffic and just didnt enable/implement the firewall correctly on them.
So all I do is hit the website several times to get through the blacklist lol.
For me, the 0-100 moment was from “oh, the swagger docs are in production” to “oh, I can call an API without being authenticated”
It kinda sends shivers down my spine that ISP's can be on such a level of incompetence. RW access to millions of devices, even FBI customers by just running a HTTP request multiple times. Jesus christ i dont know if i can ever trust anything software ever again.
@@dustee2680 don’t trust anything. Zero Trust is the best strategy out there.
xD
@@dustee2680 welcome to usa corporations
@@dustee2680 There is a joke that exists that goes along the lines of "The tech enthusiast has everything in their house linked together. The tech worker has a twenty year old printer and a gun to shoot it if it makes a noise he doesn't recognise."
I work for a Large ISP, we can push settings, pull settings, get train rates, see connected devices, i can bridge ports to WAN so I could expose anything plugged directly into our gateway directly to the public network, if you changed the passwords on the device I can force reset them, or even factory reset the whole device. If you care about your privacy and safety use your own router
Some ISPs that offer gigabit fibre have the fiber termination point and WiFi router in the same package, I had no other option than to get the UniFi Dream Machine SE and disable the WiFi on the modem and plug a local network to the wan port on the udm se, then all traffic goes through the udm and the UniFi WiFi gateways, but this vulnerability and how they reacted to it has me concerned, the modem is huge and gives off a lot of heat and is quite capable, all that power, complexity and control is breeding ground for breaches and malware, I just hope I can set up pf sense to work well
@@AkashMishra23 try searching forums about that device of your isp. It just may happen you could replace it with sfp supported your very own router.
Yeah, it seems that nothing related to the internet cares about privacy at all.
But you have to admit, from a tech support perspective the reason to have that much control is because the vast majority of customers calling in can't figure out how to set up a wifi password and just want the ISP to do it for them so they can get on with their day.
@tyler108 Either, but if you really care then maybe DIY.
accidentally, it's a backdoor
Was gonna comment the same thing
You can accidentally find something purposefully placed. Like that time I accidentally noticed a glass pane with my head while running. It was purposefully placed there to be a sound dampener.
@@stucevevo8947bonk
ran here to say the same xD
imagine not telling anyone you found this and giving everyone free internet !
Bro went out and accidentally discovered another vulnerability after he got hacked via a different method 💀
shooting the moon and reached the stars
@@ChandravijayAgrawal I think a better space analogy is that he left at the periapsis while the OG hackers seemed to have left elsewhere on the orbit
@@djshenanigans7579 🤣🤣
@@djshenanigans7579 shut up dont be annoying
Implying the ISP wasn't just making stuff up. How would they know that vulnerability wasn't the one being used to for the original hack?
Cox once put corporate domains from an email whitelist (to allow them to send emails en masse) onto a blacklist, ensuring hundreds of firms cannot send emails to Cox users, then pretended those corporations are at fault. Those corporations are blocked to this day
Not trying to defend any corporation, but these are complex systems run by imperfect humans, so vulns are bound to happen, it's inevitable.
@BillAnt the problem is that those corporations that were now blocked all asked Cox to revert it, and Cox just either ignored them or blamed them for it.
Source: it included my former employer, who is actually a household name in my country in western Europe.
@@BillAnt mistakes are expected. Refusing to admit fault or fix those mistakes? That's an entirely different thing.
If you own the mail server sending the mail it would be trivial to prove who's at fault
Living up to their namesake, no?
This reminds me of another story I read about a German ISP years ago, must've been at least 5 years ago.
The ISP in question (I believe it was Vodafone Germany, Unity Media or Kabel Deutschland; definetly one of these three) wasn't providing customers with the credentials necessary to use any router on their network (at the time they didn't had any legal obligation to do so, so they forced customers to use the rental routers).
So one of their customers didn't want that rental unit and instead wanted to use their own router, so they started digging in the rental router from their ISP to get it to spit out the network credentials. While doing so they found out that bypassing the rental router and getting direct access to the ISPs network also gave them direct access to a similar maintanance API, completely without any authentication requirements (after all, you wouldn't normally have completely free access to this part of the network). So it was fairly trivial for them to change any router from that ISPs network, just by using his own router.
Not only did the ISP fix that vulnerability quickly, it also sparked a big legal debate on whether or not it was legal for that ISP to demand usage of rental routers. The outcome was no and now everyone can use any router, even on the cable networks (which is now all Vodafone Germany)
anything network related is zero trust, but i guess that's just my network side talking
It's staggering to me that was ever considered acceptable. Were the modems and routers are least separate?
@@PhysicsGamer Nope, they were one box. Outside of Fiber networks, it's pretty common here for the modem to be part of the router
@@Chickenbreadlp Can you not just buy a separate modem? So long as it's compatible (DOCSYS version, etc.) you should just be able to put your own router behind it...
@@PhysicsGamer to use a modem you still need the network credentials from the ISP, which they weren't giving out at the time. They weren't even offering the option to just get the modem. It was rental router or nothing. Iirc only fiber ISPs provide a dedicated modem, but even that's a controversial topic, because it's another device drawing its own power, when a modem+router combo is power efficient...
Just a heads up, towards the end of the video (27:19) you said you "think Cox fixed the issue within a month or two." Using the day/month/year calendar, that makes sense based on what this post has written, but I think this post had the date in a month/day/year format, suggesting the bug was hot fixed within a day of it being reported, not a month.
I can only imagine how many heart attacks this researcher nearly had from beginning to end of this story.
I have to deal with the date differences all the time at work, because we have people in the US, Germany, Romania, Greece, etc. I'm at the point that I write dates in a format like today is 11Jun2024 instead of 5-11-24 or 11-5-24 or 24-5-11
Antvenom, what are YOU doing here?
@@TheJunky228That definitely is the safest way.
My Perl script completely agrees with Jun being the 5 month of the calendar…)
@@TheJunky228 I am in finance in the U.S. but my corporate clients are global. I spell the month out also, otherwise it gets too confusing. I generally follow the day month year rule but even then there are times you have to question it so I also went to always spelling the month out.
The scary thing is that all the described stuff is really easy and very basic stuff. Its not something highly sophisticated or really obscure, the exploit just uses the most basic building blocks anyone who ever did anything in networking is familiar with.
That's really the crazy part. I'm not exactly the hacker type, I don't have low-level hardware knowledge, but I can fool around with APIs, craft http requests, log responses, etc. This is all so basic I can't even believe it, no special knowledge required.
I could've found this vulnerability when I was 12 lmao
@@thecakeredux I do bug bounty and this happens alllll the time you'd be surprised. Almost all the bugs i've found have been a similar story, random api endpoint that has no business being exposed to the public
To be fair, it still requires both an understanding of network protocols, and experience in exploiting systems, because you will absolutely not come up with "hey does this work" prompts as the researcher did, unless you've already done similar plenty of times.
It's easy to forget that the majority of the planet's population would struggle to even explain what a network protocol is, much less do what was done above.
@@Alblaka As a teenager I could hack sites easily (not illegally!). Because you dont know stuff and you are curious to understand every details, you begin to tinker a lot. Kids come up with all sorts of unfiltered hypothesis.
kudos to Cox for eventually opening a conversation with Sam.
the poor office worker must have been very confused. "this doomsday apocalypse guy claims to have found a way to hack all of us"
Lmfao
Biggest problem with pretty much any ISP in the world is that every possible interaction interface you have to them is through these kind of workers and there's direct way to talk to someone that has any actual knowledge about their tech and those guys won't put you through because a) company policy doesn't allow it and b) they couldn't care any less anyway because they're underpaid and overworked
As someone who has worked at an ISP I laughed so hard imagining what was going through that worker's head. 🤣
If I had a dollar for every time a crazy customer told me they were hacked I'd have been retired by now.
He has definitely earned the right to have that original modem back to tinker with. It's likely been collecting dust in some warehouse for years, I'm sure they could find it with a little effort.
@@Hitman12. Whoa whoa whoa, you mean to tell me we can put a little effort in and do something good, or do nothing at all? Well hot damn, nothing at all it is, "Sorry, sir, we cannot give you back the original modem"
That article reads a bit... like someone focused on the wrong thing:
"I wonder how they hacked my modem"...
[ 5 minutes later ] ...
"Darn, this is the inside of a national bank vault, I do not want to be here" ...
[ 5 minutes later ] ...
"hacked the addresses of 5 FBI offices... also not quite what I was after" ...
"nuclear launch codes" ...
mmmh...
"Ah here... this is it! Mr Router-company, please fix your software. I do not want my router to be hacked again, this is just such a nuisance!".
i feel like once you get far enough as a pentester morbid curiosity overtakes logical thought
Cybersecurity pros are a different breed.
Yeah I'd have trolled the fbi by changing their premises SSIDs first.
When I bought my own router, my ISP insisted on configuring it to be remotely accessed, I allowed them to configure it just because I was already pissed and disabled it right after, no no no, no ramdom person access here
The way many managed services work, they reach out to the control server periodically. While it may not be "remotely accessed", it can still be remotely managed. (and if it does check in, the backend systems mark it offline.)
Considering 99.9999999999999999999999999999999999999999999999999% of customers are morons, this is the system we all have to live with.
Good that you disabled access afterwards, I wonder what "just before" you was thinking, giving access to YOUR router to someone else
Have them setup your network and tell them that you're modem will be plugged directly into a PC, and that you won't have a router. Then, once your internet is functional, setup your router, and tell your ISP to suck it!
NEVER allow your ISP access beyond the modem.
And NEVER rent hardware.
One more reason I like separate equipment...because the way DOCSIS works you can't actually override the ISP's control of the device that is part of the modem.
@@TheNefariousFoxor use a mobile router plug it to a random tree nobody knows it's yours unless they spend $$ on an operation
bro could have easily just “lost” the router and paid for the missing equipment
yeah, I'd've backed out on getting the new one right then, come back next day w/o the old router.
@@pauls5745 holy it’s been some time since I last saw a double contraction. I’d’ve just wrote ‘I would have’.
or did everything he wanted to do before getting a new one.
yeah. a computer genius, but then gives away the router lol
yeah if you thought there was some crafty zero day exploit in your router and you are a exploit researcher first thing you would do would be to pull a firmware dump off the device, probably even directly via the chip itself incase its some NSA level shit that has hijacked the normal device operation and if a firmware dump is initiated either feeds back unmodified data or quickly deletes itself to avoid the code being exposed, handing the device back to some minimum wage worker who will just tag it as damaged/faulty on the system and chuck it in a bin was a pretty silly move, you would either just pay the extra and say you lost it or dump everything you might find useful before giving it back,
Cox said they checked and hadn't seen any evidence that anyone else had used this exploit, now that's not to say they hadn't but cleared up after themselves to hide it, its also possible that whatever hacking group had infected his router possibly just paid a Cox support engineer to access his account directly and push an infected firmware out to his device via official channels rather than via this exploit, that's the risk with any of these engineer protocols, it only takes 1 low level employee with the right level of access to completely compromise the entire system for a few hundred bucks per hit, seen similar stories with people working for hacking groups actually going out and getting jobs with places like Verizon so they can clone SIM cards etc to compromise giant accounts via 2fa exploits to push crypto scams etc
Thats why I'd never ever use any hardware provided by an ISP and obviously have all remote managing protocols disabled on the router I bought myself. Just the idea of anyone having access to my hardward would drive me crazy.
Among the many reasons I use strong authentication, encryption, and firewall rules even within my own LAN, I don't trust my ISP provided modem. Every network is potentially hostile.
wow, how did you buy the required modem? your isp allows the use of an unknown device connecting?
@@par5ek Generally (in the USA anyways) you can buy compatible cable modems assuming they're on the ISP support list, but I'm uncertain on how much access they still retain as an ISP. By definition there has to be some sort of ISP side configuration or else you couldn't connect to their network.
Luckily my ISP provided router (Eero, yes the amazon one) was so Atrocious i just bought a new one back in 2021, then i started to learn about this stuff and am glad i did. Unfortunately still have ISP provided modem but im pretty sure its just a fibre to ethernet converter so i dont mind too much.
@@iotkualt I live in rural Patagonia, using internet over radio. I'm screwed 🤷♂
I worked tech support for a budget ISP a while back. We could see your wifi password. If you had email with us, we could even read your emails. No I didn't need anything to verify you gave us permission to access the account - just clicked a "yes I've identified the customer" button.
Never use an ISP that doesn't let you BYOD. Always use your own device, turn off remote access protocols you don't own, don't use the ISPs own email service.
"they fixed the issue in a matter of a month or two, which is really really solid"
Was listening to the vid on the side when I heard this and stopped what I was doing to do a double-take.
This vid wasn't scripted so I might be a bit pedantic here but I did want to make one comment. It seem that it was fully *resolved* in a month or two, but the vulnerability itself was shut down less than 24h after it was reported.
That's honestly really cool they had that fast of a turnaround given how large Cox is. I've worked at enterprise companies in the past that brush security issues under the rug, so this is refreshing to see.
I mean, that's why there's triage and priority management.
There are many kind of bugs and vulnerabilities that show up that might need to sit on the back burner while compensatory controls are put in places.
This is definitely not one of those issues lmao
Siri, what is a P0 priority?……/s
I wouldn't say it's "fixed". But they definitely put a band-aid over the hole in the fence. (changed the settings on the front-end server to Just Say No(tm). Whatever backend didn't have the correct settings very likely hasn't been touched.) It's common for places to address the symptom without giving any thought to the actual disease.
This is the kind of issue that makes everyone and their mother shit a whole ass brick. You bet they fixed it in 24 hours. Either that, or the FBI comes in and does it for them lol
@@youhackforme The FBI would only do something when that guy sends them the list of all their agents that he got from Cox.
I hate Cox, having them as my ISP. I dont use their hardware because they charge rental fees for it. They also removed all my port forwards after I told them it stopped working and then told me "Your plan doesn't allow for port forwarding."
They sell u a natted connection??
How were they able to "remove ... [your] port forwards" if you're using your own hardware?
@@PhysicsGamer Remote management. They can change the settings you can (and even more) from their helpdesk. Most ISPs can through TR-069 and PPPoE
@@devnol That should only be able to touch your modem, though - did you buy a combined modem and router for some reason? I'm not sure I've ever seen those on store shelves, since they're pretty pointless for consumers.
@@PhysicsGamer well the isp here hands out a single device to customers (it's leased with the contract) and it's a modem/router/access point/switch/voip gateway combo.
As a fellow software developer who uses spring for backend development i can assure you that i forgot more than once to dissable swagger for prod env)
I did the same too, luckily we have some verification that stops the code from running if endpoints are exposed.
What's the difference? If the vulnerability is there available or not available swagger doesn't change anything. Obfuscation is not security.
@@soanvig I think it would reduce the number of potential attackers. Having API docs that can be found by web scraping makes you an obvious target to someone just poking around for misconfigured websites.
@@Zer0ji most potential attackers scan networks for points of access. Although I'm sure some of them would read through a swagger doc.
@@Zer0ji Be on your toes the whole time when it comes to authorization. That's all. Having open Swagger just reminds you about it.
Don't give yourself any sense of "lowering chance of potential attack" by obfuscation.
Would’ve been fun to rename all the FBI wireless networks to “You’ve been hacked” and watch the chaos that ensued 😂
@@devrim-oguz although I think that would be hilarious, since he was reporting the venerability to the ISP, and then publishing it after it was fixed, it probably would have been too easy for the ISP to see that he used it himself maliciously, or just a bad idea in general, which might have opened himself up to having broken some kind of terms of service or legal action against him, which is why he probably limited it only to reading information about them that he then redacted and didn’t spread since he though it wouldn’t work anyway (but did) and only made changes to his own equipment/network which he already had legitimate authority to make changes to, just not normally via these exploits.
Just telling the fbi that is playing with fire, maybe be a little more diplomatic about the isp leaving huge vulnerabilities open.
Wow, makes you wonder how many smaller ISPs have similar vulnerabilities.
Small ISPs are in a better position to fix things. First off, their "higher ups" are likely a bunch of retired IT guys who already know how to handle this kind of thing. Secondly, the corporate environment isn't overbearing on them because its not a very big corporation. They can put orders through with little delay, emergency work can be started immediately. There is no pass the buck culture of excuses for failure in a smaller ISP.
I love my local ISP, the big guys have outages all the same and schedule a fix that can take hours maybe days to get techs dispatched. But the locals are here within minutes...once they lose access to their hardware they already know and just fire the trucks up immediately and get to work. Even if its 3AM.
I have a very small ISP (epb) and they’re light years ahead of my old isp (Comcast) in software, hardware, and security. It’s the large bloated megacorps that are very slow and sloppy with everything.
Certainly some. But I'd trust some small ISPs over the megacorps. I've worked for a small ISP and then for a company that made software for ISPs and a lot of them don't want tools like remote execution on the customer's modem specifically because of the risk that can open you up to. A lot of shadier ISP stuff like tracking everything you've ever done or injecting notices into your browser that you're almost at your bandwidth cap isn't a high priority at some 100 employee ISP. Overall I think they usually were pro net-neutrality as well. Though I'm certain there's also plenty of them that don't prioritize security sufficiently.
Everyone has these sorts of problems at one point or other. In this case, one of the backend servers doesn't have the same config as the rest, allowing unauthenticated requests. As for the nonsense that started this whole crap, pretty much every router ever made has some type of flaw. Since he gave it back, we'll never know what was going on with it.
Don't see why size of the ISP would matter. Are you assuming they use cheaper/less secure stuff because they are small?
"He's getting his Cox hacked"
Honey, they neuralinked our poultry
You need to have a Cox counter
1
2
4
7
9
I did a gig with a ISP for about a year. I've raised enough alarms about all the security holes in tr69 implementation that they had, and ultimately it got me a sac. Sad part is that this ISP still distributes their binaries with hardcoded admin passwords for TR69 implementation that can be reached from anywhere in the world ... yeah.
Needless to say I run my own router and everything past the wall is isolated from ISP.
Sounds like an ISP no one should use....
@@cutecats532 and they had circa 240 milion customers when I did a gig with them :/ unfortunately people are stupid enough to chose complete crap to save 50p a month
The back door was left unlocked, opened, with a sign that read "come on in!"
When R&D makes everything "just work" and suddenly your house has no doors or windows. Breaking is the remaining 1/10th the law.
TR-069 is a loaded gun pre-pointed at every customers foot. All it needs is an ISP fucking up a little. But there is no way publicly traded ISPs will be cheap about software and security, right?
More terrifying is that the hacked modem was very likely re-issued to another Cox customer.
TR-069? Nice backdoor for Cox
Thats used by most ISPs. And almost every router supports that and turns on automatically
@@sas408it's just a sex-related joke dog
🤣
@@sas408 none of my routers have for the past 20+ years. seems kinda crazy that people who are into computer security would let ISP's boxes into their home network
@@sas408 you missed the joke
So when a Cox worker needs free wifi, they can just disable the passwords of their customers and leech on it.
In Poland P4/UPC have "free wifi" via leaching on customer connection and set up 2 wi-fi
Why are you downplaying such a big issue? It's not even funny, it's reet harded. Who cares about leeching on your wifi when you can create a botnet of millions of devices?
Our@@JonahTheWhite
I hate it when ISP manages the network Gateway. The equipment they give always feels slow, cheap and unsecured to me. Plus the UI for a lot of the settings "if your lucky enough to have access to them" sucks, like 90% of the time.
usually no VLAN or other features either.
Yeah, if there's not a bridge mode, I say "heck it" and run double-NAT. It adds about .2ms to everything but in return, I get control over my network again.
i just use my own modem and dumping the credentials from the isp modem
@@xor128 The ISP can detect that when it doesn't react to some of their manual test requests, and then throttle your connection to force you to reinstall their proprietary equipment.
@@Charaqatso they can take their proprietary equipment back and I'm getting another ISP easy
I'm a programming beginner currently studying API development and testing and this just made my heart rate explode the more this story developed. Thank you for sharing!
5:48 that's why you always put another firewall in the front of your ISP crap device and never let it have direct access to the private network, so if it gets malware, their problem, as long as it doesn't make the internet slow
but this was on the router itself (modem mode with your own router installed wouldn't stop it)
@JimAllen-Persona uhhhhhhhh how? 500 a month is wild. Why not just use PfSense which is more secure, open source, and free?
@@leexgx A router that you control certainly won't allow TR-069 remote API access from the ISP.
It could log the websites you visit, unless you're using a VPN or encrypted DNS.
Your temp IP could end up on a blacklist.
The internet activity from the remote adversary would at first be attributed to you. "We found a server at this IP address, traced to this residential address...". That's exactly how they have found people running CSAM servers (CSAM=Child Sexual Abuse Material)
@@collectorguy3919 "It could log the websites you visit, unless you're using a VPN or encrypted DNS."
You *could* but is there any particular guarantee that your VPN provider hasn't been cracked? Seems to me it's just a question of 'is faceless corp A or faceless corp B more trustworthy'. Is the answer to that question even really knowable?
The hack was a poetic setup. Server from "Digital Ocean" + every request played back ... just like waves crushing on the sand at the beach and then slowly falling back into the sea *sighs*. Yep, true poetry.
I know it's very difficult to have a system where services like ISPs don't become monopolies, But I really think that one of the root causes of this is them being monopolies. They don't really have the incentive to make things secure, because customers can't leave anyway.
This is the main reason I love starlink, it's giving a decent competitor to almost every ISP on the planet
This is why Title II has the "right of non-discriminatory access ... [at] the same rates as competitors" clause. So that ISPs don't each need to run their own lines to set up their own networks.
@@Jason9637 Why are you lying? Starlink isn't competing with piss. I would assume atleast 30% of internet users in the US are gamers or have a gamer in their area. Good luck getting 17 ping on Starlink.
@@octav7438 There's tons of rural areas with terrible ISPs, and 25-60ms ping is definitely not bad, I game on 30ms and have no issues
@10:10 Why is the ISP special (that they can remotely affect your modem)? Answer: The ISP must provision the modem. To provide Services the Internet Service Provider must configure the cable modem through a process known as provisioning. Provisioning is done remotely. The cable company, Cox, like every ISP depends on being able to remotely provision your modem and thus configure your service without you having to physically bring in the modem for changes to your service.
He states at the end of the article that it couldn't be this service because this service went live in 2023 and he was originally hacked 3 years prior. (In response to him still not knowing the original http stuff)
Is it really new from the ground up, or did they just add to the API and have the graphics monkeys put some new look and feel to it?
@@gorak9000 yeah you could be right. My ISP has had a function where they can reset my router for at least 4.5 years, now I'm gonna have to investigate how to firewall off the fiber router without them throwing a shit fit.
@@gorak9000 it's really an "it depends" type of question. Generally speaking APIs don't have graphical interface so I would assume it's newer or they left developer permissions enabled on prod. (Seen this alot personally). Unless the actual protocol had an issue this is just bad API setup/migration
During my time at Spectrum Cable, if I was the one working behind the counter, I would have just notated the account to say "Customer states modem stolen, new modem issued, replacement fee applied" and let them buy the modem to reverse engineer. That said, I was actively working threat intel undercover when I was working at Spectrum and honestly would have loved to have let someone find some interesting malware.
Spring Boot defaults beans to application scope. Most Spring Boot apps use a servlet thread model. I've seen a lot of people miss the implications of those two statements who should know better. They're clearing using custom security based on the encryptedValue parameter. I'd guess "authenticated" is a class variable and you can trigger a race condition and effectively steal the authentication of a prior request if your request comes in close enough. I've actually seen a very similar bug.
Yeah, this is a very good point. A huge subtlety that is easy to miss is all Spring beans are singletons by default; it's so easy to just throw annotations on classes without actually knowing what is going on in the background. If you're using fields as state variables inside singleton objects, you're going to see issues. Things like Tomcat use a thread pool to process incoming requests - all of which will share the exact same reference of your Spring beans.
I think this is a pretty good guess, I've read simliar article
This is a common problem with SSR as sometimes devs store authorization states in global variables
I think the API architect will have to find another job... 💀
TBF to the guys in the ISP's shop.
They've probably heard someone absolute headcases say stuff about their modems being hacked before.
Years ago i worked in a supermarket, and a customer lost their shit about some product display watching people.
It had an infrared sensor that triggered a noise when people walked by....
The crazy part is that some vending machines theses days do actually monitor people with cameras and in the fine print on the digital displays for them almost all of the ones with cameras reserve right to send that data back to home base for whatever purposes they want
Dont expect much from minimum wage workers. They're minimum wage for a reason.
@@Snail641what
5:26 - I might have watched too many Polish Train Hackers, but it baffles me he didn’t dump the firmware before turning it in.
"Don't worry, hacker can't harm you"
The hacker in question:
Edit: everyone in the replies is a nerd
Pretty sure people say the reverse
@@prashank rly? or are you matrix
@@susstevedev what in the yap are you talking about
@@susstevedev what? Who says the hacker can't harm you? The entire point is for hackers to harm in some way
@@KerestellSmith-t7d seems like language barrier
Seeing these tools again is bringing me back. I was actually working as a tier 1 support agent for the business sector at Cox. Like the author explains we had a lot of power to remotely control the devices. However, residential often was running the short end of the stick so usually they had to follow a predefined tree script. So business customers you can say have preferential treatment and usually higher skilled agents. Outside of what I know at this time. But interesting to see this in my feed.
The picture of the COX store actually scared me for a second. The researcher must live where I live!
Orange County? Lol
@@eso210 Nah this is in the Omaha area
@@blzby6592 that narrows it down for hacermans lol
TLDR: Man got his box infected. Showed his Cox to unimpressed store staff. Friends confirmed the source of infection was someone who put it about but remained anonymous.
he showed his what
@@qoombert Showed his big eight-inch cox
"API dev forgot to turn or swagger"
or you know, the api is designed for external consumption. You'd imagine business with alot of accounts would want something like that.
Disabling swagger is security through obscurity, which isn't security.
the fact that their API wasn't validating tokens on the other hand.......
If it was obscure, the API URIs might've not been located and abused, which is a big plus for security, but if authentication/authorization security worked how it should've then it shouldn't matter. Both are important points though
Maybe, but doubtful. It's not covered in much detail in the video, but a naive load of the swagger API page just resulted in a redirect loop and shouldn't have been actually usable for external users. He had to do some work to find a way to get it to load any actual resources. In the blog post this is the heading "Loading Static Resources from Reverse Proxy API".
I mean, maybe? I get the point but if you could have the complete information about every system then you can even predict the random numbers that are generated to provide cryptography features. I know that in this case there was a bigger vulnerability hidding but also I can see how the amount of information you provide to attackers closely match the level of security you have.
@mideno7619 not if they're using hardware entropy generators for key generation which they should be.
Hiding api documentation that can be inferred by reading the Javascript is not security in any sense. Thinking it is security cause people to ignore or be overly confident about real systemic issues.
No, security through obscurity is when you're _relying_ on the obscuring to protect you.
They had """auth""" mechanisms in place, so that's clearly not the case.
Disabling swagger is good practice because it exhibits security _in depth_.
They could use the API to set up port forwarding and start snooping on your network. Security cameras that don't have good security or known vulnerabilities, unpatched workstations, cheap streaming devices with known back doors. The scary part is not them reading your network traffic, but the access they have to all your connected devices.
That's scary, now think they have access to your NAS even configured in local only, they drop CSAM and inform leo's
I don't know anything about security. I just like these videos cause they're like interesting story time. So it's crazy to me how much of this was just "I wonder if I can do this thing? ...oh, it worked. So then can I do this other thing too?" And so on.
I'm only a rookie, but yeah that's generally what the process is like. It's like any puzzle, you try different stuff until something works out.
Of course, you build intuition along the way
(for every "I tried X and it worked" in the blog post they're probably omitting five "I tried X and it didn't work"s)
@@cjbprime exactly
6:41 "I think if I say cox again I will get demonitised... Cox!" 😂
Lmao why aren't we pointing this out lol
casually embeds malware into your router via the api by renaming the modem a really weird name lmao
i wonder if it replays the http request to like fetch the page for a preview for the C2
@@tonimortoncan you explain what you said?
@@keylanoslokj1806 basically malware (a virus or malicious software)
Is installed into your internet modem box to watch your Internet traffic. They can inject ads. Or other dangerous applications into your network as well and once they control the modem it can be used to attack other machines inside your home or workplace.
There are a lot of stories regarding ISP managed devices, back in the day we use to get higher bandwidth than contracted just by hardcoding the external mac address of a router to a specific prefix.
IMHO, the author uses "signing" because of HMAC signature which is computed by using a secret key
The key isn't very secret if it's in the JavaScript code lol
"If I say Cox again, I would get demonetized."
Hacker: Of course, that's the intention, never random.
Now we have footage for a "cox compilation"
Your content is really great for advanced technical users that aren't exactly security experts. The explanations you offer are easy to follow and make a lot of sense. Thank you for breaking down and going through these articles with us!
I found an issue a week ago with my ISP that let me get any customer’s information with just an account number. Then I found out how to get an account number from either an address or phone number.
Thankfully it looks like it’s fixed already from what I could tell.
Intentionally, it’s a documented backdoor that only a few were supposed to know but they had to make it open to later claim that it’s a mistake and not intentional.
That's why been years not using any ISP modem.
I've seen this exact thing happening (let 1 of the backend servers with auth unconfigured) before. 2 times to be exact.
The first is what happens when you have draconian security rules for a service, and operators alone on weekends with boureoucracy problems to get the access.
One day, some guy, uses one incident to open one of the 10 servers and take note of the IP, so he can ensure the support.
The other time was due to some testing. A node was removed from the pool, and disabled auth to test some changes. It was put back to the pool without re-enable.
On DOCSIS, even if you use your own modem, the ISP updates the firmware on it, so you're still potentially vulnerable regardless of if you bought the modem, or if you rent it from the ISP - they're all running the same code anyway.
@@gorak9000 DOCSIS? that's CableTV?
WTF, that still exists?
@@gorak9000yep, even a neighbor could force a modified firmware to your modem and you'd never know.
The entire video is reading an article and explaining sometimes either clear implied things everyone would have got, or things nobody without technical background would understand more than when just reading the article...
Well that's how you royally f*Ck up and lose customer trust. Btw wouldn't they have thought of the consequences of using such a loose security system when they built it in the first place?.
Definitely, anyone working on anything like that understands the necessity of security.
A functional prototype would have been built and demonstrated; then management would have forced the Devs to move on to other projects before doing the invisible but essential tasks. Tale as old as time.
Hahaha its American ISPs though, they're likely the only ISP in the region
This looks like US or maybe canada from photos so: dont have to care when theres no competition.
@@CentreMetre yeah, Cox is American. At least these days, Canada requires that the large ISPs must sell wholesale access to small carriers, so there can be some competition here. (Since 2016)
@@forivall Ah ok, thats good. Tbh i dont really know much, just what ive heard from the WAN Show, thanks for telling me
Scary that it is still unclear why his modem is configured to setup auto replay network calls…
I seriously hope this guy got paid some amount of money for this.
More great stuff. Thank you!
Working at an ISP decades ago with DSL modems, we started to see the DNS entries (which would be populated out to systems of the network as they received an IP address through DHCP) being poisoned; It was beautiful hack, because if they thought their URL redirection to impersonated web pages was noticed, they simply turned off the machine they had set for the first DNS server entry (the secondary DNS entry would be untouched, so requests would then be passed to it). We set ACLs after that, but would sometimes see the modem reprogrammed from a system inside the network. Most modem manufacturers thankfully start doing a multi-tiered security privilege level to prevent critical settings being changed.
This is just about the most horrifying shit I’ve ever read. Thank you for a paragraph horror story that’ll keep me looking over my shoulder for a week
What are ACLs?
@@keylanoslokj1806 - Access Control List; A set of allowed IP addresses that is permitted to work with that equipment.
the overwhelming power of just asking again but saying "pretty please" at the end
The scary thing is that if this API backdoor was used to initially hack his modem, it means the exploit had been open and known to hackers for at least 3 years
With how many business accounts tied to it? If his router was hacked and someone was listening in, who and what else was being observed? Business emails, online shopping records, video calls? WFH logins with potential for getting further into business networks?
@@EmptyZoo393 It wasnt just business accounts residential too i think
If I remember correctly from the video, one of the API calls was to get browser traffic from the modem (maybe the modem collects this for customer service troubleshooting). Is it possible that someone was making an api call to retrieve that for his original MAC address, and then making the same requests? The delay between the requests weren’t constant. Instead of actual malware installed on the modem, it could’ve been someone else exploiting the API.
this reminds me of a news story several years back, when someone discovered that basically the same http replay was happening on any/every? Telstra (Australia’s largest ISP) connection (discoverer had just run up a new server which no one knew about, but logs showed a repeat of his own http request a few seconds later from an IP in Canada IIRC). at the time it was reported to be Telstra doing some kind of customer traffic analysis outsourced to a foreign 3rd-party.
I absolutely love this article, thanks for breaking it down to something the rest of us can understand! Hilarious that Cox just had a server that had no authentication whatsoever. More videos like this please!
Remotely accessible modem by ISP is a dumb idea
and... it's one that they all do.
My ISP remotely removed all ny port forwards last week. it was lovely.
Their average customer doesn't know how internet works and calls for tech support. It's not that crazy.
there should be a physical button that you need to press to give the support staff access.
@@kerr1221 Can you ask your ISP to do this? I'm so paranoid now lol
@@martinzihlmann822bro users can't even find the power button on their monitor and PCs sometimes and you want them to find an obscure button on a device they've probably never touched otherwise?
Some tips for the internet, guarantees in fact:
* DNS is unencrypted (except Dot/DoH). If you let your ISP be your DNS provider, they can see what websites You're visiting.
* HTTP is unencrypted (duh)
* ISP owned modems are like asking for a man in the middle attack. Own as much as your own network infrastructure you can.
* Traffic over predefined ports can easily be suspected to identifying it's purpose and potentially being modified, unless there's integrity measures for the protocol in use.
* Even if you encrypt all traffic going out of your router and to the Internet, the source and destination IP address will always be visible in plain text effectively. This is required for IP to work. If you want to avoid even this from being spied on, use a local VPN or multiple gateway VPNs for even more anonymity. Effectively form a Tor network.
Can you explain the pre-last point? Also about the last one, how do you form a tor network?
Exposing swagger on prod is a new level. It shows that any of us can do any job, the people with 5+ years of xp are doing all this.
I've found tons of exposed API docs, and if that's not available and I have a valid login I can still observe my traffic and make out as much of the schema as I have access to. At the end of the day that's fine, as long as authentication and authorization is working correctly.
So you think the swagger docs on the test system is different from the one in production?
Nothing wrong with swagger doc in production. Not having your API properly protected with authentication is the issue, and not having swagger won't protect you against that.
@@awmy3109 Still it should be masked via login. It's increases the attack surface by a mile.
I am so glad you went over the whole article and explained it. I honestly don't think I would understand anything if I were to read it.
Cox intentionally put this backdoor in their routers and then moved the management access to the routers from the intra to the clear.
I called them and told them I refuse to use their insecure router update but they refused to roll it back so I left them.
They then tried to get childish with me about getting back the modem which I own and to this day are trying to bill me for it as it sits on my shelf gathering dust.
I may see them in court over it but its not on the top of my todo list. ;)
Exactly why I absolutely refuse to buy ISP WiFI. Give me a data pipe and get the heck out of my face otherwise.
True
I’ve reverse engineered isp routers before and they have terrible security around tr69 in particular. Disclosed the issue to them and they said “it is not an issue they were concerned with” even though you could remotely compromise the router…
Then make it an issue they WILL be concerned with...
6:36 "without a doubt there was a piece of malware living in his cox"
Sounds painful
HIS COX HACKED! 7:03
Now a serious question: ISPs worldwide have similar problems when it comes to security. How can we protect our traffic? One approach might be to avoid using the modem/router provided by the ISP, although I know some ISPs do not allow this. Another approach could be to use a second router connected to the ISP's modem/router, and from there, connect to a VPN deployed on a major cloud provider, assuming they offer a safer way to pass your traffic. Could you verify if this is technically sound and correct any grammatical errors in my question?
0:18 Hi. Is Low Level Learning on your birth certificate?
As someone that worked at a cable ISP in engineering I have to say this shouldn't be a surprise. Fistt off no isp worker will allow a random customer to reverse engineer their equipment. Even if someone knowledgeable was involved they would at the most contact internal engineering/security folks on the back end. Employeees are not authorized to give equipment up for reverse engineering. In addition whether it is your modem or the ISPs make sure you have a REAL firewall between the modem and your network. This could be a server running ips and firewall code or a dedicated appliance but it gives extra protection from ISP snooping although they still have things they can try and snoop with. Obviously IPv6 is trickier if you use it because all individuals will have their own public v6 address and even v4 theu can analyze ports to estimate how many systems you are running even with PAT in play. If you care about privacy don't trust ANYONE. Also use ur own dns over tls if possible. They love reading ur dns queries.
Ive immediately thought of the tr-069 after watching for a minute or so.😅
"getting his cox hacked" 💀
The original compromised device triggering the reply is to potentially steal the user's session on some services?
Might be, especially since they were already running some phishing scheme.
This could explain a lot of what I have seen on my network. A few years back it appeared that someone was using our modem as a gateway for their nefarious whatever and causing network saturation and slowness of traffic. Once the modem was replaced with our own it all stopped.
Interesting response from Cox, I thought the standard response to this sort of discovery was to sue them for unauthorised access.
Their lawyers are drafting the papers now. Just wait.
Why would you think that, given that they have an established responsible disclosure program?
I hate ISPs as much as the next guy but that's just silly
This actually makes me wonder if other ISPs forgot to lock down their APIs as well, I am definitely expecting Comcast to have degraded infrastructure.
What baffles me the most is how did that guy have that much of free time 😅
shouldn't take more than a Sunday to get access if you are not a total noob
Doesn't take long at all
I hope that researcher got a reward from Cox. He certainly earned it.
Hate it when my Cox gets hacked.
This is why I cracked my ISPs admin password and deleted all the T-069 config.
What were the results of that?
What's with the npc spam bot comments about views. It's so dumb
A lot of those are actually this weird online pseudo-cult called UTTP. They're actually really concerning.
@@OhhCrapGuy ?
@@tcscomment I don't recall if this was the video I watched, but: ua-cam.com/video/ABIXt5R4vV8/v-deo.html
The TR-069 protocol is why I absolutely refuse to use ISP supply equipment beyond it acting as a 'modem'. Everything beyond that is mine, under my control, and under no control of the ISP. It had been common knowledge for a while that this was a capability and that given that they can access one part of the device they can probably access all of the device and not just see what is attached but it can be a gateway to those devices.
They can still see the traffic in text file though right?
@@keylanoslokj1806 If you had/have the following choices with your ISP, which would you choose?
A. traffic snooping + vulnerability opening your network to intrusion
B. traffic snooping only
I would choose B every time.
Bro reads webpage for 29 mins.
Here's an idea, make your own critic!
@@SirMartinMuriithi Critic?. You mean Critique? He doesn't though. He just reads the webpage and provides very little analysis or (Critic). He is leveraging off someone else's work for views.
When you stumble onto a poorly implemented NSA backdoor lol this is exactly what I'd expect from Cox - the ISP that uses Yahoo for their webmail. Their software engineering talent pool has less depth than a Michael Bay film
jesus christ
I dont trust any isp that didnt go through 20 staging servers and unit testing on **everything** before deploying. Especially anything API needs brutal and constant testing
6:38 "If I say Cox again, I think I'm going to get demonitized... Cox" 🤣
Instead of viewing 29 minutes, read the story in 10.