@@nomadshiba right? I see nothing remotely marxist in any of this. Lewis as a member of the market, would not bear that cost - so chose another method. Sounds quite capitalistic to me.
I find that I may need to reuse a part of a script for another project. In that light, I may also save time when solving other or similar problems in the future
This is 100% how I pick my next software rabbit-hole, I mean “project” spend 6 months working on something that I could’ve had done in a few days without code. It’s not a glamorous life, but it’s one worth doing with code
I am a noob Python user... I spent a week making a terrible Flask site so that instead of running a script off my pc I can check things off my cell when on the go. Best week of migraines ever. 😂
Fr, I made a Python script that does all the work in 7 minutes, but it took 3-4 hours to make that fking script. In the end, I was happy and satisfied that i can do stuff and am not useless 😂
yea… but he made a video and script publick. now he might saved someone hours of time. Doesn’t matter how stupid you thinkr task is, you never know when and for who it will be useful in what scenario
yea... but he can re-run his script in seconds as many times as he likes, whenever he likes. Which is useful if he decides to make a change. If doing it manually you have to spend 17 minutes again each time.
I feel that a project like this just for the fun of problem solving and writing the code for it. Yeah, you could just spend the 17 bucks to get printed copies from the bank which you'll probably end up spending half a weekend scanning it all back to digital. OR you could spend a weekend making your finances actually fun!
this. this is how i got into programming. this is the reason i recommend basic programming as a hobby to anyone who uses a computer. i love these types of videos because they're the exact type of crazy pointless things i do.
Decided to try something new :) let me know what you think of this more "shorter" style content! We have a hackathon coming up shortly :) join the convo on discord!
Lewis, I think it wouldn’t hurt to check with a lawyer or ask someone from the bank if you could do this. I am 90% sure it’s ok but when it comes to banks you could potentially be breaking your contract or something hidden in the terms of service. Just a heads up from a lawyer & programming hobbyist. Love your videos!
Ah yes - I totally relate to the "I refuse to do this manual, tedious process that will take me 20 minutes so I'm going to spend a week writing software to do it". Many times.
i made a project once, called it multiterm, you can get for python as a module thru pip, it gives you a screen based system to create terminal user interfaces
I closed my account with this same bank earlier this year. I'm not a programmer and I somehow screen scraped all my records for the past 7 years. You're right, they purposely made the UI barely workable to get you to pay them for what you need.
I myself have multiple times turned a 20-minute problem into 2 days of python project. But the benefit I got, the next time I get that 20-minute problem, it would be done in seconds!
I did this manually recently. Downloaded all pdfs, copied everything to excel and did some calculations. I found out that the "No cost emi" was not really "No cost". It was less than the normal emi plan but it was way more than what expected.
Everything you just said in the video is meaningless compared to the fact you can read your bank request in plain text through a proxy. This should never happen. Call your bank and tell them they have a major security issue and they need to stay up all night to fix it. Like today.
1. I think you really lucked out with the proxy/man in the middle thing working. Normally mobile apps these days use HTTPS with certificate pinning, which means that you cannot intercept any traffic between your device and the server without modifying the app somehow. 2. I don't think the API getting the 4 years every single time is a dark pattern, I full heartedly believe it is incompetence.
Honestly, they probably just didn't have any support for pagination. That and 20 years of bank statements probably loads only marginally slower than four years. It was the path of least resistance and meant he got to go home at 5. I think that's the real reason Edit: the lack of encryption is inexcusable though
@@twobitsnickit *_is_* encrypted, he exported the cookies and certs and just made the exact same request through his computer rather than solely his phone. There is literally no vulnerability here that could be meaningfully prevented. People really need to stop wasting time, money, and fucks on endpoint security. Once you've sent the data to yhe user, that's it, they *_can_* get it. He had full access to the device so there is nothing any company could do to stop this aside from adding annoyances, and that's just security through obscurity. The issue is, if there *_is_* a real vulnerability here then adding more shit to get in the way won't protect anyone. Again, the user has the device, they *_can_* get the data. Even if they need to transcribe it by hand on a type writer, they *_can_* get it. So if I'm a malicious actor, "boo hoo, I get a bit inconvenienced when writing my code to steal millions of peoples' life savings, I should just give up now! I mean the 2 weeks of work spent dealing with this just wouldn't be worth the millions of dollars I'd get from doing it!" Meanwhile you, as the business, are fumping millions into this futile effort, invariably also making your app slower, shittier, and less user friendly in the process. Endpoint security for public services is a myth, fullstop.
I've done something similar. I had to check how much credit accumulated in my account over years, had dozens of files to check through. Now I could check every single file, or I could do what I did, and learn how to automate it in 2 hours. The script took like a second to run. I can now use it whenever I want too.
You don't want to mess with bank apps. It's illegal I'm pretty sure and even if it's not who keeps a bone headed judge who still uses a nokia as his only "smart" device from charging you with some sort of hacking? Especially if you show everything on a video for the whole world to see?
Sort of. WireShark is more focused on inspecting TCP/UDP network packets (transport layer), while Charles/ProxyMan/Surge focus on inspecting HTTP requests (application layer). WireShark is also built for Macs.
Wireshark for Mac is Wireshark for Mac. There's some overlap here, but Charles is a lot more focused on what it snoops and is limited in scope compared to the shark.
As a teacher, I did something like this for Canvas. There was an obvious lacking feature, so instead of eating my time, I made a chrome plug in to add a button to do the feature for me.
@@senatuspopulusqueromanum Just buttons for full marks, and no marks. Weirdly this isn't a build in thing. You have to go through each question and type the grade in.
@@Mathhead2000 ohh, makes sense. How did you do it, I know theres an API thing in the settings, but I'm not sure how I would do something for my canvas
@@senatuspopulusqueromanum Oh no. I just read the source code and created a Google Chrome plug in which created the buttons and correct Javascript when it was clicked. It was brittle, but worked fine for me.
I love taking a simple repetitive task, and turning it into a multiple hour, sometimes even days, programming project to automate it. It's just a lot more fun and satisfying to automate things.
I spend a lot of time with banking apps from all around the world. People should not be surprised by the lack of SSL pinning here. You have no idea how bad it is.
The way you execute projects is has tons of learnings for a newbie like me who is really interested but gets lost during the research part but somehow creates it by not giving up! 😅
Having the ability to automate things is one of the best skill you could possible have. I always had a problem with twitter not bulk deleting your tweets and all of the external tools for that are limited to 3k tweets. In literally 10 minutes i made and script that look through all of your tweets and delete them all.
Why wouldn't using a self signed CA certificate on your phone, and MITM all requests by having the proxy decrypt it and then forward it using a forged certificate derived from your self signed CA like it does with all other HTTPS/SSL/TLS MITM attacks not work?
@@sophiophile This will not work, because the app is (or should) check the certificate of the server. The real certificate is known and saved in the app. After receiving a response, the app should check if the received certificate matches the saved certificate. So, it's not about the response having any certificate, which would allow MITM attacks, but the response should have the known good certificate. Search "certificate pinning" on Google.
"Projects like this that make me think: I love programming" I identify myself a lot with this statement. Doing this manually would take less time then programming, but there is something fun and rewarding when it comes to automate mindless tasks
The thing is, if you never do technically irrelevant stuff like this, you never gain the experience to built something when it’s really really relevant. Saving time is one thing; having once again experienced how to solve a problem end-to-end is practice on how to catch fish instead of constantly buying it. ❤
i caught a fuel theif in my area who used to take out fuel at night from motorbikes using Computer Vision and AWS, Now i walk like a superhero in my area XD
I wanted to get my 6 years of bank statements from my Romanian bank when I moved to the UK, and they only allowed me to print it. They didn't ask for money for that, but I paid a nearby office supplies shop to use a Xerox scanner with an automatic feeder like 15 euros to get a PDF of the whole stack of papers. 10 years later that PDF is still in PDF form, I ain't got time to write the OCR for that xD
I love how being a dev change your mind on these kinds of things. Before starting to dev : YEAAAAH FINE I'LL PAY. After : 17$ FOR THAT ? I'll do my own tool and get it myself ! Then you work for 10 hours straight on a project, technically lost more money worth of work time that it would have cost to pay the thing, buuuuut you got a fun project to do ! (noooo this isn't the story of my liiiiife)
I have the exact same problem a year ago. I was not in my country to get the statement so I can only access it through the mobile app. I also thought about python script, but I didn’t know where to start so I was just gave up on the idea. This is immensely helpful. Thank you.
Yes, good that you did this video, and that you did the project. The people who have commented about how much time you spent really really miss the point. You're spot on with how many institutions, banks being notorious, nickel and dime us, or $17.63 us to death. It's, annoying and costly, but not quite costly enough, usually, to stop us. And we give in. Glad you didn't. I'm sure it cost you more than 17.63 if you thought of it as your hourly programming rate. But that really isn't the point is it. So, good for you. Using that phone network traffic interceptor was a clever idea.
The real point isnt that you saved money or stuck it to the banks. The real point is that you learned some new things you'll be faster and more knowledgeable in the future. I notice this all the time, I'll spend a couple hours doing something like this and then when im done learning I'll understand how to do it in like 10 minutes next time. Your knowledge pays compounding returns.
"But, I'm a programmer, so how can I stretch these 17 minutes over hours?" That is the most truest statement I've ever heard anyone say, and I do this all time.
'clearly done so the bank can charge you $17' I don't think it's clear at all. It's pretty rare to ever need all your statements, so maybe it wasn't a "user story" in their design process and didn't get accounted for. And while it does seem a bit silly to query the database for all statements, conceivably it was just the easiest way to meet the deliverable, because the backend can just serve all the statements while different front ends (i.e. browser, app..) do the filtering? I guess I mean, as a programmer I think people overestimate how much is deliberate rather than just arbitrary in software development.
Found this very interesting. Small enough example to digest its entirety, but great example from top to bottom of multiple steps in the process and how they are related
Reminds me a little bit of that one airline website that didn't allow me to purchase a ticket because my card's expiration year wasn't in the dropdown list. So I did what a good little front-end guy does and was able to book that flight. Still crazy to think how much bisiness they'd been losing due to bad actual front-end design.
Its generally workers that doesnt know how their system works (cant expect a bank employee to know coding and how these things works) and software engineers exploiting their lack of knowledge by giving them bare minimum "workin" UI. They trying to work things out their old tech knowledge.
Love it! I also love automating stuff too! I wrote code to auto pay student loans for me since the loan servicer doesn’t allow me to set up auto pay before grace period is up.
And those banks buying software from those "lazy programmers" is just fine right? There was absolutely no possibility the conversation went something like this? Bank: we pay X Coders: but that will not be enough, we need more. Bank: This is what we offer, take or leave it. Coders: (does want make money for food to survive): we could make it, but with lower quality. Bank: WE DO NOT CARE, JUST GET IT DONE AT THIS PRICE. (the bank does NOT care about customer quality/security)
@@DarkGob It's not? You think it's a "GREAT pattern" that corporations abuse their economic power to degrade customer quality? Degrade security is GREAT because it "was cheap"? Are you a big corp lackey or an idiot?
This is precisely exactly why companies are lobbying to ban web scraping. They call it 'piracy' when you use your own tools to retrieve your own data. Also - 0:53 be careful with apps like this. This is a great way for malicious programs to harvest session cookies and ship them to third parties. Depending on the app and how much you trust the developer this can easily be used to hack your accounts.
1:39 everytime someone on youtube says "pdf file" now I get flashbacks to the current Mr.Beast evidence (no, I'm not saying "allegations". I hate that word. It's pretty obvious what they did)
Bash is perfect for this kind of throw away data processing script. Tools like curl, cut, grep, cat, jq, awk, sed, sort, uniq, tr make this a breeze and really fast. You can literally open dev tools network tab and right click a request and "Copy as cURL" to replicate the exact request in a Bash script and you have extracted and formatted your data usually within 10 minutes easily. Most developers not using Linux are missing out. When I have to use Windows for development it's a self inflected handicap, and running "Bash" on Windows is a joke.
I wouldn’t call myself a coder but I’m really interested - just feeling too silly to understand :/ But this kind of DoGoodThingsToSafeTimeAndMoney stuff makes me happy every time. Thanks man :)
Oh my... Tikes like these I'm grateful: - I can do this myself - I live in a country without these "mind games" - I'm broke so banks don't expect me to oay anything
You got a subscriber from this video everything about it was good. Please do more like this I thought I was the only one who found these types of programming interesting lol
That moment when I look at this and think: "Crap... that was a lot faster than my Selenium Test to do a similar thing on my mortgage company site." 😅 Good stuff!
I have a project for using plain text accountings tools like hledger and I wanted to automate stuff like bank statements, why it so hard for banks to provide a public API where you already have an API for your web interface?
wait, they didn't have certificate pinning? It should have been slightly more complex to use charles proxy there. Like decompiling and recompiling with a different cert.
Thank you. I am not the only one. I just want to make a button on my web server, that when I push it, it will download my utility bills, that are all in pdfs
Most marxist and anti-corporation CodingWithLewis video
Marxism is pro mega corporations and anti people and small business what are smoking
how? doesnt make sense
o7
@@nomadshiba right? I see nothing remotely marxist in any of this. Lewis as a member of the market, would not bear that cost - so chose another method. Sounds quite capitalistic to me.
@@redneckcoder It's a joke that none of them are and are quite the opposite.
This is perfect. Perfect example of, this could take me 30 minutes OR, I can spend a half day automating it.
A man of culture
auto and free? yeah ill take choice no 2
I find that I may need to reuse a part of a script for another project. In that light, I may also save time when solving other or similar problems in the future
Effort to avoid (future) effort.
in other words, i can do a borring 30 minute task orrr have fun for half a day and get what i want
"How can I take this 17 minutes... and stretch it out over hours?"
You're my inspiration
This is 100% how I pick my next software rabbit-hole, I mean “project” spend 6 months working on something that I could’ve had done in a few days without code. It’s not a glamorous life, but it’s one worth doing with code
oh shit we have the same pfp
@@Nikkuuu69 ur the 3rd person ive seen with this pfp as well lol
Difference between a developer and a software entrepreneur is knowing what pain points are worth investing time into to increase efficiency
O other hand: I will learn myself how to do this in case bank asks much more than 17$ next time.
I'm a programmer I solve 17 mins issue with 67 hours solution
High IQ
For 10000 people that could work as well
That others will use collectively saving hundreds of hours, that’s why i love coding
Always ❤
I am a noob Python user... I spent a week making a terrible Flask site so that instead of running a script off my pc I can check things off my cell when on the go.
Best week of migraines ever. 😂
Fr, I made a Python script that does all the work in 7 minutes, but it took 3-4 hours to make that fking script. In the end, I was happy and satisfied that i can do stuff and am not useless 😂
yea… but he made a video and script publick. now he might saved someone hours of time. Doesn’t matter how stupid you thinkr task is, you never know when and for who it will be useful in what scenario
yea... but he can re-run his script in seconds as many times as he likes, whenever he likes. Which is useful if he decides to make a change. If doing it manually you have to spend 17 minutes again each time.
@@michaelstreeter3125- He is closing the account though, so it’s likely a one-off activity for him.
I feel that a project like this just for the fun of problem solving and writing the code for it. Yeah, you could just spend the 17 bucks to get printed copies from the bank which you'll probably end up spending half a weekend scanning it all back to digital. OR you could spend a weekend making your finances actually fun!
For me it's the consistency. If I would need to press 1000 buttons correctly, it's bound to have some errors and so much brain strain.
Instructions unclear: The bank interpreted my python as a “threat” and said I had “intent to rob”
i like your profile picture
I'm a Banana
@@Horseyhcan i eat you
@root...... We are racist. We don't like Mushrooms.
this. this is how i got into programming. this is the reason i recommend basic programming as a hobby to anyone who uses a computer. i love these types of videos because they're the exact type of crazy pointless things i do.
Why spent 10 minutes doing something when you can take hours??
@@CodingWithLewis truly a programmer of culture (i can relate lol 🥲)
Ikr? At least you are not bored then and can save other people's time afterwards too
Well, I wish I could get a job with this 😅
You can? @@oioio-yb9dw
The hyperplexed impression was so jood 😅
This was an awesome part 😂🎉
Very relatable. Why spend 17 minutes when you can spend hours to automate the task that you will never have to do again?
Decided to try something new :) let me know what you think of this more "shorter" style content!
We have a hackathon coming up shortly :) join the convo on discord!
How old are you?
I don't see any discord link though
What discord
Lewis, I think it wouldn’t hurt to check with a lawyer or ask someone from the bank if you could do this. I am 90% sure it’s ok but when it comes to banks you could potentially be breaking your contract or something hidden in the terms of service. Just a heads up from a lawyer & programming hobbyist. Love your videos!
@@pedroandrepiccolithat's why he added in that little delay. So the bank would never know!
looks like not a secure app, you should not be able to intercept traffic in a secure app (i.e SSL pinning ), and the app should refuse to do anything
hmmm, exactly i don't understand.
In addition, he calls the app out for using “dark patterns” when it’s probably just coded lazily so it just returns all the statements at once
It's also weird that a native banking app uses regular REST endpoints instead of an RPC
And fetching every data possible in every request jeez they should innovate pagination in their api calls
And the traffic was unsecured?
I don't belive a bit, you might have dreamed you coded that
Ah yes - I totally relate to the "I refuse to do this manual, tedious process that will take me 20 minutes so I'm going to spend a week writing software to do it". Many times.
i made a project once, called it multiterm, you can get for python as a module thru pip, it gives you a screen based system to create terminal user interfaces
What you are working on is WAY more complex than this. Props!
I closed my account with this same bank earlier this year. I'm not a programmer and I somehow screen scraped all my records for the past 7 years. You're right, they purposely made the UI barely workable to get you to pay them for what you need.
I myself have multiple times turned a 20-minute problem into 2 days of python project. But the benefit I got, the next time I get that 20-minute problem, it would be done in seconds!
I did this manually recently. Downloaded all pdfs, copied everything to excel and did some calculations. I found out that the "No cost emi" was not really "No cost". It was less than the normal emi plan but it was way more than what expected.
0:47 Why is this the most relatable thing ever
Everything you just said in the video is meaningless compared to the fact you can read your bank request in plain text through a proxy. This should never happen. Call your bank and tell them they have a major security issue and they need to stay up all night to fix it. Like today.
he's using SSL unpinning, this is illegal in the US and can get you in jail if the bank wanted to.
@@eclipsek0first it's not a us bank, second do you think it being illegal would prevent actual malicious person to use that beach?
Tell them to use certificate pinning and enforce https all the time and they’re good
What's wrong with it? All apps act like this
If the bank implements a client side check to make the app close in case there's interception in process, some will still be able to bypass the check
"But I'm a programmer so how can I stretch that 17 minutes to over an hour" is so real and is my new motto!
Thank you for respecting my time enough to make this a ~5 minute video instead of trying to drag it out like other channels would.
1. I think you really lucked out with the proxy/man in the middle thing working. Normally mobile apps these days use HTTPS with certificate pinning, which means that you cannot intercept any traffic between your device and the server without modifying the app somehow.
2. I don't think the API getting the 4 years every single time is a dark pattern, I full heartedly believe it is incompetence.
100% incompetence, and it will remain in the code for another 20 years before they decide it's worth fixing 😆
Honestly, they probably just didn't have any support for pagination. That and 20 years of bank statements probably loads only marginally slower than four years. It was the path of least resistance and meant he got to go home at 5. I think that's the real reason
Edit: the lack of encryption is inexcusable though
@@twobitsnickit *_is_* encrypted, he exported the cookies and certs and just made the exact same request through his computer rather than solely his phone.
There is literally no vulnerability here that could be meaningfully prevented.
People really need to stop wasting time, money, and fucks on endpoint security. Once you've sent the data to yhe user, that's it, they *_can_* get it. He had full access to the device so there is nothing any company could do to stop this aside from adding annoyances, and that's just security through obscurity. The issue is, if there *_is_* a real vulnerability here then adding more shit to get in the way won't protect anyone. Again, the user has the device, they *_can_* get the data. Even if they need to transcribe it by hand on a type writer, they *_can_* get it. So if I'm a malicious actor, "boo hoo, I get a bit inconvenienced when writing my code to steal millions of peoples' life savings, I should just give up now! I mean the 2 weeks of work spent dealing with this just wouldn't be worth the millions of dollars I'd get from doing it!" Meanwhile you, as the business, are fumping millions into this futile effort, invariably also making your app slower, shittier, and less user friendly in the process.
Endpoint security for public services is a myth, fullstop.
HYPERPLEXED IMPRESSION LMAOOO
Hey, blurring is non destructive and can be unblurred. It is better to block out text you don’t want seen.
Bro saved $17.63, wasted a couple of hours writing the code, made a UA-cam video on this topic, and earned double, triple, or more.
it's not about the money, it's about sending a message
I've done something similar. I had to check how much credit accumulated in my account over years, had dozens of files to check through. Now I could check every single file, or I could do what I did, and learn how to automate it in 2 hours. The script took like a second to run. I can now use it whenever I want too.
Reverse engineer the bank app and add the download button next!!
hes on iphone
@Hellscaped you can reverse engineer iPhone apps too lol
@@kaajjaak much more of a pain in the ass to get them working live
You don't want to mess with bank apps. It's illegal I'm pretty sure and even if it's not who keeps a bone headed judge who still uses a nokia as his only "smart" device from charging you with some sort of hacking? Especially if you show everything on a video for the whole world to see?
@@theairaccumulator7144 not hacking if you're modifying the client side
That Hyperplexed impression was spot on! Banger of a video!
1:14 is Charles not just WireShark for Mac?
Sort of. WireShark is more focused on inspecting TCP/UDP network packets (transport layer), while Charles/ProxyMan/Surge focus on inspecting HTTP requests (application layer). WireShark is also built for Macs.
Charles intercept the web requests and gives you the ability to edit it
Wireshark for Mac is Wireshark for Mac. There's some overlap here, but Charles is a lot more focused on what it snoops and is limited in scope compared to the shark.
As a teacher, I did something like this for Canvas. There was an obvious lacking feature, so instead of eating my time, I made a chrome plug in to add a button to do the feature for me.
what did u do
@@senatuspopulusqueromanum Just buttons for full marks, and no marks. Weirdly this isn't a build in thing. You have to go through each question and type the grade in.
@@Mathhead2000 ohh, makes sense. How did you do it, I know theres an API thing in the settings, but I'm not sure how I would do something for my canvas
@@senatuspopulusqueromanum Oh no. I just read the source code and created a Google Chrome plug in which created the buttons and correct Javascript when it was clicked. It was brittle, but worked fine for me.
Hahaha love your inner hyperplexed! Great video too, love the short style! Got some inspiration out of it as well.
0:51 oh buddy, I smiled when you hit my soul with that one :)
I love taking a simple repetitive task, and turning it into a multiple hour, sometimes even days, programming project to automate it. It's just a lot more fun and satisfying to automate things.
I'm pretty sure it's illegal in some places to charge for your basic bank information (unless it's an analytical look at your account or something)
I spend a lot of time with banking apps from all around the world. People should not be surprised by the lack of SSL pinning here. You have no idea how bad it is.
The way you execute projects is has tons of learnings for a newbie like me who is really interested but gets lost during the research part but somehow creates it by not giving up! 😅
"I'm a programmer, how can I take this 17 minutes and stretch it out over hours?"
So true lol
Having the ability to automate things is one of the best skill you could possible have. I always had a problem with twitter not bulk deleting your tweets and all of the external tools for that are limited to 3k tweets. In literally 10 minutes i made and script that look through all of your tweets and delete them all.
Projects like these are why I got into programming in the first place.
Fixing annoying problems that simply shouldn't exist.
How is a bank app not certificate pinned? I wouldn't expect to get much further than the proxy step honestly
At the moment i would find out that the app is not checking the certificate i would switch to a better bank as fast as possible
Why wouldn't using a self signed CA certificate on your phone, and MITM all requests by having the proxy decrypt it and then forward it using a forged certificate derived from your self signed CA like it does with all other HTTPS/SSL/TLS MITM attacks not work?
@@sophiophile This will not work, because the app is (or should) check the certificate of the server. The real certificate is known and saved in the app. After receiving a response, the app should check if the received certificate matches the saved certificate. So, it's not about the response having any certificate, which would allow MITM attacks, but the response should have the known good certificate. Search "certificate pinning" on Google.
"Projects like this that make me think: I love programming" I identify myself a lot with this statement. Doing this manually would take less time then programming, but there is something fun and rewarding when it comes to automate mindless tasks
The thing is, if you never do technically irrelevant stuff like this, you never gain the experience to built something when it’s really really relevant. Saving time is one thing; having once again experienced how to solve a problem end-to-end is practice on how to catch fish instead of constantly buying it. ❤
Cant wait for "my wife tried to leave me, so I used python instead."
My first Python project in 2004 was a scraper. I still use it today. I just discovered your channel and am revitalized and getting back to coding.
How did you deal with SSL on the app? Or is the bank doing it in plaintext (big no-no) or ignoring SSL checks (even bigger no-no)?
First video of yours I've seen! Subbed and def would like to see more stuff like this!
Me: I should figure out a way to automate this
Lewis: I hacked a bank to automate this
i caught a fuel theif in my area who used to take out fuel at night from motorbikes
using Computer Vision and AWS, Now i walk like a superhero in my area XD
I wanted to get my 6 years of bank statements from my Romanian bank when I moved to the UK, and they only allowed me to print it. They didn't ask for money for that, but I paid a nearby office supplies shop to use a Xerox scanner with an automatic feeder like 15 euros to get a PDF of the whole stack of papers. 10 years later that PDF is still in PDF form, I ain't got time to write the OCR for that xD
Have you checked the PDF? The scanner may have OCRed the pages and embedded the text in the file when it saved the PDF.
@@mlindholm I'd be genuinely impressed, but I'm not sure it did that. Text is not selectable, which is the usual telltale sign
One of your best videos imo. I love learning practical applications that I have never even thought of.
2:30 you fricking legend 😂😂
Is that Hyperplexed?
I love how being a dev change your mind on these kinds of things.
Before starting to dev : YEAAAAH FINE I'LL PAY.
After : 17$ FOR THAT ? I'll do my own tool and get it myself !
Then you work for 10 hours straight on a project, technically lost more money worth of work time that it would have cost to pay the thing, buuuuut you got a fun project to do ! (noooo this isn't the story of my liiiiife)
2:58 "So this is when I wanted to get my Python out"
*vine boom sfx*
💀💀
I have the exact same problem a year ago. I was not in my country to get the statement so I can only access it through the mobile app. I also thought about python script, but I didn’t know where to start so I was just gave up on the idea. This is immensely helpful. Thank you.
This is amazing I like how you found a way to get the web requests through your phone via proxy
Yes, good that you did this video, and that you did the project. The people who have commented about how much time you spent really really miss the point. You're spot on with how many institutions, banks being notorious, nickel and dime us, or $17.63 us to death. It's, annoying and costly, but not quite costly enough, usually, to stop us. And we give in. Glad you didn't. I'm sure it cost you more than 17.63 if you thought of it as your hourly programming rate. But that really isn't the point is it. So, good for you. Using that phone network traffic interceptor was a clever idea.
The real point isnt that you saved money or stuck it to the banks. The real point is that you learned some new things you'll be faster and more knowledgeable in the future. I notice this all the time, I'll spend a couple hours doing something like this and then when im done learning I'll understand how to do it in like 10 minutes next time. Your knowledge pays compounding returns.
"But, I'm a programmer, so how can I stretch these 17 minutes over hours?" That is the most truest statement I've ever heard anyone say, and I do this all time.
'clearly done so the bank can charge you $17' I don't think it's clear at all. It's pretty rare to ever need all your statements, so maybe it wasn't a "user story" in their design process and didn't get accounted for. And while it does seem a bit silly to query the database for all statements, conceivably it was just the easiest way to meet the deliverable, because the backend can just serve all the statements while different front ends (i.e. browser, app..) do the filtering? I guess I mean, as a programmer I think people overestimate how much is deliberate rather than just arbitrary in software development.
Found this very interesting. Small enough example to digest its entirety, but great example from top to bottom of multiple steps in the process and how they are related
Reminds me a little bit of that one airline website that didn't allow me to purchase a ticket because my card's expiration year wasn't in the dropdown list. So I did what a good little front-end guy does and was able to book that flight. Still crazy to think how much bisiness they'd been losing due to bad actual front-end design.
Its generally workers that doesnt know how their system works (cant expect a bank employee to know coding and how these things works) and software engineers exploiting their lack of knowledge by giving them bare minimum "workin" UI. They trying to work things out their old tech knowledge.
Dude. I’ve been tryna create a good algorithm to read my banks statement pdfs also for like a year so this hits close to home!
That hyperplexed impression 😂😂
He's honestly the 🐐. Love his vids!
The fact that your bank doesnt have cert pinning is the scariest part of this video.
How does charles overcome HTTPS? Does it MITM your bank?
And no certificate pinning in banking software ?
This is great! Thanks for making a video about this, today I learned how to extract requests from a phone app using a proxy.
My community college website back in 2017 was so bad that you was able to view the “locked documents” by removing blur in inspect element
"Im a programmer, How can I take the 17 minutes and stretch it out over hours?"
This sentence right there got you the like.
Love it! I also love automating stuff too! I wrote code to auto pay student loans for me since the loan servicer doesn’t allow me to set up auto pay before grace period is up.
wait what so there is no tls encryption when using charles?
Yup wanna see more it's always great to watch engineers of your caliber solving problems
No Lewis - it's not a 'dark pattern' used by the Bank to be difficult. It's what stupid/lazy programmers do.
And those banks buying software from those "lazy programmers" is just fine right?
There was absolutely no possibility the conversation went something like this?
Bank: we pay X
Coders: but that will not be enough, we need more.
Bank: This is what we offer, take or leave it.
Coders: (does want make money for food to survive): we could make it, but with lower quality.
Bank: WE DO NOT CARE, JUST GET IT DONE AT THIS PRICE. (the bank does NOT care about customer quality/security)
@@unconnectedbednastill not a dark pattern. Words matter.
@@DarkGob It's not?
You think it's a "GREAT pattern" that corporations abuse their economic power to degrade customer quality? Degrade security is GREAT because it "was cheap"?
Are you a big corp lackey or an idiot?
Why is it not encrypted?
Yes we need more videos, code/ function where it could be used for daily life rather than for corporate scenarios.
banking with rbc was your first mistake. One time they kicked me out of a branch for asking "too many" questions
This is precisely exactly why companies are lobbying to ban web scraping. They call it 'piracy' when you use your own tools to retrieve your own data.
Also - 0:53 be careful with apps like this. This is a great way for malicious programs to harvest session cookies and ship them to third parties. Depending on the app and how much you trust the developer this can easily be used to hack your accounts.
LMAO, that hyperplexed moment got me dead!!! So funny, wish he actually was in this video lol!
1:39 everytime someone on youtube says "pdf file" now I get flashbacks to the current Mr.Beast evidence
(no, I'm not saying "allegations". I hate that word. It's pretty obvious what they did)
no certificate pinning in the bank app?
Bash is perfect for this kind of throw away data processing script. Tools like curl, cut, grep, cat, jq, awk, sed, sort, uniq, tr make this a breeze and really fast. You can literally open dev tools network tab and right click a request and "Copy as cURL" to replicate the exact request in a Bash script and you have extracted and formatted your data usually within 10 minutes easily. Most developers not using Linux are missing out. When I have to use Windows for development it's a self inflected handicap, and running "Bash" on Windows is a joke.
I wouldn’t call myself a coder but I’m really interested - just feeling too silly to understand :/
But this kind of DoGoodThingsToSafeTimeAndMoney stuff makes me happy every time. Thanks man :)
Oh my... Tikes like these I'm grateful:
- I can do this myself
- I live in a country without these "mind games"
- I'm broke so banks don't expect me to oay anything
You got a subscriber from this video everything about it was good. Please do more like this I thought I was the only one who found these types of programming interesting lol
Turning 17 minutes of manual work into hours of automation. Truly a classic.
Such a cool project. Love the onus! It sucks how paper is cheaper than paperless
That moment when I look at this and think: "Crap... that was a lot faster than my Selenium Test to do a similar thing on my mortgage company site." 😅
Good stuff!
this is the kind of shit that i love building, tools that make the things i do on a regular basis a zillion times easier
This is awesome. Even the Hyperplexed reference 👌
Hanlon's razor applies here. bank tech is probably just not really focused on this part of the user experience so they didn't care to make it good.
This video was like eating a bag of chips that we good, satisfying and a quick snack . Thanks buddy.
I have a project for using plain text accountings tools like hledger and I wanted to automate stuff like bank statements, why it so hard for banks to provide a public API where you already have an API for your web interface?
loved the hyperplexed segment and yea stuff like this is why im glad i chose to be a programmer even tho the job market is shit rn
wait, they didn't have certificate pinning? It should have been slightly more complex to use charles proxy there. Like decompiling and recompiling with a different cert.
Thank you. I am not the only one. I just want to make a button on my web server, that when I push it, it will download my utility bills, that are all in pdfs
And that's why when my dad asked me why was I looking into things one by one, I just said: believe it or not, manually is faster
if the bank is fetching ALL your records at once then it seems easy to crash their servers if your devious
With my cookie attached 😅
Sounds like a great way to get the feds to visit you
"Hi, this is a robbery, by the way here is my passport, SSN and (pulls out a heavy stack of papers) my entire life worth of documents"
Tried something similar once, got temp ip banned from accessing my bank probably because I missed a header or something eeek.
The irony here is some dev set this up in the first place.
Dev: I can make it so annoying to get statements that people will pay.
Bank: Yes please
As a long time RBC customer (terrible bank), I recognize those blurry app, website and statement layouts!
It would be cool to get a real demo on it :)
As all scenarios are not the same it would be interesting too having to solve that little problem lol
"but I'm a programmer how can I make this 17 minutes stretched out for hours", boldly true