What I love about this type of videos is that I get to see the whole chain of problems a project comes with in just 5 minutes. This video was very well done, can't believe you're under 1k subs, keep it up man!
برو محتواك رهيب تفكيرك البرمجي و تسلسلك في تطوير حل المشكله بشكل أكثر كفائا يدل على أنك ميرمج جيد لقد حصلت على مشترك جديد أنا مهندس برمجات من الأردن أستمر ❤❤
You can also control an user , not just bot programatically, and use the mtproto api instead of bot api, then can upload/download 2 gb directly (either bot or user ). Pyrogram is a python library, supports that
you can get banned for controlling the user account directly but you *can* just control *bots* with mtproto (in fsct the bot api server is just a proxy that takes bot api reauests and turns them into mtproto requests)
@@v0xl you won't get banned for using the users with any more probability than using bots. In fact all telegram apps (official or unofficial ) works in the same way. That's why I said "also control an user", and to use MTPROTO instead of bot api. I know about bot api being proxy for mtproto and other info , I read it on pyrogram website.
Telegram actively bothering to upload many files >30mb from the client. Sometimes I need to send 10-20 videos from my iPhone via TG. It starts smoothly, but on 2nd-3rd video it starts to freeze on first 30mb chunk of each video (so, first 30mb uploaded fast, and after that uploading stops with ethernal spinning of the progress bar). Resending files over and over helps. Premium subscription doesn't help. It seems that TG has some server-side algorithm that marks user as unwanted and starts to freeze uploads after first 30mb chunk with some probability. Maybe, this probability grows over time
They don't. The encryption is only enabled in "secret chats". But even if they do, their encryption is unsafe as it is unverified by experts. It's also proprietary garbage. Matrix would have been better. But filling up volunteering matrix servers with your gigabytes of data would be a dick move.
The servers are closed source, and if you've taken a look at the clients, they only encrypt things in secret chats, and the encryption is something they've come up with meaning it could have backdoors.
You can encrypt whole file, split and upload. In this way, no one can decrypt your file unless they construct it back from splits. To determine which chunk goes for which file, you can embed hash of your unsplitted file + two index bytes(65535 max values * 20mb per chunk / 1024 gives you over 1TB of encrypted data per file). Actually, better way is to embed this hash +index in message with file, so you won't need to download chunks to determine their original file. Yes, you would loose info about file name, but this can be safely splitted and embedded right into your chunks
I just saw this video after finishing my project. And yeah, for the first half of the video I went through the exact steps and issues. I created my own filesystem for my project and used a very basic CLI for interacting. I used Python for this project. Next plan in to build more features and probably a GUI
Have you considered creating a FUSE filesystem that uses this as a backend? That way you can mount the folder locally and open the files as if you had physically connected a drive to your PC. FUSE is a linux thing, but there are also windows bindings (WinFuse)
Keep the interesting side quests coming - you might stumble upon something and make a breakthrough. Who knows what useful piece of software you could come up with, perfect, and then share with us all.
Bro, I've been trying to achieve the same but for some reason I'm unable to split the large files into small chunks. When trying it gets corrupted. Someone mind helping me a lil bit? 😭
All the splitting an encryption stuff could have been solved just using standard zip segmenting and encryption. It would also decrease the size of the upload making it faster.
If you use typescript your code should be clear than js. And you can another solve problem that 20mb limit (polling, 2gb is webhook). But all of those stuffs take a more time you can imagine
I actually have made a similar project a long time ago that im hosting with a full frontend. I also encountered a lot of issues and i just wanna say whenever you do decide to rewrite it don’t do the same mistake as me and USE STREAMS
Hey, sorry mate but at the moment I haven't put out the code for this thing yet because it's a little mess. One I fix it I'll put it out and sure add a very beginning friendly setup. At the moment its hard to be able to make even a second programmer understand what I have done 😂
I did try to build a CLI app in go, and almost get it down until I saw tg bot can only upload files less than 50mb, and I know it's tricky to get the chunk not corrupted and I also don't wanna do local server or split files or encryption, they all eats a lot of resources locally especially you got a lot of files (in TBs), now it's 1 o'clock in the morning and I wasted three hours on doing this. Good night 😅 let those files reset in my HDD forever.
Completely Understandable, after I did this video I got to know that there is a library in python (pyrogram or something like that) which can upload files upto 2GB's. I tried it for a file and it worked smooth. Try using that, you don't need to split unless your files are 2GB in size, so the encryption is also quite easy in this case.
3:00 not needed at all telegram is super secure, no goverments have acces to the data u can post whatever you want and only people in the group will be able to dl/view files, although telegram is in some issues with goverments rn, so i geuss if ur that worried it isnt a bad idea
This project isn't scalable, atleast what I created isn't. It was just a quick personal use of the API. If I work more on it, I'll sure do something about the storage on the server. Thankyou for the comment :)
You do know Telegram deletes data every 6 months right? All chat history, files and media, same for bots. It's in the settings it can't be turned off but you can adjust the time, max is 6 months tho
Entire video was superb. Just didn't understand the one part. Time stamp : 4:05. "Why don't I store just a single chunk which needs to be uploaded, encrypt the chunk, upload it and just delete the chunk, that's how it's gonna store only 20MB on the server"? Why only 20MB when the actual file is let's say 2GB? Is the @StupidGeek talking about the middle server which is responsible to the the encryption and has the other APIs to upload the data to the Telegram?
Thankyou. The part you're talking about, I take the whole file from the client (frontend) split a 20mb chunk from it, encrypt it and upload it. When the upload is complete, I deleted that chunk from my server and continue to make further chunks from the same file. This process continues till I upload the whole file.
"I found some libraries but they all lacked some or the other functionality" This is literally the JS ecosystem in a nutshell
For real
And then there's me, who literally only uses vanilla js and refuses to learn any libraries
My favorite part is you didnt bother to read the docs for download size before smashing out half the code for this monstrosity. truly one of us
That's our thing, right? 😁
What I love about this type of videos is that I get to see the whole chain of problems a project comes with in just 5 minutes. This video was very well done, can't believe you're under 1k subs, keep it up man!
Thanks mate, means a lot :)
This guy motivates me to code something in python when i don't even have storage problems,
Subscribed.
:D thankyou
Bro didn't even know this is your first video, your commenting and editing skills are damm good. Keep up the good work 🙏
thankyou for the kind words :)
one day dude you'll hit that 1M subscriber. great video. i enjoyed a lot
haha, thankyou, I'm glad :)
برو محتواك رهيب تفكيرك البرمجي و تسلسلك في تطوير حل المشكله بشكل أكثر كفائا يدل على أنك ميرمج جيد لقد حصلت على مشترك جديد أنا مهندس برمجات من الأردن أستمر ❤❤
You can also control an user , not just bot programatically, and
use the mtproto api instead of bot api, then can upload/download 2 gb directly (either bot or user ). Pyrogram is a python library, supports that
That's cool, I'll consider switching to that then. Thankyou :)
you can get banned for controlling the user account directly
but you *can* just control *bots* with mtproto (in fsct the bot api server is just a proxy that takes bot api reauests and turns them into mtproto requests)
@@v0xl you won't get banned for using the users with any more probability than using bots. In fact all telegram apps (official or unofficial ) works in the same way. That's why I said "also control an user", and to use MTPROTO instead of bot api.
I know about bot api being proxy for mtproto and other info , I read it on pyrogram website.
Telegram actively bothering to upload many files >30mb from the client. Sometimes I need to send 10-20 videos from my iPhone via TG. It starts smoothly, but on 2nd-3rd video it starts to freeze on first 30mb chunk of each video (so, first 30mb uploaded fast, and after that uploading stops with ethernal spinning of the progress bar). Resending files over and over helps. Premium subscription doesn't help. It seems that TG has some server-side algorithm that marks user as unwanted and starts to freeze uploads after first 30mb chunk with some probability. Maybe, this probability grows over time
hydrogram*
It's disgusting. I love it. Keep up the work
haha, thankyou :)
Very nice POC, can't believe it's your first video, keep it up! I was here ahah
haha, thanks
First video?? Damn this is good quality content
Thankyou :)
telegram actually encrypts by default
yeah just got to know that when I opened the docs again, silly me :')
But who holds the key?
@@leandroalbero you, locally
They don't. The encryption is only enabled in "secret chats". But even if they do, their encryption is unsafe as it is unverified by experts. It's also proprietary garbage. Matrix would have been better. But filling up volunteering matrix servers with your gigabytes of data would be a dick move.
The servers are closed source, and if you've taken a look at the clients, they only encrypt things in secret chats, and the encryption is something they've come up with meaning it could have backdoors.
this is just amazing! Subscribed for this haha!
thankyou, appreciate that :)
10/10 storytelling, would watch again.
Thankyou :)
Amazing content man keep up the work 💪🏻
thankyou :)
You can encrypt whole file, split and upload. In this way, no one can decrypt your file unless they construct it back from splits. To determine which chunk goes for which file, you can embed hash of your unsplitted file + two index bytes(65535 max values * 20mb per chunk / 1024 gives you over 1TB of encrypted data per file). Actually, better way is to embed this hash +index in message with file, so you won't need to download chunks to determine their original file. Yes, you would loose info about file name, but this can be safely splitted and embedded right into your chunks
And if you want even more safety, you can multiply your hash value in bytes by your index and store encrypted hash+chunk index
That's sounds good, I'll sure try it. Thankyou :)
Dude, I would love to help you out on projects. This looks absolutely like so much fun. Subscribed for sure.
Thankyou, you can sure contribute after I open-source it.
i love this channel, keep pumping out more content
Thankyou, definitely :)
During download you should have it send files into 20 mb chunks as well, and let client merge them
yessir should've done that. Thankyou for the suggestion :)
I just saw this video after finishing my project. And yeah, for the first half of the video I went through the exact steps and issues. I created my own filesystem for my project and used a very basic CLI for interacting. I used Python for this project. Next plan in to build more features and probably a GUI
Good project, right? Keep working on it :D
The best feature about JS is its ecosystem
:D
Great idea and excellent video. Subbed.
Awesome, thank you :)
loved this idea and video format. what time tracker are you showing at 5:16? looks pretty cool
I think it's WakaTime, but I'm not sure wakatime.com
Thankyou :) and yes its wakatime
Please make this open source.
Also, loved the video! I can't believe this is your first one :o
very soon :D
Thankyou :)
So excited for your next video ?
working on it :D
Have you considered creating a FUSE filesystem that uses this as a backend? That way you can mount the folder locally and open the files as if you had physically connected a drive to your PC.
FUSE is a linux thing, but there are also windows bindings (WinFuse)
That's interesting, I'll try to use that :)
@@stupidgeek Dev Detour tried this and used this too :)
Your first upload and you are already making quality content ! :)
Where could we see more of that code ?
Very soon, maybe in the sequel to this video :)
Nice project great editing
Thankyou :)
Why didn’t you use TDlib directly without a bot to get 2gb limit and not 50mb ?
So at the end you didn't able to make it?
I did
Very cool project . You deserve the sub, and it would be very great if you released it open source
thanks mate :), the code is mess right now to be honest 😂, i'll fix it and put a link soon or maybe a follow up video.
Went to watch your other videos but sadly this was the only one, hope you upload more soon.
sorry, stay tuned :)
@@stupidgeek you could try documentary content, your voice is great
@@flixqtt haha, nah, programming is enough for me :D
This exactly the type of stupids projects I like to do and live to watch. I sub right now ! 😎
thankyou XD
Me, with slow af internet:
:(
@@stupidgeek don't worry, I can still "borrow" storage at night with this technique kekw
@@fuzzy-02 ;)
What website/software/vs code extension/whatever is that you used at 5:16 ??
It's a website called WakaTime :)
@@stupidgeek nice! Thank you :D
Somebody already made an app for this Unlim Cloud but that stores the data in saved messages and nk encryption
Keep the interesting side quests coming - you might stumble upon something and make a breakthrough. Who knows what useful piece of software you could come up with, perfect, and then share with us all.
suree :D
Bro, I've been trying to achieve the same but for some reason I'm unable to split the large files into small chunks. When trying it gets corrupted. Someone mind helping me a lil bit? 😭
fire but on an update try planting the storage as a drive in your computer using it as a hella slow drive instead of that
sure thing, cool idea, thanks man :)
All the splitting an encryption stuff could have been solved just using standard zip segmenting and encryption. It would also decrease the size of the upload making it faster.
That's a good idea, I'll give it a try. Thankyou :)
Oh man you have so great video edit
thankyou :)
Aren't you open sourcing it? 😢
Where are the graphs he showed at the end from?
It's a website called WakaTime
Sir, please tell me the project Management software you are using.
Thank you
Its a website called WakaTime :)
@@stupidgeek Thank you sir
How did you get the project metrics at the end of the video?
Its a website called WakaTime
Why is this video interlaced??
my screen recorder didn't want to work that day, had to record it on a cheap recorder :')
5:15 - but it says 23 on the picture
yea, but its time spent in the previous week, the top right says the total time :)
How do you measure the time spent in the project?
It's a website called WakaTime :)
Nice project bro, what program do you use at 5:17?
It's a website called WakaTime :)
5:15 which app you used to measure your time spent on project ?
It's a website called WakaTime
how did u track time spent in the last part of the video?
It's a website called WakaTime :)
step by step, not reading documentation, a journey.
:D
If you use typescript your code should be clear than js. And you can another solve problem that 20mb limit (polling, 2gb is webhook). But all of those stuffs take a more time you can imagine
I see, I will sure use these tips, thankyou :)
Kinda reminds me of using UA-cam as a file system but in less cursed
I actually have made a similar project a long time ago that im hosting with a full frontend. I also encountered a lot of issues and i just wanna say whenever you do decide to rewrite it don’t do the same mistake as me and USE STREAMS
I see, sure thing, thankyou for the advice :)
link it bro, I'm interested in trying it out
@@thatcat7113 Its private currently cause yk how people are they tend to upload weird stuff so i dont want to be held responsible
@@backiscute oh yeah that makes sense, true that's an issue
Hi , Can you pls provide the name for your editing software
Hey, I use Davinci Resolve
good job mate !
Thankyou :)
@@stupidgeek and u just gave me the idea for my first video xD
sure, go for it :)
How did you track the entire development time and get those graphs at the end?
just by memory, haha. The graphs are from website called WakaTime
@@stupidgeek thanks a lot
@@nahu-dev you're welcome
How did you do the time tracking?
Using a website called WakaTime :)
how did you found the time spent on this project?
It's a website called WakaTime :)
Hii is there a way for simpletons to use these? Most of them are complex and I really don’t get how to use it.
Would be nice if you could help me
Hey, sorry mate but at the moment I haven't put out the code for this thing yet because it's a little mess. One I fix it I'll put it out and sure add a very beginning friendly setup.
At the moment its hard to be able to make even a second programmer understand what I have done 😂
@@stupidgeek
Okayy thanks :)) you’re really sweet haha
I did try to build a CLI app in go, and almost get it down until I saw tg bot can only upload files less than 50mb, and I know it's tricky to get the chunk not corrupted and I also don't wanna do local server or split files or encryption, they all eats a lot of resources locally especially you got a lot of files (in TBs), now it's 1 o'clock in the morning and I wasted three hours on doing this. Good night 😅 let those files reset in my HDD forever.
Completely Understandable, after I did this video I got to know that there is a library in python (pyrogram or something like that) which can upload files upto 2GB's. I tried it for a file and it worked smooth. Try using that, you don't need to split unless your files are 2GB in size, so the encryption is also quite easy in this case.
What is the name of the music playing in the background? Anyone?
I thought about doing this with youtube before. Store data as videos. :)
Care to share the code? Some very interesting things you've done that I like to learn
Yessir, soon after fixing some of these issues :)
How do you get those GitHub analytics?
Those are my vs code analytics, you get these using a website called WakaTime.
@@stupidgeek Damn, sweet.
@@Anto-mi5pn :)
Nezt time please use completely black background for the text comments so its nicer to look at it with oled screens
sure, thankyou :)
3:00 not needed at all telegram is super secure, no goverments have acces to the data u can post whatever you want and only people in the group will be able to dl/view files, although telegram is in some issues with goverments rn, so i geuss if ur that worried it isnt a bad idea
Audio name?
Your server will store 20 MB file for each client , if there are more people won't it crash ?
This project isn't scalable, atleast what I created isn't. It was just a quick personal use of the API. If I work more on it, I'll sure do something about the storage on the server.
Thankyou for the comment :)
A very interesting project, great work! Also, what did you use to track time spent on the project?
Thankyou :)
It's a website called WakaTime.
@@stupidgeek thank you!
You do know Telegram deletes data every 6 months right? All chat history, files and media, same for bots. It's in the settings it can't be turned off but you can adjust the time, max is 6 months tho
It's only if you don't come online in the last 6 months :)
I haven't had anything deleted from me
Entire video was superb. Just didn't understand the one part. Time stamp : 4:05.
"Why don't I store just a single chunk which needs to be uploaded, encrypt the chunk, upload it and just delete the chunk, that's how it's gonna store only 20MB on the server"?
Why only 20MB when the actual file is let's say 2GB?
Is the @StupidGeek talking about the middle server which is responsible to the the encryption and has the other APIs to upload the data to the Telegram?
Thankyou.
The part you're talking about, I take the whole file from the client (frontend) split a 20mb chunk from it, encrypt it and upload it.
When the upload is complete, I deleted that chunk from my server and continue to make further chunks from the same file.
This process continues till I upload the whole file.
@@stupidgeek Got it! Thanks a lot for your response.
@@immayurpanchal welcome :)
The time you put in such projects could have been wage hours towards a NAS with no strings attached.
amazing work
Thanks mate :)
your screen capture is interlace please fix it, i find your video very entertaining
Sorry about that, I fixed it right after the upload :)
i hope you didn't use react for frontend
I did, in a hurry :')
I'll use nextjs or something now
Whats the software that you used to track your project time and languages at the end of ur vid? Im a newbie :p
Its a website called WakaTime, welcomes to the zone btw hehe
@@stupidgeek thank you! earned a sub ;)
@@stupidgeek thats cool man sub earned
@@oak___ thankyou :)
@@hkstar2871 thankyou :)
Love the video
Thankyou :)
did you roll your own crypto?
If I were you I would get this thing up and working and then I'd start selling the storage to make a profit.
That's a good idea haha
lmao i lost it at more chuncks
:'D
Great vid! What did you use to track the time spent on a project?
thanks mate, its a website called WakaTime, its like a dashboard for devs :)
yo what is teh code?
Great video
Thankyou :)
wait for limitations from Telegram ((
Nah bro give us the bot this is so cool I wanna use it
patience, won't take too long :)
but, where is the code ? :D
sorry didn't share, I'll share the code after fixing some of the issues, please wait for it or maybe the sequel to this video :)
This was brilliant 👏
Thankyou :)
unlim in andoid
well that's awesome
project repo link? :)
Need to clean the code and fix some of these issues. It'd not be worth putting it out now, I'll put it out very soon :)
Dev Detour beat you to it.
Forreal :D
@@stupidgeek :D
Nice
Thankyou :)
nicee
thankyou :)
source code plz?
I'll put it on GitHub soon, just have to clean it up and fix some of the issues.
Hey.. how did you track your work hours. Nice video btw.
Thankyou :) its a website called wakatime
File spliter is where it ends.
this technique is so great, too bad that telegram is banned in many countries.
:)
:(
how tf only 600 subs
Sad right?