You attained such speed because your "cache" is just reading a file of paths, the real and realistic time is ~50 sec like you showed since there is no cache invalidation. But hey, enough to fool most apparently
this is tik tok content stretched out to 4 minutes for no reason, every video this guy has made are superficial projects or stuff that he just gives up half way through
In practice this is more useful if you're looking for multiple files though. I wish windows would keep a cache. Sure it's a cheat, but it doesn't invalidate it being useful. It just changes the acceptable use cases
A file explorer is a pretty good project imo because it's easy enough on a basic level but it also can be made significantly more complicated if you want to torture yourself. Optimizing stuff like this is always very satisfying as it's what us - programmers - are paid for. I would love to see this project with a better UI, i think it's worth it. And finally, "wOw" - conaticus, 2023
I started playing with similar project several years ago. Now it is almost done, approaching 1.0 release... Just google for HiFile file manager... And I can confirm that it is really a lot of work if you want to have your file manager feature rich, crossplatform and supporting archiving! :)
yeah, the next level for search is to implement it in a way that it also knows about updates to the disk and you can have multiple file explorers open at the same time afterwards you want to add support for finding stuff in the file contents
@@IshCaudron It's true, look at the search settings: there's an option to enable content searches, which explicitly warns about performance degradation.
@@IshCaudron it's slow indeed, because the indexing feature is not applied to the entire drive. In your implementation you're basically caching the entire tree. This can also be done for the windows search, but you have to manually set it that way, because indexing comes at a cost. If you're on Windows 10 or 11, you can look at the indexing settings in the start menu.
@@minciNashu not slow, I'm saying it doesn't work. It doesn't find things on the drive even if I have an explorer window showing them. It's just broken.
@@IshCaudron file explorer search based on indexing. if your file doesn't get indexed, it won't get searched. regardless whether the file exists or not.
Wanted to say the same. Like querying your database without an index, just going from beginning of storage to the end of storage looking for whatever you're searching for lmao.
Does it also beat Everything? I am asking this because if I am right that application does raw NTFS searches, without indirectness API of the Windows operating system in between, making it practically unbeatable, unless the application in this video is doing the same thing.
Yo super cool project. BTW there's something made just for this called as "Everything" which also searches your entire file system faster than windows could ever imagine.
Yeah, he could have just written a explorer with everything integration. But his results where pretty impressive as well while utilizing a similar approach. But I would say it's not possible to search faster than everything as it literally adapts the result by the keystroke. I actually installed it on computers of all my friends when I just wanted to search 1 file as it was faster to install everything than search it in any other way. Those who understood it's power continue to use it for everything 😉
Yeah... But what if your file isn't cached? Does it just not appear? You should do an analysis of the Windows search, indexing and all. Then see where it leaps ahead.
If the file isn't found in the cache, it must be in a modified or new folder, so all you need to do is not check where you know it isn't, based on the modified folder time and the cache update time, and whether it was in the cache or not before the search. Or something like that. Comparing this to Windows search isn't really fair, as it searches file contents as well, like txt files, etc., idk.
@@johnmarston2474 this is exactly why I'm saying you should research the system you're saying you're improving. It's not fair to say you made a fast Explorer if you implemented it and missed a massive part of how it works. Windows search might be slower but it checks file contents and thus they didn't actually make the same functionality but fast.
It's pretty much unbeatable as it literally searches multiple disks at the speed of a keystroke. It's literally faster to install it and initialize it that waiting for a windows search result.
this is only practical if you are planning on searching stuff in bulk... its undeniable that the windows search is slow af but comparing it to your caching aproach is pretty like comparing apples to oranges. windows search is used after an unknown ammount of changes were made to the file system meanwhile in your approach you need to re cache everything whenever you make any change so the boot time of the program is similar to a normal windows search (please correct me if i m wrong, i have no clue what your caching times are since you didnt really show them) so yes your code is faster id you need to make 2+ searches but for what people usually use it for its pretry useles
Probably worse than windows search, as that is designed to index the files in the background when you're not active so as to be as unnoticeable as possible - enabling indexing for your entire drive is probably a much better solution
@@CorrosiveCitrus Even with search indexing on, explorer takes at least a minute to find a file even when I give it the exact name and the start menu takes 10-30 seconds, and much more if the file is in multiple sub folders. I don't know how the explorer search manages to be so bad, Everything can index my entire computer faster than explorer can find a file.
Nice work! There is also a program called search everything that runs in the background. It's whole purpose is for seraching (you can use wildcards and that stuff) and is also superfast. I am mentioning this as a piece of information, I don't mean to insult or anything.
Yep, I was going to mention this. It's a fantastic program. Even the initial run only takes a minute to cache all the filenames, and after that the searching is instant.
Yess! You can also sort, add exceptions, hotkeys, everything (see what I did here) you want! I like to leave the sort from "date modified" , with this I can see in real time what's happening in windows. It's useful for me because I sometimes just save a quick word or paint file fast and forget where it is. In this case I just pull up search everything and see what was last editet. Game changer
Хаха, да :) find находит файлы очень быстро, обходит весь мой системный диск, диск с играми и находит нужные мне файлы. А ещё слышал про fd который с многопоточность ю и работает ещё быстрее ⚡ На самом деле не смотря на то что есть GUI, мне все равно проще искать файлы, открыв терминал и используя find, он позволяет более гибко указать что тебе нужно найти
As far as I know, Windows' search is significantly slower as it also searches file metadata for supported file types. There are apps that implement pretty fast searching using indexing as well though, like voidtools' "Everything" tool is blazing fast if you give it the time to index all your drives, and it's pretty good at picking up changes in your FS almost immediately and indexing those changes.
yes!!! more rep for the best search tool and the reason Everything can pick up changes so fast is because it reads the changes journal of the filesystem directly, which you can see by doing Ctrl+` (backtick) (closing the debug window will close Everything, do the keybind on the search window again to hide the debug window)
This search is not fast because of the Rust programming language. Rather, it is fast because of the data file cache data structure you are using and the algorithm you are using to access it. Searching for strings inside a single json file will be way faster than running file search queries across a file system on a hardware disk for sure. Windows should be able to achieve similar or better results. I have no idea if the search is using the windows index or not as it can ve turned off
He just cached everything and searched there, but there could be a case when cached data is old and not up-to-date (some files/folders could be already deleted or renamed, new files could be already created at that time). So there must be some syncing that runs every n minutes/seconds. Am I right?
@@ulanaitbay afaik there are "watchers" that allow your program to know when a file/folder is modified, one could use that to add/remove cache entries dynamically
I would like you to note that the windows file explorer's search function is a little more complicated than that, as it searches through the contents of the file as well. It looks for keywords in the files, or uses certain search terms to match with the file name. It's not just a simple .includes/.contains match.
Rust developers in nutshell: I don't get how turn on Windows indexation, and it's a reason to write it by myself from "scratch" and bunch of libraries...
Windows indexing is shit. It takes loads of performance in the background and can't handle large amounts of file changes. If you try indexing an entire disc you can literally trash the pc. Use "Everything" there is literally no discussion on what the best file search is. It's Everything period.
@@redcrafterlppa303 Yep. Windows indexation a lot worse in way of metafile size per file, and content indexation by default as i know. But as "From a box solution" it's enough, especially on "modern" hardware
@@CorrosiveCitrus not if u have 20-30 tabs open with vscode and notion running alongside it. The indexation is bad specifically because my pc just can't take it with all the other thjngs going on.
Glad to know that the file search is not slow only on my computer. Honestly, how come Microsoft never fixed that? Windows after windows and it's all the same, you have to install third party software if you want to find files in your computer.
Because there's nothing to fix. It's a raw-dog search, not an indexed search. Nobody fails to understand that indexed searching is faster than unindexed searching, but indexing operations are expensive and you don't want to be running them on your system nonstop for ages. If your files are constantly changing, indexing can easily be out of date, and you then lose the benefit of indexing because you have to re-index before every search, which, surprise, is basically what windows search does.
There is a "fix", just go to your start menu and search for "indexing options" and turn on indexing for whatever folders that you want faster search in. And as @@Sammysapphira mentioned, indexing is expensive, so it will take a while for windows to build a full index in the background. But when it is done, searching in that folder is usually under a second.
@@SammysapphiraYeah if not for the literal fact that Windows already has indexing (at least partial) for other features and a boatload of "maintenance" or otherwise labeled services that spin all your drives at random, even when not needed at all for the current operation.
Enabling full disk indexing will be answer It's not enabled by default because it's taxing to the system and is supposed to I my run while you're not active, so needs to complete in a short time, but with a decent enough pc, or one you leave on, it should work fine
One improvement you could do for NTFS file system is use NTFS USN Journal. Everything already uses that and it's insanely fast! Other than that, great project, good work!
This looks like a really fun project to put together, but as others have said there are some comparisons to be made. Windows explorer works without extra disk space, works on even the slowest of drives, and takes no extra boot time even after thousands of file changes. For this program to work at full speed, you'd have to have an insanely fast and empty disk to begin a cache and be ok with it loading/flushing that cache every time. Most people who use the search feature dont have such hardware which is why Windows had to sacrifice performance for full combability with all hardware.
what happens if a file is created after the cache? does it know to recache? also i think windows search is understandably slower than this since it searches for non exact matches (like if i search for eclipse it will show me eclipse-workspace too), and it doesnt have a cache since it needs to find recent files. so basically yours was like a bit faster but then you added caching, which in my mind is kind of pointless since it constantly changes, but i guess it helps.
As was shown, substrings are also searchable. The cache doesn't change much. I believe the hybrid approach is the best in this case: first load the cache and ask the filesystem if the file still exists. If it does, congrats, you found a match. If it doesn't, delete it from the cache. And in parallel the slow search should happen which will re-evaluate the cache
@@phoenix-tt true i guess the hybrid solution is best sorry i missed the substrings being searchable this video is pretty cool though its inspired me to make my own in rust
There's a really good program called "everything" that I use to search for files normally. It's been my go-to over default windows search for years now.
Awesome solution to an interesting problem. It's also interesting how voidtools implemented their filelist index/cache in Everything. It takes the program 1 sec - few minutes to index the files initially, and then only monitors NTFS changes, not missing them even if it isn't running.
Something you could do to account for changes in the file tree is have it redo the caching when it finds nothing. Only thing you should keep in mind is to show the file that's found during the caching immediately, not after the caching is done.
"There are only two hard things in Computer Science: cache invalidation and naming things." Unlike most of the usages of this quote, this video is not about the second one
It is really funny that windows explorer is literally the slowest file explorer possible. Try deleting the js 'node_modules' folder in it vs in terminal. The difference is sometimes in minutes.
@@phoenix-tt I just use wiztree and wizfile. Literally the best file searchers/drive indexers out their. Wiztree is especially good at showing huge folders of junk that you should delete and wizfile can find any file in less than a second. Its fucking insane how a multibillion dollar company cant at least improve one of the main tools of windows instead of piling and piling more shit ui changes that can be pulled off to reveal windows xp. Im so frustrated that i rely on windows for most of the apps that i use. Linux will always be king.
@@phoenix-tt If you add the cache indexing time to every search then the speeds are almost identical. There's no change here. Caching is just a "cheat".
Yesterday I also starting write a directory file indexer library for indexing all file from sveltekit svelte build and gonna map each file content with path and serve them using actix web.... I know i could have used some library for this but still to learn rust better I started implementing this myself......
This is cool, but can it be set to just update the cache every now and then? Perhaps on the creation or movement of a file? If a file gets deleted while the explorer is running, will it notice the file no longer exists when you run a new search in the same instance of the explorer?
"so I just spent the entire afternoon looking at fancy searching libraries just to find that I can just do this with a hashmap" hashmaps are in fact so overkill sometimes it isn't even fun to use them
You don't understand, Windows cares so much about user experience that they know the average user is gonna want to go to the kitchen and grab a cup of coffee while they are using file search so they make it slow on purpose. You've taken all the fun out of searching for a file.
The problem with this implementation (as far as i know) is that if you want to keep all files findable, it is impossible to rely on the file cache alone. If you download a file and haven't cached the filesystem after that, you won't be able to find it, so you will have to cache very, very often, which would slow down the system. Another option would be to use the cache to show those results first (which would work most of the time) and then keep looking for files that match the search that are not cached, so at first it would quickly show you the files that are already cached, and after about 50 seconds show the rest (it would be an improvement compared to Windows Explorer, but in the base, in certain cases it would be the same).
Love you're work. I believe Microsoft apps do some data analyzing also, that's why it's slow, even video player loads slow. Why i'm watching this is, because i wanted to learn web programming more and i'm making my music, video player in Tauri, cause i'm so angry at Microsoft monopoly and i want to make more webapps that can be used on any OS. Like for example this file explorer is super nice.
What about cache invalidation. What if we edit file delete or create from another application like Visual Studio, will the data be updated in the cache
There are a lot of “programmers” in the comment section who have absolutely no idea what they’re talking about. My guy, what happens when your file structure changes and now your cache is invalid?
Bro make it a quad file manager that saves the state with a preview system and make it that when you click ctr+k the search box opens and you can switch local and global
It makes sense that a file explorer Witten in a modern language is gonna be faster than an ancient FE in C/C++ with no proper optimization or bugfixing. Explorer is so slow and unstable on windows 10, i rather use Teracopy or Total manager to move files between slow storage devices. Also explorer has random annoying bugs like it freezes for no reason when opening showing options for a file. I got it freeze multiple time when I wanted to open a RAW sound file in a sound player m
Search time = search time -+ indexing time- (....ummmm .... what!? 🤨) *Meanwhile:* Windows file explorer search -> searches files and directories. Rust file explorer -> searches through the cache Fair comparison imho 🤝 🚶♂️🚶♂️🚶♂️🚶♂️🚶♂️🚶♂️🚶♂️🚶♂️
The cache generation took more than the explorer search. And what if a new file appear, do you have to wait 260 more seconds? Its not faster, it just slower at the start and faster in the end.
Now write a whole OS
thank you so much
Yea, Would love to see that
Yes please!
And make it Android, GNU/Linux, MacOS, Windows, FreeBSD, IOS apps compatible
This comment gives hard ‘Do a flip!’ vibes.
You attained such speed because your "cache" is just reading a file of paths, the real and realistic time is ~50 sec like you showed since there is no cache invalidation. But hey, enough to fool most apparently
this is tik tok content stretched out to 4 minutes for no reason, every video this guy has made are superficial projects or stuff that he just gives up half way through
gpt "programmers" are easily impressed
In practice this is more useful if you're looking for multiple files though. I wish windows would keep a cache.
Sure it's a cheat, but it doesn't invalidate it being useful. It just changes the acceptable use cases
@@CatMeowMeow as I'm modifying and removing files too often it's useless for me, luckily visual studio code has a pretty powerful search system
@@gonzalologwouldn't you be able then to use windows file events to update the cache?
A file explorer is a pretty good project imo because it's easy enough on a basic level but it also can be made significantly more complicated if you want to torture yourself. Optimizing stuff like this is always very satisfying as it's what us - programmers - are paid for. I would love to see this project with a better UI, i think it's worth it.
And finally,
"wOw" - conaticus, 2023
I started playing with similar project several years ago. Now it is almost done, approaching 1.0 release... Just google for HiFile file manager... And I can confirm that it is really a lot of work if you want to have your file manager feature rich, crossplatform and supporting archiving! :)
@@vladimirkraus1438 I just searched for it and didn't find the repo. Link it, please?
@@vladimirkraus1438 but is it fast?
@@vladimirkraus1438 That file manager looks really good!! Congratulations!
yeah, the next level for search is to implement it in a way that it also knows about updates to the disk and you can have multiple file explorers open at the same time
afterwards you want to add support for finding stuff in the file contents
The explorer search does more than look at file names. It also does some content reading.
I'm not sure that's completely true, but it doesn't really matter since the search barely works.
@@IshCaudron It's true, look at the search settings: there's an option to enable content searches, which explicitly warns about performance degradation.
@@IshCaudron it's slow indeed, because the indexing feature is not applied to the entire drive. In your implementation you're basically caching the entire tree. This can also be done for the windows search, but you have to manually set it that way, because indexing comes at a cost. If you're on Windows 10 or 11, you can look at the indexing settings in the start menu.
@@minciNashu not slow, I'm saying it doesn't work. It doesn't find things on the drive even if I have an explorer window showing them. It's just broken.
@@IshCaudron file explorer search based on indexing. if your file doesn't get indexed, it won't get searched. regardless whether the file exists or not.
If you enable windows search indexing (enhanced mode), searching via explorer or start menu works amazingly fast.
Wanted to say the same. Like querying your database without an index, just going from beginning of storage to the end of storage looking for whatever you're searching for lmao.
The speed is honestly insane compared to the usual search! Great job man!
@@RadinTheFroggta 6?
Does it also beat Everything? I am asking this because if I am right that application does raw NTFS searches, without indirectness API of the Windows operating system in between, making it practically unbeatable, unless the application in this video is doing the same thing.
@@jongeduard Everything caches the data too so the result should be ~the same.
@@jongeduardit's just cache and there is no invalidation
find / | grep "your search" is faster lol
Yo super cool project. BTW there's something made just for this called as "Everything" which also searches your entire file system faster than windows could ever imagine.
i use this every day also power toys has this feature too
Yeah, he could have just written a explorer with everything integration. But his results where pretty impressive as well while utilizing a similar approach.
But I would say it's not possible to search faster than everything as it literally adapts the result by the keystroke.
I actually installed it on computers of all my friends when I just wanted to search 1 file as it was faster to install everything than search it in any other way. Those who understood it's power continue to use it for everything 😉
SSD users can just use dir.
@@TariqSajid what? did you mean powertoys run? or is there something like everything inside powertoys
@@BarraIhsan yes i was talking about powertoys run
Yeah... But what if your file isn't cached? Does it just not appear?
You should do an analysis of the Windows search, indexing and all. Then see where it leaps ahead.
If the file isn't found in the cache, it must be in a modified or new folder, so all you need to do is not check where you know it isn't, based on the modified folder time and the cache update time, and whether it was in the cache or not before the search.
Or something like that. Comparing this to Windows search isn't really fair, as it searches file contents as well, like txt files, etc., idk.
@@johnmarston2474 this is exactly why I'm saying you should research the system you're saying you're improving. It's not fair to say you made a fast Explorer if you implemented it and missed a massive part of how it works.
Windows search might be slower but it checks file contents and thus they didn't actually make the same functionality but fast.
They could do a fast search and then do a file content search, a sort of two stage search. That would make more sense for this title.
@@johnmarston2474oh so it's like that meme explaining how the missile algorithm works.
@@StiekemeHenk linux file explorers are still faster, and they also index content
What a lovely thumbnail, the group that designed it have to be quite tallented :O
100%
It's such a shame that they'll never see our appreciation
I should work on a file explorer, seems like a fun project to do. Great job on how fast it was
That's why i always use "Everything" for searching, it's aeons ahead of the default file explorer searcher
Literally the first thing I install in windows 😂
Same :)
It's pretty much unbeatable as it literally searches multiple disks at the speed of a keystroke.
It's literally faster to install it and initialize it that waiting for a windows search result.
For those that want a native solution, you can also enable full disk indexing within windows search to achieve the same thing
that's pretty awesome lol, makes me wanna do something similar
rust programmers be like: "so I found this library that does everything for me" and posts it as a video
this is only practical if you are planning on searching stuff in bulk...
its undeniable that the windows search is slow af but comparing it to your caching aproach is pretty like comparing apples to oranges.
windows search is used after an unknown ammount of changes were made to the file system meanwhile in your approach you need to re cache everything whenever you make any change so the boot time of the program is similar to a normal windows search (please correct me if i m wrong, i have no clue what your caching times are since you didnt really show them) so yes your code is faster id you need to make 2+ searches but for what people usually use it for its pretry useles
Windows search is still bad, Everything (by voidtools) caches things and updates immediately and can update the results as fast as you type
Probably worse than windows search, as that is designed to index the files in the background when you're not active so as to be as unnoticeable as possible - enabling indexing for your entire drive is probably a much better solution
@@CorrosiveCitrus Even with search indexing on, explorer takes at least a minute to find a file even when I give it the exact name and the start menu takes 10-30 seconds, and much more if the file is in multiple sub folders.
I don't know how the explorer search manages to be so bad, Everything can index my entire computer faster than explorer can find a file.
@@daviddube9215 yeah, explorer somehow manages to be slower than just brute force searching it for every new term
give the kid a break. He is learning and this is great initiative. He is prob gonna be a better problem solver than 99% of engineers.
Nice work! There is also a program called search everything that runs in the background. It's whole purpose is for seraching (you can use wildcards and that stuff) and is also superfast. I am mentioning this as a piece of information, I don't mean to insult or anything.
Yep, I was going to mention this. It's a fantastic program. Even the initial run only takes a minute to cache all the filenames, and after that the searching is instant.
Yess! You can also sort, add exceptions, hotkeys, everything (see what I did here) you want! I like to leave the sort from "date modified" , with this I can see in real time what's happening in windows. It's useful for me because I sometimes just save a quick word or paint file fast and forget where it is. In this case I just pull up search everything and see what was last editet. Game changer
This is awesome! I've thought about doing this for a while but never got around to it haha
Linux Users (like me): "Lol, we just use the terminal! And our GUI explorers are fast anyway! XD"
love dolphin
@@xvnexus8814 Dolphin is good.
Хаха, да :)
find находит файлы очень быстро, обходит весь мой системный диск, диск с играми и находит нужные мне файлы. А ещё слышал про fd который с многопоточность ю и работает ещё быстрее ⚡ На самом деле не смотря на то что есть GUI, мне все равно проще искать файлы, открыв терминал и используя find, он позволяет более гибко указать что тебе нужно найти
As far as I know, Windows' search is significantly slower as it also searches file metadata for supported file types. There are apps that implement pretty fast searching using indexing as well though, like voidtools' "Everything" tool is blazing fast if you give it the time to index all your drives, and it's pretty good at picking up changes in your FS almost immediately and indexing those changes.
yes!!! more rep for the best search tool
and the reason Everything can pick up changes so fast is because it reads the changes journal of the filesystem directly, which you can see by doing Ctrl+` (backtick) (closing the debug window will close Everything, do the keybind on the search window again to hide the debug window)
Thank you for making this file explorer! Now I wouldn't have to wait 7 minutes just to find one file anymore.
everything is a... thing and it does seraches by reading the ntfs data directly.
much faster then this.
@@v0xl just heard about it
Try enabling full disk caching for windows search, they have it disabled by default even if you have a powerful pc
This search is not fast because of the Rust programming language. Rather, it is fast because of the data file cache data structure you are using and the algorithm you are using to access it. Searching for strings inside a single json file will be way faster than running file search queries across a file system on a hardware disk for sure. Windows should be able to achieve similar or better results. I have no idea if the search is using the windows index or not as it can ve turned off
He just cached everything and searched there, but there could be a case when cached data is old and not up-to-date (some files/folders could be already deleted or renamed, new files could be already created at that time). So there must be some syncing that runs every n minutes/seconds. Am I right?
@@ulanaitbay afaik there are "watchers" that allow your program to know when a file/folder is modified, one could use that to add/remove cache entries dynamically
@@ulanaitbay You are indeed correct.
@@m.projects watching every single file on a drive is very inefficient.
Thanks!)
I would like you to note that the windows file explorer's search function is a little more complicated than that, as it searches through the contents of the file as well. It looks for keywords in the files, or uses certain search terms to match with the file name. It's not just a simple .includes/.contains match.
you also could just use "Everything" and "Explorer" together (Everything is like a super fast search engine for your files)
I was about to say this lol
Rust developers in nutshell: I don't get how turn on Windows indexation, and it's a reason to write it by myself from "scratch" and bunch of libraries...
Windows indexing is shit. It takes loads of performance in the background and can't handle large amounts of file changes. If you try indexing an entire disc you can literally trash the pc.
Use "Everything" there is literally no discussion on what the best file search is. It's Everything period.
@@redcrafterlppa303 Yep. Windows indexation a lot worse in way of metafile size per file, and content indexation by default as i know. But as "From a box solution" it's enough, especially on "modern" hardware
@@redcrafterlppa303most pc's today can handle a bit of indexing
@@CorrosiveCitrus not if u have 20-30 tabs open with vscode and notion running alongside it.
The indexation is bad specifically because my pc just can't take it with all the other thjngs going on.
Glad to know that the file search is not slow only on my computer. Honestly, how come Microsoft never fixed that? Windows after windows and it's all the same, you have to install third party software if you want to find files in your computer.
Because there's nothing to fix. It's a raw-dog search, not an indexed search. Nobody fails to understand that indexed searching is faster than unindexed searching, but indexing operations are expensive and you don't want to be running them on your system nonstop for ages.
If your files are constantly changing, indexing can easily be out of date, and you then lose the benefit of indexing because you have to re-index before every search, which, surprise, is basically what windows search does.
There is a "fix", just go to your start menu and search for "indexing options" and turn on indexing for whatever folders that you want faster search in.
And as @@Sammysapphira mentioned, indexing is expensive, so it will take a while for windows to build a full index in the background. But when it is done, searching in that folder is usually under a second.
the fix is to install "everything" and search using that instead
@@SammysapphiraYeah if not for the literal fact that Windows already has indexing (at least partial) for other features and a boatload of "maintenance" or otherwise labeled services that spin all your drives at random, even when not needed at all for the current operation.
Enabling full disk indexing will be answer
It's not enabled by default because it's taxing to the system and is supposed to I my run while you're not active, so needs to complete in a short time, but with a decent enough pc, or one you leave on, it should work fine
One improvement you could do for NTFS file system is use NTFS USN Journal. Everything already uses that and it's insanely fast!
Other than that, great project, good work!
I think putting "Everything by Voidtools" would help provide context here
Such great content. This is so cool and interesting. I need to come up with smaller project ideas that can be tackled in a few days like this one.
Amazing 🤩🤩🤩
You're absolutely inspiring and smart
Pls do more videos like this
Great video, and I found it really interesting :)) Also, you became a victim of a premature optimization with your BK-tree :P
Next: I made whole GNOME Desktop Environment but faster (in Rust)
This is actually so sick. I need this so much in my life. Please keep updating this
Look up Voidtools Everything :)
It's not really a file explorer but it definitely finds files quickly
This really inspires me to try Rust ! Thanks for sharing and good trail and error!
Microsoft, please hire this guy.
Good job lad, you got a subscriber. Hope this goes viral.
This looks like a really fun project to put together, but as others have said there are some comparisons to be made. Windows explorer works without extra disk space, works on even the slowest of drives, and takes no extra boot time even after thousands of file changes. For this program to work at full speed, you'd have to have an insanely fast and empty disk to begin a cache and be ok with it loading/flushing that cache every time. Most people who use the search feature dont have such hardware which is why Windows had to sacrifice performance for full combability with all hardware.
Great work!
you don't even know how happy I was when I saw that you're using CLion, not a lot of creators use it, but it's such an amazing IDE
This is awesome. Great stuff!
Honestly a clean FM using the Everything SDK for searching in the background would be my preference.
what happens if a file is created after the cache? does it know to recache?
also i think windows search is understandably slower than this since it searches for non exact matches (like if i search for eclipse it will show me eclipse-workspace too), and it doesnt have a cache since it needs to find recent files.
so basically yours was like a bit faster but then you added caching, which in my mind is kind of pointless since it constantly changes, but i guess it helps.
As was shown, substrings are also searchable. The cache doesn't change much. I believe the hybrid approach is the best in this case: first load the cache and ask the filesystem if the file still exists. If it does, congrats, you found a match. If it doesn't, delete it from the cache.
And in parallel the slow search should happen which will re-evaluate the cache
@@phoenix-tt true i guess the hybrid solution is best
sorry i missed the substrings being searchable
this video is pretty cool though its inspired me to make my own in rust
@@lythd Yeah, it's a cool task, good luck mate
@@phoenix-tt thank you marcel
that moment you use your whole cpu/gpu and ram to just find that 1 file on your harddrive.... i love it
And tomorrow, Spacedrive, a file manager that also uses Tauri, should finally be released. Great video and nice timing xD
cant wait the official release from you and i only need to execute the installation
There's a really good program called "everything" that I use to search for files normally. It's been my go-to over default windows search for years now.
Forgot to add that the Rust Project did not review and approve your work and does not support it.
lmfao he better get rid of the logo on the thumbnail xD
are you using jet brains? I wasn’t aware that actual people used jetbrains
Nice work!
great project
thanks for uploading videos
This is really good but you need a background thread that’s constantly updating the cache
Kind of funny that you basically recreated the unix utility “locate” (more easily googleable by it’s indexing command “updatedb”)
Wise Jetsearch is an amazing tool to search multiple drives at the same time under 10 seconds
that sounds fun, I wanna do something similar
4:08, If it took 303ms to search, imagine how much faster it would be when compiled with optimisations enabled
but that's rust what compiled optimization does it have is it like c++? were we pass flags
@@ko-Daegu rustc is llvm based, so yes, it's somewhat like c++ with clang. Yes you can pass flags to rustc through cargo
@@ko-Daegurust by default compiles with no optimizations in debug mode. Release mode it does use things like -o3 on llvm
@@sohn7767 ah thankx that's what i was actually looking for
also why the fuck all reddits are down can't get into the r/rsut communities?
@@ko-Daegu pretty big news, there's a blackout from protesting subreddits. Read about it, it's long to post here.
Awesome solution to an interesting problem. It's also interesting how voidtools implemented their filelist index/cache in Everything. It takes the program 1 sec - few minutes to index the files initially, and then only monitors NTFS changes, not missing them even if it isn't running.
Finally a project which makes use of tauri on UA-cam. We need more tauri fr
Something you could do to account for changes in the file tree is have it redo the caching when it finds nothing. Only thing you should keep in mind is to show the file that's found during the caching immediately, not after the caching is done.
"There are only two hard things in Computer Science: cache invalidation and naming things."
Unlike most of the usages of this quote, this video is not about the second one
I see the Tauri logo! Actual walking W
ur my fav youtuber now.
there are excellent file search softwares like everything from voidtools
"I Made"
> Proceeds to have chatgpt and random libraries do all the work while he just assembles the kid puzzle.
This is absolutely beautiful
Yay. Fix what Microsoft cant
It is really funny that windows explorer is literally the slowest file explorer possible. Try deleting the js 'node_modules' folder in it vs in terminal. The difference is sometimes in minutes.
@@phoenix-tt I just use wiztree and wizfile. Literally the best file searchers/drive indexers out their. Wiztree is especially good at showing huge folders of junk that you should delete and wizfile can find any file in less than a second. Its fucking insane how a multibillion dollar company cant at least improve one of the main tools of windows instead of piling and piling more shit ui changes that can be pulled off to reveal windows xp. Im so frustrated that i rely on windows for most of the apps that i use. Linux will always be king.
@@phoenix-tt If you add the cache indexing time to every search then the speeds are almost identical. There's no change here. Caching is just a "cheat".
more ctrl-z for notepad ?
Love your vids
Nice that you used Tauri! How was the DX for you?
Alright underrated channel I just found!
Yesterday I also starting write a directory file indexer library for indexing all file from sveltekit svelte build and gonna map each file content with path and serve them using actix web.... I know i could have used some library for this but still to learn rust better I started implementing this myself......
This is cool, but can it be set to just update the cache every now and then? Perhaps on the creation or movement of a file? If a file gets deleted while the explorer is running, will it notice the file no longer exists when you run a new search in the same instance of the explorer?
"so I just spent the entire afternoon looking at fancy searching libraries just to find that I can just do this with a hashmap" hashmaps are in fact so overkill sometimes it isn't even fun to use them
Dude, you inspired me into learning Rust and writing my own OS. I’ll try to at least
🤡🤡🤡
These videos are pure QUALITY!
You don't understand, Windows cares so much about user experience that they know the average user is gonna want to go to the kitchen and grab a cup of coffee while they are using file search so they make it slow on purpose.
You've taken all the fun out of searching for a file.
The problem with this implementation (as far as i know) is that if you want to keep all files findable, it is impossible to rely on the file cache alone. If you download a file and haven't cached the filesystem after that, you won't be able to find it, so you will have to cache very, very often, which would slow down the system.
Another option would be to use the cache to show those results first (which would work most of the time) and then keep looking for files that match the search that are not cached, so at first it would quickly show you the files that are already cached, and after about 50 seconds show the rest (it would be an improvement compared to Windows Explorer, but in the base, in certain cases it would be the same).
using json to store the disk cache is crazy
what would you use
@@NithinJunea custom file format.
@@rastaarmando7058 yea but surely you can see why he used json given the time limit he gave himself
Love you're work. I believe Microsoft apps do some data analyzing also, that's why it's slow, even video player loads slow. Why i'm watching this is, because i wanted to learn web programming more and i'm making my music, video player in Tauri, cause i'm so angry at Microsoft monopoly and i want to make more webapps that can be used on any OS. Like for example this file explorer is super nice.
**unsatisfied trait bounds** ahh the true rust experience
I require this searching speed
I use a search program called Everything and it's the best thing ever. Blazing fast and can find any type of file.
I use this app called everything
That's actually slow as crap compared to Everything from Voidtools which gives you search results simultaneously
A good example that windows can be optimized WAY more than it currently is, but M$ being M$ hardly gives two shits about it
So you made spotlight search
Both use a cache
What about cache invalidation. What if we edit file delete or create from another application like Visual Studio, will the data be updated in the cache
no
I will show this video every person which will say that rust is over hyped. Great work.
There are a lot of “programmers” in the comment section who have absolutely no idea what they’re talking about.
My guy, what happens when your file structure changes and now your cache is invalid?
Bro make it a quad file manager that saves the state with a preview system and make it that when you click ctr+k the search box opens and you can switch local and global
This is so cool
You're a G
wait till he finds "everything"
Configure your indexing options on windows. My laptop has the minimum requirements for windows 10 and it searches less than a second.
Best music!
"I made a file explorer in rust" repo being 60% typescript.
Bro you sound exactly like how i remember wilbur soot
"Premature optimisation is the root of all evil" - Knuth, The Art of Computer Programming
Fun fact: explorer doesn't search only files. It searches network locations, file contents and extensions to it
Nice video... But windows doesn't just search for file name it also search to data inside a txt or any readable file.
If it impacted performance that much, they should've given us the option to use the faster search algorithm.
@@ArnabAnimeshDas it does lol, just enable indexing
@@ArnabAnimeshDasfull disk indexing is your friend
Next time: Building a blazingly fast Windows in Rust
It makes sense that a file explorer Witten in a modern language is gonna be faster than an ancient FE in C/C++ with no proper optimization or bugfixing.
Explorer is so slow and unstable on windows 10, i rather use Teracopy or Total manager to move files between slow storage devices.
Also explorer has random annoying bugs like it freezes for no reason when opening showing options for a file. I got it freeze multiple time when I wanted to open a RAW sound file in a sound player m
I love tauri
Search time = search time -+ indexing time- (....ummmm .... what!? 🤨)
*Meanwhile:*
Windows file explorer search -> searches files and directories.
Rust file explorer -> searches through the cache
Fair comparison imho
🤝
🚶♂️🚶♂️🚶♂️🚶♂️🚶♂️🚶♂️🚶♂️🚶♂️
Linux find - ⚡
fd - ⚡⚡
Windows explorer - centuries :)
2:30 basically what Everything (void tools) does.
The cache generation took more than the explorer search. And what if a new file appear, do you have to wait 260 more seconds? Its not faster, it just slower at the start and faster in the end.