Some third-party file copying tools, like TeraCopy or FastCopy, allow you to adjust buffer settings and may provide improved performance over the default Windows file copying process. Command-Line Utilities: Windows includes the built-in command-line tool Robocopy, which can be used to copy files and directories with more control over the copying process, including specifying buffer sizes. The /MT switch allows you to use multi-threading, which can have an impact on buffer usage.
It’s worth mentioning that write speeds are slower than read speeds, sometimes a lot slower. The process of reading proceeds apace, but when it comes to writing, it slows down, during which time the buffer is still full and reading cannot continue. One other thing that is often forgotten is that theres is a fixed, and significant, amount of time taken to copy each file, regardless of its size. Therefore, copying thousands of small files will proceed much more slowly than the same amount of data held in a few large files.
This was an somewhat common issue with WIN. Pre vista. Never experienced with win 7 or newer. Could be an issue with faulty memory or a drive that is close to failure.
i think that's my problem a faulty memory *ram* because i used to copy files all the time as big as 20gb and it gets done in 10 mins tops but now i've been experiencing some crashes with games and bluescreens and what came to my mind that it was the memory because i updated everything the gpu and mb and cpu except for my rams they r old i bought them in 2018
how could i let windows use more ram for file copy or network copy got on my file server around 20gb ram , and ustileze around 5 some time go up to 6 ,
Any activity that involves accessing files on disk, one should consider the time taken by the security software to analyze the file being copied. Nothing happens in a windows computer without the eye of the "Big Brother is watching you".
if you have a Seagate HDD use sea tools to check for a lot of errors WD has their program as well all vendors do that will pull up the logs from the disk controller
My copying of files between 2 external USB 3.0 hdd was at 70MB/s in the morning for hours. But now copying files again it slowed down to 7MB/s. Granted they are both encryption based. But don't know what the issue is exactly.
Hi Very well explained My daily real life experience is as follows My laptop has a D-Drive partition, this is were i work all day End of the day ( if i remember ) i grab the entire d: and copy it to the server in \\Accounts \George If i use windows explorer it takes close to 15 minutes ( about 4g data ) Now at the end it asks to copy pre existing files - i say No It them will finish in 20 seconds Now i use FastCopy program, it prechecks the copy attributes and copies only the altered files Understandably it takes under 1 minute as the actual files transferred will be about 20-30 and 500meg So it is not just the number of files and what it skips, the Fastcopy program uses multiple cores ( possibly 4 i dont remember ) So even it i turn of the write attribute and have all files marked as new , the entire 4g transfer is relatively fast 3/5 minutes - much faster that the windows explorer copy speed Also to note, transferring 1 file = 100 meg if faster than transferring 100 files at 1 meg
the otherthing that can slow cping large files is you not being on a 5ghz Wifi network that is not wireless AC or higher like AX or known as wifi 6 and 6E make sure your router is wifi6 and computer wifi adapter as well otherthing that can cause this issue is not haveing updated chipset drivers and wifi drivers outdated more than a year and motherboard firmware as well can cause this issue too if using data update the data drivers too
Crab copy Programms like win Explorer, I fergotten what the name of a older win program was that copyed everything without problems for free, was is carbon copy or so 🤔
I copied some huge video files to my TrueNAS server. Once that had finished, I waited for a while then tried to delete the originals on my PC. It keeps reporting that "it is still in use", but the server only ever reports back a finished transfer to disk. Perhaps Windows is at fault.
The user gave no information about his hardware. But it sounds like he has a configuration issue, due to him needing to restart his computer. 1) For mechanical drives, files being written to (or read from) the outer portion of the disks will be far faster than the inner portion of the disks. For example: An empty drive will probably write at ~275 MB/s, and when it is nearly full, might drop to ~135 MB/s. The outer portion of the drive simply has more platter real-estate with each rotation, resulting in faster performance for files on the outer portion of the drive. 2) SSDs have deceptive manufacturer advertising, and even many reviewers either do not know, or they intentionally leave out an important design aspect of SSDs when providing reviews. Those high performance numbers you see on the box (or the web site that is selling the SSD) are for (let's call it) the caching section of the SSD. Without exception (as far as I know), 100% of consumer level SSDs are comprised of two types of NAND cells (which is where your data gets stored). And different types of NAND cells will perform at wildly different speeds. The manufacturers use (it varies from model to model) fast (more or less) NAND cells for approximately 10% of the SSD's storage capacity. The rest of the SSD is comprised of slow NAND cells. The slow NAND cells are inexpensive, which is why ~90% of the SSD will be comprised of the slow NAND cells. And good luck finding how an SSD is comprised when scouring the manufacturer's web site, white papers, technical notes, etc. They bury that information. When you write to the SSD, it will always go to the fast section. So you will virtually always experience the drive's top speed. When the SSD is not busy (which is most of the time), its controller moves the data to the slower portion of the SSD (you will not see that happening) -- resulting in always having free space in the fast portion of the SSD. If you want to see just how slow a supposed fast SSD can be, write enough data to it, non-stop. When you fill up the fast portion of the SSD, the drive will have no choice but to write data directly to the slower NAND cells, and you will (depending on the SSD model) likely see the performance drop like a brick. You could go from 3 GB/s to 100 MB/s. When you see Crystal Disk's benchmarking results, they typically involve using 1 GB of data. That will always test the fast, and only the fast, NAND cells of the SSD. Change it to use 250 GB of data, and when you wake up the next day, you will likely see lousy performance results. For 99.99% of users, the above will never be an issue, as few of us ever write enough GBs of data, non-stop, to experience such a slow-down. If you want an SSD that will not slow down much, there are a few high-end, consumer level drives that do a very good job, no matter how much data you throw at them: -- Samsung 990 Pro (also the 980 Pro, if you find one). -- Western Digital Black SN850X (although if you ever need warranty service from WD, you are in for the ride of your life). -- SK hynix Platinum P41. There are Enterprise / Data Center level SSDs that are made 100% of the fast stuff. They will never slow down. They will also break the bank. If you are interested in doing further research, NAND cells come in these verities (from fastest and most expensive, to slowest and cheapest): -- SLC (single layer cells) -- also the most robust. -- MLC (multi layer cells (not sure why they did not name it DLC for dual layer cells)) -- TLC (triple layer cells) -- QLC (quad layer cells) -- also the least robust. The SSD's controller also plays a critical role in the drive's performance. 3) I mentioned the above, because the user's question pertained to large file copies slowing down. But that user has either a hardware issue or a configuration issue. -- Are his file copy issues limited to a specific drive? If yes... has he tested it in a different computer? -- Does he have a spare drive that he can connect to his main computer (that has the issue), to see if the spare drive also has the issue (and if it does, then the problem is with the computer). -- Backup his computer, and do a fresh OS installation. If the problem is no longer there, then he had a configuration issue (maybe a driver issue). Restore his computer from his backup, and happy hunting for the configuration issue. -- Boot to a live Linux distro, and do a large file copy. If all goes well, then it is a configuration issue with his normal OS.
I'm trying to copy a 15 gig file to a 3.0 flash drive. It starts out fast then slows down to like 5 megs a second. I even went out and bought a brand new flash drive just to see if it was my old flash drive, nope, it still gets down to 5 mps. Lame...oh dam, it just went down to 4 megs... talk about usb 1.0 blah...
Some third-party file copying tools, like TeraCopy or FastCopy, allow you to adjust buffer settings and may provide improved performance over the default Windows file copying process.
Command-Line Utilities: Windows includes the built-in command-line tool Robocopy, which can be used to copy files and directories with more control over the copying process, including specifying buffer sizes. The /MT switch allows you to use multi-threading, which can have an impact on buffer usage.
It’s worth mentioning that write speeds are slower than read speeds, sometimes a lot slower. The process of reading proceeds apace, but when it comes to writing, it slows down, during which time the buffer is still full and reading cannot continue.
One other thing that is often forgotten is that theres is a fixed, and significant, amount of time taken to copy each file, regardless of its size. Therefore, copying thousands of small files will proceed much more slowly than the same amount of data held in a few large files.
Particularly if you are writting to a SMR harddisk. It is slow.
This was an somewhat common issue with WIN. Pre vista. Never experienced with win 7 or newer.
Could be an issue with faulty memory or a drive that is close to failure.
i think that's my problem a faulty memory *ram* because i used to copy files all the time as big as 20gb and it gets done in 10 mins tops but now i've been experiencing some crashes with games and bluescreens and what came to my mind that it was the memory because i updated everything the gpu and mb and cpu except for my rams they r old i bought them in 2018
Many possibilities.
how could i let windows use more ram for file copy or network copy got on my file server around 20gb ram , and ustileze around 5 some time go up to 6 ,
Any activity that involves accessing files on disk, one should consider the time taken by the security software to analyze the file being copied. Nothing happens in a windows computer without the eye of the "Big Brother is watching you".
if you have a Seagate HDD use sea tools to check for a lot of errors WD has their program as well all vendors do that will pull up the logs from the disk controller
My copying of files between 2 external USB 3.0 hdd was at 70MB/s in the morning for hours. But now copying files again it slowed down to 7MB/s. Granted they are both encryption based. But don't know what the issue is exactly.
Hi Very well explained
My daily real life experience is as follows
My laptop has a D-Drive partition, this is were i work all day
End of the day ( if i remember ) i grab the entire d: and copy it to the server in \\Accounts \George
If i use windows explorer it takes close to 15 minutes ( about 4g data )
Now at the end it asks to copy pre existing files - i say No
It them will finish in 20 seconds
Now i use FastCopy program, it prechecks the copy attributes and copies only the altered files
Understandably it takes under 1 minute as the actual files transferred will be about 20-30 and 500meg
So it is not just the number of files and what it skips, the Fastcopy program uses multiple cores ( possibly 4 i dont remember )
So even it i turn of the write attribute and have all files marked as new , the entire 4g transfer is relatively fast 3/5 minutes - much faster that the windows explorer copy speed
Also to note, transferring 1 file = 100 meg if faster than transferring 100 files at 1 meg
the otherthing that can slow cping large files is you not being on a 5ghz Wifi network that is not wireless AC or higher like AX or known as wifi 6 and 6E make sure your router is wifi6 and computer wifi adapter as well otherthing that can cause this issue is not haveing updated chipset drivers and wifi drivers outdated more than a year and motherboard firmware as well can cause this issue too if using data update the data drivers too
Sir is it bad to fill up the hard disk nearly full if its only for a buckup drive?
Nope. As long as your backups continue to work.
@@askleonotenboom thank you very much sir 👍
Pulling files is always faster than pushing files. Avoid the GUI file copy for performance gain' get to the command prompt.
Crab copy Programms like win Explorer, I fergotten what the name of a older win program was that copyed everything without problems for free, was is carbon copy or so 🤔
I copied some huge video files to my TrueNAS server. Once that had finished, I waited for a while then tried to delete the originals on my PC. It keeps reporting that "it is still in use", but the server only ever reports back a finished transfer to disk. Perhaps Windows is at fault.
This technique may help: askleo.com/how_can_i_find_out_who_is_using_a_file_in_use/
The user gave no information about his hardware. But it sounds like he has a configuration issue, due to him needing to restart his computer.
1) For mechanical drives, files being written to (or read from) the outer portion of the disks will be far faster than the inner portion of the disks.
For example: An empty drive will probably write at ~275 MB/s, and when it is nearly full, might drop to ~135 MB/s.
The outer portion of the drive simply has more platter real-estate with each rotation, resulting in faster performance for files on the outer portion of the drive.
2) SSDs have deceptive manufacturer advertising, and even many reviewers either do not know, or they intentionally leave out an important design aspect of SSDs when providing reviews.
Those high performance numbers you see on the box (or the web site that is selling the SSD) are for (let's call it) the caching section of the SSD.
Without exception (as far as I know), 100% of consumer level SSDs are comprised of two types of NAND cells (which is where your data gets stored).
And different types of NAND cells will perform at wildly different speeds.
The manufacturers use (it varies from model to model) fast (more or less) NAND cells for approximately 10% of the SSD's storage capacity. The rest of the SSD is comprised of slow NAND cells.
The slow NAND cells are inexpensive, which is why ~90% of the SSD will be comprised of the slow NAND cells. And good luck finding how an SSD is comprised when scouring the manufacturer's web site, white papers, technical notes, etc. They bury that information.
When you write to the SSD, it will always go to the fast section. So you will virtually always experience the drive's top speed.
When the SSD is not busy (which is most of the time), its controller moves the data to the slower portion of the SSD (you will not see that happening) -- resulting in always having free space in the fast portion of the SSD.
If you want to see just how slow a supposed fast SSD can be, write enough data to it, non-stop. When you fill up the fast portion of the SSD, the drive will have no choice but to write data directly to the slower NAND cells, and you will (depending on the SSD model) likely see the performance drop like a brick. You could go from 3 GB/s to 100 MB/s.
When you see Crystal Disk's benchmarking results, they typically involve using 1 GB of data. That will always test the fast, and only the fast, NAND cells of the SSD.
Change it to use 250 GB of data, and when you wake up the next day, you will likely see lousy performance results.
For 99.99% of users, the above will never be an issue, as few of us ever write enough GBs of data, non-stop, to experience such a slow-down.
If you want an SSD that will not slow down much, there are a few high-end, consumer level drives that do a very good job, no matter how much data you throw at them:
-- Samsung 990 Pro (also the 980 Pro, if you find one).
-- Western Digital Black SN850X (although if you ever need warranty service from WD, you are in for the ride of your life).
-- SK hynix Platinum P41.
There are Enterprise / Data Center level SSDs that are made 100% of the fast stuff. They will never slow down. They will also break the bank.
If you are interested in doing further research, NAND cells come in these verities (from fastest and most expensive, to slowest and cheapest):
-- SLC (single layer cells) -- also the most robust.
-- MLC (multi layer cells (not sure why they did not name it DLC for dual layer cells))
-- TLC (triple layer cells)
-- QLC (quad layer cells) -- also the least robust.
The SSD's controller also plays a critical role in the drive's performance.
3) I mentioned the above, because the user's question pertained to large file copies slowing down.
But that user has either a hardware issue or a configuration issue.
-- Are his file copy issues limited to a specific drive? If yes... has he tested it in a different computer?
-- Does he have a spare drive that he can connect to his main computer (that has the issue), to see if the spare drive also has the issue (and if it does, then the problem is with the computer).
-- Backup his computer, and do a fresh OS installation. If the problem is no longer there, then he had a configuration issue (maybe a driver issue). Restore his computer from his backup, and happy hunting for the configuration issue.
-- Boot to a live Linux distro, and do a large file copy. If all goes well, then it is a configuration issue with his normal OS.
So how to fix
Depends on the issue. See the video.
I'm trying to copy a 15 gig file to a 3.0 flash drive. It starts out fast then slows down to like 5 megs a second. I even went out and bought a brand new flash drive just to see if it was my old flash drive, nope, it still gets down to 5 mps. Lame...oh dam, it just went down to 4 megs... talk about usb 1.0 blah...
Why talk about mechanical drives its obvious why
The real question is about SSDs