Вставка
- Опубліковано 1 жов 2024
- SD cards in Raspberry Pi computers wear out and die after a while because they are not made for computer usage, and many write cycles. In addition, the SD cards can become corrupted if the Pi loses power without a proper shutdown. We have several possibilities to avoid this disaster. Only one is simple and cheap. But, unfortunately, it has its caveats. So let’s get an overview and use a nearly unknown Raspbian feature to help us solve the problem.
Links:
Raspberry Pi 4: amzn.to/3kBF5nv
Barefoot method 1: / make-your-raspberry-pi...
Barefoot Method 2: www.raspberry-...
Adafruit's method: learn.adafruit...
The links above usually are affiliate links that support the channel (no additional cost for you).
Supporting Material and Blog Page: www.sensorsiot.org
Github: www.github.com...
My Patreon Page: / andreasspiess
Discord: / discord
If you want to support the channel, please use the links below to start your shopping. No additional charges for you, but I get a commission of your purchases to buy new stuff for the channel
My Amazon.com shop: www.amazon.com...
For Banggood bit.ly/2jAQEf4
For AliExpress: bit.ly/2B0yTLL
For Amazon.de: amzn.to/2r0ZCYI
For Amazon UK: amzn.to/2mxBaJf
For ebay.com: ebay.to/2DuYXBp
www.facebook.c...
/ spiessa
www.instructab...
Please do not try to email me. This communication channel is reserved for my primary job
As an Amazon Associate, I earn from qualifying purchases
#no#midroll#ads
there is another option using write-behind caching on EXT4 so it caches all the writes to ram and then using the COMMIT=xx seconds mount option it will write the changes from ram to disk every so often, the best of both worlds
Good point! And it seems to be easy. Similar to log2RAM, I think.
It doesn't ensure that writes will be cached to ram though. Just that it will cache up to "commit" seconds of writes before automatically syncing the filesystem so if your system suddenly crashes, you know that you have lost at worst this amount of writes. Applications can request their writes to be immediately recorded to the disk/SD Card. Applications who use databases (such as SQLite) or unbuffered logs will do that frequently.
@@_gawen i forgot to mention, that its a setting to enable journal_data_writeback using "tune2fs" on ext4 filesystems, as well as "nobarrier". this will run it all to ram first at the cost of data security and reliability, obviously
I have a pi3 running on the same sd card I started with, probably 6 years of running 24-7 with not a single issue. Pi4, has already burned 3 sd cards up in less than 2 years.
Can't for the life of me remember what commands I did to limit the card writes, or whatever I read at the time, but it's worked. Wondering if this is it...
I was able to reduce my sd card consumption drastically by logging to my remote homeserver. (almost) No local logs anymore. There is another advantage in remote logs: you can find suspects, even after the device in question died. Since there is a homeserver up and running 24/7, it was just a matter of some simple configuration.
Remote logging seems to be an interesting alternative (or extension) to this concept. Maybe I have to try it too...
@@mixxx2005 For anyone considering this: make sure to put security in your network config. Unless you want a random device in the wild to give an attacker access to the files on your home servers/computers.
Have any details on how you did this? I'd love to do the same.. have an NFS share or similar on a pre-existing NAS that would support logging from all of my various PIs, which could then run completely read-only.
@@SteveJones172pilot I enabled one of my Linux hosts to accept remote syslog messages, and enabled all clients to log to this remote logging facility. This is quite straight forward. How you have to do i depends on your software, of course.
SD cards are not by definition unreliable. They come in different flavors for different use cases. If you pick the right one, an SD card can be more reliable than a big cheap SSD.
The most important criteria are the cell technology used and the Bill of materials.
Cell technology:
* Most consumer cards use TLC or QLC (3D) cell technology. A single cell can typically be rewritten approx 200 to 300 times.
* Some consumer cards advertise with 'long endurance' or 'industrial' terms. Often such cards use MLC cell technology (but this is not guaranteed!). In this case a single cell van typically be rewritten approx 2000 to 3000 times. This type is also often used for EMMC memory and modules.
* For more demanding environments (automotive, medical, extreme conditions,..), often (p)SLC cards are used. For these type of cards, a single cell can be typically rewritten up to 30000 times. That's >100 times more than a typical consumer card!. You will not find these cards in a regular (online) shop. Cards utilizing this technology often have more advanced flash controllers onboard providing additional security and features compared to consumer grade cards (e.g. power failure protection, better wear leveling...).
Bill Of Materials:
* Most consumer cards do not have a fixed Bill Of Materials (BOM). This means the internal components can change without notification and the manufacturer will choose whatever is available and the cheapest that complies more or less with the specifications. So for one batch of cards, the cells can be TLC, and a few months later, QLC becomes cheaper and the manufacturer switches to QLC components, or uses another flash controller... This is also the reason why almost no manufacturer will give the complete specifications of the used components for these cards.
* Most manufacturers also offer fixed BOM cards. These are cards where the used components are fixed and will not change without notice. This guaranties that if you buy the same card next year, it will behave exactly the same as the ones you already have.
An example of a 'cheap' (p)SLC card with fixed BOM is the SDSDQED-016G-XI model by Western Digial/Sandisk.
It is a 16GB SLC card with an extended temperature range (-40+85C) and an endurance of 480TBW. It costs about 30 euro's at Mouser. This is typically cheaper than a usb->sata/M2 adapter + decent SSD.
For comparison: a Kingston A2000 SSD of 500GB has an endurance of 360TBW. And if you only look at smaller very low cost ssd's, then the endurance is often 40 TBW)
So for projects that do not depend on very high speed disk access or require a lot of storage, such SD card is unrivaled. Both for price and reliability.
(The explanation above is a bit simplified... there are other factors playing also a role, but this comment is already long enough for now :-)
Thank you for bringing light into the dark of the SD cards! So far I have only used SD cards from online suppliers. Good to know other variants exist.
Concerning SSDs: Even my home automation server only writes MB and not GB because I store sensor data and not videos. So with my Raspis, I can live with very low TBW values (other than with my UA-cam production machine ;-) ) I also only use the smallest SSDs I can get because the volume is small (
Best comment ever! Thanks.
I see that there is also a SDSDQEC with up to 3200 TBW. Search for "brochure-removable-flash-storage.pdf" for a good summary of current offerings.
I use RAMFS and mount it over the top of the log directories. Then, when a program writes to the log files in the log directory they are intercepted by RAMFS instead. When the max size is reached they are discarded. They are never written to the SD card. My SD card is in its 4th year of operation (as a LoRa gateway) with no problems so far.
I never used RAMFS, only log2ram which seems to have a similar strategy. This is another valid way to solve this problem.
I have a bunch of raspi 3's running on the field and they have been working nice for the past 4 years with this tmpfs trick. A couple of sd-cards have became corrupt during this time, but I suspect its due to power outages, since after dd'ing they seem to work ok again.
I would love to see any scripts or configs you have to accomplish this.. Sounds cool.. Also sorry you're "not" mark Knopfler.. it would be cool to be Mark Knopfler! :-)
Can you write the command for setting this up?
I have found this:
sudo nano /etc/fstab
And added
tmpfs /var/log tmpfs defaults,noatime,nosuid,mode=0755,size=20m 0 0
Is this correct or are there other directories?
Thx.
YES! Finally I can cut the power to my rasppi whenever I want without the fear of destroying the SD-Card!
True. I like this, too!
I tried it with hdd. I've had to re-set it up twice. 😒
So, why isn't there an option to write the ram files back to sd on close? Thus dramatically reducing the writes
My guess is it’s too hard to guarantee that. If people expect their data to be saved when the pi reboots then they will be angry if 100’s of megs get erased when it looses power or crashes. If the system expects files to be written on shutdown then the file system would be corrupted on ever single unexpected reboot. It’s much safer to just assume all data is temporary in the RAM disk.
@@CodeMonkeX Yes I see if the rewrites were automatic, there could be a problem. My thoughts were for a Manual/programmatic option - a simple 'do you want to save' or cmdline save
I am sure a lot of things can be done. But, as a Linux noob I am happy to have a simple way. All my use cases can be appointed to one or the other scenario: No longterm storage needed --> SD card or longterm storage needed --> SSD
Interesting question. That would be a nice option. I'm all about those SSD drives because of the speed. I only use SD cards for the portable builds like my hotspots.
In some way a similar functionality can be achieved with "apt install log2ram". If I'm not mistaken "log2ram" is installed by default in armbian-os, on raspberry-os we can install it ourselves.
I use a different approach, and I find it much simpler and scalable: I use a NAS, so shared directories to save logs and data. So I get automatically backups (I need to set up only once for the base directory in the NAS), fast disk (but it depends also on network), cheap disks. And I'm able to split or merge services and rpi much easier.
Storing the files on a NAS is for sure a good thing for many applications (if you own one).
When a Raspberry is booted without connection to the NAS (e.g. NAS is down, or no network connection), the Raspberry runs without NAS connection forever! No retries to connect later.
Solution: autofs mounts the NAS directories when needed, and releases the connection after 5 minutes when not in use, so this also costs less memory. Thus a retry is provided after connection-loss.
When speed is not needed, you can make a NAS from a Raspberry and 2 USB disks (even USB-thumbs) as RAID-1 using mdadm.
@@AndreasSpiess It is super easy to make your own NAS, just install Linux on a machine (could be another raspberry pi) and configure a RAID 5 (preferably not on USB, which is too unreliable for it, as it can disconnect randomly from time to time, which breaks the RAID and makes it go into degraded mode) and then share certain folders via NFS, very easy. The advantage is that you don't have a proprietary NAS like Synology that is both inflexible, hard to fix when it breaks and expensive and often uses a non standard proprietarily modified Linux.
I have a NAS as well and I prefer using it as well. I just need to get 1 that's a few teraflops my problem is the $.
@@PvGeens Sounds like I will need to be careful with doing that.
Thanks Andreas for the tip. What I do is to use a high endurance SD card, specially designed for high number of writing cycles ( they are used in security cameras). Is a very lazy solution though.
I hope this works (because I ordered a few of them after comments of other viewers...)
@@AndreasSpiess I personally have been lucky using Samsung pro endurance in Rpi for a while after testing them 1+ year on dashcam. Only 1 out of 10 have made one file corruption (as far I have noticed). Before that the Sd cards was not saving all the files or saving them empty. Just a note, that bigger sizes have bigger large ratios before failure.
Other option is to add some USB drive, which comes in micro size, so fairly extends out of devices itself
@@AndreasSpiess Nothing is failure-free. I used to work with MDVR (mobile video recorder) and moved from a Sandisk Ultra type SD to a SanDisk high endurance. From a batch of 20 units, we used to have one SD dead every 2 weeks; after the change the first failure happened 6 months after. Using Transcend High Endurance SDs in three Raspi working as LoRa Gateways for one year, no problem so far.
@@jdsan6009 Thanks for the info about Transcend SD card. I will try that one next time. Currently using SanDisk 128GB High Endurance in my 4K dashcam. Working fine so far. My old 64 gig SanDisk SD card worked fine for 2 years before I replaced it. I always get the biggest SD card that the device can support to extend the life out of it.
@Andreas Spiess. You will like to know Tesla uses industrial type SD Cards to save logs. Here is the reference ua-cam.com/video/ODdIRr5RzI8/v-deo.html
It is also very useful to mount the filesystems with noatime,nodiratime options in order to avoid updating access time in the file and directory attributes each time the file or directory is accessed, no matter it was just read.
Another viewer also mentioned noatime. I am not aware of this feature. So I have some reading ahead ;-)
@@AndreasSpiess This is how I use these mount options, on each and every extX filesystem which resides on SD, SSD or CF on my machines.
root@raspberrypi:~# cat /etc/fstab
proc /proc proc defaults 0 0
PARTUUID=60650d97-01 /boot vfat defaults 0 2
PARTUUID=60650d97-02 / ext4 noatime,nodiratime 0 1
@@AndreasSpiess I just noticed on the mount(8) man page that nodiatime is implied if noatime option is used. There is always something new to learn. From the mount(8) man page:
noatime
Do not update inode access times on this filesystem (e.g. for
faster access on the news spool to speed up news servers).
This works for all inode types (directories too), so it
implies nodiratime.
This came just as I needed it!
Releasing 24 dashboards driven by raspberry pi's next week and was worried about configs being changed and other (python) code config that needs to write to the memory! Much appreciated Andreas!
Indeed this should prevent bad things from happening.
I still prefer to use an ssd. You can buy a 128 GB M.2 disk for 20-30 € and a small enclosure for a similar amount. On the other hand I don’t have Scottish ancestry….
For me it is mainly about the form factor. As you write, the prices are ok these days.
I also tend to install ZRAM on every pi I own. It removes the need for swap space on the sdcard and it allows you to fit about 50% more stuff into the physical available RAM.
I never used it because I only have 2GB and bigger Pi4. So memory is no more a big problem for me. Probably good for a Pi zero if it does not consume a lot of CPU.
@@AndreasSpiess I even use it on my rpi 8gb model. Sometimes I notice raspian swapping stuff out even with minimal applications running and plenty of main mem free.
@@misteragony For some reason I think the default "swappiness" value is a bit too high in raspbian (how likely the os is to use swap). I adjusted it to 10 from its default 60 and have only noticed minimal swap usage since then, but it is still there for emergencies when you are running close to the limit (I have also done this on my laptop with similar results).
I've usually mitigate writes to the sdcard by making sure the noatime and nodiratime option is on the sdcard in the fstab and mounting a small flash drive to the /var directory, and using a larger flashdrive for the home directory (you could put both partitions on one flashdrive). This should eliminate all writes to the sdcard except when installing and updating software. Not on PIs, but on PCs, I frequently use a virtual disk and keep my work on flashdrives, and have scripts with clickable icons to setup, load, and when my session is done, save and destroy the virtual disk.
Thank you for sharing how you do it. Virtual environments are also a good thing if you want to protect the original.
Sometimes, at good points, I hit the save work icon, and it saves only the files that have changed, using rsync, and I continue on.
After 2 SD cards busted, I put an SSD for my Home Assistant RPi4 and it is running since then very reliable, hope I don't jinx it. But very good tip to make the SD card last longer.
I also use an SSD on my server. Also for speed...
Very useful video, Andreas👍- this setup is indeed ideal for 24/7 applications that do "something" and send the results to a remote server or such. Or another idea: Run Minikube on the Pi4 and push containers to it from remote. That way the Pi would become application-agnostic?
So far I never used Kubernetes. Maybe a project for winter ;-)
It's a complete BS that there is still no integrated M.2 slot on pi5.
M.2 slot is also much better way of exposing pcie lane for other devices, and there is already a large market of M.2 to 100 different things, like 6x sata ports, risers for huge GPUs and lastly can conform to gen3 standard np (dodgy fpc connector they used is guaranteed only for gen2)
The Pi5 seems to have that possibility.
@@AndreasSpiess via hat yes I mean built in
Instead of a SSD, you can also use a network filesystem and redirect things there through symbolic links. Handy if you have a NAS on the same network. Beware that SAMBA/CIFS does not work well for that and some services will randomly fail if their data are put on a SAMBA mount. NFS on the other hand works fine with everything.
You can also use the iotop command for instance 'iotop -o -b -a' to log the cumulative writes of the different processes so you can see who is writing and how much.
Thank you for your input. Remote logging indeed would be a possibility. Maybe even with these switches (if you want to keep your logs)
Add commit=300 as a mount option in /etc/fstab. EXT4 default is to commit all writes every 5 seconds. If you mount with commit=300 it will cache writes in memory then write them to disk once every 5 minutes. You can reduce your SD wear by 60x by adding that line.
Good point. Thank you. I will try it
Years ago, my daughter had been using my core 2 duo laptop running ubuntu 12LTS booting(with pre-loaded apps) from the dvd drive loading it on ram for several years till the internal cooling fan conked out. You could still save on the hard drive but everything else resets after shutting down.
Others also suggested that the overlay FS was used for USB stick booting which was probably similar to your concept. So nothing new, but still useful and now integrated into Raspbian (good for noobs like me)
The overlay technique is actually pretty old. They used it for bootable Linux CD-ROMs back in the day.
Good to know. And now they made it simple to use it.
With the newer Raspberry Pi 4 with the higher RAM this overlay makes a lot more sense now. I guess you could use read/write Boot drive and periodically copy the overlay to the SD or to an external network drive for data that may need to be kept longer term. Thanks for the great information.
You are right, many combinations are possible if you include network disks in your setup.
Thanks Andreas, I was worrying about my sd card in. my headless Raspberry Pi 4 recoding NOAA weather images 24/7 so I made images of the card if it fails, I haven't had any card fail yet but you saved the day and me worrying, thanks for this tip will implement it : )
As said: A backup is always good! But your application probably is a good use case to reduce the chance for failure.
The easiest trick to protect your filesystem on the sd card is using a usb stick.
In my opinion using an sd card for the first pis was a terrible design decision
I do not know if USB sticks have wear levelling because they are made for the same purpose as SD cards. Do you know?
I want to download new app on sd card how can I do it? Or I want to transfer old apps to sd card automatically how can I do it?
We do not have apps on a Raspberry. So I do not understand.
@@AndreasSpiess -_-
Clever ways to save on writes :) Phones uses F2FS file system to save on writes. F2FS supports ZSTD compression. Image with bear minimum of applications would write less compared to stock Rasbian.
There are MLC SD cards that last ~10x at same capacity compared to most micro SD cards in the wild. Jet MLC cards cost ~4x (on SSDs the price difference is more reasonable ~2x). Higher capacity cards will last longer (grows lineary with the size). As for disconnecting the drive unsafely - I had luck with usb sata bridge and microSD readers, they usually go before the storage.
Thank you for the additional info!
Ooooo DMR radio hot spot. What radio chat room # do you haunt? usually 93 or scanning is what i'm doing.
I am on our Swiss TGs plus the Redit TG.
@@AndreasSpiess So 228 and TG950 .
228 is only used by non-Swiss guys calling Switzerland. Very quiet TG. 2280 / 2284 are the ones used. Reddit is now on 98003
What would be the difference when leaving the boot partition on read/write while having overlay enabled?
I never thought about that :-(
after the loss of 2 SDcards, ive switched to using a USB Stick in my Pi, worked sofar without issues.
Thank you for sharing your experience.
Another astonishingly simple option is to use a bigger SD cad. Let's say that your card is 80% full, then there are 20% storage/repair capacity left. If you chose an SD card with twice as much capacity, you'll get 120% storage/repair capacity left relative to your original one. This leads to a 6 times as much lifetime of the SD card as before.
I agree with Jan. But bigger cards usually also are faster (at least in the past).
@@AndreasSpiess
Wear-Leveling If Not done by Hardware is a Problem a OS should handle... But Looks Like they dont.
Maybe writte a Script to do monthly a copy of the Files that have IO writtes and rename the old File to dead Disk space +timestamp
If i build a System i Always keep more then 50% Disk free
And If the Disk get to 20% free i search for Upgrade Options.
Have a nice Day
And greatz from Germany
opo
In addition to these tips I would strongly recommend getting an SD card that's more suitable for this kind of usage. I've personally had good results with the WD purple SD cards, I've got 3 pi's that have been running 24/7 for about 3 years now on an unmodified raspbian installation with full logging enabled (and one of them is constantly writing due to a pihole installation). So far I haven't had any issues. But... I would recommend partitioning only a small part of the SD card (i.e. 25% or so) so the wear leveling algorithms can do their thing and make sure your SD card won't run out of working blocks.
Thank you for sharing your experience. Because other viewers also recommended these WD purple cards a few are in the mail by now. I hope they implemented wear levelling. The descriptions I found were not clear. Just that they monitor "bad cells".
@@AndreasSpiess I've read on reddit that they are supposed to have wear leveling according to WD suport. Anecdotally however, I've had many SD cards fail before I switched. Even high quality ones
I do not believe having a small partition would make any difference to the wear leveling since it uses all the cells to write data regardless what file system you use. The idea is get the biggest card the device can support to extend the life out of it.
@@Darkk6969 You're right, assuming you have TRIM enabled it won't make any difference in that aspect. But if you only partition a small part of the disk you leave the SD card with enough space to keep wear-leveling without any worries and you can't accidently fill up the disk with logs or other files. Given how cheap memory cards are and how much space you actually need, this has been a good tradeoff for me.
Sandisk do a range of rugged IoT targeted SD cards using more robust flash. Makes sense.
IIRC, OpenWRT uses OverlayFS, but their implementation is pretty static without lots of hackery. Really neat the Pi Foundation was able to simplify it down to a toggle switch! 👍
As a Linux noob I was not aware of it until it was simplified ;-)
Hi. Can I do this with twister OS? This OS is so beautiful. I ve seen a review but I want to follow your advice. If Twister OS has this set up for overlay, would be great.
I only use Raspbian:-(
Just mount the folders like log, tmp and stuff into a RAM Disk mounted Filesystem
For something like that there's log2ram in deb-repository
AFAIK this is done by the "barefoot" method. It seems to work, too.
@@AndreasSpiess It works, unless you need to keep the logs after reset or powerloss. But I only need to use them, while developing, so I don't care.
Didn't know, its called barefoot :D
Nice hint I was not aware of! It's there a known command to write down all memory changes to disk aka SD? That would be great if you are experimenting and at the end think, ah I should have saved this. As I understood it, writing or not ALWAYS requires a reboot first, correct?
Log2RAM does something like that. It stores the data in memory and saves them every hour or so as well as before a proper shutdown.
@@AndreasSpiess awesome, that is what I'm interested in, I will look into it. Thank you!
More complex than the other methods, but wired Ethernet allows diskless PXE (network) boot. Unfortunately wifi doesn’t support that.
Others also mentioned network boot. Also a good idea.
I have few application that move files from 1 folder to another in my Pi. I assume that the overlay file system will revert everything back on restart?
I getting a SSD for my 3B+ soon, which is a better solution for my situation (i think)
1. Yes, it will revert to exactly the initial state
2. An SSD is a good thing. However it will not be very fast on the USB2 of a Pi3B+
I saw this option before in dietPI, I think it's a cool feature, glad the ported it on raspbian too
Another viewer asked if it is available on dietPi too. So here we have the answer...
@@AndreasSpiess Not sure if it is exactly the same. I know that in DietPI logs are not saved directly to SD and when this functionality was introduced, it was not yet in Raspbian. It is a really interesting concept to preserve SD life expectancy
Good video, no mention of emmc SD card adapters?
I would classify them as SSDs (price, form factor)
@@AndreasSpiess eMMC SD keys or adapters are small and directly replace the micro SD card, much smaller than an SSD, although I expect the SSD will beat on performance
I did not know they are so small! I have to look at them
Some clever guy should build a case with UPS and SSD built in, with a few extra USB ports.
Some clever guy should build an RPi hat with GPS, RTC, and an UPS built in. An instant internet independent NTP server.
You find many additions for the Pi. Maybe also one which covers your requirements? (GPS probably could replace a RTC)
This is a discussion about two topics:
1. How flash works.
2. BC/DR.
Lets solve 'two' first. If your data/system is critical make sure you take/keep a remote copy and keep it off site. End of story. 'Local resiliency' doesn't help you if your data center - or in this case raspberry pi - goes up in smoke.
For 'one' let's talk about flash's 'peculiarities.' Flash devices - and by the way both memory cards and SSDs work the same way - have the following characteristics:
1. Fixed block size for writes. Unlike spinning disks flash can has a fixed write size. i.e. If a device writes at 32kb and your write is 4kb. The device writes 4kb of data and 28kb of zeros. If your write is 128kb the same device writes 4 × 32kb writes.
2. SLC, MLC, TLC. Denotes the number of bits in a cell. 1, 2 or 3 respectively. Additionally, it indicates 'burn rate' how quickly cells those cells die. The greater the number of bits in a cell the faster they wear out. Most consumer grade SSDs and flash cards are MLC.
3. Overprovisioning. Every consumer flash device SSDs and flash cards are overprovisioned to take wear and tear into consideration. SSDs have wayyyyyy more than Flash cards as they have to deal with the writes an OS throws at them. So don't buy an SSD buy a bigger memory card. It will last a lot longer, be easier to deal with and work out cheaper.
4. Fade. As magnetic media is hit by 'bit rot' flash gets hit by 'fade.' Over time the bits flip or the cells fail and the files become unreadable. Magnetic interference and heat accelerate this process so keep your pi cool.
4. Device firmware. Not all flash devices are created 'equal' and - no disrespect to the engineers who do this - the firmware on the a given generation of devices often gets written in a hurry. So the lifespans of two different cards with the same size advertised size and identical workloads may differ drastically in terms of lifespan.
5. Reads are free, writes are poison. High levels of random writes are absolute cancer for flash. Everything slows down if your device is bangoed with high write traffic. So if you are logging tons of data? My advice send it to a NAS backed by spinning disk DON'T store it locally on your memory card it will kill your flash device quickly.
So in summary:
1. Keep a back up.
2. Buy a bigger memory card than you think you need.
3. Keep your PI cool.
4. For all random write traffic send it to spinning disk.
5. Check the product reviews for angry consumers complaining about 'failures within weeks of purchase.'
How do I know this? I was employee 186 for the best enterprise flash array vendor on the market. As an aside - but related to this - 'middle out' from the 'Silicon Valley' series is based on a real algorithm that optimised wear and tear on MLC using meta data references to avoid writes if possible.
Thank you for your additional information. Everything is true. Do you have information about the wear-levelling of SD cards? I read a lot that they do not have such an algorithm and manufacturers explicitly forbid to use them in file system scenarios (and refuse warranty as one viewer wrote).
Just a few additional thoughts about the use cases of the video:
I talk about Raspberry Pi computers which are not often used in data centers nor do they have to deal with lots of data, because its performance is weak compared to servers. On the other hand its data often is not very valuable for the long run or does not change a lot. Many of our Pis just do some data transfers (sensors, gateways). or are used to drive displays, etc. and are sometimes operated in harsh situations.
Fortunately, we will hardly hit any of the specified transfer volumes of SSDs. However, we all experienced crashes of SD cards. Sometimes also because the power was switched off during a bad time which corrupted the FS.
I have installed "remote-iot" to my pi3B/3B+ and am monitoring total writes -the only tool I found that would read microSD stats. I added "/dev/root / ext4 rw,noatime,commit=60 0 0" to mtab" to minimize writes. I have computed writes over many days using the "total writes" count, and it is about 10/sec. I am trying to figure how "many" this actually is...... Doesn't seem like much. Hard to translate this. Thanks for your video! Wish stretch/jessie had overlay FS.
I did not know of remote-iot. Its SD monitoring function is very interesting! The only problem: Nobody seem to know how many write cycles a typical SD card supports before it dies...
Wouldn't a high durability SD card be a better solution? Justaskin'..... Yeah, as already suggested...
I do not know. Different results were reported by commenters.
My solution - buy A1 class card that is NOT SanDisk. They are counterfeited most often. I have several genuine A1 class cards working for several years already with Armbian (that is tuned to minimize writing)
Good tip. Others suggested to use WD purple cards wich are made for extensive overwriting.
Eh, I just turn off logging and use zram at (current available memory plus 25%).
You are right. This can help with some problems (as Log2RAM).
1. download Raspberry Pi Imager
2. Flash sd as usb boot
3. Flash image to usb
4. Insert sd+usb and boot
5. green screen, unplugged power and sd card
6. Plug the power and go
Unfortunately, I do not understand:-(
Indeed interesting. Guess I should make work of adapting my RPi asap.
If you have one which does not have to keep data it is a good thing.
great features
Thanks for sharing your experience with all of us 👍😀
Thanks for watching!
That guy doesn't work in IT. He has no beard, his hair is combed, and I suspect he is wearing socks, too. And he's smiling. I think he must be an Engineer. ;-)
:-)) Everything true. Except the "combed hair". There is a reason for the hat ;-)
Add autofs against an NFS box with your homedir, we did diskless workstations that way for years back in the 90’s.
You are right. Remote file systems can be a good solution instead of an SSD if you need to keep the changing data.
Have you tried using high endurance SDCards sold for devices like dashcams that repeatedly overwrite the storage?
Not yet. But a few viewers suggested to use WD purple. So they are in the mail now ;-)
@@AndreasSpiess They should be a lot better, but they will also wear out at some point. This also depends a bit on the file system you're using. Unfortunately I don't think LittleFS can be used seriously with Linux yet.
I know I stated that SSD would wear out and that harddisks would last longer. This also depends on the use case. If for instance your harddisk is placed in a -20° shed, then the SSD will probably last longer. Harddisks may die if turned off and put on the shelf for 15 years (SSD likewise). -So it all depends on things we know and things we don't know. :/
Disable file date updates. This will beat the heck out of sd cards. It will also speed up access times.move logs to ramdisk and dump and rotate logs to external storage.
Agreed. I assume that the instructions I mentioned do such things.
Puppy & other distro's can be setup to load into RAM thus SD card can be removed or just to store personal files
Thank you for the tip. As a Linux noob I only use Raspbian :-(
Why not provide an option to dump back the content of the overlay fs back to the sd card? This would just narrow it down to one massive write on shutdown, instead of billions of tiny ones?
Log2RAM and other software support such a concept. It would be in-between a read-only and an SSD solution.
I feel you should look at using a network drive so you do not need to use so much RAM. One simple way to make a network drive is to use an old desktop as a Linux box and have it share the hard drive to the raspberry pi.
You are right. Other viewers also mentioned remote logging as an opportunity. I assume it would need at least some Linux knowledge.
strange i dont see this Performance tab in my OS of desktop raspian
Strange. Maybe it changed place in newer releases...
I usually use a μSD card only to boot from and I have the storage on network with Ceph.
Also a good possibility if you know what you do...
Thank you Adreas for the easy to understand ex[;amation of the Overlay File System. It's just what I needed for my weather station which uses an ePaper display controled by a Pi Zero W.
That seems to be a perfect fit!
Is it a perfect fit, though? You mentioned in the video that 2G of RAM is recommended, and the Pi Zero W only has 512M.
I don't know how the Overlay File System decides how much RAM to use: is it dynamically allocating memory based on the amount of data written, possibly running out of memory over time? Unfortunately, Googling doesn't return any tips on reducing its memory usage, and I couldn't find confirmation that anyone is using it with a Pi Zero W successfully.
I guess the only way to be sure is to try it.
Thanks for pointing out the option, though: I didn't know it existed.
Can you share the scripts for stopping logs or move them ?
Look at the barefoot method 2 in the video description. It is in German, but easy to translate
thanks to the guy with the Swiss accent, another great tip and information
Happy to help!
Does this also help in the event of a sudden power loss? I imagine as long as there's no writes occuring at the time of loss it could be okay?
Yes. Because nothing is written directly to the disk (or SD card). So, your Pi always starts with a "fresh" image
Disable swap permanently. Pi's 4Gb/8Gb do not need swap.
Agreed. I proposed this method also in other videos. But often it is not sufficient.
I was able to reduce my SD card usage by using virtual box with Raspberry pi iso
This for sure works. But is it what we want?
Are any of these options available on the old Raspberry Pi model B boards (not Pi 3B but the original one)? Would it for instance be possible to boot from SD card and then re-mount to an SSD on USB? I'm not a unix expert, so commenters forgive me if my question is very stupid.
The feature presented in this video is a software feature and should be available also for older models. But I do not own such boards anymore. So I do not know if there are restrictions.
@@AndreasSpiess thank you kindly for your reply, Andreas.
I always experience "error in writing storage". How can I fix that error. Please help
Unfortunately, I cannot do remote debugging :-(
On aRaspi 4 8GB I disabled swapping to reduce disk operation
This is also a good thing. I recommended it also if past videos. But with the bigger memories it does not help a lot because they usually do not use the swap file a lot.
Thank you very much! It was useful and interesting 🙂
Glad to hear that!
Hi Andreas,
Last week I wanted to update a 2017 installed Raspbian Jessie on its 8 GB µSD card. Of course, I dumped the card beforehand ... or at least I planned to. Because dd reported many defective blocks. The dump ran for 48 hours because of the many retrys.
Out of interest, I checked the file systems in the dump: were both okay. Obviously the mechanism that hides defective cells in the µSD card works very well. But I'm also sure that the card was on the verge of total failure ;-)
vy 73 de Micha, DD0UL
Good that you were able to get the content. This was not always the case here...
I stopped to use my 8GB cards when I saw how slow they are compared with the bigger cards, not because of unreliability. So I spent a few bucks.
Wow..I didn't know about this feature..?...thanks!
You are welcome!
Nice Video, learned something new. Now it would be great if specific log operations could be made to go to the SD card and the majority just to ram...
You an du that by using RAMFS or log2RAM to only store logging in RAM. But it needs a little more Linux knowhow. Or you mount a network drive for data as other viewers suggested.
If you're going to go to all this effort just build a completely in-ram system with buildroot. A very basic system with python3 etc is going to be about 16-32MB as an initramfs. You can stick that on an SD card, have it load into memory at boot and never do any writes to the SD card.
That is an interesting idea. Would it also run all the examples I mentioned or only Python programs?
@@AndreasSpiess
You can run anything you can on a normal distribution. You just have to select the pieces you need for your build. Packaging your own bits is easy too. IIRC Tesla use buildroot in their cars.
An example: I have a beaglebone black based LoRaWAN gateway in my roof. It's impossible to go an fix that thing without getting out ladders etc. So it has just u-boot in it's eMMC. It downloads a buildroot based image over TFTP on power on and runs from that in memory.
If there is a problem and I need to update the OS I can just build a new image, load it to the tftp server and then power cycle the board via it's PoE port.
Sounds like what we call "OTA" on our ESP32s ;-) I did not know this exists, but it is very handy for situations like yours!
@@donpalmera Is that also know as PXE boot?
Warum in englischer Sprache? Es gibt doch genug englische Videos - aber gute deutsche sind Mangelware.
Aber leider gibt es viel zu wenige Menschen die deutsch verstehen… (Für mich ist Deutsch auch meine erste Fremdsprache)
i not understand i has make sd copier whit rasperry pi 4 new copy at card and not make, say ewery sd card or usb memory same no can mount card. or make but after 1-2 time boot then not boot newer again, and need make new image again. vor not mount or make copy but not boot newer no start run. i has test ewery day last 3 week now make good image and boot good but no can, only one old sd card run raspi now, all other 13 new sd card 4 different size usb memory, 32gb,64gb,128gb,256gb usb all not can make boot image. no can sd card copyt no boot.
I think not that the SD card is the problem. It is not possible that you have so many bad cards. But I cannot do remote debugging :-( I would start with downloading the official Raspbian for your Raspberry version and test if the problem persists. If so, Your Pi probably has a problem. Or the power supply.
@@AndreasSpiess io uses official raspi bullseye and official imager software. not working good or notr mount or verifed fail error etc. andf 13 new sd car uses. all have genuine sandisk or sansung what raspi say need use. 2 month and need have ready working raspi i need because i go to travelling to kambodia and need i have computer were looking e mails and connec t to home and speak whit peoples, ewery hotel have TV hdmi display not need carry. raspi have small if working.
Are SD cards that fail in this mode of operation covered by the warranty?
In short no. I made a claim for SanDisk memory stick. They asked for the use case. I said it had been used to transfer files and to write a large Linux ISO file. I think they interpreted the ISO file for a complete working OS system as they replied:
"Please note that the pen drive you are using has not been manufactured for use in devices or applications that perform constant read and write operations like car cameras, security cameras, bootable drives etc."
They did replace it as a one-time courtesy. I normally buy SanDisk but I worry about buying a fake. I only buy those in a full retail blister package but you never known what happens to the ones that fail the quality test in the factories.
@@Alan_UK I still believe in SanDisk for quality but as they said the product have limitations (and IMITATIONS).
They probably can check the operation history kept somewhere in the drive.
Anyway, I learned something from your case.
another downside of ssd is current draw. well done herr spiess
Correct. I never measured it. Maybe because I do not care too much about power on my Pis.
I use a SANDisk SD Card in al my Pi for 6 years now. Some of thm write in every minute a few bytes. I never had a SD Card failure yet with SANDisk.
So you are a lucky guy! Maybe the older (and smaller) SD cards used different flash technologies?
I think it is the Brand SANDIsk. They make great Cards. I use only 32GB ones. Though, I know the pain with no name cards. Saw some failing. But not with me.
What will happen if we set Overlay: Enabled, but Boot Partition: Read-write? Any use case?
Hmm, I might be wrong, but I don't think it'll write to the partition.
You might be able to doubly-mount a partition on the overlay disk if you wish to write to it.
Eg. On your overlayfs...
mkdir /mnt/seethru; mount /dev/sda2 /mnt/seethru
... don't know if it'll work of if it needs some fiddling.
I never tried it (because I did not have a use case).
Toll währe es wenn die Ramdisk beim runterfahren oder andere Trigger, z.B. auf einen USB Stick gespeichert werden würde und es beim Neustart wider eingelesen werden würde.
Hab sowas früher in MS DOS gemacht.
Auch eine gute Idee. Das geht teilweise mit log2ram. Auch wurden Vorschläge gemacht die Periode bis zum speichern zu verlängern. Das hätte ebenfalls einen ähnlichen Effekt. Die Gefahr eines Datenverlust ist natürlich höher. Deshalb verwende ich es nur für unwichtige Daten.
You can enable iscsi drive from network for writing.
Unfortunately, I do not understand :-(
@@AndreasSpiess As far as I understand, Raspberry Pi 4B supports iSCSI in the boot-firmware, so it might be able to just boot off of an iSCSI disk.
Alternatively, one could OverlayFS mount an iSCSI volume on top of the SD-card.
(and if you want to make things complex, which is off-topic, then you could include docker and mount network-shares only inside containers).
4 Minutes until you get to the actual point of the video?
I did not count…
That’s why my sdcard for gaming is always crashed and get error
Well possible...
Not Pi related but Linux related. In the early days of SATA SSDs I had one at 16GB from Imation. I had the most 'write intensive' volumes mounted on an LSI RAID volume (e.g. /tmp /var/ and SWAP). It lasted over 6 years in 24/7 use before it was replaced by a new computer. I did pull the drive before sending the computer to freegeek. Those are the 'high write cycle' mounts for a system drive. Data drives are not addressed.
Thank you for sharing your experience. So far I also had no problems with SSDs, also not on the PC. Even if I do some video editing. For the Pi I do not expect any issues because the data volumes are very small (mostly sensor data and logging).
I have the same SD-card running since my Raspi 3B+ was new. 2018? 19? Never got corrupted (headlesss lite raspian doing DNS and Cups). I still did that, thanks.
So you were lucky and got a good SD card, I think.
If you know what l mean 😂😂 that's great
:-)
Write files to USB Flash & update as needed if data logging with minimal data loss. ...There exists a "HAT" that goes under the Pi and is screwed on, supports an M.2 SSD via a small USB 3.0 adapter. The bonus: you get SSD speeds with low power draw.
Do you know if USB sticks use wear-levelling? Otherwise we probably have the same issues.
Good that the "hat" goes below the Pi4. I still see too many "top hats" which create heat issues.
How do we get docker to function when using read only?
I never tried Docker with this system
Does this only work with Raspberry Pi 4? I don't see the Performance options on my 3B+ which is running the latest OS.
I now tried it with a Pi3 (desktop version) and it was available as with the Pi4.
@@AndreasSpiess It seems to be missing from the lite version. I’ll try again with the full desktop version. That opens up a lot of possibilities for automotive use.
I had to update the firmware and then was able to get the the latest OS installed. I have it working the Pi I used for Fruitbox (jukebox software) and it works great! This should greatly prolong the life of the card. Thank you!
If you want to store data and you have a NAS available, maybe ist worth to create a network share, where you can store your variable data.
Also a good idea!
@@AndreasSpiess next step with my raspberries: booting from network ...
• 2:15 - Another disadvantage of SSDs is that as flash-media, they wear out like a memory-card does, so it's no better.
• 3:19 - The write-protect switch on SD cards is just a piece of plastic, the write-protect function is only in the card-reader/writer, and not all card-readers (*cough*Chinese*cough*) honor the switch, so it doesn't guarantee to protect a card from being written (this is the same as with floppy disks).
• 4:22 - This makes the system even more vulnerable to data-loss than ever. 🤔 You assume reboots are intentional.
1. SSDs use wear leveling which was not the case for SD cards. This is why they were not specified for computer usage.
2. You are right
3. I do not agree because I only use it for relatively stable applications. So reboots for configuration changes are very rare.
RAM disks are faster, and never "wear out".
You are right.
How would this work when using a cluster hat?
I do not know :-(
Just use pifs so it never writes to the sd card
What is pifs?
Thank you! Extremely useful, extremely inexpensive, reversable and can be done in seconds. Love it! The overlay wins for me.
Glad it was helpful!
Hey: what's a LoraWAN gateway and what do you use it for?
Maybe you watch my LoRa introduction video?
@@AndreasSpiess good idea!
thanks for the knowledge
You are welcome!
Awesome, so I guess no UPS will be needed if uptime is not desired? am I correct?
Yes, I think so.
@@AndreasSpiess excellent discovery , thanks and hats off to you sir!
thats a neat trick, plus is more secure, becasue if you loose your rapsberry pi whilst transporting it, you wont have to worry about still being logged into google etc and saved passwords. just enable it on a new install after you have setup wifi and customized the dektop
Good point!
great video. happy christmas
Thank you! Happy Holidays, too.
Does the SSD have the same corruption issue as the SD cards resulting from power outages? I have corrupted Octoprint a few times from someone (me) cutting the power before shutting down. I am definitely going to try the Raspian Function. Thanks!
Corruption often comes from the file system (the computer cannot finish a transaction) and can also happen to the SSD.
Very educational content. Just wanted to point out that I'm personally using High Endurance SD by SandDisk which meant for CCTV application (run 24/7 + consistent read & write application). So far works well (more over 8 month with multiple time of trips by thunder storm - in turn will be a sudden power loss) for my simple LAMP project without overlay file which is running 24/7 when ever possible.
Maybe can give those a shoot for extensive test?
Other viewers also commented about those SD cards with different results. I do not plan such a test because I do no more use SD cards for heavy usage. I changed to SSDs, also because they are faster.
what about special endurace sd cards?
A few other viewers posted their (mixed) experience with them. I never used one.