For dropping some files onto a drive at work and “sneaker-netting” it home, sure, the USB Flash drive is quite good. For long-term storage? Nope. In fact, nearly any other kind of storage will be more reliable in the long term.
The Architecture of a Flash Cell
Flash memory is a relatively recent invention, so we don’t have as much history as you might think about longevity. But enough to know that flash is a temporary thing. This wasn’t a shock or even a problem, for several reasons, but largely because in its initial use, flash memory replaced EPROM (erasable programmable read-only memory), which itself had a limited longevity.
A flash memory cell is made of a MOSFET transistor with a “floating” gate. This is a gate surrounded by insulators. To write a bit, electrons tunnel through the insulator and are stored on the floating gate in a process called channel hot-electron injection. To erase a cell, Fowler-Nordheim tunnelling is used.
The original sort of flash was retronymed “SLC”, for single level charge. This is what you’d expect out of a binary thing: there’s either a programmed charge on the gate (0) or no charge (1). An SLC flash cell should last about 90,000–100,000 program/erase (P/E) cycles…. some specialized parts can last 1,000,000 P/E cycles.
However, flash designers got more clever, and came up with “MLC” flash, for multi-level charge. Now they could write three different levels of charge, plus no charge, to deliver two bits per cell. However, this made the cells more fragile, less robust against corruption over time. More robust “enterprise” qualified MLC flash memories used in more expensive SSDs have a write/erase lifetime of about 20,000–30,000 write/erase cycles. Normal consumer oriented MLC flash memories have delivered about 5,000-10,000 P/E cycles, and they’re down to around 3,000 P/E cycles at the 15nm node. And even more recently, we have “TLC” flash, which puts in seven levels of charge or none, to store three bits per cell. That’s one of the reasons flash memory got so cheap. But you’re going see cells failing after 1,000–3,000 P/E cycles.
And here’s the other thing… flash has, for awhile, been getting worse. The highest density 2-D TLC chips may only deliver 300 P/E cycles these days. The wear mechanism in flash memory is primarily due to stress on the tunnel oxide layer in each cell. The oxide layer degrades as electrons, over time, become trapped in the oxide. And as we’ve been shrinking the chip geometries of these cells, their longevity has been dropping. Curiously, when a drive runs hotter, there’s less stress on the insulator, so the overall effect is a longer lived drive… but shorter data life. The 3-D chips used in high density flash have increased longevity and data retention, as they’ve mostly gone to larger chip geometries.
Data Retention
So as mentioned, there’s a charge — a bunch of electons — placed on an insulated gate. That’s your storage mechanism. There are only so many electrons, and over time, they’re gradually going to leak off the gate. At the high end, the specialized, small flash memories in embedded microcontrollers can be expected to last 20 years to 100 years at 25C. On the other hand, Intel recommends unpowered consumer SSDs for data retention of only a year. And this is exacerbated as the drive takes on wear.
Now, for SSDs, it’s a bit different when the drive is active. Between wear levelling and data enhancing routines, an active drive will last considerably longer than a year, even the static data stored on that drive. There’s quite a bit going on within a modern SSD that you never actually see, which is designed to spread wear out across the drive. USB drives, though, don’t run data-enhancing routines.
And the trend toward less reliable flash has started to reverse itself, at least for now. The 14nm or so node is about as small as a practical NAND cell can go today, even though we have smaller chip geometries. They just get too unreliable. So the larger memories coming out today, 256GB per chip and so, are actually 3D chips using larger geometries. Some of these stack 48 bits worth of data on separate layers, and the reliablity is up.
Mechanical and Environmental Issues
Flash drives of any kind will lose data faster in warm environments. Most consumer USB dongles are not recommended for use over 40C or storage over 50C, but they’ll last longer in colder temperatures.
The standard rating of a USB Type A connector is only 1500 plug/unplug cycles. This should never be an issue for any USB stick you’re using for longer term storage, as you’d like to use those drives just once. But it’s a consideration if you have a drive you’re using every day. And of course, that’s a rating — you could find your real-world life is more or less than the USB specification.
Real World Examples
You don’t have to take my word for it, either: a drive, whether USB, SD, or SSD should offer longevity data in the drive’s data sheet. Or in their warranty. However, you’re probably not going to find it mentioned at all for most drives, because the manufacturers don’t warrant data retention. That should tell you a thing or two.
Here’s a USB flash drive designed specifically for long life and long retention. This Swissbit drive is SLC, so only charge or no-charge on each memory cell, and it’s rated for 100,000 program/erase cycles. And when new, it’s rated for a 10 year data retention. When at the end of its useful life, though, it’s only rated for a year of data retention. So if you must insist on using USB flash drives for storing imporant things, use a new one and put it right on your shelf. Oh… by the way, the Swissbit 4G SFU24096E3BP2TO-I-DT-121-STD drive is only USB 2.0 and costs $97.58 in single quantities on Digi-Key.
For about the same cash, I can buy a PNY Technologies P-FD512ELX-GE Elite-X Fit USB 3.1 flash drive… 128x the storage, and USB 3.1 performance as well. Only one problem: PNY has a one-year warranty for the operation of the device, and absolutely no data retention warranty. None. The JEDEC specifications for NAND flash memory require an SSD in power-off to retain data for one year, in consumer devices, and 3 months, in Enterprise devices, through the life of the device. So yeah, you can expect newer devices to retain data longer. There’s no requirement that the NAND flash memory in your USB stick meet full JEDEC specifications, but major brands most likely do. Those are not the numbers you want, but that is the fact.
SanDisk warrants some of their actual USB flash drive hardware for a “Lifetime”, whatever that actually means (30 years in countries that don’t allow lifetime warranties). You would think that might have them offering a data longevity warranty or even an official guideline. Nope, they don’t.
Kingston warrants more of their USB flash drive hardware for 5 years, but they don’t have a data retention warranty, either. In fact, they offer this guidance: “Kingston Flash Data Retention: Kingston Flash Storage Devices primarily use MLC/TLC Flash Memory. Data retention on Flash memory is dynamic since the amount of time the memory has been cycled affects data retention. Important information should always be backed up on other media for long-term safekeeping.”
And sure, you can hear ancedotal stories of USB flash dongles from 10 or 15 years ago being completely good today. And sure, possible but not likely and absolutely not guaranteed other than on specialty memory like the Swissbit. And the memory chips made 15 years ago were almost certainly more reliable, being both SLC and much larger chips.
So, Where to Archive?
Well, I’d pick a hard drive over an SSD any day of the week. The magnetic storage on an HDD is extremely robust, likely to last 10–20 years or more, but do check your specific hardware’s data sheets. The problem is, you’re dependent on the mechanical bits of the drive tolerating that 10 year sit. You may be fine, or you may be subject to “stiction” problems, where lubricants get sticky over the years and prevent a normal spin-up.
My long-term medium of choice is Blu-ray, with qualifications. The minimum you want is “HTL” Blu-ray. The original Blu-ray formulation starts out shiny, and the laser melts a bit of silicon and copper together, forming a less reflective material that’s stable against sunlight. Avoid “LTH” discs, which use dyes similar to those used on CD-R and DVD-R.
Even better is M-Disc, a proprietary non-organic HTL disc using a material engineered specifically for longevity. The disc is even HTL for the DVD-R version, one reason it needs a special drive to burn it… regular CD-R and DVD-R is always LTH…. the laser is making the dye layer clear, allowing the reflection layer through. M-Disc is described as “glassy carbon”, it’s unaffected by oxygen — the enemy of the aluminum layer in a CD-R or DVD-R — and rated for 1,000 years life. Even if that’s an exaggeration, if that’s not effectively “forever” for me, then I’m certain all my old media will be stored in 0.001% of my positronic brain upgrade. But this also points out a universal truth — nothing is forever and under every circumstance. There’s no such thing as “waterproof”, you only have degrees of water resistance. There’s no such thing as “timeproof”, you only have degrees of time/entropy resistance.
You can also consider managed archival… online storage. Storing things in the cloud, you will get the slight benefit of enterprise-level hardware, perhaps. But what you’re really getting is active data management. You’re paying that company to mind your data, replace drives (on a RAID, of course) before they fail, etc. I never would have considered this a year ago when I was on satellite internet at 12Mb/s and low data caps, but today I’m on a gigabit no data caps. So it’s an actual option for some.
UPDATE: After I wrote this, I did self-examine my former stance on online backup. I was not a fan, because it was slow, expensive, and based on your faith in the company still being around when Godzilla stomps on your house. But I did some research and signed up for a trial backup with a cloud backup company, and then paid the annual low flat-fee. So we’ll see. This service has very little user interaction.
Their main purpose is backing up media, not making an archive of your HDD, which is my main concern. So they don’t back up program installs. They automatically scanned 1,882,572 files worthy of backup, at 13,454,758 MB. They have so far backed up all but 25,522 files, but those files comprise 10,288,725 MB of data. I started this three weeks ago, and I’m on gigabit fiber to the internet. So the throttling is probably intentional, and not unexpected. Nearly everything they’re archiving will never change at my end, so ones is enough.
Now, of course, if Godzilla did stomp my house, I would not relish the idea of spending several months downloading my stuff. This company will provide all that data on 8TB hard drives… so two drives and I’m good. Not terribly expensive, particularly compared to my house and all my meat-space stuff. At some point, I’ll probably update this again.
Rot and Redundancy
No matter what you do, you’re subject to data loss. The best archival medium is redundancy. When I store my photos on those pricey BDXL M-Discs, I write overlapped… so half of the last backup goes onto the next backup. Every photo is on at least two discs. And my RAID. And my backup HDD.
While SSDs can show data rot in a few years, HDDs and tape certainly can as well. I shot analog video in 1994 on high quality tape that was showing rot by 2008 (multiple playings on different 8mm decks got it all converted to digital). Some of the old CDs we made in the early 1990s are still good, some of the CDs and DVDs I made in the later 1990s and early 2000s are dead. Particularly for anything using organic dyes, both light and heat are your long-term enemies.
Then there’s format rot. I backed up onto TR-4 tapes in the 1990s. It wouldn’t matter if the tape’s good if I can’t find a compatible drive. Or a PATA hard drive for that matter… you can probably still find a PATA interface, but your modern PC may not have one. One of the best reasons for CD/DVD/BD is the fact it’s a consumer format, not just a computer format… that’s the primary reason the discs stayed upward compatible over three decades and counting. But will that last? UHD Blu-ray is out, with 66GB and 100GB discs, based on the same tech used in BDXL, so that is a consumer format. But it’ll never be as popular as HD Blu-ray, which isn’t as popular as DVD. Will we have another consumer optical upgrade? But I digress…
So archive often, on different media, and in different places. That’s the only way to prevent data loss for certain. And flash drives are cheap, but not archive-grade. If you must use them, back up redundantly.
Read More
https://www.anandtech.com/show/9248/the-truth-about-ssd-data-retention
Dave Haynie's answer to Is SD card a good idea for long term data storage?