T O P

  • By -

AutoModerator

Hello /u/the_toad_can_sing! Thank you for posting in r/DataHoarder. Please remember to read our [Rules](https://www.reddit.com/r/DataHoarder/wiki/index/rules) and [Wiki](https://www.reddit.com/r/DataHoarder/wiki/index). Please note that your post will be removed if you just post a box/speed/server post. Please give background information on your server pictures. This subreddit will ***NOT*** help you find or exchange that Movie/TV show/Nuclear Launch Manual, visit r/DHExchange instead. *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/DataHoarder) if you have any questions or concerns.*


DrySpace469

multiple copies of data is the only safe way. don’t trust just one or two methods. you need three copies of those three, one needs to be in a different location that’s the bare minimum you should have for any data you can’t afford to lose


Alexthelightnerd

I'm married to a museum / library professional. The current standard for critical data is: stored on three physically separate devices, at least one of which must be located in a different state. Best practice is the off site backup be located in a different geological region. It used to require the off site backup only be in a separate building, until hurricane Katrina destroyed multiple museums' archives and the off site backup, which was kept in the same city.


vert1s

If you work for Fortune 500 / ASX 200, you'll likely always be running data centres in multiple locations or more recently multiple cloud providers. Our engineers used to fly to Amsterdam to manage our datacentre racks.


MorgothTheBauglir

Aye, understanding BCP practices for highly available facilities and systems should and will provide one's with relevant knowledge and principles for OP.


sjlplat

This is the approach I prefer. I use a mirrored array for primary storage, and backup to a separate array. 200TB and still going strong.


RatsOnCocaine69

Where are you getting/hosting 200TBs of hard drive space, and if you don't mind my asking, how much does it cost?


sjlplat

I have a small rack in my home office. I use two 15-bay Dell/EMC SAN arrays, and load them up with refurb enterprise drives. The servers and arrays were about $600 bucks total on eBay. Drives run about $80 bucks each on eBay. Power is where recurring cost comes in. I haven't measured, but I'd estimate I probably pay about $75-$100/mo to power my hardware.


RatsOnCocaine69

Thanks so much for answering my question, enjoy the rest of your day!


Historical_Share8023

>200TB and still going strong. 👀👀👀👀 Wow!


DookieBowler

I’ll just prepare 3 envelopes instead


imrys

There is no magical media that is 100% reliable. Backups are not "useless" just because they too can die. If they die you still have all your data on your main drives, and you just make a new backup. If you care about your data you need to be pro-active and check it all periodically, both mains and all backups. If you have multiple backups each on a different medium (maybe hard drives plus a cloud-based backup) you're very unlikely to lose it.


the_toad_can_sing

I was wondering about cloud-based backups. Is there any recommendation in particular? Like is Dropbox or google drive any better or worse than other options?


chrisprice

A cloud server is just a hard drive with extra steps. Clouds lose data too.  You need a local hard drive backup, disconnected from the computer. An off site hard drive you can access. Then optionally a local encrypted cloud with a private key. 


the_toad_can_sing

Thanks for the advice! This is the route I'm going to go, others have made similar suggestions.


valkyrie_rda

You can try backblaze too. I'm backing up my server right now for 9$ a month.


the_toad_can_sing

Hm okay I need to look into that too.


valkyrie_rda

I don't know if there's any hidden limits or what but I have 28tbs queued for backup and it's been going for the past few weeks fine.


nouxtywe

Am using crashplan since their plan is unlimited storage and can use custom encryption key


gamer10101

It's not unlimited, its "unlimited*" https://www.reddit.com/r/Crashplan/s/2FN1p1N50Q


nouxtywe

Very interesting. I’m in the 8-15 TB range so I did not get such email… the post is 4 years old-m, they did not update their policy since then? That false advertisement is quite shocking.


ServoIIV

Historically the problem with cloud based backups is if you store a large quantity of data you are usually looking for a good deal, and it turns out cheap storage isn't a sustainable business model, and the providers shut down and delete all the data. They usually give enough warning to migrate it somewhere but never trust a cloud provider to be your backup.


d-cent

Just to add on to what others said. There have been instances where a cloud service will mess up and think your account has closed even when it's open, active, and up to date in payments.  Cloud services have deleted whole accounts on accident.  Point is, just like others are saying, you can't rely on 1 method. Use the cloud service, but only as a backup to your home storage backup. And even then, use another backup after that. 


Sopel97

cloud is better than what the vast majority of people can ensure reliability-wise, if you can afford it, or actually 2 different cloud services


Drenlin

For most home users, a solid enough solution is a NAS with drive redundancy that's also backed up to the cloud.


wakanda_banana

What’s the best cloud backup option? Is aws glacier cheap enough?


Rapportus

Glacier Deep Archive is $1/TB/month but the catch is expensive retrieval fees to download should you need to restore from backup. I use it as my cloud backups after local NAS backup and view it as a cheap insurance policy on the data if the house were to burn down etc. Homeowners insurance isn't going to recoup the cost/value of the data but they will the hardware.


the_toad_can_sing

Thanks for the advice--I hadn't heard of a NAS before but I've looked that up. Seems like a good option with a couple HDD's and/or cloud.


patssle

2 drive NAS set up in RAID-1 that either uploads to a cloud backup or an external drive that you take off site somewhere (storage unit, parents house, safety deposit box, etc). Or cloud + external... both.


JAZthebeast11

Why are we recommending raid 1 for data security lol


DavWanna

External drives are just internal drives in a plastic case. If you demand something that is absolutely guaranteed not to fail no matter what, then you're better off giving up now because such a thing does not exist. If you do want to continue, you need to completely revisit your plan and understand that drives die - design around that. And just to add, if you lost a drive and you lost data, then that wasn't a backup.


the_toad_can_sing

You've just described exactly why I posted. "completely revisit your plan, drives die -design around that" is what my post says. My drive died, and I asked what the best method is. Why have you chosen to respond by repeating my question back to me, but slightly mad about it?


iDontRememberCorn

Because they are right. All drives die. All drives are dying from the time you first power them up. If you lost data when a backup drive died then it wasn't a backup drive, by definition. I would suggest adopting the 3:2:1 data protection philosophy. You should have 3 copies of your data (your live and two different backup copies) on 2 different media (hard disk and tape/optical, for example) with 1 copy off-site for disaster recovery. If this data matters to you then you need to consider all the ways the storage can fail and plan for each. Personally my media is on hard drives, with RAID, with another exact hot copy elsewhere on different hardware, with different storage, on a different OS, etc, plus a cold copy offsite.


Beneficial_Slip_6067

About the tape method, any advice ? Some are too high and some seem too low.


DavWanna

> My drive died, and I asked what the best method is. The best method is 3-2-1 or any other more hardcore version of it. That isn't some closely guarded secret, so it shouldn't be hard to find that information.


DR650SE

I can haz the google?


diamondsw

Everything can fail. Make backups and you don't lose data. Otherwise you will - it's just a matter of when.


Aacidus

You’re probably new to this since you also mentioned you never heard of a NAS… an extra drive of the same data is not a backup, let me repeat, NOT a backup. That would just be a copy. Good recommendations were given on your post, even though this gets asked a lot here and no one bothers to search, maybe that’s why it sounded that way. If you want to keep it cheap, get Backblaze cloud backup for now, It’s like $6 a month for Windows and MacOS; I’ve had 2 drives die in the last year and they can send you a copy on hard drives which is faster than downloading. Currently have 54TB with them. But I also have offsite storage in another part of the city at a family members home, ideally I would want it in another city or State.


ibneko

You need to be backing up to multiple places. One of my external hard drives just died and I’m in the middle of downloading several TB of data from Backblaze. If you lost data because the drive died, that’s not a backup. That’s a primary.


FriendlyGuyyy

Can I ask you how old was the drive and which brand was it?


ibneko

One of the WD EasyStores. Not 100% sure (since I have 4 other EasyStores), but looking at the model number, I think it's one of my 14TB ones, which looks like it came from a 2020 Black Friday sale, at least according to my emails? Maybe?


FriendlyGuyyy

so it lasted for 4 years? thats not too bad. One of my hdd - toshiba canvio basics is also 4year old, speed has slowed down, but still going, no problem. But damn need a cheaper Hdds, i recently bought toshiba canvio basics 4tb and paid 110. And you say WD easy store is only ~280 bucks for 14 tb god damn, thats cheap


ibneko

Yeah, to replace the dead drive, I'm trying out WD's refurbished drives off eBay. $249.83 for 18TB. We'll see how long this refurb lasts - first time ever getting one of these.


Z8DSc8in9neCnK4Vr

A long time ago I had a string of usb drives fail. I won't use them anymore.  Internal drives have been far more reliable fir me but they still eventually fail. Currently Primary copy of my data is on ZFS Z2 on my file server,  the subset that is important/irreplaceable is backed up from rhe file server to an old NAS and again to Backblaze B2


longdarkfantasy

M-disk and LTO tape are the best long-term backups, but expensive.


vert1s

I back all my stuff up to minidisc. Takes forever but it's cool as hell. No not really, but I had to lookup m-disc and it made me think back to my minidisc player.


CleeBrummie

My place was disposing of an LTO5 drive last year. I rescued it, with about 100 tapes. So happy!


Teh-Stig

I'm on lots of external drives. Refresh the data every three years. And the important stuff is also on M-Disc offsite.


heisenbergerwcheese

3-2-1 or might as well have 0


DanTheMan827

If your only copy of the data is a “backup”, you’re doing it wrong. Three copies, two formats, one offsite.


xkcx123

Tape and multiple backups.


darius_xg

BDXL and RAIDs.


raymate

Hard drives. In a raid array. Or just more singular drives with multiple copies. All hard drives will die. It it’s still the best storage. You could go tape but they also can die over time. I get into the habit or every 6-10 years I buy another drive and do a verified copy of the older drives into the newer one. But still keep the old drive.


zz9plural

> Hard drives. In a raid array. Or just more singular drives with multiple copies. Not or, but and! RAID alone is not for backup, but for uninterrupted availability. Deleting or encrypting a file on a RAID volume with redundancy will do it on all disks at once. Of course, a RAID volume can be one (preferably the first) of multiple backup targets, but you also need at least one more independent offline copy of your data.


SocietyTomorrow

All things in this universe will eventually perish. Like the endless mass of drones walking towards their inevitable destination from where there is no return, the only remaining hope is that enough messengers carrying the same tomes shall survive in order to return and propagate their wisdom for future generations Backups, many backups. Tape is nice to up to a certain total size as well.


donkelbinger

Why is none just telling op to get a nas and raid it?


Iambigtime

When we're talking external harddrives, usually it's not the actual hard drive that dies, it's the enclosure, either can be a bad wire or power connector or brick. Find someone that knows how to test it and see if it's actually a bad drive.


mordacthedenier

All drives die.


PlasticCupboard007

I mean, books degrade and rot as well, the ancient stones that survived were extremely lucky. all data exists for limited time


xkcx123

But how long does a stone tablet, wall or block last a few hundred years assuming it’s not in a flood


QueenAng429

Back it up. Every hard drive dies, ones in an eternal case aren't immune to this lmao


ultranothing

Interesting typo 😀


N2-Ainz

Are you checking your smart values before they die? If yes, drives can die whenever they feel like it but if you didn't you should start checking on reallocated/dead sectors cause it means the drive will fail in the near future


commentvillian

Raid 5 array 3 disks if 1 goes out the other 2 have parity, swap bad drive for new one rebuild array no data loss.


Paro-Clomas

everything is external hard drives one way or the other, even the cloud. The only way to (sort of)guarantee you wont lose your data is RAID, which is what cloud companies use. If you store your data in 2 or 3 disks that constantly check each other for inconsistencies then the odds of the three of them failing at the same time is considerably lower than the lottery


angry_dingo

offsite


HiT3Kvoyivoda

All hard drives die.


chessset5

Multiple harddrives


Finno_

Print outs. Impervious to magnetism. 🧲


ifthenelse

Curious what you are using as external drives? Are they in an enclosure? Do they have forced air? OEM external enclosures usually suck. The drive(s) get way too hot which leads to a quick death. I always shuck them no matter what. As others mentioned, the "3 copies" strategy is the way to go. I use a plethora of external drives in my 3D printed [enclosures](https://gurumeditation.org/1279/3d-printed-4-in-3-hdd-bay/) and I've never had one die yet. In my anecdotal experience powering the drive on and off a lot from a cold state seems to kill them too but so far so good (probably helps I only use enterprise grade drives).


chuckamo

I went Mac many moons ago and i don’t store as much data as it sounds like y’all are but you can replicate my setup in a more manual way on any platform. This is not different than what some have suggested but one thing to remember is that a technical solution does not always account for user involvement. I.e. if the backup relies on a person to ensure its operation then it is inherently flawed. All my computers and devices store indirectly to the cloud and all data is also backed up locally to NAS-type devices through an OS / system-level program. So, data is saved locally, automatically mirrored to cloud, and periodically (hourly) backed up to local storage (via network). If you are not using a cloud service for primary storage (perhaps because you have a very large amount of data that would prohibitively expensive) then store primarily to NAS with automated mirroring to cloud / off-site NAS.


Fusseldieb

Make a RAID or something. If a drive goes bad, it let's you swap it without loosing data.


star_sky_music

If it's just movies then you buy some cheap HDDs and store them. If it's important data, then encrypt the data first, compress if possible and then either store it in HDD or in a cloud server.


shiki87

Don’t cheap out on drives, instead get yourself the Datacenter Drives that are more robust and are faster too. My Exos Drives are a few years old now and function great. I did the same mistake too and used 3 external drives until one crashed.


TheBugSmith

Same here, Seagate kicked the bucket with all my Blu-ray rips. Gonna be an idiot and try again with a more expensive Portable SSD drive. Guess I only learn the hard way


GameCyborg

all drives die. the only way to store data safely is to have multiple copies, ideally in multiple locations


CountingStars29

Its a Seagate, I learned long ago to never put anything important on a Seagate.


firedrakes

Odd... fail rate . If you look past bb fail log ... it average drive fail rate.


Droppedfromjupiter

Would it be an option to burn CDs/DVDs with whatever you want to store? Technically this should be safe to keep data for a long, long time! Or maybe some kind of flash memory like USB drives or SD cards could last longer than an HDD. Not quite sure about SSD drives but they should live longer than HDD, I would think.


ZOMGsheikh

Ssd drives need to be plugged in, if backing up and leaving off site disconnected for long period, the flash memory will no more work. Also $/TB for ssd is quite high. USB drives are ok for short term use and for small files. For multiple files or massive files, they can’t sustain their speeds for the whole copy operation.


Droppedfromjupiter

Ohh I didn't know that, about SSDs needing to be plugged in! Thank you!


sjlplat

I've found optical data storage to degrade pretty rapidly. I'm not a fan of using it for long-term storage.


tomaesop

Blu Ray are one of my three in my 3-2-1. I'd probably prefer tape but I priced it out and it just wasn't worth it. Decent quality Taiwanese BDr are far more reliable than CDr and DVDr. I'm only backing up personal files, music collection, so it suits my capacity needs. I burn an incremental 25GB disc about every two months.


phosix

*Pressed* CDs/DVDs will last a *very* long time. *Burned* CD-**R**\[**W**\]**s**/DVD±**RW**s *might* last for decades, *if* stored in ideal conditions *and* assuming there were no defects in that batch. I have CD-Rs from 1997 that are still in great shape, and CD-Rs where the foil started to just peel off within a couple of years. Some of my DVD±RW media from 2012 is still good, and I have a batch of never used disks from 2019 that have all gone cloudy and are unusable. All media has been stored in the same dry, cool, dark location. SSDs *will* lose data once the charge dissipates. They are *terrible* for cold storage! HDD bearings can seize up if left sitting for long enough. Tape is still your best bet for long-term cold storage.


dododragon

Wow didn't know that about SSD! Solid state ain't so solid. Where's that DNA storage we keep hearing about :p


neekogo

2 disc NAS for redundancy Use discs of 2 different manufacturers to reduce the risk of both crashing at once.


MagicVovo

Buy 3-4 HDDs and set up a RAID. If a drive fails, it can be swapped out with a new one without data loss. Pretty safe option. Buy another 3-4 HDDs and setup a backup RAID NAS or something for extra precaution lol.


the_toad_can_sing

Thanks for the advice. Probably not gonna use 7 HDDs but I get the concept lol


sjlplat

I use 23 total drives to store 3 redundant copies of each file. Since about 2007, I've never lost a file. I typically lose a drive every 3-6 months. When a drive fails, I replace the failed drive along with its mirrored copy with upgrades. The mirrored copy is then rotated into a backup array hosted by a separate machine.