Backing-up data

[corus]

Running Abaqus models the files created are huge and require a massive amount of storage. I usually back up the model/input/result files on to CDs, or lately DVDs, for future (possible) retrieval. I've been told by our IT people (computer folk for those old enough) that this is unacceptable as the CDs could be damaged or lost, and that they should be backed up on a server, which is also backed-up every night.
What do you do to store old data files (or current in-use files) and how long do you keep them for?

corus

 

[cbrn]

Hi,
I understand the IT guys' grief, but this is nonsense: even magnetic backup on tapes can be damaged, even a backup on redundant hard-drives can be damaged, everything mankind has invented is susceptible of corruption.
The best way to backup, IMHO, is to be redundant and use several different media: CD + hard drive is the optimum, I think, at the condition that the backup hard-disk is normally switched-OFF and connected ONLY during the periodic backup operations.
Moreover, you can take advantage of the redundancy check with which data are formatted: if you copy data before it becomes too much corrupted to be corrected by the integrity checks, then the copy will be "corrected" (I mean that, as long as the error length is below a certain threshold, the checksum and the intrinsic redundancy allow to understand that, for example, one bit should be 1 even if the read value is 0).

Regards

 

[EdR]

Corus:

A suggestion.....Instead of backing up the data files why not back up a compressed version of the ASCII input files...It will be much smaller and will only require time to rerun a model to get to where your data files are at...ASCII files will always be readable whereas the binary files may not be.....As for the current in-use files you are probably stuck but then you are not concerned with long term storage for those files so a DVD should be OK....

Ed.R.

 

[prost]

Guessing that some of the files such as results files can be quite large, and agreeing that a better backup plan would have at least one level of redundancy, is there a minimum set, so to speak, of files that you can backup on CD/DVD that will allow you to recreate any analysis? In most cases, I would guess you could throw away most of the files that any software uses, saving just the bare minimum of input files that would allow you to recreate the analysis. Saving what can be very large database/solution files on two sources seems excessive to me. Saving the input files on two sources seems wise and prudent. Is one of the backup places off site, so that a fire wouldn't ruin all backups?

 

[corus]

EdR,
Some jobs take a week or so to complete so the input and output files are retained, just in case someone wants to see the results again. Normally the input text files would be no problem but the output and model files are as some people just won't wait for an answer.

cbrn,
I understand the idea of redundancy, and given the probablility of failure of one or other components, this way would virtually guarantee the safety of files/data. Although what the probability of the data surviving is I don't know. I don't understand the remainder of your post though but will refer it to someone who knows wtf a checksum is.

corus

 

[Jordonlaw]

I work remote from my offices and backup servers so I've been through a number of backup options. I run big FEA models that take many days to complete so saving the results files (many GBs)is important as Corus states. DVDs aren't an option as the results files are too big so I use a combination of Iomega Rev drives (35GB capacity) and USB2 hard drives of 500GB. I can use either to backup to the office server every few weeks but still keep at least one copy, preferably two locally. The USB2 drives are excellent, pretty cheap and can be fully recommended.

As for how long to keep the data, that depends on your requirements. In aerospace it is typically over 20 years.