×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Jobs

Backing-up data

Backing-up data

Backing-up data

(OP)
Running Abaqus models the files created are huge and require a massive amount of storage. I usually back up the model/input/result files on to CDs, or lately DVDs, for future (possible) retrieval. I've been told by our IT people (computer folk for those old enough) that this is unacceptable as the CDs could be damaged or lost, and that they should be backed up on a server, which is also backed-up every night.
What do you do to store old data files (or current in-use files) and how long do you keep them for?

corus

RE: Backing-up data

Hi,
I understand the IT guys' grief, but this is nonsense: even magnetic backup on tapes can be damaged, even a backup on redundant hard-drives can be damaged, everything mankind has invented is susceptible of corruption.
The best way to backup, IMHO, is to be redundant and use several different media: CD + hard drive is the optimum, I think, at the condition that the backup hard-disk is normally switched-OFF and connected ONLY during the periodic backup operations.
Moreover, you can take advantage of the redundancy check with which data are formatted: if you copy data before it becomes too much corrupted to be corrected by the integrity checks, then the copy will be "corrected" (I mean that, as long as the error length is below a certain threshold, the checksum and the intrinsic redundancy allow to understand that, for example, one bit should be 1 even if the read value is 0).

Regards

RE: Backing-up data

Corus:

A suggestion.....Instead of backing up the data files why not back up a compressed version of the ASCII input files...It will be much smaller and will only require time to rerun a model to get to where your data files are at...ASCII files will always be readable whereas the binary files may not be.....As for the current in-use files you are probably stuck but then you are not concerned with long term storage for those files so a DVD should be OK....

Ed.R.

RE: Backing-up data

Guessing that some of the files such as results files can be quite large, and agreeing that a better backup plan would have at least one level of redundancy, is there a minimum set, so to speak, of files that you can backup on CD/DVD that will allow you to recreate any analysis? In most cases, I would guess you could throw away most of the files that any software uses, saving just the bare minimum of input files that would allow you to recreate the analysis. Saving what can be very large database/solution files on two sources seems excessive to me. Saving the input files on two sources seems wise and prudent. Is one of the backup places off site, so that a fire wouldn't ruin all backups?

RE: Backing-up data

(OP)
EdR,
Some jobs take a week or so to complete so the input and output files are retained, just in case someone wants to see the results again. Normally the input text files would be no problem but the output and model files are as some people just won't wait for an answer.

cbrn,
I understand the idea of redundancy, and given the probablility of failure of one or other components, this way would virtually guarantee the safety of files/data. Although what the probability of the data surviving is I don't know. I don't understand the remainder of your post though but will refer it to someone who knows wtf a checksum is.  

corus

RE: Backing-up data

I work remote from my offices and backup servers so I've been through a number of backup options. I run big FEA models that take many days to complete so saving the results files (many GBs)is important as Corus states. DVDs aren't an option as the results files are too big so I use a combination of Iomega Rev drives (35GB capacity) and USB2 hard drives of 500GB. I can use either to backup to the office server every few weeks but still keep at least one copy, preferably two locally. The USB2 drives are excellent, pretty cheap and can be fully recommended.

As for how long to keep the data, that depends on your requirements. In aerospace it is typically over 20 years.  

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Eng-Tips forums is a member-only feature.

Click Here to join Eng-Tips and talk with other members!


Resources