Quote (geesamand)

...

For the last 15 years I've simply kept all email within my Inbox and relied on searching.

...

Wow! You are going to have fun cleaning that up.

I firmly believe in archiving emails. I occasionally look for stuff I wrote a long time ago. Given the cost of disk storage these days, I suspect that your company is determined to not have old emails lying around, for legal reasons. Maybe the best thing is to wait and let the purge do its thing.

--
JHG

Our company uses GFI archiver. I literally don't have to do anything. It automatically archives everything. I'm just obligated to file emails into project folders as appropriate. GFI archiver is essentially the backup.

Anything directly project related is filed through our Newforma Project Centre. This program keeps track of correspondence, transmittals, shop drawing reviews, etc. It's also a fairly slick program. It's one button in Outlook to file emails.

Just wait until the bean counters see the bill for work that needs to be redone due to a purged email... it won't happen overnight, but it WILL happen. Then the real fun begins.

Dan - Owner
http://www.Hi-TecDesigns.com

No. The cost to a bean counter of a potential imaginary lawsuit always supersedes realworld cost no matter the amount.

Work needing to be redone is very real but also extremely squishy and arguable. The time required to adhere to a tight email retention policy (non value-add time) is similarly hard to pinpoint. Cost of lawsuits is comparatively easy to quantify and scare higher management. I'm too pragmatic to charge that hill and go to battle over it.

I would like to know how other engineers assess what is "essential" and worthy of a longer retention vs. simply keeping it all or let it all get deleted. And with historical emails - what you did about those.

To be clear, I do not have several years' email in one folder. I have been segregating it by calendar year to keep it manageable. However I had not categorized my emails in a way that relates to its retention value.

I was a PROFS user on the mainframe from 1980 to 1992 when we went to Outlook. I still have all of my PROFS e-mails (about half of the people I remember from those tomes are dead) and all of my outlook files from before I retired. The lawyers hate me. But still today I get a "do you remember ..." question that I can answer in one of the archives (I had one just today). I never had a company lawyer say "you HAVE to purge", they always said "you NEED to purge" and I always felt like "need" was low enough grade that I didn't really need to. Some of that crap is still interesting. I have purged the SPAM many times over the years, but I've gotten rid of very few e-mails that were addressed to me instead of "occupant". This document-retention garbage is counter to effective access to information.

David Simpson, PE
MuleShoe Engineering

In questions of science, the authority of a thousand is not worth the humble reasoning of a single individual. Galileo Galilei, Italian Physicist

To be fair... Being counter to effective access of information is exactly /why/ lawyers suggest it.

Dirty laundry stays dirty if you keep it in a bin. Getting rid of it ensures that someone else doesn't find a big stinking pile of stuff.

As the numerous email leaks over the past few years have demonstrated, even relatively innocuous email chains can be re-interpreted and re-imagined into nefarious conspiracies.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert! https://www.youtube.com/watch?v=BKorP55Aqvg
FAQ731-376: Eng-Tips.com Forum Policies forum1529: Translation Assistance for Engineers

Ethically, it's not your decision whether to keep it or not, in my opinion. If you decide not to follow company mandate and end up keeping stuff that later is found out, your posterior may be in a sling, not to mention whatever pitfall it may create for the company.

It is better to have enough ideas for some of them to be wrong, than to be always right by having no ideas at all.

some people actually print and file emails...

you can purge all your emails but there are nearly always copies on another device somewhere that you are unaware of. recent events have proven that

I ran into this issue too. Problem was, IT set the policy so email couldn't be kept more than 1 year....and I had projects lasting more than 5 years.

I go against policy and save my email anyway. I know it's there to protect the company... I keep mine to protect myself. I'm not going to rely on anyone else to remember what happened...not even Wikileaks...

______________________________________________________________________________
This is normally the space where people post something insightful.

Geesamand, are you the type to put the correct cover page format on your TPS reports?

Quote:

... I used to download email by year to a .pst file that I'd keep as a DVD in my desk drawer. I'm told off-line copies are against policy...

Who's looking through your desk drawer?
And if they do find a DVD that you have carefully labelled "cutest cat photos of 2014", why should they get excited?
Even if they do try to access the DVD, they will be disappointed that the video won't play, despite your efforts to carefully name that 12 GB file with the correct ".VOB" extension.

Simple steps like these will rule out discovery by anyone who is neither nosy nor resourceful enough to discover what's on that DVD.

You've asked this question because you know you have your own interests to look out for. Read the advice from Controlnovice above - ask yourself if you believe the same thing.

STF

Actually I read the retention issue differently, in terms of keeping records rather than ensuring that they're purged, so as to keep appropriate records in archive and with the project.

That said, at least one of my former employers had a much smaller storage allocation (i.e. 400Mb ) so my mailbox would stop receiving emails when full and I'd have to purge them. I also did as controlnovice did, and kept my own copies of correspondence. I was most unimpressed when a later employer blocked the Outlook install so I couldn't archive to .pst either. That company had their own archiving system per users mailbox. I also recall issues with local .pst storage and Outlook losing its brain if the local storage gets beyond a certain size, which was also motivation for IT to enforce retention restrictions.

To be honest though, I don't see what the issue is with turfing emails or having a short retention period (even 90 days). In all the cases I can think of, the email should be stored separately (ideally indexed and recorded centrally so others can see it) if its work related, rather than relying on individual mailboxes as a proxy storage medium for work related correspondence. If that's done, then short retention periods are no issue at all. Same with archiving to relevant formats for personal records as needed.

I get to keep emails for 2 years. Then it is automatically deleted.
Oh and the archiving and exporting functions are completely disabled. The only way to actually save an email is drag it to the local hard drive as a file... meaning you lose all timestamps/searchability/metadata, etc.

Super great fun especially since major shutdown/turnarounds are 2+ years apart.

Do "Whistle Blower" laws apply? I guess this should be on a legal site. And I guess it depends on the information you may have and who it help or hurts.

with the low price of storage these days, the only reason to purge or "archive" files before a project is finished is liability. And archiving really does not reduce liability or cost. that said, 99% of all emails are not worth keeping. The important ones can be easily pdf'd, or printed, or copied to the project files on a one by one basis. Better yet, any decisions should be documented in a decision log and not just in an email.

I might question whose liability. If a bridge collapsed and I was the PE on the design, having an email that shows that I instructed the contractor to do such and such would be very helpful if it was my head on the stick.

Emails go to someone or are received from someone. In many cases, for legal purposes, those "someones" might not have a retention policy and have a zillion emails that someone can subpoena to get what YOU said anyway. While I try not to keep a lot of emails, there are some that are worth keeping for definitive purposes, as controlnovice noted.

Retention policies are just another way of someone in a corporate position showing how important they are to the company. Mostly worthless.

My employer imposed a retention policy after one court case involved the opposing counsel reading (and charging for reading) 7000 boxes of documents, many of which were identical, just from different people's desks.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?

People seem to forget about the fact the other guy you sent the email to has a copy of the email and will probably be keeping it for a century and you can't make them erase it. I fail to see the point in forcing the deletion at just one end. The lawyers will be at the other end.

I spent a lot of time trying to find a easy way to organize emails for myself and to facilitate transfer to our public project email folders. I found a method for me that keeps me from pulling my hair out when retaining emails and has really helped me stay organized.

My firm has a public folder organized by year with project subfolders labeled with job number. I use the "categories" labeling with a color. I create a category when a new job comes in and also create a search folder that captures incoming and outgoing mail. Each time I send an email or receive one that I consider worth keeping, I label with the appropriate job label. Apparently, there is an endless number of colors in Outlook to label with - I have yet to find a limit; so far so good. Every 2 weeks on Friday, I copy the labeled emails to their respective public folders. THEN...I label those emails with a category called "copied to public folder". That way I know which emails I already copied over and how many are remaining to do so.

I used to dread keeping track of emails, but now I really, dare I say, enjoy it.

"It is imperative Cunth doesn't get his hands on those codes."

I've got a couple of scripts we use at work to either save selected emails or save a whole folder of emails. They save the email with the file name "sent date - sent time - sender - subject.msg". Basically, add the script as a VBA module and then create a custom toolbar button to access it. For the save individual emails we just select the emails and then hit the save button. It asks for the directory and saves them. Pretty easy to use.

On a Windows server or even a PC, you turn on the indexing and add on some search filters and then you can search the bodies of all the different common file types - very quickly too. The directories we use are per order or project so we also save any other files related to that project into the same directory (or a sub-directory under it). In Windows Explorer, you can use the search to find various files and also turn on the previewer to look at them. The Windows search has some fairly decent abilities if you learn how it works.

Lionel,
I use your second recommendation frequently.
Can you elaborate about your first recommendation: is this script custom-written?

STF

Quote (Agent666)

People seem to forget about the fact the other guy you sent the email to has a copy of the email and will probably be keeping it for a century and you can't make them erase it. I fail to see the point in forcing the deletion at just one end. The lawyers will be at the other end.

This strategy is not about hiding possible evidence. In fact it's well agreed that in legal discovery, email is usually just sifting through immense stashes of garbage w.r.t. admissible evidence. Email is a huge part of the cost of defending yourself in court. The reason is to keep the stash of crap to a reasonable volume so that the plaintiffs can't use that as leverage to pry a settlement out of you.

I suppose if you or your company make nefarious decisions by email, you could have a second reason to purge your email.

Jeez...I wouldn't trust a guy named "lionel Hutz" for much of anything!

"It is imperative Cunth doesn't get his hands on those codes."

They're modifications, well really more a few tweaks, of scripts found on the internet. They're really not that complicated. I could post the scripts if you want either one.

The save one you just select the emails and run the script which brings up a window to select the location and then saves them.

The save all will save emails into a file structure on your disk that matches the inbox structure. You can select to only save a certain email folder or group of folders, but it still matches the file structure to the inbox structure. It would work perfectly for David if he wanted to dump everything onto a hard drive.

https://www.youtube.com/watch?v=HODjrjb32fU

Dan - Owner
http://www.Hi-TecDesigns.com

At the end of this, here's what I did.

I chose a handful of topics that were of crucial importance to me. Basically a couple of product lines I was involved with and a couple of customers in which I had a number of email correspondences. Then I used search terms to pull emails from my past into those folders.

After this I found there was too much email in the archive area, so I sorted the new archives by size and went through the largest ones individually to delete ones I did not care about.

Warning: if you use Outlook, sometimes it copies emails and other times it moves them. If it's copying the emails, some of those emails can match more than one search and leave you with duplicates in your archive. It's not smart at all, and it doesn't warn you about when it's creating duplicates.

Warning 2: if you use Outlook, your Calendar counts against your storage use. My calendar was purged a couple of years ago and it's still 1GB in size. That's a bit frustrating, since I can't control the size/quantity of attachments to meeting invitations...

I wanted to save some stuff to PST to clear up my archive storage limits but PSTs are completely disabled.

If your company thinks that deleting Emails is going to lessen liability, they're going to be shocked. What would they rather have happen, be out front and turn over everything they have or be surprised about something incriminating from another source? Why don't we shred all documents after 10 years? It's a new world and we need to get used to it. Everything is saved somewhere.
We have a archive search function and I routinely turn up Emails that are from 1998.

It's not about limiting the liability. All parties could have a copy of an important email. The facts are the facts.

This is really about the cost of the legal discovery process and keeping it contained enough that you can still defend yourself.

Even when you have a winning argument and evidence, the amount of documentation review that the process can mandate makes defending yourself prohibitively expensive. You must pay your lawyers to go through "everything" in preparation. If "everything" is sufficiently large, you can't afford to even begin a defense. I suspect too, that once the company finds itself settling, other lawsuits will become optimistic about their success.

I'm not legally trained so I hope I used the terms correctly.

geesamand,

Now that your old emails are deleted and are no longer an apparent threat, what does your company do with its old backup tapes?

--
JHG

I expect they discard them beyond that window of time. They might retain the backups for a few months after this "big purge" just in case someone made a major mistake. Would you like me to confirm?

Limiting liability... I brought up an issue once with compliance. All my emails corresponding to a particular exchange as well as all of my other emails for the month of August just vanished once my complaint got to someone that understood what I was bringing up. I still don't know what to make of that other than more effort was put into trying to sweep up evidence than maintaining compliance.

I just did a search on destroying backup tapes, and I found an interesting link.

NIST Special Publication 800-88 Guidelines for Media Sanitization.

--
JHG

It seems so unlikely, with the advances in search and parsing, that there is a significant cost to going through emails.

See:
http://www.aid4mail.com/email-ediscovery
(consolidated list) http://www.capterra.com/electronic-discovery-softw...

With this in mind, it's more about hiding data than the cost of digging through it.

Also, an article written to law firms as a guide to the process:
http://www.americanbar.org/publications/law_practi...

The reason that these tools exist is because sometimes defendants will dump TBs of data expecting to 'hide' information. That worked well with printed material, but not so much with electronic data.

One thing that is true is that (beyond the size where it is trivial) the size of the data doesn't affect the skill level to dig through it.

On the topic of where lawsuits affect companies, anyone else remember Blitz USA? Nothing to do with emails, but an example of being found liable. I haven't seen details about the evidence that would tell if the company had internal documentation that influenced the verdict or if it was entirely external actions.

My company has a 60 day rerention policy on inbox and sent box -no restrictions on user created folders

3DDave,

The only way email would not be searchable would be if you encrypted everything, and were not going to be required to give up the key. I have done several searches through my email archives at home. On my Linux box, my emails are stored in mutt format, which is a plain text file for each email. With the Linux command line, I can search on dates. I can search on keywords. Within my email tool, I can search on sender and subject.

I don't know how Microsoft Outlook works. Even if the files are binary, either they are encrypted as noted above, or there is plain text embedded, and all I have to do is strip the binary stuff. I have written a very simple C program that does exactly that.

--
JHG

The search software also looks for misspellings, and uses a Thesaurus for similar terms, et al. Much more than a straight text search. As the related pages indicate, there is also the problem of extracting attachments. They also make a traceable package of emails, which is the required deliverable.