×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Jobs

Small firm file server
3

Small firm file server

Small firm file server

(OP)
Has anyone tried using a Network Attached Storage (NAS) for a small engineering office with some remote freelancers logging in?

We have been using Dropbox, which works very well in most respects. It has zero maintenance and setup, it allows access to anyone with an internet connection, and its backed up. The thing that DB does badly is it doesn't lock out other users when someone is working on a file.

I am thinking to have all our files on an NAS which is then backed up with Dropbox.

Does anyone have experience with this?

RE: Small firm file server

I'm in the middle of moving my network storage (about 950 GB) from a computer that is dying to a new WD My Cloud NAS. Started Friday afternoon and I'm not quite to 100 GB moved. Hopefully the problems are with the dying computer and not with the whole "cloud" process. Some of the files are pretty old and some early attempts to scab on metadata were pretty primitive and that is what is crashing to process. Hopefully once the move is over life will be better. I'm hoping to use the WD cloud manager for the remote access thing so I can't help with dropbox questions.

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

(OP)
zdas04 - awesome, hopefully NAS is the way to go. Do you know if you can work on a CAD file directly sitting on the NAS, or do you have to download it locally first? Let me know how it goes with the WD My Cloud. It was on my short list of NAS possibilities.

RE: Small firm file server

I will let you know as soon as I get the damn files over. So far I'm learning not to take too big a chunk at one time (my first cut was to grab the whole drive and drag it to the NAS, ran for 15 hours building file structures before it crashed the old server without copying a single file). Now I've build the first level folders and am coping the second through 25th at the second level folder level. When I crashed this after noon after 10 hours I had successfully moved 100 GB. Should be done by Wednesday.

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

(OP)
Yes, small chunks are key with big file transfers. It always screws up. The thing that always gets me is the 250 character filename limit. If it chokes after 1 hour and you come back 10 hours later, you have just wasted 9 hours

950GB is a lot, btw...what kind of files do you have?

RE: Small firm file server

I use Dropbox and WD MyCloud. Both work fine for my purposes. My group and I have access to almost everything from anywhere. My goal is to get to the point where I have only executables on my laptop and little or no data storage...at least long term.

The biggest drawback I've seen to Dropbox is that if you sync it with any of your computers, you need a lot of free space on those computers. I ran out of space on my laptop from Dropbox taking it all! I changed the hard drive and went to a 1tB drive, which solved that issue but have an occasional windows snafu from the OS transfer.

I did the same as IRstuff...used a dedicated synchronizing program for transfer. Faster and more reliable.

RE: Small firm file server

(OP)
Ron - with your WD MyCloud, can you work on a file like a Word file or ACAD directly on the NAS, or do you need a local copy? I ask because I am sick of the conflicted file copies we get with Dropbox.

RE: Small firm file server

I've been an "active" photographer for over 40 years. I've scanned all the pre-digital negatives to jpg. That is about half of the space. The rest is project files from 11 years in business. I keep everything (I converted the mainframe PROFS archives to Outlook so I have every non-spam e-mail I've gotten or sent since 1980 on there). When I first did the conversion It was on "Bernoulli drives", migrated to CD, then DVD, then as space got cheep (3 TB for under $200, the first 10 MB Bernoulli drive I bought in 1985 was around $50/platter ($5/MB, $5 million/TB) to digital media. When I look back at the old e-mail, we were just as facile 34 years ago as today (most e-mails are simply worthless, but every now and then a search will turn up a discussion that bears on a current project or reminds me of a way to solve a current problem).

IRStuff,
I'm copying over a dedicated ethernet cable which is advertised to be about 100 times the speed of USB3. I think the problem is the dying server I'm coming off of. When stuff works right, this is the fastest copy I've ever seen, problem is that when it craps out, it sits there for hours looking like it is working and then crashes. I'm betting that my transfer should have been under 8 hours without this wonky computer. Day 3 is going better than Day 2 (the copy I started at 8:00 last night is 70% finished at 7:30 this morning, It is over 200 GB copied.

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

glass99..direct access in My Cloud. I believe you can control editing to allow file to only be opened and changed by single user at a time. Check sharing options.

David...I have similar problem of keeping lots of stuff. Not pleased with dedicated scanner results on my negatives. How did you do yours?

RE: Small firm file server

Got a high-end slide/negative scanner from Nikon. Great results.

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

(OP)
Thanks Ron.

Seems like you don't really need Dropbox if you have an NAS plus syncing software like GoodSync to give you a local copy. Dropbox would only then be useful for backup and web access.

RE: Small firm file server

We are using a Synology NAS with Google Drive. The files we work on are local. Not sure if Drive works exactly the same as Dropbox, but it does have the option to use Dropbox instead. Synology automatically syncs any time a file is changed. But you can always just map the network drive and work right off of the NAS.

A mapped network drive setup could involve making a freelancer account that sees specific folders. Synology is pretty easy to setup for either method.

I just put together a freenas server to see how that can work too. In case you want to research some other options out there.

I prefer working off of local files and letting everything sync in the background.

B+W Engineering and Design
Los Angeles Civil Engineer and Structural Engineer
http://bwengr.com

RE: Small firm file server

David...I have a Nikon 4000 slide/negative scanner, but I didn't like the results...but then I hired a college student for the summer and that was one of her tasks. I'll do a few myself and check.

RE: Small firm file server

Mine is the CoolScan V ED. I've had it a few years and the results have been exceptional

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

Thanks, David. Does that one operate on a 64 bit system? Problem I had with my CoolScan IV (4000) is that is will only run on 32 bit system, Windows Vista or earlier.

RE: Small firm file server

It is running on a 32 bit system, but that is the server that is on its last legs. I never even thought about that, I'll look into it tomorrow (or whenever the copy completes).

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

zdas04, can't shut the server down, pull the drive and copy off that way?

As for backups, versions and file locking, there are a multitude of software development tools that do exactly that, the biggest problem with them is that they're not designed for project recording, and thus the configuration and operation tends to be a bit confusing. SVN is one such tool, also solves some of the storage space issues as it only keeps differences in versions of files rather than complete iterations. Of course, a huge issue with them is that they don't run on a NAS system (at least not that I'm aware...).

RE: Small firm file server

Have you considered just continuing to use Dropbox with a better internal communication system between your employees to ensure they're not simultaneously working in the same file?

I use Dropbox for my entire body of corporate files. When I work with other folks, I merely share the project folder with them, not the whole thing. Then I back up everything once a week to a local drive simply using an old DOS batch file I wrote with "XCOPY" (yeah, I know). To make sure me and my subs or collaborators don't step on each other's toes when working on a project, I make sure everyone has an email account with my server (which is run through the google aps 'gmail' style interface) and then I just shoot them a gchat every time I want to work in the same project they're working on, to find out which files they have open.

One thing I will say about backing up to Dropbox - don't do it. It's slow as all get out to have a weekly backup file try to sync through the cloud, because then every week your entire server's contents have to be transferred through your internet connection to the cloud. Huge pain in the rear. I found it much, much more efficient to host all my corporate data on Dropbox, allow it to sync in bits and pieces throughout the week as files change (as Dropbox was intended to be used), and then 'backup' all my corporate files to a local external drive. This gives me redundancy in four locations. One, my main computer, two, my laptop which stays plugged in all day when I'm not out and about catching the Dropbox cloud updates, three, the local external drive, and four, the cloud itself.

Hydrology, Drainage Analysis, Flood Studies, and Complex Stormwater Litigation for Atlanta and the South East - http://www.campbellcivil.com

RE: Small firm file server

(OP)
beej67: I love the convenience of Dropbox. Its really quite amazing how easy and powerful it is for distributed groups and for working on the road. However, the file locking and occasional 24 hr delay in syncing gives me heartburn. Someone not working on the latest file version is potentially very expensive. I am time poor, and this feels like something which should be automated. The other small problem with Dropbox is that file references in AutoCad do not have the same path. File references seem to work half the time.

I understand there are some higher end cloud systems like Huddle and Egnyte which deal with file locking, but they are much more expensive. So: I am thinking that remote staff should do a VPN into an NAS in the office, and we keep Dropbox for backup and for occasional working when offline.

RE: Small firm file server

FreddyNurk,
I thought about it, but the drive is SATA 3.0 (not USB 3) and needs an active computer. I started another step last night and the process crashed an hour after I went to bed. Now I'm having to copy 4th level folders.

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

We are a 5-person office and have been using a Netgear NAS for about 6 years. No adverse issues with small CAD files being worked on directly on the NAS.

Also have a router set up so that people not in the office can VPN in and work as if they are here. Works better than the built-in ReadyNAS remote option.

Works pretty well for us. Have had the drives go bad, so I always have a spare ready to hotswap.

We do a local backup to an external hard drive, and also cloud backup.

RE: Small firm file server

(OP)
greenone: Great to hear that the NAS+VPN combo works in the real world. I think I might dive in and get one.

Can anyone recommend specific models of NAS's? Which Netgear/Synology/WD should I get? Assuming max 5 people using it, no more than 2TB required.

RE: Small firm file server

I have heard that you should always size your NAS for a minimum of 3x total system size.
*** 950 GB server X 3 = 2.850 TB... But a 3TB only allows 2.7 TB of usable space, therefore bump up one size.
-Not a rule only guidance and sometimes the cost is small for increasing in size now vs later.

RE: Small firm file server

glass99 -

I hear you. Good luck with the NAS thing, and report back once you've got it set up. I may switch to something like that eventually.

On the AutoCAD path issue - I always keep all references in the same folder, and I don't reference a path when I drop in the reference. That usually keeps things loading properly.

Hydrology, Drainage Analysis, Flood Studies, and Complex Stormwater Litigation for Atlanta and the South East - http://www.campbellcivil.com

RE: Small firm file server

<tangent>
Re AutoCAD paths

One of my former employers made much use of standard blocks, all kept in a few directories on the sole file server.

... which worked okay for everyone else, except lucky me, who got assigned the fastest computer in the place as my workstation. ... yep, the file server.

I could not find a way to stop AutoCAD, or MSDOS, I never knew which one, from helping me by substituting C:\blockpath whenever I put in \\\fileserver\blockpath when it happened that I was working at the fileserver. So every time I edited a file containing blocks at my computer, I had to then go to someone else's computer and fix the damn block references.

... else the boss got all pissy about not being able to find the blocks. That would be the same boss who wouldn't 'waste' the money on a dedicated fileserver stuffed in a closet.

</tangent>

Mike Halloran
Pembroke Pines, FL, USA

RE: Small firm file server

(OP)
Mike: yes, the time and frustration cost of futzing with things like filepaths on an ongoing basis is significant. Unless of course you are billing by the hour, in which case its amazing.

RE: Small firm file server

zdas, have you considered using ZFS or other bitrot resistant file systems?

RE: Small firm file server

Referencing blocks in a central location is a holdover from the days back when I wasn't using terrabyte external drives as doorstops.

Hydrology, Drainage Analysis, Flood Studies, and Complex Stormwater Litigation for Atlanta and the South East - http://www.campbellcivil.com

RE: Small firm file server

(OP)
beej67: Curious how you organize your CAD files on your server. Do you have references for blocks, titleblocks and architectural backgrounds all in the same folder as your current CAD files? Do you copy the whole thing to an "Issue" folder every time you issue a revision to the drawings?

We don't have large numbers of sheets in our drawings, and try to put all the sheets in one dwg with minimal XRefs.

RE: Small firm file server

TomDOT,
I'm not familiar with ZFS or the term "bitrot". After 34 years of computers being a major part of my practice, I am pretty familiar with the sins of our past causing problems with our futures (that is my main problem right now, improperly formed meta data on photographs that end up corrupting files). I will look into ZFS and see if I can incorporate it or similar technology.

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

zdas, Wikipedia is telling me I should call it "Data Degradation" - but "bit rot" is the term I hear used.

Basically, a single random bit flip will often corrupt an entire file (photo, whatever) in a conventional file system. While it's not likely to be totally unreadable, there will often be significant negative impact. With a "next gen file system" (ZFS or btrfs) the bit flip is automatically detected and corrected.

Ars Technica has some good articles, including example photos with a single bit flipped in both conventional :

http://arstechnica.com/information-technology/2014...

http://arstechnica.com/information-technology/2014...

RE: Small firm file server

Freenas uses ZFS. I was able to build a test server for about $850. That was with five 3TB Western Digital Red drives and 12GB of ECC ram (ECC is a must with ZFS).

I did write a small followup to my experience with our Synology NAS, which works very well. http://bwengr.com/blog/google-drive-cloud/ 100GB of Google Drive is now $1.99/month.

Relative xrefs are your friend.
project/xrefs
project/sheets
Previous revisions just get the bundle of sheets e-transmitted to an Archive folder.

B+W Engineering and Design
Los Angeles Civil Engineer and Structural Engineer
http://bwengr.com

RE: Small firm file server

3

Quote:

beej67: Curious how you organize your CAD files on your server. Do you have references for blocks, titleblocks and architectural backgrounds all in the same folder as your current CAD files? Do you copy the whole thing to an "Issue" folder every time you issue a revision to the drawings?

We don't have large numbers of sheets in our drawings, and try to put all the sheets in one dwg with minimal XRefs.

I copy everything I need to reference into the same project folder. Blocks, tifs, images, title blocks, USGS quad rasters, whatever. I do not save CAD files from previous issues, because those always get mucked up anyway by changes to the references. Instead, I keep PDFs of each issue, which are dated and then secured against modification with a password. In fact, I don't print from AutoCAD at all, I send the PDFs themselves to the print shop. Mostly I find that clients just want the PDFs, so I only ever print if I'm submitting for permit or issuing CDs.

Most of the old methods regarding referencing blocks and the like in a central location were holdovers from back when storage space mattered. It no longer matters, period. Most of the old methods regarding keeping revision history in DWG format were likewise holdovers from back when PDFs weren't a thing, and everyone had to plot out of AutoCAD. I just dumped the holdover policies and moved on with my life. Revision history in DWG was a nightmare, especially if you have your blocks in a central location, because suddenly new changes to your blocks show up in old already-issued drawings. And if you are pulling an old revision for some reason, chances are that reason is some kind of disagreement about what you sent out, so you definitely need to know exactly what you sent out.

I come from a background where we would have three or four XREFs blended so we could have multiple designers working on the same project at the same time. One doing utilities, one doing profiles, one doing grading, etc, in separate DWGs that reference each other. As I'm a smaller shop now, I much prefer to have everything in one file. I'm iffy on survey - sometimes I keep it in an XREF, and sometimes I just drop it in as a block. It's nice to be able to move survey text around, but you run the risk of accidentally deleting something important, so I usually lock the layers.

my .02

Hydrology, Drainage Analysis, Flood Studies, and Complex Stormwater Litigation for Atlanta and the South East - http://www.campbellcivil.com

RE: Small firm file server

beej67,
Thanks for that, You covered a bunch of topics I've been struggling with. I like your solution.

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

(OP)
beej67: This is great information. We have been eliminating xrefs to the maximum degree possible too, including putting title blocks directly into the drawing. I have always thought of xrefs as being over complicated and fiddly. We do keep the CAD file with the old revision though because from time to time we have to go back. Its kind of like a big undo button. We accidentally deleted a bunch of work one time, and were able to reconstruct from the previous issue.

RE: Small firm file server

That's what backups are for. Version tracking in AutoCad is simply a nightmare.

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

(OP)
zdas: speaking of which, how do you back up? An online service?

RE: Small firm file server

There are commercial packages around for revision tracking and so on for AutoCAD (Vault is but one), all of them have their benefits and drawbacks as well.

A lot of the issue with XREF stuff is whether or not the data contained in the XREF is likely to change. A manual process to import and bind all the XREF data, and then remove it again when the client issues an update (building services for example, hopefully survey stays static...) as opposed to just linking the new file in when the client changes something, is often labour intensive.

I've largely been for the use of XREF, though it hasn't suited all my applications and others have ended up binding in the data and then having to scrap the revision. There are also options for exporting all relevant drawings including XREFs and static blocks in a package so that they're kept clean and unencumbered by such issues as what beej mentions, but that leads to where to store them (i.e. version them). Later versions do funky things like sheet sets with automatically generated drawing lists and title block properties, but not many places use them.

Conceptually there are heaps of good tools that do the equivalent for software development, and it does get frustrating to have to wrangle through AutoCAD's methods instead of having a system similar to code and libraries, but it sounds like most of us are in similar situations.

RE: Small firm file server

I use an online service, it runs an incremental back up every night and keeps 30 days of incremental backups. If you fail to notice a bust in a month then it is lost. I've had to recreate files 2-3 times in the years I've been with them and the process worked well.

David Simpson, PE
MuleShoe Engineering

Law is the common force organized to act as an obstacle of injustice Frédéric Bastiat

RE: Small firm file server

note that NAS and Dropbox (and similar) are not true backup solutions.
a virus like Cryptolocker could still infect your files, as it could connect to the NAS without any problems.
dropbox is a little better though, as you are able to get the previous version of the file. i don't think that google drive has such options.

i still think that NAS with raid 1 + cloud is the best solution for small companies.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Eng-Tips forums is a member-only feature.

Click Here to join Eng-Tips and talk with other members!


Resources