You need a good understanding of structural behavior to build a good model. If you don’t know the differences between trusses, beams, plates, shells, etc, how can you choose the right elements for your structure? I guess you get this from college classes and years of experience.
You also need to know something about the theory of finite elements. If you want an overview without too much math, consider the following books:
Finite Element Modeling for Stress Analysis by Cook
Building Better Products with Finite Element Analysis by Adams & Askenazi
To gain confidence in the software you’re using, run simple test cases and try to reproduce expected results. Most software comes with example problems, or better yet, devise simplified cases relevant to your problem. If you’re dealing with plates, get a book on plate theory, make a simple model of a square plate with uniform pressure, and see if you get the textbook solution. If not, don’t give up until you find out why. You'll learn a lot from this process.
Also, use the simplest possible model that adequately represents the behavior of the actual structure. It’s easy to make an overly complicated model, but it will be more difficult to de-bug. Try to get in the ball park with a simpler model, and add complexity only if necessary. And you can’t put too much faith in the answers if you don’t know the input data very accurately. You might have reliable material properties, but how well do you really know the loads? Seemingly small details can have big effects.
Besides NAFEMS, here are a few web sites that might be useful. The last one contains course notes for a college FEA class.