INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

Engineering Software: Verification and Validation

Engineering Software: Verification and Validation

(OP)
Hi there,
Clause 7.6 of ISO 9001:2008, says:"... When used in the monitoring and measurement of specified requirements, the ability of computer software to satisfy the intended application shall be confirmed. This shall be undertaken prior to initial use and reconfirmed as necessary. NOTE Confirmation of the ability of computer software to satisfy the intended application would typically include its verification and configuration management to maintain its suitability for use."

On the Other hand, in ASME V&V 10-2006, we have guidance for Verification and Validation in Computational Solid Mechanics.

Is there anybody out there that has an experience in preparation of a procedure for V&V of computer packages which are used in engineering design process, e.g. Caesar II, Ansys, etc.?
 

RE: Engineering Software: Verification and Validation

Well, you might start by comparing the item under test (IUT) with its specifications.  You then might, for something like an FEA, compare its outputs on canonical problem sets that are run on other, proven software.  The latter falls under the broader category of regression testing.

As with image processing, one would use the equivalent of a "Lena" image or images.  Lena is a headshot picture of a model wearing a hat.  I'd guess she's been featured in hundreds of papers on image processing.  Likewise, one might start off with simple beams that have analytical solutions and go from there.

TTFN

FAQ731-376: Eng-Tips.com Forum Policies
Chinese prisoner wins Nobel Peace Prize

RE: Engineering Software: Verification and Validation

(OP)
Thank you IRstuff for your reply. Would you please tell me more about Item Under Test (IUT)? Considering a broad range of application for engineering software, are you going to say that every item should be tested physically and the result of, say FEM for example, be compared to it? If i'm modeling a storage tank and making an FE analysis for it how can i make a stress (or strain) test on it before the construction begins?

RE: Engineering Software: Verification and Validation

You can't. But you can compare your FEA results for your previous design with tests on the previous design. If you haven't ever done that you aren't doing FE /A/ you are making pretty cartoons for management.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies  http://eng-tips.com/market.cfm?

RE: Engineering Software: Verification and Validation

(OP)
But Greg, isn't it somehow paradoxical? What about validity of the analysis for previous design, if it's the first one?
Did anybody implement the content of ASME V&V 10 for his/her company?
 

RE: Engineering Software: Verification and Validation

It's not common practice to verify proprietary software as it's assumed that they are from a 'reliable' provider (such as ANSYS, Abaqus etc.) and that they have their own QA standards to have checked the software. This doesn't mean that the odd bug occurs but it'd be nigh impossible to find these under some system. Validating the results is another matter, as you're really checking the modelling assumptions, input data etc. You're really down to establishing a level of confidence in the model (and the modeller for that matter), and you can only really do that with actual physical measurements. If that's not possible you can only compare the results from the FE model with theoretical values where these can be taken from the model. Alternatively you can compare the results with other FE models that take a different approach to the problem.   

RE: Engineering Software: Verification and Validation

OK, I thought the OP was developing the software.  The criteria for V & V as a PE are considerably less.  You need to show due diligence, so a reasonable smattering of design examples coupled with a customer list of the software would seem to be apropos to me.   

TTFN

FAQ731-376: Eng-Tips.com Forum Policies
Chinese prisoner wins Nobel Peace Prize

RE: Engineering Software: Verification and Validation

If the work is done for a Nuclear Power Plant, the NRC has some stringent QA/QC requirements that the FEA software has to meet and the software has to have documentation on file with the NRC that it meets their requirements.  Each new version of the software must be re-validated with the NRC.

_____________________________________
I have been called "A storehouse of worthless information" many times.

RE: Engineering Software: Verification and Validation

Corus - I disagree. We check each new release of software that is used for sign-off against the results for the previous release. That's why I'm currently using 2008 software!

This would apply to vehicle dynamics software, crash, and probably general FEA.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies  http://eng-tips.com/market.cfm?

RE: Engineering Software: Verification and Validation

(OP)
IRstuff says: "OK, I thought the OP was developing the software.  The criteria for V & V as a PE are considerably less."

From ASME V&V 10-2006:"...Verification and validation (V&V) are the processes by which evidence is generated, and credibility is thereby established, that computer models have adequate accuracy
and level of detail for their intended use."
So, I think "ASME V&V 10"'s expression is not only for softeware developers. Am i right?

RE: Engineering Software: Verification and Validation

No it's not.  But no one expects a lone PE, let alone a raft of them to validate software to the level that a developer would, particularly with proprietary software, since you would have no clue where the potential weak points of the software might be.  As I posted earlier, you have to do due diligence, but it's not your responsibility to do formal software qualification, although you still need to have proven some level of applicability and validity of the software for your intended application.

Otherwise, of course, we'd be talking about thousands of companies running essentially the same identical tests on the same software, which seems a bit pointless.  Running the same test 5000 times does not make the software 5000 times better.

TTFN

FAQ731-376: Eng-Tips.com Forum Policies
Chinese prisoner wins Nobel Peace Prize

RE: Engineering Software: Verification and Validation

ISO 9001 means that we (vendors) have regression testing in place. Purchasers require 9001 but still do their own tests. Why?

- Steve
 

RE: Engineering Software: Verification and Validation

It's a question of liability.  If a PE is sued resulting from errors in calculations, he generally cannot claim that the software was at fault if he has no proof of due diligence validation testing.

TTFN
Eng-Tips Forum Policies

RE: Engineering Software: Verification and Validation

So customer demands 9001 compliance from vendor and then:

a) Does their own (redundant) testing.
b) Moans that vendor can't respond quickly to change requests.

- Steve
 

RE: Engineering Software: Verification and Validation

Every warranty on MSC software I've ever seen disclaims ANY repsonsibility for mathematical or programming errors and their consequences. So we have to check, I guess.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies  http://eng-tips.com/market.cfm?

RE: Engineering Software: Verification and Validation

I thought I had posted a further comment, but it appears to have gotten waylaid.

Even if the software company does its own due diligence regression testing, that does not ensure that it's applicable to every other application, or even an application that's only numerically different.

The once was math processor bug on the Pentium that was undetectable for 99.9% of the people using the chip.  Likewise, one version of popular math program had a rounding bug  in the 5th, 9th, and 11th decimal places, but only if the preceeding digits had a certain pattern.  This is why one has always be wary of software results, and do as much testing of your own to ensure that your particular problems are correctly solved.

TTFN
FAQ731-376: Eng-Tips.com Forum Policies
[IMG ]

RE: Engineering Software: Verification and Validation

(OP)
From my point of view, this discussion should be divided in two different way (if only Solid Mechanic software are targeted):
1- V&V of software for its declared capabilities.
2- V&V of model, B.C.s and loadings, which depend on capabilities of user, and not software.
BTW, my previous post also has gotten waylaid,it seems nobody already engaged with ASME V&V 10, seriously.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Eng-Tips forums is a member-only feature.

Click Here to join Eng-Tips and talk with other members!


Resources


Close Box

Join Eng-Tips® Today!

Join your peers on the Internet's largest technical engineering professional community.
It's easy to join and it's free.

Here's Why Members Love Eng-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close