×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!
  • Students Click Here

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

Jobs

Ethical question regarding tolerances on test plan
4

Ethical question regarding tolerances on test plan

Ethical question regarding tolerances on test plan

(OP)
I'm new to engineering, so I'm looking for the opinions of some more experienced engineers regarding the following ethical concern, which I will phrase as a hypothetical.

Suppose there's an aerospace company that's trying to get a product into the market. There's a specification that defines the required performance, and a test plan that's been approved by the would-be customer as a means of demonstrating compliance with the spec. The spec requires the product perform at a temperature of 100°C, so the test plan calls for the unit to be placed in an oven at 100+/-5°C for three hours (+/- 5 minutes), then removed from the oven and operated within five minutes.

The engineering team discovers that the product will not function satisfactorily at 100°C. Engineer A suggests setting the oven to 96°C, and lowering the temperature of the test stand room to improve the product's chances of passing the test. Engineer A further suggests leaving the unit in the oven for exactly 2 hours and 55 minutes. He further suggests building a "quiescent oven" to use instead of the standard convection oven that the company has always used to test similar products.

The question is: Is Engineer A's proposal ethical?

It will be Engineer B's responsibility to execute the test. If Engineer B's boss says that Engineer A's suggestions should be followed, what should Engineer B's response be?

RE: Ethical question regarding tolerances on test plan

Did the customer approve the test plan, or is it internally defined.

Just FYI if I had specked something to perform at 100°C then I'd expect that the minimum temperature used during testing would be 100°C.

Posting guidelines FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm? (probably not aimed specifically at you)
What is Engineering anyway: FAQ1088-1484: In layman terms, what is "engineering"?

RE: Ethical question regarding tolerances on test plan

(OP)
Greg,

Sorry to peg your irony meter, but I did tell you I'm new at this. It's not from a textbook, but it is against company policy to post details of a real case.

Matt

RE: Ethical question regarding tolerances on test plan

Technically, it would appear that the test was being conducted within the letter of the spec, i.e., within the tolerances given, but only if the specs were understood to be the TOLERANCES for the test environment. However, you already know that your company is on shaky ground since the product is just barely able to pass the test, even by shading the tolerances toward the best possible result, so if later some of the items delivered did fail and someone audited your testing procedure it could be argued that your tests were invalid if what the customer was relating to you were NOT TOLERANCES for the test environment, but rather the TOLERANCES for the functional environment which the product was going to subjected to in production. That is, the expectation was that the testing was done to assure that the product would actually work in an environment where the temp could vary from 95°C to 105°C and could be expected to be in such a working environment for time periods of 2:55 to 3:05. If THAT's the case, then you have failed to verify that your product will perform as the customer had expected it to.

So I guess to put it another way, unless you can get them to put it in writing that the tolerances given are for the testing environment ONLY and are not being given as an actual RANGE across which they're expecting the tests to be conducted, you should be prepared to accept the consequences that may occur if you're assumptions are later questioned as whether you've meet the letter or the intent of the contract.

John R. Baker, P.E.
Product 'Evangelist'
Product Engineering Software
Siemens PLM Software Inc.
Industry Sector
Cypress, CA
Siemens PLM:
UG/NX Museum:

To an Engineer, the glass is twice as big as it needs to be.

RE: Ethical question regarding tolerances on test plan

Depends on what you report. If the report states that the unit was heated in a "X" oven at exactly 96 degrees for 2 hours and 55 minutes before testing. Then the client should be able to accept it. But are you able to exactly state that the temperature never was below 95 degrees in the oven? Also what information can you supply that comfirms that you were able to. I don't think the oven manufacturer would state that it would.

Garth Dreger PE - AZ Phoenix area
As EOR's we should take the responsibility to design our structures to support the components we allow in our design per that industry standards.

RE: Ethical question regarding tolerances on test plan

Interesting question.

Apparently an aluminum manufacturer decided they could save money by adjusting alloy content from nominal values of the aircraft sheet metal specification. If you have ever examined the alloy content ranges on such a specification, you know they are quite wide. With better specification control, they realized a more consistent product and increased profits. Meanwhile an aircraft manufacturer that had developed a massive library of strength, fatigue, and other material allowables for the aircraft sheet metal specification started to notice unexpected issues in the field since all their test data was based on the previous alloy properties. Needless to say this caused quite a row...

In your case, the stakes are likely nowhere similar but there are lessons to be learned. Honestly, I would say that Engineer A's approach is more resourceful than unethical and this is more a training/knowledge issue than anything else.

Arbitrary specifications such as this 100°C requirement come about general because past experience has shown them to be reliable. We can "sharpen the pencil" or otherwise try to squeeze into the requirement but in doing so, you really are shaving away from the reliability of the system.

You can go too far the other way as well, adding conservative requirements. One time I was told by an experienced certification guy that he always likes to have the techs drag flammability samples around on the shop floor to make sure they have a good amount of dirt and oils to simulate degradation of the samples in service before testing. Well, that skews things away from the requirements which again have been established as reasonable.

Stick to the nominal.

RE: Ethical question regarding tolerances on test plan

A good analogy for any sort of tolerance scheme is to look at it as if the numbers represented a 'budget' which can be 'spent' across the lifecycle of a product's production. If the first guy in the loop 'spends' all of the 'money', that doesn't leave much flexibility for the rest of the downstream contributors.

John R. Baker, P.E.
Product 'Evangelist'
Product Engineering Software
Siemens PLM Software Inc.
Industry Sector
Cypress, CA
Siemens PLM:
UG/NX Museum:

To an Engineer, the glass is twice as big as it needs to be.

RE: Ethical question regarding tolerances on test plan

I think it's baloney to game the specs like that, but if a contractor did it I wouldn't have a leg to stand on.
Tolerances are for things that can't be controlled.

RE: Ethical question regarding tolerances on test plan

2
Then again you may have a good client who needs the unit to work at 80 or 90 degrees maximum. So they added a safety factor to 95 degrees as a mimimum and spec that 100 degrees plus/minus 5 degrees is the testing they wanted done. Someone really needs to talk to the client and I nominate "Engineer A" first and the "Boss" second.

Garth Dreger PE - AZ Phoenix area
As EOR's we should take the responsibility to design our structures to support the components we allow in our design per that industry standards.

RE: Ethical question regarding tolerances on test plan

100+/-5°C means the oven is held to 95 to 105 as the thermostat cycles. If you run it at 96, what is the tolerance on that? Drop anything over a degree before the stat kicks on and you'll be under the 95. What's the tolerance on the thermostat itself? Will it always kick on at precisely 95.1? Every time during the three hours? And whats the accuracy and tolerance of the thermometer you are monitoring with? Tested and traceable to NIST? Crowd the edges and the nice loose tolerance tightens.

RE: Ethical question regarding tolerances on test plan

(OP)
SAITAETGrad, you say to stick to the nominal, which is probably a good idea. But what if my boss says to game the tolerances? Can I convince him that he's wrong? Do I have an ethical obligation to stick to the nominal, or to inform the customer that we intend not to?

I have a feeling that if my boss was intent on hiding this from the customer, he'd write the test report himself. It's not good when your boss feels he has to do your job for you. On the other hand, gaming the spec creates a culture of dishonesty that could ripple through all the testing we do.

Woodman, if the customer were on board with this, my conscience could rest easy. But I think it's unlikely that "A" or "boss" would agree to tell them, and it's not "B"'s role to inform them.

RE: Ethical question regarding tolerances on test plan

Unless the "would-be customer" agrees to Engineer A's changes, I would stick with the "agreed to test plan" to demonstrate performance compliance. Never change agreements w/out the customer's full knowledge and consent. Further, I would tell the customer the part doesn't function per the "agreed to test plan" at 100C/3 hours and determine a course of action to meet their requirements, unless they agree to change the test. It's a test to determine performance in its intended purpose, which is eliminating your product.

All of the specifications I've seen and used were stated as XXX +/-Y, which is the mean and its tolerance. The customer expected that and not XXX-Y +/-Y. There is a difference because you're stating different means. If your customer expects XXX +/-Y and routinely gets XXX-Y +/-Y, he will eventually tell you your process is out of control and look for another supplier, if he understands manufacturing. For example, some of the parts I've worked on the in past now have a spec of XXX +/-0. They now have to be dead on. XXX-Y +/-Y is far from dead on in some parts. In some other parts, we moved the mean to gain other improvements.

Is it unethical to be ignorant? No. It is unethical to be devious? Yes. Only you know what's going on completely and can provide your ultimate answer.

Engineers should conduct themselves as professionals. You always treat your customer fairly and honestly. That's the right thing to do. If you're looking for guidance here, you know something stinketh and you probably know what it is. Whatever you do take the high road and protect your customer as well as your working relationships. It's nothing personal. It's good business. Through good business and professionalism, you will gain a good reputation and those are hard to get.

Pamela K. Quillin, P.E.
Quillin Engineering, LLC

RE: Ethical question regarding tolerances on test plan

I would suggest that if the product tested was the absolute worst example of the product to ever be produced... you should have no problem... If, however, there was a failure and the testing were audited... you're baked... at whatever temperature! Could be something else... totally unrelated that fails and brings down whatever part you are testing... it might then be 'put under a microscope'.

If life safety were involved... talk to your lawyer... you could be criminally responsible.

I don't agree with the methodology, but to the letter of the spec you have achieved a passing grade... this time!

Dik

RE: Ethical question regarding tolerances on test plan

mdgates,

You may need to read carefully a contract. If you guarantee that you will pass the tests you describe, then you must test the device at 105°C for 3 hours and 5 minutes, or whatever the worst case is.

--
JHG

RE: Ethical question regarding tolerances on test plan

Call your lawyer now or later!! You pick..

RE: Ethical question regarding tolerances on test plan

Is this a for a civilian or military application? If this is a civilian application, how are you obtaining FAA approval?

"On the human scale, the laws of Newtonian Physics are non-negotiable"

RE: Ethical question regarding tolerances on test plan

The 100 C in the spec is a minimum.

The test procedure is not compliant, and should not be approved.

Engineer A is a scum bag.

Your warranty costs will be high.

RE: Ethical question regarding tolerances on test plan

I agree with MJ. The product specification is 100°C, so knowingly testing to less than that as an acceptance test is unethical. The bottom line is that the product design is marginal, and needs to be redesigned. Barring that, you can ask for a waiver and see if the customer will accept that. I don't see anything that even suggests that it will work at 96°C; it just only happens to be within measurement tolerance of the lower limit of the test procedure.

The customer should not have accepted those limits as valid. Assuming ±1°C tolerance on the thermometer, the minimum acceptance test temperature should have been 101°C.

I should also warn you that marginal designs may have other flaws. We came across a sensor that was specified for operation up to 100°C, and it operated to spec at that temperature, but failed to operate at 90°C.

TTFN
FAQ731-376: Eng-Tips.com Forum Policies

RE: Ethical question regarding tolerances on test plan

Oh, you agree with MJ but no one said anything when I brought that issue up originally.sad

I don't think the OP ever really answered the question of if the customer approved this test plan or not.

Without having seen the exact wording of the specification the 3 of us are assuming that the 100°C is really a minimum, which seems likely/reasonable, however we don't know for sure.

Certainly if this is right it seems unlikely the customer explicitly approved the test plan, in which case why try to fudge it?

Posting guidelines FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm? (probably not aimed specifically at you)
What is Engineering anyway: FAQ1088-1484: In layman terms, what is "engineering"?

RE: Ethical question regarding tolerances on test plan

mdgates (Mechanical)
Do you mind telling me the name of your aerospace company, so that I can avoid buying whatever it is you are selling.thumbsup2
B.E.

The good engineer does not need to memorize every formula; he just needs to know where he can find them when he needs them. Old professor

RE: Ethical question regarding tolerances on test plan

I do a lot of testing and I go by the rule of “when in doubt, ask the customer”. You want to be as transparent as possible. Surprises like this down the road will kill your trust and career with anybody. If the customer is really hot for this product, they will either lower the spec or give you a waiver.

Tobalcane
"If you avoid failure, you also avoid success."
“Luck is where preparation meets opportunity”

RE: Ethical question regarding tolerances on test plan

To add, for thermal testing, the chamber will never be at a constant 100 degC. The chamber will fluctuate by +/- 5 degC depending how big or small it is. The larger the chamber the more the chamber will fluctuate. So even if you lower the setting, your tolerance will go out of bounds of the spec at the low end. Now it depends who is singing it off on the customer side. If s/he was not fed, did not have their coffee yet, and in a grumpy mood, this would be a fail. If s/he has some sensibility, it would pass. For me, if I see a gross fail, maybe more than 5% drop, I would fail it.

Tobalcane
"If you avoid failure, you also avoid success."
“Luck is where preparation meets opportunity”

RE: Ethical question regarding tolerances on test plan

I'm sorry, more than 10% I would fail it. Also, some times the spec you get is higher than the original spec the customer was suppose to meet just so the customer will have margin.

Tobalcane
"If you avoid failure, you also avoid success."
“Luck is where preparation meets opportunity”

RE: Ethical question regarding tolerances on test plan

On the converse side, there can be pathological air flows within the chamber that can result in absurdly high local ambient temperatures. We once attempted to test about 200 parts in the thermal chamber, where each part dissipated about 5W each. That was full 1 kW, and because the testing boards were tightly spaced, we later estimated that while the programmed temperature was 125ºC, the actual part ambient was closer to 170ºC, which was well past the maximum allowable temperature. Needless to say, we had substantial life failures. Reducing the number of boards, improving the air flow, etc., resulted in the expected life test performance.

TTFN
FAQ731-376: Eng-Tips.com Forum Policies

RE: Ethical question regarding tolerances on test plan

The spec can easily be 100C +/- 5C, it doesn't need to be a minimum.

The test procedure should state something to the effect of "set temperature to 100C" not "set temperature to 100C +/- 5C". The customer should have read and approved the procedure. If you are changing it you need to get their approval again.

Where did you determine that your product would fail the test? Was it during the formal testing? If so then that is a failure already, you don't get to keep trying until you pass. You and your customer would need to get together and determine what to do about the failed test, re-design or spec change.

I assume your customer will also be reviewing and approving the test results, what happens when they see the output of the temp chamber droping well below 95C?

Doug

RE: Ethical question regarding tolerances on test plan

(OP)
Thanks for all your feedback. It's clear that in this case, the spec requires that the product function at 100°C, and we know that it's wrong (and generally bad business) to propose to ship product that doesn't meet the spec.

If I were in Engineer B's shoes, I'd stick to the nominal, i.e. how the test plan should have been written, and suggest that the customer be contacted about a waiver. I'd explain this to Engineer A and the boss first, and appeal to my QE and Head of Engineering if needed. My name wouldn't go on any document suggesting that A's test plan demonstrates compliance with the specs. However, I suppose I would conduct any testing I was ordered to, even if I thought they were going to use my test data for evil instead of good.

RE: Ethical question regarding tolerances on test plan

Just a note

When your boss wants you to do something illegal or unethical, it normally ends up bad for you no matter what.

Your best chance of minimising personal consequences is to play real dumb and claim you don't exactly understand what he wants you to do.

To push the point you can even say (in writing) that you must be misunderstanding his direction as from your interpretation, it seems he is asking you to do something dishonest, so could he please give you written instructions so that you can study them and ensure you are not misinterpreting them.

Keep copies of request and reply at home. Make diary notes of verbal exchanges on the subject.

Regards
Pat
See FAQ731-376: Eng-Tips.com Forum Policies for tips on use of eng-tips by professional engineers &
http://eng-tips.com/market.cfm
for site rules

RE: Ethical question regarding tolerances on test plan

Quote (patprimmer)


Keep copies of request and reply at home. Make diary notes of verbal exchanges on the subject.

Where I used to work, it was called "keeping a Pearl Harbor file"...

John R. Baker, P.E.
Product 'Evangelist'
Product Engineering Software
Siemens PLM Software Inc.
Industry Sector
Cypress, CA
Siemens PLM:
UG/NX Museum:

To an Engineer, the glass is twice as big as it needs to be.

RE: Ethical question regarding tolerances on test plan

It occurs to me that you are creating a situation where your product will pass when you test it, but if any other lab ever tests it, or the customer tests it, it will fail everytime, which is not a good situation.

I would also think that as you get close to the pass/fail borderline, that there wouldn't be a sharp division, you'd just be changing the percentage pass/fail. So at 100 degrees, maybe 20% of them pass, and at 96, 80% of them pass. Either way, it's not an indicator of good reliability.

RE: Ethical question regarding tolerances on test plan

Warranty cost is embedded in the sell price, so you really don't want to have more than what is allocated for warranty costs, which should be less than 1.8% of sell price (http://www.entigo.com/pdf/Cost-of-Warranty-White-Paper.pdf). The good news is that not every user will ever hit the max operating temp condition, so if those users are only 1:10, then even a 15% failure rate would only incur about 1.5% of sell price. Nonetheless, if that 1:10 happens to be a single, large user, then you're going to get a gigantic reputation hit and a potential loss of a large customer for future sales.

TTFN
faq731-376
[IMG http://tinyurl.com/7ofakss]

RE: Ethical question regarding tolerances on test plan

JStephen has it.

If it is an accelerated life test then you don't really know whether knocking 5 dgerees off the test temperature, and 5 minutes off the oven time, equates to a reduction of X% in real life performance, without performing additional tests. The relationship between real life and accelerated life testing is measured and expensively known, for many systems, but not in the open literature.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?

RE: Ethical question regarding tolerances on test plan

I have passed and failed integrity tests, and perhaps been graded on the curve as well. Power, authority, profit motives, wishful thinking, the list goes on. When you think a lot at work, integrity issues may get bypassed or ignored because so little thought is frequently required when it comes to right and wrong. Certainly very much less than self deception, just google 'Sandusky'.

I'd ask a couple 5 year olds. You have a pair (of something) and a backbone. Start your career by using as required, people will take notice.

RE: Ethical question regarding tolerances on test plan

To really solve this ethical problem is to have a third party test the product per the customer spec (http://www.dtbtest.com/). They are not bias and will run the test by the spec. What you do in house can be pre-qualification for confidence. Once you feel good, send it out for formal qualification.

Tobalcane
"If you avoid failure, you also avoid success."
“Luck is where preparation meets opportunity”

Perception is reality: You may build your skill set, but what you think of it doesn't really count. The only thing that really matters is what your boss and or customers think of it. Likewise, the real you is how others perceive you, not

RE: Ethical question regarding tolerances on test plan

Quote:

To really solve this ethical problem is to have a third party test the product per the customer spec. They are not bias and will run the test by the spec.

No.

Test labs follow test procedures.

If the test procedure is ambiguous or poorly written they will do their best, or seek clarification from whoever is paying the bill. If that clarification comes from an unethical or knowledgeable source then it's not clear what will happen.

RE: Ethical question regarding tolerances on test plan

Well that is my point is that they will seek clarification from both parties particularly the one who wrote the spec (most likely the customer). The test engineer has no interest if the product passes or fails. Their job is to produce the environmental spec requirement. If it says 100 degC +/- 5 degC, that is what test engineer will dial in, no fuddy duddy or second guessing. There business relies on their accuracy to meet what’s on paper and not what the designer wants. Now shock is best effort and will depend on experience of the test engineer. Depending on the magnitude and duration (and sometimes velocity change if you are dealing with the Navy) the test will be different with every blow. Thermal and vibration can be programmed in per the spec. Random, depending on the table and fixture, can be challenging to meet the spec, but is programmable.

Tobalcane
"If you avoid failure, you also avoid success."
“Luck is where preparation meets opportunity”

Perception is reality: You may build your skill set, but what you think of it doesn't really count. The only thing that really matters is what your boss and or customers think of it. Likewise, the real you is how others perceive you, not

RE: Ethical question regarding tolerances on test plan

I’m sorry, I meant test procured, not spec in my last two post…

Tobalcane
"If you avoid failure, you also avoid success."
“Luck is where preparation meets opportunity”

Perception is reality: You may build your skill set, but what you think of it doesn't really count. The only thing that really matters is what your boss and or customers think of it. Likewise, the real you is how others perceive you, not

RE: Ethical question regarding tolerances on test plan

The test lab has no contractual relationship with the end user.

If there is an end user representative present to witness the testing then usually everyone will defer to his or her decrees. But there is not always an end user representative present, and even when present they may not be the right person to notice that the test procedure is not compliant with "the spec". Often the end user representative is an inspector or junior engineer and all they'll look for is "approval" of the test procedure.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Eng-Tips forums is a member-only feature.

Click Here to join Eng-Tips and talk with other members!


Resources