Checking of calculations in the aircraft industry
Checking of calculations in the aircraft industry
(OP)
Hi All,
There is an interesting thread under Structural Engineering regarding checking of calculations; some of the replies are quite scary (think Hyatt Regency Walkway 1981).
How do you go about checking of calculations in the aerospace industry?
Andries
There is an interesting thread under Structural Engineering regarding checking of calculations; some of the replies are quite scary (think Hyatt Regency Walkway 1981).
How do you go about checking of calculations in the aerospace industry?
Andries





RE: Checking of calculations in the aircraft industry
but then the certification rests with the static test. the stress calcs are pretty much supporting data.
and of course in any company there is usually lots of checking, and usually (hopfully) enough experienced guys to shoo away those pesky project "engineer"s.
RE: Checking of calculations in the aircraft industry
A very worrisome thing, at least to me, is checking spreadsheet calculations. People tend to panel-beat data to satisfy the requirements of the spreadsheets instead of first looking at the fundamentals of the analysis.
Andries
RE: Checking of calculations in the aircraft industry
Brian
www.espcomposites.com
RE: Checking of calculations in the aircraft industry
if you develop your own analysis, this usually needs to be valiadated before someone will believe it ... this'll include philosophy and math ... usually a hand calc of one loadcase.
with all the checks and balances we still get satellites crashing into Mars 'cause guys make fundamental mistakes in their calcs. fortunately for us, most of our strcutural problems are revealed in test ... a la 787 side of fuselage "issue".
RE: Checking of calculations in the aircraft industry
Another fried was project manager on an F-15 special fuel tank for 20 years or so.
Bet both got analyzed to death!!!
RE: Checking of calculations in the aircraft industry
RE: Checking of calculations in the aircraft industry
RE: Checking of calculations in the aircraft industry
For critical applications we stuck to this, however for a lot of other stuff the Stress guys calcs were the only detailed analysis.
What is Engineering anyway: FAQ1088-1484: In layman terms, what is "engineering"?
RE: Checking of calculations in the aircraft industry
but then that's just me, being cranky ...
RE: Checking of calculations in the aircraft industry
yes you are cranky :>)
would canned software make less mistakes.
than hand calcs?
mfgenggear
RE: Checking of calculations in the aircraft industry
That said, the design engineers in question were mostly HNC/HND/Apprenticeship so in the US probably wouldn't be allowed to be called engineers. Then again the govt approved signatory Stress Engineer was HNC/HND/Apprenticeship not degreed or Chartered - go figure.
What is Engineering anyway: FAQ1088-1484: In layman terms, what is "engineering"?
RE: Checking of calculations in the aircraft industry
i was voicing my opinion that they have their own body of knowledge to absorb and usually don't have the time to become proficient stress analysts. particularly where FE is concerned; in fact, if i come across a designer stress-wantabe who is good with FEA (not someone who thinks he is), then the question would be "why are you designing ?" and "what have you given up to become proficient in stress analysis ?"
no disrespect was intended or implied.
as for canned FEA ... yes, i am very suspicious of "instant stress analysis". the models are usually limited in how they can apply constraints, how they can load it, what elements you can use, the users are generally not well schooled in subtleties ... but they do produce pretty pictures.
RE: Checking of calculations in the aircraft industry
> yes! it will probably have worse (and undocumented) assumptions than hand calcs, and is much more likely to be misused - the GIGO effect - too easy to think that if the input produces a "pretty picture" that it must be correct.
> the worst thing ever invented is one button FEA inside CAD programs
RE: Checking of calculations in the aircraft industry
The saving grace with designing to code is that there are a lot of hidden safety factors, whereas if you are designing for function there is nothing to stop you designing any old thing, and (hopefully) relying on a good testing program to catch any errors in the design.
Approximately 25% of my work is the final safety related signoff for a car. My results and process and correlation are reviewed, but I've never had anybody actually check the runs themselves. For all they know I'm giving them the graphs of a completely different car (the input decks are stored, so down the track it is possible to replicate the work pretty easily).
Cheers
Greg Locock
New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?
RE: Checking of calculations in the aircraft industry
And the second worse thing is the mass use (abuse) of general FEA programs.
Brian
www.espcomposites.com
RE: Checking of calculations in the aircraft industry
As an Aussie, I deal with both civil and military regulators who are both quite anal about their requirement for independent verification of the adequacy of designs.
I personally like to see my analyses verified. It saves grief later.
I have found that the regulators seem to use language which aligns with ISO9000 type design philosophy, where the independent verification of the adequacy of a design is a basic requirement.
I know some individuals resent review for items which are non-structural and passed as acceptable by comparison. In such cases it means that another suitable individual is required to review the design and also find that it is acceptable by comparison (at least), FROM HIS/HER PERSPECTIVE.
From experience, I have found that items reviewed, certified and approved in such a manner will be more likely to be suitable for end-use than items where a review has not been performed.
In fact, an experience that I had with an aircraft OEM in the mid-80's (now defunct) cemented my views on this subject. They designed and certified a HF installation without a design review, only to find that the Tx/Rx was failing in-use. The chief avionics engineer responsible for the design swore black and blue that the design was good and the fault was with the box. After frying a couple more boxes (and with national stock levels in jeopardy), he finally agreed to a design review, whereupon a design fault was found.
Pride cometh before the fall.
I recommend that the review and validation process be built into the design process and not tacked-on after. It has positive effects on the quality of the design for function and cost as well as the engineering of the strength and durability for certification. If done properly it saves money and time and improves reputations.
I'm about to change employment modes shortly.
Watch out for Melbourne Aeronautical Design, operating in Melbourne (Australia), and anywhere else someone has a problem that needs a solution.
By the way, if anyone needs access to ANC-5, MIL-HDBK-5, MMPDS-01 for designs on aircraft which have referred to older issues (and they almost all have), I have a personal copy all of them except MIL-HDBK-5E Vol 1. If anyone has it I would love to obtain a copy.
I have personal copies of a lot of texts and other material. I will share if anyone wishes to examine my copy of anything.
I am cognisant of the need to avoid breaching copyright, so I cannot copy, loan or re-sell where the document is copyrighted (MIL-HDBK-5 is approved for public distribution by the way) or but I like to help as I have been helped myself and numerous good people have given me access to their libraries when I have been searching for info.
"Let me bring order to your chaos"
RE: Checking of calculations in the aircraft industry
i worked (breifly) for CAC ... good gig, particularly the lawn bowls !
RE: Checking of calculations in the aircraft industry
Certainly over reliance on FEA without validation (which in some cases is classical calcs, some cases testing, some cases all 3) or understanding is problematic.
We didn't do this, in fact we didn't have access to FEA most of the time.
What is Engineering anyway: FAQ1088-1484: In layman terms, what is "engineering"?
RE: Checking of calculations in the aircraft industry
Formal design verification was done by an outside, independent* group. Rather than checking the calculations of the design group (which supposes that they calculated the right stuff and didn't overlook anything), the V&V people started with the system requirements and developed their own analysis, wrote their own s/w and did their own work.
*The whole question of independence came up from time to time when I was at Boeing. Where the design and certification groups work for the same company and eventually reported to the same first level supervisor.
RE: Checking of calculations in the aircraft industry
From the responses one realises that the problem has many aspects to it. I’m sure the problems being highlighted on this thread are being experienced worldwide.
I’d like to relate the situation experienced by me with regard to analysis report checking and what I call “gate-keeping”, or lack thereof, of company methods and processes being used to substantiate structural designs. It would appear that those checking other engineers’ work do not have sufficient knowledge of “what” they should be seeing. There is no point in checking another person’s numerical accuracy if the checker doesn’t know how the analysis should be done in the first place.
There are 4 signatures on final certification reports. They are: - (1) original analyst, (2) peer checker, (3) lead stress engineer and (4) department manager. Only the peer checker does a half-decent job. If he’s not experienced enough, mistakes and application of incorrect methods are not picked up. The lead “trusts” the peer checker and the manager “trusts” those below him to have “caught” any bugs/errors there may be in the analysis.
The company has been around a long time and are still providing daily repair support to their products that have been in service for 30 years or more. These old structural analysis reports contain stress methods that are clearly origin-referenced and any deviation from the standard is explained and justified. Needless to say they are all hand written and where early computer methods were used, a complete set of printouts necessary to replicate the calculations is provided.
However, a change has occurred with the advent of work-station automatic stress analysis methods and primarily the use of VB-based spreadsheets. Every component now gets its own dedicated multi-worksheet workbook that produces MS values at predetermined critical areas of the component. Most often that spreadsheet can only be “run” by the creator/analyst. It rarely is accompanied by an explanatory “front sheet”. The checkers therefore have to be familiar with the workings of the workbook or it can’t be checked. The problem is aggravated when these spreadsheet/report combinations are handed over to MRB/sustaining.
The company has its own structures manual, which is an excellent reference document based on classical analysis theory and each subject entry is backed up by a report outlining the theory which it is based on and, where appropriate, laboratory test reports.
The fly in the ointment occurs however, in that many of the stress engineers don’t know what the structural manual contains and the implications and/or limitations of the methods presented in it. The more complex analysis methods can be performed using online program versions of the method. However, some of these are often blindly used to solve problems that that do not always conform to the original analysis assumptions made. A similar problem exists with the “Roark Formula Grabbers” as I like to call them.
Another project-based, primary assembly item phenomenon called “The Stress Guidelines (or Methodology) Document”, has reared its head. (This form of document is now apparently widely used in the Industry) This document contains the stress methods that shall be used by the stress engineers. These documents often contain questionable methods that do not appear in the public domain literature or the company structures manual and are therefore not covered by in-house supplementary substantiation reports.
As examples amongst others, at a detail level, there are two frequently occurring problem areas that are not captured by checkers. The first is the application of bolt group analysis to joints that do not comply with the assumptions which bolt group analysis is based on. The second is lug analysis that suffers from the same problem.
Bolt Groups: Items that should be calculated as multiple support overhang beams are treated as a bolt groups. The flexibility of members between fasteners in the joint excludes the necessary assumption that fasteners further away from the bolt group centroid carry more load than those close to the centroid. In fact the opposite is most often the case.
Lugs: The in-house method used is based on the widely-used Melcon/Hoblit method, not Ekvall. Stress engineers incorrectly apply force systems that ignore the geometric symmetry definition of the axial and associated transverse in-plane axes. In a recent case a non-symmetrically placed bearing was analysed with the in-house lug analysis tool as if it was symmetrical. Out-of-plane offset moments caused additional stresses at the critical section behind the pin. This resulted in a substantial MS reduction. Three tiers of checking did not pick up the error.
While the above examples have not (yet) resulted in serious failures, margins have been dramatically reduced that will affect MRB and/or sustaining engineering functions down the line.
Are similar problems being experienced elsewhere? How are they being dealt with?
Ed.
RE: Checking of calculations in the aircraft industry
I could have written your (edbgtr) piece myself such are the similarities. I have been working at a EU/NL based firm for some years as a consultant trying to get the stress methods and analysis into line with a major customer (who assemble a/c in Toulouse). Much of the customer's methods for the UK projects are based on familiar BAe practices and procedures and are similar to EU/NL in-house methods. However, rather than just adopting these methods there is this overbearing requirement to make the tools 'idiot proof' - making black boxes or canned tools.
This means the poor (sob) engineers trying to actually learn how to do stuff are confronted by an array of black boxes (KBE is the buzz word - knowledge based tools)into which they blindly plug numbers and read off the MS/RFs. Few of these guys get the chance to actually ensure the result matches the expectation.
In some cases these KBE tools have been used even though the internal assumptions are inappropriate for the problem being analysed. As the KBE tool becomes 'secret' over time, people move around, so unless a thorough effort has been made in documenting the KBE tool it is inevitable it gets used again and again, potentially giving bad results.
In terms of checking and signing off these calcs (black boxes are often locked VB routines within spreadsheets or Mathcad) present a nightmare for the Lead Stress (me). Criticism of taking too much time does occur from management. However, l insist on all data flows being documented and 'front sheet descriptions/scope statements' added. Significant oversight of the methods used is also embedded within the certification dossier.
Despite all of the measures mentioned above a 3rd party auditor would still have significant challenges in understanding all and every assumption, source of input/data in those calcs, etc.
So to answer Andries original question - how do we check Aerospace calcs?
a) The author of the dossier completes a checklist that prompts him/her to include standard and specific data, inputs, method descriptions, etc. The emphasis being on content and traceability of data. It includes embedding examples of calcs into the dossier itself. He prepares and documents all supporting files. He adopts a means of doc configuration (via issue and version number).
b) A third party checker (usually part of the same team but ideally from a different project) works through the dossier and underlying supporting files. He/she utilises a checksheet which categorises the issues found. These are documented in a tabular form.
c) The compiler receives the checksheets from the checker and, after discussion, normally makes the changes required.
d) The checker gets to re-check the items in the checklist to ensure all are adequately covered and/or corrected.
e) The Lead Stress confident of the compiling/checking process to that point then takes a view on the dossier and supporting files. The amount of additional checking (dip checking or backwards checking from an RF/MS) depends on the assessment of the people in the chain before him. A rough guide is that 10%-20% of the content is rechecked/dipped. This can lead to putting the dossier back to the checker if the required quality/content is not in place.
f) Once satisfied as to the content the Lead Stress then approves or authorises the dossier and it is delivered to the client (Chief Engineers office if internal or maybe client/Airbus Cert team). They then go through a similar checksheet checking process with feedback to the Lead Stress as needed.
And that is why aerospace projects burn so many hours. An example being 20,000+ hrs stress work on (part 25 a/c) wing components with an additional 2000-3000 hrs on FEM validation and documentation, 5000 hrs of test justification plus design/ME/PV/MRB support.
I hope this helps answer the original question.
RE: Checking of calculations in the aircraft industry
Are similar problems being experienced elsewhere?
>Yes. And its worse when margins are written directly from FE stress contour results.
How are they being dealt with?
>If there are experienced leads on the project, then the problems are (hopefully) minimized. But in many cases the leads don't have the experience, or the time, or are pressured to worry more about schedule/cost over technical correctness. The aerospace structures community has gotten complacent - unfortunately it will take a major structural failure resulting in a severe aircraft accident to wake up the industry.
And if you think the state of the initial stress analysis practice is of concern, you should see what constitutes "analysis" in the liaison (MRB) and repair areas.
RE: Checking of calculations in the aircraft industry
Many thanks for the valuable replies to my query. Looks like SWComposites has touched on a potential can of worms in his second post!
RE: Checking of calculations in the aircraft industry
In my case, I'm the one responsible for some of the sins he lists (creating a stress guidelines document, adding VBA to Excel workbooks to automate solutions, etc.), but all with the intent of reducing mathematical/numerical errors. The problem is not that the automated methods exist, but that stress engineers may not understand the assumptions which are made when they use a particular method.
The challenge I see is in training and then later checking to ensure that the analyst understood what assumptions they were making when they employed a particular method or formula and how the real world varies from their model.