×
INTELLIGENT WORK FORUMS
FOR ENGINEERING PROFESSIONALS

Log In

Come Join Us!

Are you an
Engineering professional?
Join Eng-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Eng-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Students Click Here

The Fallacy of Extrapolating with Computer Models
7

The Fallacy of Extrapolating with Computer Models

The Fallacy of Extrapolating with Computer Models

(OP)
I have vocally and repeatedly proclaimed that computer models cannot prove anything. As I was working on updating my 5-day Engineering course I came across a perfect example of what I'm talking about.

This model started with the best dataset ever assembled. Really. The best ever. The underlying data was all "coincident" to this model. By that I mean that the people collecting the data had a strong motive to make it as accurate and complete as it could possibly be. Those folks got paid and they paid partners and mineral owners based on the data (so they had no significant incentive to illegally adulterate it), oh yeah, the data is required by law to be complete and accurate and has been since the 1940's. Further the data was collected each month on upwards of 400,000 discrete entities operated by nearly 100,000 business entities, all with an explicit license to operate that is not trivial to acquire. In other words the data entering this model has had financial and legal incentives to be accurate and complete. Of course, the dataset is monthly U.S. gas production by well.

If you start with this high quality data and bring in:
  • Historical wellhead price and consumer price data sets along with an independent forecast of those prices into the future
  • A detailed data set containing historical new-well permits that can be compared to the price data over time
  • A detailed data set containing new facility permits that can be compared to new well permits and the price data
  • A detailed list of issued permits for facilities (that take up to 10 years to build after the permit is issued) with their projected completion dates and projected capacities (see the big uptick in the attachment in the Alaska data in 2019 representing the pipeline coming on line)
  • Independent forecasts of inflation
  • Historical and (independently) projected steel pipe worldwide manufacturing tonnage and prices
  • A team of very talented, very experienced Engineers, Economists, Statisticians, and Computer Modelers
  • A project deadline that the team felt was very liberal
  • No limits on budget for manpower, computing equipment, or software
It really doesn't get any better than this. They published the attached forecast in the 2007 Energy Outlook there were some glitches in the first version so they updated it and published the attached in the 2008 Energy Outlook. This chart was reprinted hundreds of times over the next few years. I haven't seen it much since 2011. I pulled in the data yesterdat and added the actual production between 2006 and 2011 (last data available that breaks out the various gas types).

So at year 5 you find:
  • Unconventional gas under predicted 92% (using the Unconventional Gas forecast as the denominator)
  • Onshore conventional over predicted 41%
  • Offshore over predicted 60%
  • Alaska over predicted 55%
  • Total gas under predicted 19%
  • If you remove the Unconventional component, total would be over predicted 55%
This is at year 5 of a 24 year forecast. In early 2008, Gas prices were over $10/MSCF, the drilling in the Marcellus, Hanesville, and Fayetteville shales had accelerated and rig counts were approaching all-time highs. All of this data was readily available to the modelers, but they didn't quite believe it and tweaked the model back to a slight increase followed by flatish with offshore taking up the slack.

I don't mean to ridicule these guys, they did a workmanlike job. I made a similar blunder in 1990 when I failed to include a group of wells (that I had already built pipe to) in a forecast of the value of a company that was on the market. A competitor did include those wells and offered $15 million more than we did--the group of wells I excluded produced that much profit the first 6 months and 23 years later they are still on production.

My point is that with a superb set of clean data, unlimited time and budget, a team with all of the requisite skills and no incentives for reaching a particular conclusion couldn't predict something as "simple" as gas production within 55%, how can anyone put any credence in the climate models that have questionable data, intense time pressure, intense budget pressure, and intense pressure to reach a specific conclusion? Hell, they could even be "right", but I won't be willing to accept that until we can look back at a body of predictions that have the same shape as the actual (raw) data for that period. So far we are not even close.

David Simpson, PE
MuleShoe Engineering

"Belief" is the acceptance of an hypotheses in the absence of data.
"Prejudice" is having an opinion not supported by the preponderance of the data.
"Knowledge" is only found through the accumulation and analysis of data.
The plural of anecdote is not "data"

RE: The Fallacy of Extrapolating with Computer Models

One thing I struggle with often is getting others to realize that there are assumptions, limitations and uncertainty in every model. We need to always be ready for what we did not know.

RE: The Fallacy of Extrapolating with Computer Models

3
Anyone with any scientific training knows that a predictive model cannot 'prove' anything, it's a tool to predict something that could happen.

That said, could you do me a favour, and start 324,567,890,000 more threads on here that state you don't agree with the current popular climate predictions and surrounding politics? I don't think we have quite enough of your viewpoint posted on here yet.

RE: The Fallacy of Extrapolating with Computer Models

The most insidious thing is scientists who intentionally calibrate their models against a specific variable with a known correlation, but then use those models to claim causality.

Hydrology, Drainage Analysis, Flood Studies, and Complex Stormwater Litigation for Atlanta and the South East - http://www.campbellcivil.com

RE: The Fallacy of Extrapolating with Computer Models

(OP)
TenPenny,
That sounds like sarcasm. I can't help the direction that the threads I start go. The "consensus science" discussion could have gone to a discussion of historical consensus topics, it isn't my fault it fell into the AGW discussion. This one is the same. I saw an excellent example of a best in class model failing to predict a pretty simple system (compared to the climate of the globe). Thought I'd share. I would guess that there is some interest in these discussions since most of them get over 100 posts in a week. If you don't want to participate, there is a really effective technique that I use all the time--don't open the damn thread.

David Simpson, PE
MuleShoe Engineering

"Belief" is the acceptance of an hypotheses in the absence of data.
"Prejudice" is having an opinion not supported by the preponderance of the data.
"Knowledge" is only found through the accumulation and analysis of data.
The plural of anecdote is not "data"

RE: The Fallacy of Extrapolating with Computer Models

Having a bad day, Tenpenny ? curse

It is better to have enough ideas for some of them to be wrong, than to be always right by having no ideas at all.

RE: The Fallacy of Extrapolating with Computer Models

It really depends on what models you are talking about. For example models have been used successfully for years to control many different physical processes. If you have flown lately on commercial airlines, then you most likely experienced model predictor control. Also know as fly be wire.

In the case of the OP example, the model was trying to predict human behavior. I do not know of anyone, or anything that can predict human behavior.

RE: The Fallacy of Extrapolating with Computer Models

(OP)
djs,
I'm a modeler. I understand the power of models better than most. I also understand the limitations. The big limitation is extrapolating any human or natural system forward more than a step or two. Weekly projections of any system going out decades are just random number generators after the first month.

David Simpson, PE
MuleShoe Engineering

"Belief" is the acceptance of an hypotheses in the absence of data.
"Prejudice" is having an opinion not supported by the preponderance of the data.
"Knowledge" is only found through the accumulation and analysis of data.
The plural of anecdote is not "data"

RE: The Fallacy of Extrapolating with Computer Models

"" The big limitation is extrapolating any human or natural system forward more than a step or two""

I don't think so as you might guess.

Thermal conductivity models of homogenous stationary media and inherently stable. The PDE solution to a heat flow problem will be stable forever if it converges in the first place.

I think you miss the difference between stability, marginal stability, and instability as types of dynamic systems.

For instance dissipative systems are stable and the solutions to such converge to a point and stay there.

See this nice wiki article.

https://en.wikipedia.org/wiki/Attractor

RE: The Fallacy of Extrapolating with Computer Models

Google
"extrapolating with computer models"

867,000 hits.

apparently most are unaware of this fallacy.

RE: The Fallacy of Extrapolating with Computer Models

One reason interpolation works for predictions is that you can be wrong and still get reasonably good results. If you approximate a logarithmic or polynomial function with a set of trig functions, the interpolated results can be tolerably good. However, the divergence as one gets outside of the known data can be, well, logarithmic.

RE: The Fallacy of Extrapolating with Computer Models

zdas04 has 1 model that doesn't predict very well.
zdas04 extrapolates to conclude that no models will predict well.

One could just as easily point to somebody like Nate Silver who has had great success in political forecasting with his models.

RE: The Fallacy of Extrapolating with Computer Models

Of course, when it comes to certain situations, you have NO choice but to use computers and software to 'model/simulate' systems which CANNOT be tested using prototypes or even first production models.

A recent case in point:

http://www.smithsonianmag.com/video/Curiositys-Sev...

Note that for the people where I work, this topic would be more than a simple intellectual exercise:

http://blog.industrysoftware.automation.siemens.co...

John R. Baker, P.E.
Product 'Evangelist'
Product Engineering Software
Siemens PLM Software Inc.
Industry Sector
Cypress, CA
Siemens PLM:
UG/NX Museum:

To an Engineer, the glass is twice as big as it needs to be.

RE: The Fallacy of Extrapolating with Computer Models

Your Smithsonian link reminded me of another link to that site. This one shows how well the Limits to Growth predicted 30 years out.

RE: The Fallacy of Extrapolating with Computer Models

John - loved the Smithsonian video. However, in EACH of the phases of that, do you have any idea how much testing and prototyping was done? I think the engineering teams spent over 1 year modeling, prototyping and testing the parachute. The science behind the aerobraking maneuver has been tested thousands of times with hundreds of thousands of computer model simulations validated by testing. (Disclosure - I was involved in the boundary-layer stability numerical simulations of Martian and Earth re-entry hypersonic flow for blunt and semi-blunt configurations. The validation against experimental work was beyond extensive).

Mars wasn't done with only models, but models that were validated by extensive prototype testing. AND, nothing was extrapolated - the prototype testing was done to create an envelope of possible scenarios so that, if anything, interpolation and not extrapolation was done.

That's the thing about interpolation vs. extrapolation - we usually test our systems beyond their expected performance, to safely predict what they'll do under normal operation. Case-in-point: pressure-containing system (pressure vessels and piping). We test those to 1.25-1.5 (and way more, if they normally operate at high temperatures) the design pressure, to validate the integrity of the design and fabrication. Nobody says, well the last 2000 vessels we made haven't blown up, so let's not test this one - the model says that it's OK.

These modeling problems can be broken down to boundary-value problems and initial-value problems. Boundary-value problems have (obviously) known boundary conditions and are essentially interpolations - and nobody asks questions about what happens beyond the boundaries in these closed systems. As 2dye4 indicates, they are typically stable or semi-stable and are well-behaved. Compare that to weather. Weather is an initial-value problem; the more you know about the initial conditions (magnitude, derivative, and integral of all initial values), the better your prediction is. However, in these types of problems, random boundaries may pop up - say a huge increase in a variable such as unconventional gas production, that is essentially unpredictable a priori. Same with climate - solar variation (TSI, GCR, etc), vulcanism, ENSO, other not-well-understood cyclic variations, etc may pop up and make a huge difference. Heck, a 30-minute timing difference in tropical cloud formation has more effect (on a W/m² basis) than a doubling of CO2. Furthermore, the resolution needed for those initial conditions is vastly greater than what is currently available.

RE: The Fallacy of Extrapolating with Computer Models

Yea gotta wonder when they'll be including the next 10 years of data...

John R. Baker, P.E.
Product 'Evangelist'
Product Engineering Software
Siemens PLM Software Inc.
Industry Sector
Cypress, CA
Siemens PLM:
UG/NX Museum:

To an Engineer, the glass is twice as big as it needs to be.

RE: The Fallacy of Extrapolating with Computer Models

2dye4 - very interesting wiki article about Attractors in chaotic systems. One of the fascinating aspects about the so-called strange attractors is that they are exceedingly stable even in the presence of large perturbations - perhaps there may be a leap from one stable attractor to another, but they usually don't "blow up".

Here's a thought exercise for you. I don't know if this is right or wrong or otherwise. What if the earth's climate, the chaotic system that it is, has a stable or pseudo-stable attractor that keeps our overall climate reasonably stable in the presence of huge perturbations? What if something like tropic thunderstorm daily timing is sufficient to add or shed heat to stabilize the system, either in the perturbation of volcanic eruption emissions or increased CO2?

Considering that we've never experienced a run-away before, even with ice-ages (and possibly a "snowball earth") and interglacials of almost complete loss of inter-annual ice, is this a possibility?

RE: The Fallacy of Extrapolating with Computer Models

But in the end, it all depended on a lot of computer cycles to pull it all together into a single scenario simulating exactly what was going to happen during those last seven minutes of the delivery phase of the mission. And they apparently go it right thumbsup2

But I agree with your comments about "interpolation vs. extrapolation" and the need to determine how accurate your designed-for safety factors are. After all, in much of engineering it's often true that you never really know HOW MUCH until you know what's TOO MUCH. Which reminds me of several books I've read by Prof. Henry Petroski, primarily about ther failure of structures, usually bridges since that's his area of expertise, but he touches on many other famous, or should we say, infamous examples of engineering failures. I think one of the most profound comments that I remember him making was that we learn an order of magnitude more when something fails than when something does not. Another good thesis covering a famous failure was the comments made by Dr. Richard Feynman as he described the events, which included a 'Mr Wizard' like experiment before the Congressional committee looking into the Challenger disaster in 1986.

John R. Baker, P.E.
Product 'Evangelist'
Product Engineering Software
Siemens PLM Software Inc.
Industry Sector
Cypress, CA
Siemens PLM:
UG/NX Museum:

To an Engineer, the glass is twice as big as it needs to be.

RE: The Fallacy of Extrapolating with Computer Models

John, was it models and simulations that put Curiosity on the ground on Mars, or was it real-time on-board computing with sufficient flexibility to deal with the "unknown"?

And check out JPL's Mars travel log summary - http://mars.jpl.nasa.gov/programmissions/missions/... There's a whole lotta "Failure"s in there. Although the trend is improving, I don't know if I would extrapolate that trend... smile

RE: The Fallacy of Extrapolating with Computer Models

  • Historical wellhead price and consumer price data sets along with an independent forecast of those prices into the future
  • So one of the inputs to the model was the output of another model....
  • A detailed list of issued permits for facilities (that take up to 10 years to build after the permit is issued) with their projected completion dates and projected capacities (see the big uptick in the attachment in the Alaska data in 2019 representing the pipeline coming on line)
  • Permits do not equal construction. Construction started does not equal construction finished. Projected capacities are again the output of another model.
  • Independent forecasts of inflation
  • The output of yet another model
And they didn't even attempt to model factors such as:
The development of technology to make "unconventional" sources more accessible.
The rule of law by an arbitrary and capricious government.

So the question to ask is not "How could such a great model be wrong", but "how could anyone really expect that this would have any chance of being accurate" - which I guess is your point in the first place

I get "Can't you model it?" questions frequently from clients. My most frequent answer is "The equations to characterize the system are trivial. However the outcome is entirely dependent on initial conditions and external inputs; and we will be guessing about most of those."

RE: The Fallacy of Extrapolating with Computer Models

(OP)
MintJulep,
Of course you are right. It has been a while since last I saw a model that purported to project some non-trivial data set that didn't not brag about using bits and pieces from others to show that the biases couldn't possibly be theirs. That may be cynical, they are probably bragging about using the other models to show the vast quality of the results. There were probably 15 forecast data sets (model output) used as input to the model.

Of course permits do not equal facilities, but the argument (that I kind of support) is that it is so expensive to get a permit to build Oil & Gas facilities that once you have it in hand it is an asset and you will either build the facility or sell the permit to someone who will (in a way that doesn't invalidate the permit).

David Simpson, PE
MuleShoe Engineering

"Belief" is the acceptance of an hypotheses in the absence of data.
"Prejudice" is having an opinion not supported by the preponderance of the data.
"Knowledge" is only found through the accumulation and analysis of data.
The plural of anecdote is not "data"

RE: The Fallacy of Extrapolating with Computer Models

But you have to ask, would things be better if there was NO 'modeling' whatsoever? In other words, doesn't having a 'model' which is based on current and relevant data, but is recognized as being potentially inaccurate due to so many unpredictable variables, still better than just shooting in the dark?

John R. Baker, P.E.
Product 'Evangelist'
Product Engineering Software
Siemens PLM Software Inc.
Industry Sector
Cypress, CA
Siemens PLM:
UG/NX Museum:

To an Engineer, the glass is twice as big as it needs to be.

RE: The Fallacy of Extrapolating with Computer Models

Shouldn't we define what we mean by a 'model'.

Which is a computer model.

1 A program calculating force on an object given its mass and acceleration as inputs.
2 A program calculating the numerical value of an integral that doesn't have closed form solution.
4 A PID feedback controller designed around a particular system model implemented in a computer.
5 A control system on a modern aircraft that stabilizes the plane in a way a human could not fly implemented in a computer.
6 A hydrological model of a lakes level given initial state and recent heavy rainfall.

All physical relationships are just models.
F=ma
V=IR
Now I don't think zdas04 would object to calculating f=ma with his handheld calculator or a spreadsheet even though
these are just models implemented in a computer.
So somewhere Zdas04 mind between these and navier stokes equations, models become the subject of scorn.
Maybe Zdas04 ( who has more experience than most with compute models ) could tell me where models go bad and
earn his scorn.


RE: The Fallacy of Extrapolating with Computer Models

(OP)
The title of the thread kind of covers it. I use pipeline models. I calibrate a model based on the best set of data I can get my hands on. I pull whatever levers are available to me to get the output of the model to match measured conditions. Then I bring in another data set and leave all the adjustments as with the first dataset. If the model matches the pressures on the second set (without adjustments) then I have some confidence that when I put in a pipeline modification that the model will indicate performance. I put in a very limited number of modifications and evaluate what those piping modifications might do to the pressure distribution on the system. Eventually I get to a point of recommending physical modifications to the pipe based on the model results. Then I build the new pipe and calibrate a new model. Basically I am willing to extrapolate one time step with some confidence. A second time step would use model results as input and I have far less confidence that that step would relate to reality in a meaningful way. A 20,000th step is just random numbers.

In my mind, complex computer models are fantastic tools to evaluate potential modifications to the physical world. They are also very good at illuminating areas for further experimentation. They do not "prove" things since a model cannot be anything more than the knowledge of the person who wrote it. If someone was to say "AGW is a fact because I know it is a fact", you wouldn't have much faith in that proof. That is exactly what the computer modelers are saying. They wrote a model. I shows a different temperature rise with man-generated CO2 than without man-generated CO2 so AGW is "proven". The proof in this case has no more validity than the statement that someone "knows" it to be true.

David Simpson, PE
MuleShoe Engineering

"Belief" is the acceptance of an hypotheses in the absence of data.
"Prejudice" is having an opinion not supported by the preponderance of the data.
"Knowledge" is only found through the accumulation and analysis of data.
The plural of anecdote is not "data"

RE: The Fallacy of Extrapolating with Computer Models

I'm reminded of financial modeling and widespread use of gaussian copula centered models by hedge funds and larger intermediaries as 'mortgage-based' [plus commodity-backed, etc.] structured finance ramped up during the 90's and 00's -- Surely worked well didn't it.

But, on topic[?] -

Climate change impact on available water resources obtained using multiple global climate and hydrology models

''..future climate change impact assessments are highly uncertain. For the first time, multiple global climate (three) and hydrological models (eight) were used to systematically assess the hydrological response to climate change and project the future state of global water resources. This multi-model ensemble allows us to investigate how the hydrology models contribute to the uncertainty in projected hydrological changes compared to the climate models. ...''

Earth Syst. Dynam., 4, 129-144, 2013
www.earth-syst-dynam.net/4/129/2013/
doi:10.5194/esd-4-129-2013
© Author(s) 2013. This work is distributed
under the Creative Commons Attribution 3.0 License.

Effective - accurate, predictive - models, are, I believe, possible.

RE: The Fallacy of Extrapolating with Computer Models

From my perspective...

Many people think of models as being designed to predict the future based on initial conditions and a set of governing physical equations. Except for very simple systems, we all know that this isn't generally possible, beyond a very small advance in time. Otherwise we'd all be able to plan for future weather.

For other models, there may be boundary conditions that constrain the evolution of the system being simulated. I work in the engine performance simulation industry. It is very rare for a perfomance model to be dependent on initial conditions, since it has a strong, cyclic boundary condition applied to it. Nearly all engine flow/combustion models converge to a cyclically repeating solution (and if they don't that's probably ok too). We can make changes to the model and believe the changes to the solution.

- Steve

RE: The Fallacy of Extrapolating with Computer Models

I haven't read any serious climate scientists who claim that a computer model 'proves' anything.

But, like politics, it's more fun to claim people say something, and then disparage them for what you pretend they say.

RE: The Fallacy of Extrapolating with Computer Models

Computer models don't even have to be based on any physical laws to get things right. Google "Artificial Neural Networks" and spend a couple days reading. All you need, to create a model that has great predictive power, is lots of historical data to use as inputs and outputs. Then train the model to that data.

The idea that models can't be used to predict is silly. Of course they can be used to predict. Otherwise they'd be pointless. And they don't even need to be based on science if you have enough data. You can use ANNs to predict football games.

What they CANNOT do is prove causality. This is always the fun example, with which most engineers will have some familiarity:



Now sloppy graphing aside, one can show an inverse relationship between the number of pirates vs global warming. One can even tune a computer model to show that, and get a correlation between the two that's not terrible. What you can't do, is use that model to prove that one caused the other, when there may be entirely different factors in play that could be causing both.

In the global climate environment discussion, it's not just carbon that's increasing, it's everything else that mankind does to manipulate our environment that's also increasing. The things tracking 'upwards' are temperature and people. That antropogenic carbon is on the rise is incidental to the fact that *everything* humans do is on the rise. The model trained to carbon proves nothing. You could train a model to the length of roads, or the number of buildings, or the number of printed books, or CFCs (apparently) or who knows what and still get a high correlation. But correlation does not mean causality.

A model specifically trained to follow a correlation is not proof. It can't be proof, by its very nature. Lots of scientists are forgetting that these days.

Hydrology, Drainage Analysis, Flood Studies, and Complex Stormwater Litigation for Atlanta and the South East - http://www.campbellcivil.com

RE: The Fallacy of Extrapolating with Computer Models

i think the problem extrapolating historical trends is that things change in the future.

sure you can predict the statistical chance that one team will beat another. would you use the same methods to predict the next ten meetings of those teams ? or just the next game ??

but in complex and chaotic systems things change unpredictably (there might be another outbreak of Somali pirates, and no doubt another cooling, because more guys and boats and guns became available.

Quando Omni Flunkus Moritati

RE: The Fallacy of Extrapolating with Computer Models

"would you use the same methods to predict the next ten meetings of those teams ? or just the next game ??"

Or would you not try to predict the individual games at all and instead predict the overall record at the end of 10 games.

RE: The Fallacy of Extrapolating with Computer Models

To clarify.

Open loop (predictive) models are all rubbish.

- Steve

RE: The Fallacy of Extrapolating with Computer Models

To take the sports game analogy a little further...

brad1979 makes an interesting point - what resolution do you want in the predictions? I suppose that's why not only are there climate models, but ensembles of climate models - supposedly none of the individual climate models are exceedingly good, but the multi-model mean is somehow very good.

Since fluid mechanics is not only an initial-value problem, but also a boundary-value problem, it is conceivable that one could be fooled into the thinking that it doesn't matter what the outcome of the next 10 games are, only the record at the end of the 10 games. However, when the outcome of each individual game is the seeding for the next game (initial-value problem here - all sporting games have an initial-value of identically zero), then the results of EACH game are important.

That brings me to an interesting issue of averaging. If I had a winter and spring that was exceptionally below-average, and a summer and fall that was equally above-average, what does averaging tell me about my year? Nothing. How about Phoenix, AZ being hotter than average, while Miami, FL being below-average. What does that tell me about CONUS temperatures? Nothing. Or even more dramatically, how about Miami being 1°F below average (at 89°F and 90% RH) vs Tuktoyaktuk, NWT being 5°F above average (and 30°F and 25% RH)? Can you even average temperature? What does that mean?

RE: The Fallacy of Extrapolating with Computer Models

Interesting column on watts up

http://wattsupwiththat.com/2013/06/03/climate-sens...

Basically rather than looking at the actual temperature record the author looked at the various computer models that have been used.

Rather interestingly, for all their complexity, and the claims that they were developed independently, they all behave in accordance with a simple one line equation with a couple of tuning factors in it.

In my little patch of the world this happens quite often - for example I can spend months building and correlating a complex multi body dynamics model that predicts when or if an SUV will roll over, or I can multiply the height of the cg by the coefficient of friction of the tires, divide it by the vehicle's track, and make a prediction with much the same accuracy.

This doesn't mean more complex models aren't worth having, necessarily, but it does indicate that in the case of climate models their apparent complexity is not doing any real favors accuracy wise.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?

RE: The Fallacy of Extrapolating with Computer Models

It depends on what kind of system we are trying to model. Nonlinear systems can be difficult to model and you could also be dealing with chaos. For nonlinear systems there is often a time horizon beyond which prediction breaks down. Nonlinear systems are also very sensitive to small changes in initial conditions.

The time horizon is related to the error in the initial conditions for the model. There is no such thing as 100% accuracy in any measurement device. There will always be some small amount of error. This error represents the difference between our measurements of the initial conditions and what the initial conditions actually are. As time progresses in the model, this discrepancy will grow. The time horizon is the period of time which the model remains accurate to within some tolerance. The time horizon has a logarithmic dependence on the discrepancy between the measured initial conditions and the actual initial conditions. In grad school I took a course on chaos theory. Absolutely zero practical application in my career thus far, but some very mind blowing stuff. One example I remember from class on this time horizon stuff is that if you were to invent the most accurate measurement equipment in the world-intsturments that are ONE MILLION times more accurate than what is currently available to measure the initial conditons of a chaotic system, the accuracy of the extrapolation of the model would only increase by 2.5 times.


In nonlinear models that are solved iteratively, round off error can also grow very rapidly over successive iterations. One very simple nonlinear iterator can demonstrate the sensitive dependence on initial conditions. Let's say we have a model that takes an input, squares it, and then feeds the result back into the model. Take three numbers:0.99999,1, and 1.00001 and use them as initial conditions for the model. What happens? All three numbers are essentially 1. The difference between the largest and the smallest is 2e-5. However, our results from the iterator will be shockingly different after a few iterations. When you start with 1, 1^2=1. You can iterate forever and the result will always be 1. Squaring 1.00001 gives 1.0000200001, and after 20 iterations you have 35800.15749030058, and for 0.99999, after 20 iterations you have 2.79299091957e-5. Even the difference between 1.00001 and 1.000006 will be large after 20 iterations (35800 vs 539) Miniscule difference in initial conditons, huge difference in results after 20 iterations.

RE: The Fallacy of Extrapolating with Computer Models

How about an outbreak of Nigerian Pirates?

If you create a model, can't you predict how accurite it is?

All I know is the local model of traffic flows seems to look like I get every red light, and I do every morning. I know it's a model because lights turn green when no cars are present, while I sit at the intersection with a red light.

RE: The Fallacy of Extrapolating with Computer Models

Actually - that was kinda the point of the wattsupwiththat.com post - that the super-duper-complicated GCMs were no different from a 0-dimension simple equation. And the deviation of the actual numbers from the prediction (from 1981) falls well outside the 2σ bands from the Hansen81 paper, especially when you consider actual CO2 concentrations.

Conclusion - the GCMs can be reduced to a simple equation. Neither the simple equation nor the GCMs are particularly accurate. Also, could it be that for however complicated the GCMs are, they have hard-wired into them the simple equation, which means that they will likely do no better than the simple equation.

Model fail.

RE: The Fallacy of Extrapolating with Computer Models

Well i think the heat flux effect of GHG is very well established, what the complicated GCM models are trying
to do is estimate how this heat flux will effect the variables that effect the amount of heat flux.
In other words what are the feedback ramifications??

And how does this excess heat flow into the Earth 'mainly the oceans'.

So simple heat flux equations tell part but not all of the story.

RE: The Fallacy of Extrapolating with Computer Models

Overall heat flux is one thing. Spatial variability and transport (convection, advection, ocean currents, Rossby-waves, jet-stream, tropical cyclones, etc), as well as timing (daily onset of tropical thunderstorms, for example) should be the areas of interest in GCMs. If energy is flowing into earth's system mainly in the tropics, but is flowing outward in the tropics and the poles, the transport issues are critical.

I am of the opinion (I cannot confirm this) that the feedback mechanisms should be left to the physics of the simulation. However, they appear to be more hard-coded to better hindcast existing temperature histories. Of course, that leads to the "tropical tropospheric hot-spot" that is hypothesized, but isn't measured, hypothesized changes to humidity, etc.

Since the regional resolution and success of the GCMs is poor-to-fair, we get phenomena such as mis-estimating the seasonal Arctic Sea Ice area/volume and variability in one direction while mis-estimating the Antarctic Sea Ice area/volume and variability in the other direction. We still have no idea why phenomenon such as blocking highs occur and persist or even why an one emergent low-pressure system in Ethiopia can grow into a Cat 5 Atlantic hurricane while others fizzle over the Sahara.

Since the 0D equation does not hindcast or forecast even remotely closely, and the current GCMs are no better than the 0D equation, what does that really say about the GCMs?

RE: The Fallacy of Extrapolating with Computer Models

Quote (TGS4)

Actually - that was kinda the point of the wattsupwiththat.com post - that the super-duper-complicated GCMs were no different from a 0-dimension simple equation. And the deviation of the actual numbers from the prediction (from 1981) falls well outside the 2σ bands from the Hansen81 paper, especially when you consider actual CO2 concentrations.

Conclusion - the GCMs can be reduced to a simple equation. Neither the simple equation nor the GCMs are particularly accurate. Also, could it be that for however complicated the GCMs are, they have hard-wired into them the simple equation, which means that they will likely do no better than the simple equation.

Model fail.

In other words, they could have gotten the same results from plotting CO2 vs global temp in Excel and told it to best fit curve the results.

And if that's the case, it's quite likely they could get similar results by plotting human population vs global temp in Excel.

Hydrology, Drainage Analysis, Flood Studies, and Complex Stormwater Litigation for Atlanta and the South East - http://www.campbellcivil.com

RE: The Fallacy of Extrapolating with Computer Models

Or pirates - never forget the pirates.

RE: The Fallacy of Extrapolating with Computer Models

Or pirates.

Which means "proof" of CO2 as the primary driver isn't proof at all.

Hydrology, Drainage Analysis, Flood Studies, and Complex Stormwater Litigation for Atlanta and the South East - http://www.campbellcivil.com

RE: The Fallacy of Extrapolating with Computer Models

It would be interesting to run the modern climate models with the sensitivity to CO2 set to 1, ie measured physical value, and then seeing how good the correlation with the historical record is, and I suppose their predicted temperatures for the next few decades.

I imagine somebody somewhere has done this.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376: Eng-Tips.com Forum Policies http://eng-tips.com/market.cfm?

RE: The Fallacy of Extrapolating with Computer Models

(OP)
Model runs of that size are very expensive, I wouldn't count on anyone having spent their budget money on a model with a sensitivity of 1.0.

David Simpson, PE
MuleShoe Engineering

"Belief" is the acceptance of an hypotheses in the absence of data.
"Prejudice" is having an opinion not supported by the preponderance of the data.
"Knowledge" is only found through the accumulation and analysis of data.
The plural of anecdote is not "data"

RE: The Fallacy of Extrapolating with Computer Models

Past performance is not indicative of future results.

Bruce Youngman
http://www.dynamicsolutions-mi.com

RE: The Fallacy of Extrapolating with Computer Models

Au contraire.

History repeats itself. It has to. No one liatens.

- Steve

RE: The Fallacy of Extrapolating with Computer Models

that link is exactly why i think we're all doomed.

i think that we're focusing on the wrong thing, CO2, and controlling it or "mitigating" it's perceived problems.

i think we should be focusing on new energy so we can get away from burning fossil fuels. and for me that means new fission reactors (low pressure designs) or better yet fusion reactors.

Quando Omni Flunkus Moritati

RE: The Fallacy of Extrapolating with Computer Models

Nice 'trick' picking a high spike in observed temperatures as the 'baseline' then
normalizing all to this peak.

Second nice trick is ignoring the statistical qualities of the random component of the measurement.

Roy sayz
""For years the modelers have maintained that there is no such thing as natural climate change""

Direct lie or intentional misrepresentation. Where have they said this??

Roy sayz
""Forgive me if I sound frustrated, but we scientists who still believe that climate change can also be naturally forced have been virtually cut out of funding and publication by the ‘humans-cause-everything-bad-that-happens’ juggernaut""

And yet no private funding from the companies with soooo much to lose??

Sorry Dr Roy is a political hack and nothing more.

RE: The Fallacy of Extrapolating with Computer Models

Quote (2dye4)

Sorry Dr Roy is a political hack and nothing more.
Prove it.

Put up or shut up - compare your credentials to Dr. Spencer's. Compare your published articles to his. You're nothing more than some anonymous hack who gets their information from a third-rate website.

Quote (2dye4)

And yet no private funding from the companies with soooo much to lose??
Oh puleeeze! If he did, you would cry and scream that he's a paid shill. That is one ad hominem that you certainly cannot pin on this guy. Besides, do you really think that the likes of ExxonMobil will lose money?

Quote (2dye4)

Nice 'trick' picking a high spike in observed temperatures as the 'baseline' then normalizing all to this peak.
Evidence, please! Shift the modelers' graph down a little bit, if you like. Guess what? Same result!

I understand that you still can't wrap your head around the fact that the climate models that constitute the bulk of the "proof" of CAGW were wrong, are wrong, and will likely remain wrong, because they are based on incorrect premises. Dearly-held beliefs are the ones that are the most difficult to modify.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Eng-Tips Forums free from inappropriate posts.
The Eng-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Eng-Tips forums is a member-only feature.

Click Here to join Eng-Tips and talk with other members! Already a Member? Login



News


Close Box

Join Eng-Tips® Today!

Join your peers on the Internet's largest technical engineering professional community.
It's easy to join and it's free.

Here's Why Members Love Eng-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close