electricpete -
I concur that the worst case mechanical failure by definition is a locked rotor, which the motor is rated to withstand for a stated period of time.  Any current in excess of the locked rotor value MUST be due to an electrical fault condition.  
Was there any evidence of such a fault?  What about a relay oscillogram - any info from that quarter?  (Perhaps you could post an oscillogram as you did for your previous motor starting post)
Just a thought - you say that the instantaneous is set well above the locked rotor, but does the setting allow for full offset in the locked rotor current waveform?  

peterb good discussion - In this case the instantaneous function is set for 1.4*LRA.  But it's an electronic relay which incoporates an inverse characteristic for the instantaneous element (sound like a contradiction but it's not) for times less than about 0.1 sec.  The intent of that inverse characteristic is to mimic the shape of ac plus decaying offset so that the instantaneous "asymptote" can be set closer to LRA.  Manufacturer literature originally recommended setting at 1.1*LRA... we went to 1.4LRA for extra margin against spurious trips. Our total peak current including dc offset is higher than the asymptote of the instantaneous trip but well within the curve published by the manufacturer. This makes me a little skeptical because we test only the asymptote portion of the instantaneous feature... too difficult to test the inverse portion.  So in order to prevent tripping, we are "relying" on a portion of the curve that we don't test.  Mfr now is hedging a little bit... suggests that we should bump our setpoint if we're seeing trips. This is first trip in 10 yrs among 6 similar motors started approx once per month.  There are a number of reasons to expect possible variability in instantaneous performance.
These would include phase closing angle, motor residual magnetism (although I've never heard of this as a factor except possibly after dc testing).  

Motor, relay, cables, everything checked fine (only abnormal thing was mechancial system lineup). Relay was double-checked by vendor.

The only things that were slightly different during this particular start was the ac power supply lineup and the mechanical fluid system lineup (which I believe to be irrelevant as discussed).

Regarding the power supply lineup:
#1 -  Voltage was in fact being monitored and was lower than normal (should reduce starting current).  
#2 - with equipment powered from different sources we might possibly  expect somewhat different R/X ratio (dc decay) depending on the source?  

Suggestion: When it comes to a peculiar situation like this, one should not rule out the relay or protective device malfunction, unless one is biased in favor of motors.

jbartsos - relays tested ok when tested both on-site and by the vendor. (see 2nd paragraph 8/21 post). That is to say the instantaneous element picked up when a current slightly greater than the asymptote was applied.  

It is one of those non-reproducible things that makes you rack your brains. The oscillographs posted on the other thread are of this exact motor, on a subsequent test start (started normally).  Unfortunately, there was no permanently-mounted oscillography (or digital relays) to capture this one-time event.

It is of significance because the sole purpose of this pump is to auto-start in order to fulfill it's plant safety-function.  If it trips during start, it fails when needed most.

electricpete -
Based on your experience here & the criticality of the application, I would strongly suggest that you increase the instantaneous setting to 1.7x locked rotor.  What you gain in terms of certainty of successful starting will far outweigh the increase in clearing time for actual faults.  Also, installation of a microprocessor relay could be considered - if the cost is justifiable on the basis of security of operation and information available.
 
Of course, you need to look carefully at the effect on coordination with upstream relays.  Confirm that the motor cable is still adequately protected with the higher setting (this should not be a real concern, but needs to be checked).

Thanks peterb. You give me confidence that I have assessed the situation correctly. First on the mechanical effects, now on the setpoint change. I have recommended a setpoint increase.  There are still some folks that are skeptical of that recommendation and reluctant to proceed for various reasons. We're going to double-check a few facts and do one more motor check.

Interesting problem, electricpete, I hope you will keep us informed.  My first thoughts are that the analysis here ruling out mechanical causes are quite logical.  Ruling out the trip relay as a cause was a good step too.  My second thoughts however, concern the instaneous trip and inverse time functions.  Are you sure this curve is limited to less than 0.1 seconds?  With the motor checking out ok, something is different here, and the only differences you have pointed out are the supply, and pump loading.  If you were already on the edge of your locked rotor curve vs. trip settings for example, I would not be very confident that no mechanical loading problems could have put you over your trip curve for time periods in the tenths of seconds.  I would not absolutely rule it out without monitoring the motor starting currents.  

Suggestion: Please, would you provide more information about the motor control and power supply quality.