I found the issue is that the 180 uH inductor is at 85 uH when tested in the circuit and that is too low for the buck to start properly; other PCBAs that are working properly show 180 uH in circuit. Now here is where it gets interesting...
I removed one inductor and had to use a lot of heat to do it, mainly with a hot air gun before I used a two-iron trick to remove it. After removing that part I tested it again and now it reads 180 uH! Is this some Neel temperature effect(ferromagnetic to paramagnetic) or something? This is a Coilcraft LPS4018-184MRB ferrite inductor.
So after I saw that I put the inductor back in the circuit; it reads 180 uH in circuit. I powered up and the buck started properly. Once. After that it failed multiple (every) times. I measured the inductance again and see 85 uH. Can this be some type of saturation effect? I need to check with the circuit designer, The inductor has a 240 mA peak current and the TPS54061 has a 350 mA typical current limit (500 mA max), so that might be a start up issue while charging the 10 uF output capacitor.
I pulled the inductor back off the PCB to repeat the heat test, and after about 60 seconds of heat the inductor is 100 uH, 120 seconds and it is at 110 uH but appears to level off there. After about five minutes of heat it went open circuit so that isn't a very good test.
So I've replaced the inductor on the two failed PCBAs and they both start fine 10 out of 10 times. But for how long? Did I just get bad parts or is there a root cause here that I'm missing? Is this a typical inductor failure mechanism in a buck regulator?