Let me start by saying I am NOT a transformer expert. But I suspect the issue is related to the material from which the transformer is made.
That a transformer becomes warm (or even hot) is an indication that there is conversion of electrical energy into heat from hysteresis loss. The hysteresis loss is a function of the coercivity of the material - - actually the area within the hysteresis loop as the material is polarized first in one orientationa nd then in the opposite direction. Since coercivity is a major descriptor of the hysteresis loss (though not the only one) let's stay with it.
Coercivity is also a measure of the material's "resistance to de-magnetization." This resistance to demag is greatest in permanent magnets, less so in (permament) magnetic steel, even less in materials such as screwdrivers, and minimal in Electrical Steel (both grain oriented and non-grain oriented). (Strictly speaking it is the materials Intrinsic coercivity that is the resistance to demag, but in soft magnetic materials Hcj ~ Hcb).
Transformers are made from many different grades of steel with the less expensive ones being low carbon steel, intermediate grades using silicon steel and the highest grades using the very expensive materials such permender.
I should note for us that a permanent magnet remains magnetized, essentially unchanged, until exposed to a demagnetizing stress: high temperatures, reverse field, a ringing AC field, etc.
The second issue is that since the transformer is essentially a closed magnetic circuit, even a very low coercivity will result in retained magnetic field.
As an interesting side note: nanocrystalline transformer core material is excellent, but is so sensitive to physical strain increasing the Hcb that it must be annealed after forming into the transformer.
There is a paper at
which discusses silicon iron and the loss factors. You might find it interesting.