As mentioned, the best concept of Antenna Factor is that of a transducer transfer function, very similar conceptually to a calibrated microphone for measuring acoustic amplitude based on a voltage output.
Wikipedia has a reasonable introduction to Antenna Factor, and of course you can dig further if you like.
Antenna Factor is (more or less) a concept confined to calibrated 'EMC' antennas used to measure field strengths. In fact, that's typically how one would measure field strengths. Some all-in-one field strength measuring instruments may have it all built in, including the calibrated antenna.
Antenna Factor is not a concept usually applied to transmitter stations. For those, it's all about output power, cable losses and antenna gain pattern.
Antenna Factor doesn't enter into the relationship between a transmitter's EIRP and the resultant Field Strength in dBu - simply because those two parameters exist even before you get the calibrated measuring antenna out of its carrying case, before you even arrived on site.
The other distinction to clarify the concepts is that the EIRP exists at the transmitter antenna, while the field strength of interest is at some distant location, X miles from the transmitter. Those two parameters are related by the pathloss equation.
Antenna Factor is for the measuring antenna used at the point of interest, at that location.
Sometimes there are so many variables in the field measurements (reflections, multipath, operator experience) that one might reasonably wonder if the predicted field strength might be closer to the truth than the messy, possibly unreliable, field measurement. If you get an unexpected value, it might just be a measurement problem.