I'll acknowledge the elephant in the room that MartinSr00 is pointing out. There is no definition of where to measure a real part to confirm that the dimension is within tolerance. It's not "A", it's not "B", and it's not a dumb or overcomplicated question. The answer is not defined anywhere.
The "A" dimension in Martin's diagram is an example of a "directly toleranced dimension". Y14.5 does a good job of explaining how to specify these on a drawing, but doesn't explain what they actually mean on a real part. The Y14.5.1 "math standard", which defines conformance and actual values for geometric characteristics, doesn't deal with directly toleranced dimensions. For good reason - they don't stand up to rigorous mathematical analysis.
I agree with the assertions that datums have nothing to do with this. Any attempt to superimpose a datum scheme on direct tolerancing is purely arbitrary. The scheme Dave suggests would greatly improve the repeatability of measurement, but other schemes giving different numbers would be equally valid.
To me, directly toleranced dimensions are a carry-over from traditional drafting. They offer a simplified (and sometimes oversimplified) view of the world, which causes specification uncertainty. The implicit assumption is that the form and orientation error of the surfaces is small compared to the location tolerance. In cases where this is a valid assumption, direct tolerancing is workable. This is why many designers still cling to it - because the errors it causes are insignificant in their particular applications. In other cases where form and orientation errors are significant, direct tolerancing falls apart completely. The effects shown in Martin's diagram are always present to some degree, but that degree depends on the magnitude of the tolerance and the geometric variation in the real parts.
If you want something with a well-defined meaning that you can hang your hat on, then you have to go to geometric tolerancing.
Evan Janeshewski
Axymetrix Quality Engineering Inc.