It seems to me that the tolerances are specified at some point because they are needed, not just to have them for the sake of having them. If no one knows what the tolerance is, or should be, you probably don't need them in the first place.
Suppose you're making Frisbees. And say, due to manufacturing slop, the weights vary by +/- 5%. What does it matter? It'll throw pretty much the same anyway, and there's no need to try to put a tolerance on it unless there's some massive problem.
But suppose you're making part A of something, and it has to connect to, or fit inside part B made my someone else. All the sudden, tolerances become important. Or you're making 500 little widgets that all assemble into some device. They have to fit together, and the idea of tolerances becomes useful.
I work in an industry that doesn't use geometrical tolerances. What we dimension is usually measured with a tape measure and cut with a torch, and that effectively sets the tolerances for us. Minor errors in cutting just mean additional welding time, but usually don't get the item rejected.