I expect that many here have a different experience. Based on the questions from many visitors on this forum there are a lot of people who have experience more similar to mine.
I worked in one place where the producer didn't even use the drawing and neither did inspection. The source had built to the original model and to smaller variations than were originally called for. Production problems had occurred and they asked for changes, changes they made to the producer manufacturing models, that were never incorporated into the company drawings. This became evident because there was a mirror assembly version that the changes were, partly, incorporated into. When I was brought in to fix a pile of other problems with the drawing package I noted the differences and then, comparing them with finished items on the factory floor, found a bunch more.
There's not a diminishing returns place - one might never use FCFs at all and get a high quality product. Like I said, the only reason for FCFs is to predict what you might get and confirm that's what you got, but if you can put a part into assembly production without inspecting it and it works, it doesn't matter at all what's on the drawing.
In the industry parts I've dealt with, both military** contracting and FDA controlled medical equipment, they could have eliminated all use of FCFs and seen no change at all. More time was spent arguing over minor details of FCFs while simultaneously making horrific datum reference choices than I care to recall. Those choices would allow completely useless parts to be accepted. And then forgetting to specify any tolerances at all in many cases. And it made no difference because the CNC equipment was far more reliable at producing near perfect form and orientation than the tolerances allowed.
What I would do if money was the biggest deal would be to just put plain dimensions and tolerances on sizes and locations; mostly the tolerances on dimensions would be for close/bearing fits, not on typical dimensions. Only if an assembly problem came up would I bother with an FCF. But it pays to put a few of them on a drawing so management thinks some actual work is being done.
The only time I would care about using FCFs is if the company was managing yield and using factory measured variations to drive the engineering design yield calculations. Note this is not process control charting; that just makes Quality managers happy. If the company is not closing the loop on factory floor quantitative results and their manufacturing costs with design engineering, then nope. Not worth it. I'd keep the Envelope principle and Continuous Feature and chuck the rest. If someone wanted something special, I would put a fat note on the drawing.
The dilemma, and I've had my career bashed in over it, is that if you 100% use FCFs to control all aspects of parts you get kicked for doing too much. And then if you take even one FCF off a different faction can use that to complain that you don't know what you are doing at all; the same faction that will select bad datum references and refuse to look at simulation results because "manufacturing would never use all the tolerance anyway." But make the tolerance smaller and a different faction demands the cost will go up and the tolerance can't be changed. And so the ignorance goes.
Like I said, I'd scrap it all unless the company is closing the loop and your job specifically involves using production data on process variability.
**Worked specifically for the US Air Force, the US Navy, TACOM, and the US Marine Corp, all with different procurement expectations for documentation. TACOM was the one accepting drawings without any angle tolerances between nominally perpendicular surfaces/features.