I've never seen that kind of formula, probably too difficult to determine through impirical data. Most gearmotor manufacturers (Dodge, SEW-Eurodrive, Nord, Falk, etc) will list "generalized" efficiencies for a particular type of gearing configuration. For example, helical gears = 0.96, worm gears = .85, etc.
TygerDawg