From memory (I haven't looked at the Parker handbook for over a week now), and from practical experience, o-rings are best installed with a slight stretch on the i.d., and then compressed (squeezed) through the cross-section. If you squeeze an o-ring such that the ring i.d. is reduced, you put compressive stresses on that side of the o-ring, and it may buckle or squirm, and create a leak path. The buckling can (especially when the cross section is large compared to the ring i.d., which is true for a -011 ring) be very local, creating small ripples along the surface. We call that sphincterizing when we see it.
The dovetail groove is really the best option for retention, unless you can find a compatible grease sticky enough to hold the ring, and as ornery says, you need to be sure that the grease that inevitable leaks/extrudes out, and the dirt and debris the mess attracts, won't mess up other parts of your system.