I was recently involved in a project where we were investigating the behaviour of a rubber seal in an underwater structure. The seal was an extrusion, with a hollow, D-shaped profile. We removed a small length and tested it, up to and beyond the point where the D-section had closed on itself. We then set up an FE model, and analysed that under an increasing series of enforced displacements, again, up to and beyond the point where the D-section closed on itself.
I was surprised by the extent to which the results from the physical model and the FE model were in agreement across the full range of displacements, despite the gross deformations involved. FE material models for rubber have come a long way.
Based on this, admittedly limited, experience, I would suggest you could use FE to explore different seal shapes and materials. Presumably you think likewise, or you would not have placed you post in this forum (although I note you hedged your bets by posting in another forum as well).
As for your specific question of how much force would be required to retain a given water pressure, first principles suggest that water will not be able to force itself through a rubber-to-steel interface provided that the normal pressure across that interface exceeds the pressure of the water (by an appropriate factor of safety). The FE analysis will give you that interface pressure.
Minor irregularities in the smoothness of the mating surfaces will confound this a bit, but I have no idea how much. Perhaps some testing is needed? Or merely an increase in your factor of safety?