The orifice plate idea gives you more control over the discharge, and it's what I'd go with. Make the culvert half again as big as the orifice, and size the orifice plate with the orifice equation based on an upstream head of 3.5 feet.
You do realize you'll probably need to route a hydrograph through your pond to determine if its volume is adequate for your design storm, right? Also, don't forget to include some sort of emergency overflow device, for storms that exceed the design storm or if the orifice clogs. If it's a permanent pond, consider an anti-seep collar on the culvert, even if it's dry. If it has a permanent pool, there's water table and wildlife issues. If it's a sediment pond, there's usually state specific rules on how they want it done for sediment capture. If you're doing a hydrology analysis for post development to match predevelopment, don't forget the effects of the watershed that bypasses the pond. Etc.
If you're in over your head, don't be shy about hiring a hydrologist to design your pond for you. A simple hydro analysis with all the necessary bells and whistles is usually only a couple grand, unless you're in a municipality that's overbearingly tedious.
Hydrology, Drainage Analysis, Flood Studies, and Complex Stormwater Litigation for Atlanta and the South East -