Before I start, I think you have left a factor of 2 off the first term in your formula. And there is a much easier form of the formula in terms of the angle that the wetted perimeter subtends at the centre of the circle:
A = 0.5 * r^2 * (theta-sin(theta))
Solve this for the angle theta, then use simple trig to get from theta to H.
But your question is more general. How to solve an iterative equation in Excel, and how to do it so that the answer will appear instantly rather than requiring the user to push a button, activate a menu item, or run a macro. There are several ways, and which one you use depends on the nature of the equation being solved, the skill-level of the spreadsheeter, and the required robustness in the end product.
(1) You can write a VBA subroutine that will invoke the Solver or GoalSeeker for you, and set it up so that this subroutine is automatically executed every time the sheet (or any key cell on the sheet) is changed. This does not provide you with total flexibility, because the subroutine needs to know the address of the target cell, the target value, and the cell to be changed to meet the target. (Furthermore, if you use the Solver in a context like this, you might have troubles porting your spreadsheet between different versions of Excel. I certainly do when I call the Solver from VBA.)
(2) You can set up a small iteration scheme within the body of your spreadsheet. The drawback is that you have to pre-determine how many iterations you will use, and that means you will usually proved too many. The iteration scheme would typically take the form of a table, one column per iteration. Row 1 contains the latest estimate of the answer. Rows 2 through n-1 contain some intermediate results based on that latest estimate of the answer. Row n contains an improved estimate of the answer based on the various intermediate results. Then the next column takes this improved estimate in its row 1, and the process repeats. With a good iteration scheme and a good starting estimate of the answer, you will not need too many columns to be assured of adequate convergence. Develop the table in the business area of your spreadsheet, then when it is finished and fully tested move it off to some out-of-sight area. You should always do some sort of accuracy check after the final iteration, and have the spreadsheet alert the user to any convergence failure. (The "accuracy check" could be as simple as comparing the final value with the value from the previous iteration.)
(3) You could write your own VBA user-defined function to implement the iteration scheme to whatever degree of accuracy you deem appropriate. (Here your UDF does not attempt to use the solver or the goalseeker.)
HTH