Transmission shift valve force balance equation?

Simple hydrostatics and a force balance. Area1 x pressure1 + area2 x pressure2 + ... = whatever force the return spring is applying + whatever friction in the seals is doing to you + whatever force the solenoid, if present, is applying.

Don't forget to account for the sign (i.e. which direction the force is being applied).

If there is significant flow through the valve then the fluid may not be in a hydrostatic state and then life becomes very complicated. If it is a spool valve and it is letting fluid in and out through one groove in the spool then the pressure can probably be considered the same throughout that groove.

The old-school transmissions that used hydraulic pressure for shift timing can be really hard to understand because the force balance with the spool in the default spring-loaded position may not have the same effective areas exposed to the pressure as when the spool has shifted (that's how they positively stay in one position or the other after a gear change). Good luck. The new ones just use them as directional valves actuated by solenoids. The clutch is either applied (exposed to line pressure) or released (vented to the housing). Some of them may use duty-cycling and be in-between applied and released, but the computer handles that.