I think it might help you to understand me if you were to draw lift, velocity, and acceleration diagrams for a simplified cam profile. On both of the flanks, when valve acceleration is positive (in the opening direction), there will be a greater force applied to the cam than would be applied if the valvetrain were massless. Over the nose of the profile, when valve acceleration is negative (in the closing direction), there can be force between the follower and cam, but there doesn't necessarily have to be. For a given spring, geometry, and lift profile, the larger the inertia of the valvetrain, the less force will be applied between the cam and follower over the nose. Nonetheless, the closing flank will see a large follower force. With a typical cam profile, the middle of the negative acceleration part of the profile (nose) is where the drive torque will switch from positive to negative. Inertia will reduce the force a little bit on both sides of the nose. If there is no separation at all, then the force between the cam and follower will start increasing immediately when the flank starts. Even with a moderate-to-large amount of separation, the follower should be back on the cam before you move very far along the flank.
A good way to picture this is to hold a ball in your hand, and make a motion as though you are going to toss it. If you move your hand slowly up and down, gravity holds the ball in your hand. If you gradually increase the speed of this motion, you will find that there is less and less force between your hand and the ball, not when your hand is moving down, but when it's acceleration is in the downward direction. Eventually you will find that the ball leaves your hand completely for a while, and you must catch it. If it weren't for wind resistance, the energy transferred from your hand to the ball when you accelerate it will be the same as the energy transferred from the ball to your hand when you decelerate (catch) it. There may be a period of time when the ball leaves your hand completely, but the end result will be the same.