I would set up a spreadsheet or simple computer model with all your load cases input (panel load, top handrail load and one large concentrated point load), and make sure all your elements have capacity for these loads. In our office we try to limit our deflections to H/125 which allows us approximatley 10mm deflection and the top of the posts.
We had to retrofit and new handrail to an existing exit platform at a university and had a not-so-great detail where we bolted the handrail posts to the web of a channel section. The client asked us to come and and have a look at it because in there opinion it "flexed" a little. We went out and gave it a shake and I didn't think it was moving to far.
I can't upload the video, the size must be to large. I will just post a photo instead, the job is in Australia.