The only thing that comes to my mind is perhaps bending the entire antenna into a curved shape (a segment of a cylinder). Even that would need to be confirmed by modelling or experiment.

What bandwidth? very narrow frequency band gives some simpler options.

Blockage or shaped lens will work.
I've used a Donut, dome or shaped dielectric in front of antennas to widen beamwidths.

If you have alot of S/N ratio, cover up three of the elements with absorber (sounds like your antenna is a four element patch antenna) and only one will be used. That'll widen the beamwidth, but drop your gain 6 dB plus gain change.

Other options for blockage is a disk(metal or dielectric) in front of the antenna. Blocked boresight radiation (or phase delayed lens) is equal to widened beamwidth.

If you have Ansoft HFSS or CST, you could model it and build confidence, though proof is in the build of it.

Basically oyu are talking about using dielectric lens.
Interesting idea even though I afraid it will be losses.
Meanwhile I experimented using conductive scatterers.
I simulated single patch (the antenna is 16 patches) and found out that i can adjust effectively pattern of one polarization but not another. Specifically, B-plane of the pattern can be adjusted and E-plane - not
Also, match goes off
(I am limited by ~ 1 wave length how far I can place the scatterer0

Interesting point about dielectrics

16 patches can't be fed all equal, must be amplitude distribution taper, otherwise you'd have 30 degree beamwidth. With that many elements, it will be more difficult to widen the antenna beamwidth.
Seems you should buy/design a single patch antenna, it'll be easier and less expensive. Or are you forced to use this antenna by management who made a purchase error and don't want to admit it?

120 degree beamwidth antennas are very difficult to create with Smooth patterns if you have any structures near them, since RF Energy will bounce off them.

Good luck.