It's not the best combination. The VAV turndown (min box values) would be limited and the RTU would have to ride up the fan curve, increasing system pressure. There are packaged systems without VFDs that allow a VAV system to be attached. They basically have a modulating bypass damper that short cycles the supply air to the return. They modulate open at higher duct pressure while the VAVs modulate closed. Look up Carrier Parker or Trane VariTrac systems.
If you do add a VFD to one of these simpler systems, you have to be careful of the means of DX enabling – pressure switches are often used and you might lose cooling (via safety interlock) if the unit modulates to too low a speed. It is likely that your minimum VAV values could not bee too low: maybe 60% of maximum, rather than a typical 25-50%.
For your system, barring any big changes you can probably allow some VAV control (e.g., minimums 60-80% of maximums) but you will have very limited versatility.