I don't know of any books dedicated to VSAT design, but have seen a handful of AIAA papers on the subject, you might try their website or see if a technical library close has a subscription to AIAA services.
Take a look at the following website. Click on "search" on the left side of the web page and under there is a link to satellite communications. The link under satellite communications has several books on satellite communications.
Satellite Communications, Robert Gagliardi, 2nd Ed 1991
ISBN 0-442-22745-0
This book has a chapter discussing autotracking earth station antennas, a follow up to your other previous post on azimuth and elevation tracking.
A more recent book I have in my library is:
Satellite Communications, Dennis Roddy, 2nd ed. 1996
ISBN 0-07-053370-9
I have other books in my library. Some authors saying the same thing in other ways. I hope this helps. Let me know if you have any other questions.