And there are no capacitors on the input or output of your regulators? Check the data sheet for your regulators; I'm sure they'll show bypass capacitors. You'll probably need bypass caps at the chips that are fed by the supplies as well. The type and value of the caps will depend greatly on whether the device(s) are digital or analog. Higher inductance, due to long leads or traces, will make caps less effective for bypassing digital chips. Higher ESR (dependent on capacitor type and value) will also affect the effectiveness of the bypass caps, especially in switching regulators (although I'm assuming you're using linear regulators here even though you didn't specify).
So, when you say "I read somewhere that two capacitors 220n and 1u on the power supply removes the high frequency spike noise in the system", it's a rule of thumb that works in some places, but it's really dependent on your particular circuit. If you want better info, it would really help to give us a more complete description.