Are the parts in your assembly very complex ?
If so you could try simplifying them and let your draft views show the simplified versions.
The spec for your machine should really be up to doing the size of assemblies you refer to, but if you have draft and assembly open and flip between the two it can gradually clog up your system.
Whe doing drafts of big assemblies I've found it best to close and re-open the draft file before doing updates. That way you have freed up as much memory as possible.
At the moment I'm doing drafts of assemblies with something like 18-20K parts, and there are several ways of reducing the amount of data being processed (eg by only displaying visible parts, using simplified parts and/or simplified assemblies).
Most of the drawings only show one view per file, although if it's a section it requires a source view for the cutting plane.
The wokstation is an Opteron 180 with 4GB of RAM and a 256MB Quadro FX something graphics card.
bc