ykee,
Go for it. The world of IP law is tame, unlike other sectors such as family and personal injury.
If you are 'going to the dark side', it will be as a beacon of light. I think the more technically astute an attorney, the better for the profession. I work in the field of Construction Defects law as an expert witness. The attorneys who are familiar with construction and engineering are far better than the ones who are not.
My suggestion is find out what sorts of cases are most prevalent and immerse yourself in the technical details as well as the legal aspects.
"If you are going to walk on thin ice, you might as well dance!"