I believe the problem lies in the fact that you're trying to make a design table do something it's not really intended to do. Design tables are intended to drive changes to part geometry, not report the result of changes. If you are making changes to an external Excel file that a part file is linked to, after each change to the Excel table you will have to:
1. Open the part file and choose the option to update the model based on design table changes. SolidWorks will open the design table, read the changes that drive part geometry, update that geometry, and close the design table. SolidWorks will then rebuild the part, which will update the Mass information.
2. You might then have to do an additional rebuild (ideally using Ctrl-Q) to make sure that the mass information is transferred into the custom property.
3. You will then have to open the design table again to allow SW to load the new mass information into the table.