As you are using the NX11Custom setup. Did you check if the "NX_CUSTOM_TOOL_BARS=${NX_CUSTOM_LIB}\ToolBars\" variable is pointing to the correct location? (as described in the readme file)
Ronald van den Broek
Senior Application Engineer
Winterthur Gas & Diesel Ltd
NX9 / TC10.1.2
Building new PLM environment from Scratch using NX11 / TC11