WBH: is it the only piece of code pipe being done in that shop? Is there only one piece of "silver pipe" of that size and schedule in the whole shop, still sitting on the saw where it was cut? Then it's relatively easy to know! You might still be wrong, of course.
Let's think a little further: how do you know that a short piece of pipe marked with a colour code, job number, and/or heat number and grade, had these marks transferred faithfully from the source piece of material, prior to the welding rather than after the fact?
How do you mark a pup piece of 1/2" pipe used between two socket welding fittings, in any meaningful way?
Even if the pipe and fittings are of the right material, how do you know that the right filler metal was used on the root pass?
How do you know that the marks the mill put on the pipe, and the matching mill cert, weren't fraudulent in the first place?
Let's say that despite doing the best that you could during fabrication, there's still a failure that might have been caused by bad or incorrect material. Sure, you do forensic work on the failed component. But what do you refer to when you're doing your investigation, to see where things went wrong in the fabrication process? The mill marks and temporary marks are both long gone by that point.
A fabricator needs a designed process to procure and inspect incoming material, assign it to jobs per the specifications, store it properly until it is used, and ensure that properly trained and qualified staff carry the rest of the fabrication through to completion in accordance with the required procedures. A fabricator also needs a quality program which inspects the work in process and verifies that those steps are being followed, which carries out NDE and other examinations in accordance with the code and the client's specs, and which records the results.
The owner or their engineer needs to review the fabricator's quality program and decide whether or not those systems are likely to be followed, are adequate to the task at hand, and to what extent they need to be augmented with additional inspections by the owner's engineer or inspector. There's no one single right answer to this: the right answer depends on the size, scale, design life and risk profile of the work at hand.
I know everybody wants their work reduced to an algorithm so that no engineering judgment or thought is required to deliver a safe piping system, or more properly so that if they followed the algorithm and something still goes wrong, they can't be blamed because they followed all the rules. Thank goodness the people on the B31.3 committee haven't obliged them.