Yup, the old wireline drum problem! I have worked on this problem about seventeen years ago, crushed eight or nine drum designs, then gave up. It is one of the most difficult problems i'e tried to solve, probably the most frustrating.
You are correct, a said length of wire rope is spooled onto the core of the drum, line tension adds a compressive stree on the drum. This can be calculated fairly accurate as the length of wire rope. The next few wraps are spooled onto the drum, obviously under the same line tension. Doing the math, the new radius of the layer sits in-between the individual wraps below. This is common for all subsequent layers. Known as the Keplar Packing Problem, or simply the Cannon Ball Problem, you can determine the geometry and calculate the new radius for that layer of spooled wire.
But the second and layer numbers thereafter add a lateral force to the outside flanges, hence a bending moment to the drum. This is the trick, depending on the drum core geometry, sometimes the wire rope is in full contact with the flange, other times not, or partially cradled by the layer below. The literature is clear that you reduce layer tension below by half, and add it to the new core pressure by the above layer at that new radius. Plus the bending moment converted back to core pressure induced by the ens of the layer, depending upon contact with the flange. And that depends on drum geometry allowing the rope to be cradled or not, or partially.
And that is the problem I have been working on, intermittently for seventeen years. There is no literature to fully describe the mathematics, particularly if the wire rope spooled to the drum is jarred. That is even a bigger headache.
There are past threads, yes that welding book is probably the best source of information, API specifications help somewhat, also CSA. I'vecome very close in solving this in FEA, but never fully. I do not buy into the mentioned model of forgetting the layers contributing to core crush, as they are relaxed from above compressive forces. Actually flange loading increases and the end OD the drum fails due to bending. Also, the problem is best solved as an iterative solution as I have described above. I leave the details for you to rework based on our dialogue.
Definitely one of the most entertaining and difficult questions in the oilfield industry. Good luck, LOL!