Why did the downgrade occur?
It is not widely known, but the W87 was actually originally developed as a ~500 kt warhead, and it had originally been planned for it to be much more widely deployed than it actually ended up being, with over 1000 W87 warheads planned for deployment aboard Peacekeeper (MX) missiles.
The issue that appeared was that this was the same time period that several other major new warhead programs were being developed, chiefly the W88 and the B83. While the number of B83s being procured was somewhat limited, the exceptionally high yield of the weapon demanded a huge quantity of high enriched uranium (HEU) per weapon, and multiplied across 650 warheads, this was a huge quantity of weapons-grade HEU needed. There were multiple other smaller new warhead programs other than the B83 which also needed correspondingly large quantities of HEU. But the single biggest issue was the W88, which was expected to be deployed in quantities ranging from 3500–4600 warheads or greater aboard the forthcoming Trident II missile, which was planned to be carried across the entire Ohio class submarine program – a program that at one point was planned to procure at least 24 submarines. Therefore, a total of potentially as many as 4000–5000 W88 warheads would need to be manufactured for the Trident/Ohio fleet.
Each W88 required copious amounts of HEU in order to meet its planned yield of approximately 455–475 kt, and when this was multiplied across as many as 4000–5000 W88 warheads, this represented an enormous drain on US stockpiles of HEU. When this was combined with the projected need to manufacture a minimum of 1000 W87 warheads, it became clear that the US simply did not have enough available weapons-grade HEU remaining in inventory to be able to construct the 5000–6000 500 kt warhead secondaries needed for both programs.
For various reasons, it was eventually determined that either the W87 or the W88 would have to modify the weapon's design to remove the HEU in its secondary, as the US simply did not possess enough HEU to supply both programs with HEU secondaries at their planned production run sizes.
While it would be quite easy to replace the HEU with NU or DU in either weapon, and such a change would be unlikely to require any significant additional nuclear testing, this change would reduce the yield of either warhead from nearly 500 kt (455–475 kt) to only around 300 kt.
It was eventually decided that the Air Force would have to take the hit despite their program's much smaller number of warheads. The Air Force was not at all happy about this decision, and vigorously protested, but it was to no avail, and they were forced to have the W87 redesigned to use a NU/DU secondary in lieu of the originally planned HEU secondary.
This was an especially painful defeat for the Air Force, as the MX had long been envisioned as carrying 10–11 very high yield 500 kt warheads, and indeed much of the justification for building the Mk21/W87 instead of sticking with the Mk12A/W78 (the missile was actually initially planned to reuse the Mk12A/W78 until fairly late in its development) had been based in the substantial increase in warhead yield from switching to the W87.
Indeed, warheads of a variety of yields were considered for the MX when it was decided to build a new warhead instead of continuing with the original plan of reusing the Mk12A/W78, as the ABRV had been designed to be capable of being scaled up to carry larger warheads. Specifically, 600 kt and 800 kt warheads were considered before the 500 kt warhead design was chosen as the final candidate. There are no indications that any design with a yield under 500 kT was ever considered. This was of course in part because the W78's 340 kt yield, while considerable, was still not perfectly ideal for cracking ultra hard targets, especially given the Mk12A's inherently significantly inferior accuracy relative to the ABRV.
While the MX possessed admirably high accuracy, a rather large part of the accuracy of any ICBM lies in the design of the reentry vehicle, and in fact a very large part of why the MX ultimately attained as high of an accuracy as it did was due to the exceptionally large amounts of effort that had been invested into developing the ABRV, which was carefully optimized from the start to minimize every possible source of accuracy loss to the maximum extent that was technologically feasible. This went as far as to replacing the entire RV separation system with a completely new design, because the existing system on the Mk12A imparted a slight amount of error during separation. There are dozens if not hundreds of other micro-optimizations like this one that were incorporated into the ABRV. All of that added up to make the ABRV an absurd leap in RV performance.
While you could in theory have integrated the W78's warhead into the ABRV, doing so would have been a waste of money. One major goal with the W87 was to improve safety and reliability, and part of that goal was achieved by redesigning the entire warhead. The W78 used conventional high explosives, lacked advanced safety devices, lacked a fire resistant pit, and had various other deficiencies. The W87 had been built from scratch with insensitive high explosives, incorporated all of the latest and greatest safety devices (drastically improving safety compared to the W78), had a fire resistant pit cladding added, and incorporated various other improvements. Unlike the Navy, the Air Force missiles were under constant maintenance, and the amount of warhead handling and long-distance transport required to maintain the ground based ICBM force was immensely greater than what was needed for naval SLBMs. Accordingly, the Air Force placed a very high value on obtaining a safer warhead that was less hazardous to its personnel.
Also, by using IHE, different methods could be used for constructing the warheads and installing them in their reentry vehicles. These different methods allowed for drastically higher throughput and much less labor and facility space requirements to perform construction steps, which significantly reduced assembly costs, and therefore significantly reduced the cost to construct each warhead vs the cost to construct a warhead utilizing traditional conventional high explosives.
The biggest issue was the one of using bays vs cells for construction.
With IHE, bays can provide adequate isolation, as the probability of an explosion is extremely low, and warheads in adjacent cells are unlikely to detonate as the shielding between cells is sufficient to protect from the blast igniting the IHE in other warheads.
With CHE, cells are the only option that could meet minimum safety standards. If bays were used for CHE, then a blast in one bay could conceivably detonate the CHE in adjacent bays even through the blast shielding between bays (admittedly it may be more likely to act by bypassing the shielding and channeling through the corridor rather than by propagating through the shielding, but the effect from either is identical), and that was an unacceptable safety hazard.
There was also a concern about warheads being moved between bays and their exposure to blast effects. With IHE, a warhead in transit may not go off. With CHE, a warhead in transit is all but guaranteed to also go off.
You can look up information on the Pantex plant bays and cells and explosive handling procedures to find out more about this.
Anyways, once the Air Force decided that they needed a new warhead, they decided they wanted a 500 kt warhead instead of a 600 or 800 kt warhead, likely because this allowed for cramming more warheads on the missile. As for why they didn't go for a smaller warhead, well, the existing W78 was already a 340 kt warhead, and they had a 500 kt design available that wouldn't be much larger or heavier than the W78, so it seemed like a no-brainer to adopt that design, especially so as doing so would even further increase the SSKP of the new warhead against the hardest of target classes. MX was intended from the start to be the ultimate hard target killer, and a 500 kt warhead paired with the accuracy of the ABRV+MX combo would substantially outperform every other land based missile that had ever been fielded in terms of hard target performance.
But when the Navy designed the W88, they wanted basically the same thing, except they didn't give a rat's ass about warhead safety (or cost), didn't need it to survive being fired from as long of a range as the land based missiles, and placed such a high premium on slashing every kilogram of weight (and liter of volume) from the RB that they demanded a number of special design features be incorporated into their warhead to optimize it for naval use.
(I originally started going into detail on everything involved in the naval W88 program here, but I excised that section from my final answer as it was still an incomplete draft and was promising to end up being
far too lengthy to incorporate into this comment without veering way too far off topic and making it too hard to read my reply.)
You can sum this up as "the Navy demanded that they get a special variant built just for them, which reused the same secondary design and much of the same RV design (albeit heavily modified), but changed tons of other things to adapt it for naval use and incorporate a newer upgrades (some very major, some less so) into the design".
Now, a solid argument exists that the Navy went overboard on yield. However the Navy wanted a hard target killer, and the 500 kt Cursa/Almendro secondary was a perfect fit for their ambitions. And with this being the Navy, if they downgraded to a more reasonable 300 kt, they'd have probably demanded that this be done using a scratch-designed secondary in order to optimize warhead weight and volume. It's unclear if a well-tested 300 kt secondary option existed at this time, and thanks to the TTBT, new secondaries for these higher-yield warheads could not be developed anymore. There is a good argument that they could have used a W78 secondary, but the Navy wanted 500 kt, and that's what they got.
You may be wondering why the Navy won out over the Air Force, especially when the two platforms had fairly similar CEPs (and even more so considering how the Naval platform was the one packed with CEP-enhancing tech like the SSNT and the RUPL).
From a technical standpoint, there was actually a robust technical argument that the HEU should have been allocated to the W87 instead of the W88.
However from a political standpoint, the W87 was married to the hyper-controversial MX program (which had little political capital left to defend itself when what little political capital they had was tied up in defending the MX missile itself, and therefore minimal leverage to protect against the loss of HEU), while the W88 was tied to the comparatively totally uncontroversial Trident submarine/missile programs (which had tons of political capital, and therefore far more leverage to demand that the HEU be allocated to them).
So unsurprisingly, the Navy won the fight, and the HEU was allocated to the Navy's W88 program, with the Air Force being ordered to redesign the W87 to remove the HEU from its secondary (producing what we now know as the W87-0 or W87 today). The resulting decrease in yield left the warhead with even less yield than its predecessor (300 kt vs 340 kt), but the dramatic increases in accuracy from the massively improved missile guidance and reentry vehicle meant that it was still a far more capable warhead than the W78, even if it still wasn't quite as powerful or effective as the initial plan had called for.
This all happened during the early development of both programs.
The earliest reference to the yield downgrade that I am aware of occurred in an article published on January 17th 1983 that was covering the decision by the Navy to develop a new warhead for the Trident missile (ie the W88).
The decision by the DoD to replace the Peacekeeper's original Mk12A/W78 warhead with the new Mk21/W87 warhead was only made in 1982.
Due to the lack of reliable detailed information regarding the timelines of the W87 and W88 programs (largely because most documents on these warheads are still highly classified as active weapon systems and therefore largely immune to FOIA/MDR), it is not possible to say exactly when the HEU shortage issue was first flagged, exactly when the conflict between the two forces happened, exactly when the conflict was first resolved, etc.
Part of this stems from the fact that the warhead secondary was developed far in advance of the actual warhead itself, and part stems from the fuzzy boundaries between weapon system development start, weapon system development authorization, design finalization, etc.
If I had to hazard a guess though, I'd assume the problem likely first cropped up and got addressed sometime in the 1980–1982 time frame.
Probably something like XW-87-X1.
Developmental designations can vary. If it had been developed normally, it would have likely been assigned a XW-87 or XW-87-X0 designation, and the actual W87-0 would likely have been assigned the XW-87-X1 designation. But we don't know at what phase this issue occurred. If it had been early enough, there may not even have been two variants by the time that the designation was being assigned. If it was later, then odds are the X0 variant is the original design, the X1 variant is the W87-0, and either the X0 or a X2 variant was the original W87-1 design. If that's the case, then the current new W87-1 variant could potentially be labeled the XW-87-X3. But without FOIA'd/MDR'd files (which are unlikely to get released within either of our lifetimes at this point given how entrenched both warheads are in the long term plans for the US nuclear arsenal), we'll never know for sure.
The EPA and FBI storming the Rocky Flats production area and shutting them down for massive violations of more or less every environmental and radiological-controls law.
Nah, this is simply why the W88 program ended up only producing ~10% of the planned production run. It has no relation to the original decision to force either the W87 or the W88 to give up their HEU secondary in order to ensure adequate HEU stockpiles would be available to construct the full planned production run of both warheads, the resulting decision to force the Air Force to be the one to drop their HEU secondary, the subsequent redesign of the W87 to use a non-HEU secondary and consequent loss of yield relative to the originally planned design, or any of the other related events.
It is highly ironic though, since thanks to the rocky flat fire the government was forced to deploy the Mk4/W76 aboard the D5 missiles, and as a result all of a sudden we had a huge surplus of weapons grade HEU due to the de facto forced cancellation of the W88 program.
Then the Cold War ended and subsequent reductions in warhead numbers thanks to the peace dividend and multiple new arms control agreements flooded the DoE/NNSA with more surplus WGHEU than we could ever use, to such a degree that we started downblending huge amounts of it with DU tails and selling it to commercial reactor operators just to get rid of the stuff.
(Okay, that last bit was technically part of a bilateral arms control agreement to permanently draw down surplus military HEU stockpiles via disposal in commercial power reactors rather than a necessity, but the point still stands – it's kind of hilarious how much WGHEU we simply declared as surplus to requirements and disposed of, and just how much still remains even after all of that.)
