I think the question being asked is probably flawed. Insurgencies (even quite capable ones) are not in fact especially effective against effective modern militaries. And an insurgency solely targeted against a modern military is almost bound to loose.
In practice Insurgencies are generally primarily aimed at the infrastructure and operation of a state or occupation civil apparatus, and the wider civil-society of that state or those co-operating with an occupation.
What insurgencies also do in parallel is inflict a level of attrition on the state or occupation military while continually denying the state or occupation military a military solution/ victory in respect to the insurgencies primary focus/ activity as noted above.
There are of course differences of focus and activity for one insurgency versus another. However insurgencies are best not seen as primarily a military problem but as primarily a political problem with a military component that primarily needs a political solution that in many cases is also likely to have a military component.
And leaning on the military rather than political side of the equation (or just pretending there isn’t a political component) may well just make an effective solution harder/ practically impossible to arrive at.
Almost all of this is wrong. The question being asked is flawed, I'll give you that, because it assumes politicians make smart decisions.
The first part to understanding a "counter-insurgency campaign" is to ignore anything written by modern Western armies and their sympathizers, because they consistently lose. What do they know? Nothing, clearly, or else they would have won in Afghanistan and Iraq. For one thing, "hearts and minds" is something you do prior to the COIN war, because it's to prove you have no ill intentions, whereas an insurgency assumes this automatically, so it's rather pointless.
Look at Socialist campaigns instead, like the kinds Stalin and the Ba'athists fought, in the modern (post-WW2) era. They mostly won.
As for counter-insurgency wars, they come in a few typologies. It's either a localized ethnic cleansing campaign like Indonesia's democide of the Indonesian Communist Party under Suharto with foreign support, or it's a foreign invasion and subsequent settlement of the area like the British in the Orange Free State.
Afghanistan and Iraq could possibly have been won if they'd been fought as annexations like the Indian Wars, for instance.
Promise every NATO soldier who fights in Afghanistan he will get 25 acres with a farmhouse, a robot donkey, and a wife, and suddenly you have much more serious investment in infrastructure and a more compliant population. The locals will need to go, though. Put them in micro-reservations surrounded by barbed wire on economically unvaluable land, like a Central Asian Oklahoma, or send them to neighboring countries. Once the settlers reach a certain critical point, it will be impossible to revert, and the situation inverts: the settlers become the natives and the natives become the colonizers.
That's how Andrew Jackson would have fought it, anyway, and he probably would have won after a few decades. America has a big economy, so it can sustain a serious colonization effort in a few places at once, but only if it wanted to. It has done so in the past, from the Louisiana Purchase to the Gadsden Purchase, so it's not a totally alien concept, just one alien to modern Westerners immersed in LOAC adjacency.
Incidentally, the capacity of the Native American tribes to survive as cultures behind enemy lines, for over 100 years, is astounding. So Andrew Jackson can probably be considered a counter-insurgency loser in that regard, but if we go by that metric, then no one has ever won a counter-insurgency war in history besides maybe the Romans in Iberia I guess.
Simply put, to win in insurgent wars, you actually need to be very gloves off, because you're ultimately exterminating a large portion of a population, because that population does not like you and never will. This is what Western counter-insurgents simply don't understand.
It's also why the most effective Western counter-insurgency campaigns have been via proxy elements, such as in Indonesia or Taiwan, while the Soviets were more effective in both direct interventions and proxy campaigns, with a few exceptions to either side of course.
On the whole, the Marxist understanding of insurgent warfare is far more comprehensive, as it recognizes the innate brutality of "COIN".
The appearance of automatic rifles, which allowed light infantry to produce high volume of fire. This made ambushes and hit-and-run attacks extremely efficient.
Mortars and land mines are probably more significant tbf.
FPV drones might give more serious anti-tank capacities (ability to hit moving targets) to insurgents with relatively limited expense too.
There wasn't much thought given to civilian casualties in times past.
Read what Charlemagne did to the Saxons or Germans to the Prussian tribes.
They were playing for keeps and there was no internet to beam images of Roman's slaughtering an entire rule village to media outlets around the globe.
I'm pretty sure Westmoreland and Zumwalt would be hanged for war crimes these days, or at least cashiered, considering what they did during Vietnam (not because they lost), and the same with Kitchener and Slim for the Boer War and Malaya.
