To summarize the early VLS problems and challenges:
1) Most 1960s missiles are rather heavy, and their rocket boosters aren't very powerful. Launching the missile vertically - without the aid from wings lifting force - would require significantly more massive booster;
2) Stabilization of 1960s missiles during subsonic flight stage was generally achieved passively, with the help of large booster fins. Putting such fins into vertical cell would require cells of totally impractical diameter. So some kind of very fast opening folding fins must be developed, which was a significant design challenge then;
(It might be cirumvented a bit by using packet scheme with boosters wrapped around missile body, nozzles slightly inclined aside to induce spin stabilization - like on Seaslug SAM - but again, it would require very large cell to fit)
3) Most of 1960s missile required either to be pointed toward the target before launch (the 1960s seekers weren't exactly fast to lock on...) or been gathered by guidance radar beam immediately after launch and aimed toward the target - while missile is still boosting. With arm launchers it was simple; the missile was pointed in required direction by launcher. But for vertical system, it would basically require a separate control system just for boosting. And in 1960s, such control system would be bulky, heavy and costly;
4) Missiles in 1960s weren't still fully reliable, so vertical launch presented rather nasty problem; what if the missile motor failed to ignite, and the missile came crashing down on the ship? This problem could be partially solved by having launchers inclined to sides, but this would create "danger area" for other ships nearby;
(same problem with spent boosters)
5) Finally - what's the point of having fast-working vertical launcher, if fire control system could not fully exploit its advantages? Most of 1960s fire control systems have no more than 1-2 target channels, which meant that even if you could fire missiles very fast, you could only fire then against 1-2 targets already tracked. After they would be dealt with, your launcher would sit idle, while fire control radars are seeking and aquiring new targets.
Yes, you could fire more missiles against the same targets in less time, but frankly, it isn't exactly big advantage. And, for example, Soviet SAM's were mostly command-guided, i.e. limited by number of missile control channels also - so there were no point in launching more missiles that could be tracked & guided;
To suumarize, for 1960s the VLS systems were deeply impractical. Designing them would cause enormous numbers of engineering & practical applications, while advantages were not fully clear due to fire control systems limitations. It was MUCH more simple to circumvent all those headaches by just using rail launcher and mechanized ammo magazine; while heavier they were just a simpe and reliable mechanics.
P.S. Should point out that interest toward VLS become practical in 1970s due to same reason for USSR and USA: major improvements in fire control capabilities. The USSR developed multi-channel S-300F system, which could guide up to 12 missiles against several targets. The USA developed SM-2 missiles and Aegis system. In both cases rail launchers weren't sufficient anymore, and much faster rate of fire become practical.