The analogy with a denial of service attack is weak at best. The activity involved did not cause anyone a denial of service. Instead, the risk of counter-party bankruptcy caused the intermediaries to require 100% collateral. Essentially, Robinhood and TD Ameritrade would become liable for all losses instead of the hedge fund should the hedge fund seek bankruptcy protection. Other brokers took the risk. It wasn't the volume, per see, or the pattern of trading, but the risk of massive counterparty losses to the intermediaries. Had the short hedge funds been better capitalized, it is unlikely trading would have been suspended for any broker.
As to when it becomes illegal, that isn't an easy question to answer. There are a lot of ways a legal action could flip to an illegal one in securities and commodities law. It will likely depend upon intent. There may be some additional legal issues if some traders unintentionally form a group.
If everyone is strictly honest with stating their beliefs and acts independently, then a repeat would not likely be illegal. The reason GME was an opportunity had nothing to do with Reddit.
Whenever someone short sells, a primary risk is that they will drive down the price so far that an outside investor may find the firm attractive. If they would purchase enough of a firm's float, that could trigger a squeeze on its own. If they are willing to take control of the firm and inject money or resources, they put the short sellers at risk.
Nonetheless, any such investor has it in their self-interest to keep the price low until they have bought their desire target number of shares. They don't want a short squeeze any more than the short seller does, at least in the short run.
That is part of the deeper story here. An investor came along and put their people on the board with a plan to turn the firm around. A short seller makes the most money if a bankruptcy court liquidates the firm. They keep the entire amount shorted because the court will vacate any claims on the short seller.
The next thing is that there is a hardware cycle and X-box was about to come out with the next iteration. Furthermore, Gamestop made a deal to get a percentage of the total revenue resulting from X-box sales from them. Every X-box they sell is now has a permanent revenue stream tied to it.
At that point, it became foolish to continue shorting and the funds should have exited on their own. Indeed, they should probably have started exiting when a credible investor entered the story. Instead, they defended their position by selling even more shares short. The number of shares borrowed and sold short was 140% of the total number of shares in the float.
That is an insanely vulnerable position to be in. People think the individual buyers bought up the shares increasing the price, but that is not the whole story. That had been happening for some time before it burst through.
Two risks existed for the short sellers and each was equally bad.
The first was that shares previously borrowed will be sold to someone that will take them in certificate form or in a cash account. The only shares that can be borrowed are shares in a margin account. In exchange for the right to borrow money, which is what a margin account is, the broker has the right to borrow your assets and loan them out. With a cash account, those shares become off-limits. The same is true for shares in certificate form. Any shares that had been loaned out by the broker of the shares' owner may have to be repaid if put in a cash account by the purchaser if there are no available shares to borrow from someone else.
The second is that shares previously borrowed will be sold to someone that simply holds them. Although those shares can still be borrowed and loaned out if inside a margin account; the act of holding shares pulls liquidity out of the market. It becomes slightly more difficult for buyers to find shares to buy. If the short seller wants out, they still have to find shares somewhere to buy.
Technically, there is also a third risk, but it isn't present in this case. If GameStop had been profitable and capable of declaring a dividend, it could have broken the short-sellers. The short sellers would be liable for all dividends paid to the true owners of the shares. Furthermore, there was an additional 40% of float that represented non-existent shares. As far as the owners of those shares were concerned, they were real shares and they also needed to be paid.
In addition to the cash payments to cover the dividends, the short-sellers would have to pay out additional money to cover the tax losses of the dividend owners. Dividends are generally taxed in a different manner than interest. The people receiving money from the short-sellers really are receiving interest and would owe additional taxes since it wouldn't count as a qualifying dividend.
So they would have to pony up 140% of the float's worth of dividends, they would have to cover all additional taxes and they would have to pay interest on that money unless they had cash to cover the costs. The cash demands on those portfolios could easily have broken them.
Finally, the Redditers set off a cascading reaction that had almost nothing to do with their trades. Machine learning based funds use algorithms to identify profitable trades. When this started coming apart, the algorithms started placing orders built around the patterns in the trades driving demand way up.
That is the point where the Redditors should have escaped. Once it was no longer in their hands, they should have sold everything as their volume wasn't enough to matter anymore compared to total volumes.
