Could a malicious actor (e.g. government) attack the network by introducing significant compute power into the network, and once the difficulty grows very high, withdraw all resources from the network.
This could dramatically increase the time it takes for transactions to be confirmed, as well as makes it not profitable for other miners. Granted this would only last until the difficulty is readjusted, but perhaps it could be enough to shake public confidence in the network.
Yes it is possible. You already answered your own question:
"This could dramatically increase the time it takes for transactions to be confirmed, as well as makes it not profitable for other miners. Granted this would only last until the difficulty is readjusted, but perhaps it could be enough to shake public confidence in the network."
There are other attack vectors (such as certain Sybil attacks, Finney attacks, 51% attacks or attacks on blockchain immutability such as the ETH hardfork proposal in response to the recent attack on the DAO) that could have a far greater impact on public confidence than a temporary difficulty adjustment.
If the attacker could generate enough hash rate to engage in a double spending attack, that would also have a much larger impact than changing the difficulty as you describe.
