According to How do I generate a Ledger Monero wallet with the CLI (monero-wallet-cli)? exporting the private view key when generating or using the monero wallet with a ledger device speeds up block scanning, but at the same time:
if your system gets compromised, the adversary will potentially be able to compromise your private view key as well, which is detrimental to privacy
Is the private view key removed from the computer once the wallet application is closed? If not, can it be manually removed?