The purpose of overwriting secrets in memory once they're no longer needed is to avoid or limit information leaks if an application vulnerability leaks memory contents. This is not particularly related to side channel attacks. A side channel can leak the content of unused memory, but then typically it can also leak the content of used memory, and in fact most side channel attacks leak memory contents when that memory is used for something.
You can mitigate the risk of memory disclosure by using a language with enforced memory management (i.e. most languages other than C and C++). Those languages make it impossible to access the content of a variable except through that variable. However, there is still some risk of accidental disclosure. A logic errors in your application might expose data that you didn't intend to expose (e.g. an SQL vulnerability that allows attackers to craft arbitrary SELECT requests). Bugs in the implementation (e.g. in a JIT compiler) might allow attackers to do things that are impossible in a correct implementation of the language.
For long-lived secrets, you can mitigate the risk of exposure by isolating the secret from your main application. Make the secret accessible only to an isolated “partition” whose sole job is to do one thing with that secret. For example, if the secret is a signature key, the isolated partition's only interface is one that takes a message as input and returns the signed message. Hopefully, due to the small attack surface, attackers won't be able to leak the secret, only to make calls to use the secret. Even that is made harder because the attacker would need not just to disclose some memory of the main application, but to cause it to perform a request to the isolated partition with chosen input. Furthermore, if that happens, the isolated partition can make logs so that you'll at least know the extent of the damage after the fact. Also, once the attack is discovered and halted, you know that the attacker is no longer able to use the secret, e.g. they can't keep making more signatures forever.
The nature of the isolated partition depends on your system architecture. Even using a separate process helps (make sure to disable any debugging facilities that would allow the main application to read the other process's memory). That still leaves you at risk of a privilege escalation vulnerability in the operating system, which you can mitigate by using virtualization. For a higher level of isolation, put the isolated partition on separate hardware, to avoid leakage through side channels. (Though if the isolated partition is only doing cryptography using a high-quality library, it might be adequately protected from at least timing leakage.) For high isolation of a simple cryptographic process such as a signature, use a smart card, or a HSM for higher throughput.