Sudden proliferation of 'Too many open files in system' errors. Can't do backup

Question

My computer is showing signs of impending hard drive disaster. Time Machine recently killed all my backups and has not cooperated since. I tried backing up with Super Duper, but it quit after about a minute. The Super Duper log reported 'Too many open files'. I attempted a backup with Rsync and it hummed along fine for about an hour until it too started spitting out 'too many open files' errors and not copying anything. Carbon Copy Cloner worked longer but skipped a bunch of files.

Based on other posts here, I have maxed the open file limit and am running rsync again. Though this process has not completed yet, I am posting here because I do not feel optimistic. I am seeing this 'Too many open files' error all over the place suddenly, so I fear that maxing the file limit is a workaround at best. (assuming it even works)

The first time I saw the 'Too many open files' error, I was running a shell script that involved uploading an archive to a remote server and unzipping it. I had done this numerous times before without incident. The number of files hadn't changed. I got this error again while running a Python script to build files using Jinja2. This was also a script I had run previously without any problems.

The sudden proliferation of errors of this kind coincides with a general degradation in the performance of my computer, which most frequently takes the form of app and system freezes that can only be resolved with forced restarts. At the very same time rsync choked this evening, Safari was hanging also. As ever, Force Quit was not available and so I had to force a shutdown. There are also more crashes than usual. A recent Apple update that crashed my computer on restart seems to have made everything much worse.

Carbon Copy Clone's Disk Center said my Hard Drive seemed to have Read/Write errors, though the Apple Hardware Test and Disk Utility have reported nothing out of the ordinary. Activity Monitor also does not show anything unusual, though I am not an expert in interpreting its output. I think there may be more disk activity than usual. CPU and Memory stats all seem fine.

Any ideas on what's going wrong here? Is it appropriate to assume the hard drive is the problem? It came with the computer, which is now four years old. Any other possibilities? Any recommendations on how to get everything backed up before the drive -- or whatever -- dies altogether?

I am running Snow Leopard on a mid 2010 15" MacBook Pro. I am attempting to back up to a brand new portable USB-connected portable drive which seems to work fine.

Answer 1

I have several suggestions that I'll put into two different posts.

Here's the first one:

This seems to help with practically everything, so why not try it.

Open your Disk Utility app, and after selecting your main hard drive (or partition), follow the steps in the picture below. This operation will attempt to find and repair any disk and disk-permission errors.

Please note that step 1(b) probably will not work unless you are booted into the Network Recovery Mode, or running off a different hard drive (not the one you are trying to fix). If running step 1 reveals a lot of errors then doing 1(b) would definitely be recommended.

The easiest way to do this is to borrow someone else's Mac and connect your computer to it in Target Disk Mode. This will allow you to run Disk Utility on their computer thus letting you perform step 1(b) to your hard drive.

If you don't have another computer to borrow, your will need to put you computer into Network Recovery Mode (aka. OS X Internet Recovery). Please see the OS X Internet Recovery section in this hyperlink before proceeding with this fix. Also note that Network Recovery Mode is different from Recovery Mode; the regular recovery mode will not help in this case. To start your Mac in the Network Recovery Mode, please see this link.

If none of the above are available on your computer, or Network Recovery simply takes too long on your internet connection, there is a third option. You can make a Bootable OSX Recovery Drive from a USB stick as described at the bottom of this page from Apple.

Hope this helps! :)

Answer 2

If none of the other backup options work, I would recommend trying the dd command as described here. The benefit of this command is that it brute force clones your hard drive and continues running even if it encounters corrupt sections on your disk (depending on the settings you use, as described in the link and in the comments there).

I think you might be able to run this directly from your computer, but I would actually recommend running it from an external hard drive or the Network Recovery Mode like I described in my previous post.

Please note that this operation can take up to several days to complete, and you should not use your computer during this time, because otherwise some files may get changed while the dd command is trying to save them.

Here are some other less-brute-force recovery tools to try:

Data Rescue for Mac (It's not free, but this is what our local Mac repair guy uses.)

Data Rescue 3 also has a free demo version if you'd like to try it.

Here are some freeware recovery tools that are actually quite good:

PhotoRec recovers most standard file types (not just photos) and is pretty easy to use.

TestDisk is made by the same people who made PhotoRec, but it's way more advanced (and therefore more involved to run). I would recommend reading the guide on their site if you want to use it.

Best :)