How robust are Git commits when the power goes out?

Question

One day I was using Git (I'm still using it) and the electricity went down while I was committing.

When I (actually, the electricity) came back, the git repo was corrupt. I don't remember the exact name, but it was something like "invalid refs" or something like that.

It's easy to guess that the commit was broken in the middle of the operation (I was committing through IntelliJ, which does the index additions automatically). It was also easy to guess that, actually, 'commit' is not as ACID as the DBMS operation with the same name.

Q: Is there a way to ensure repo-altering operations respect atomicity? i.e. if the electricity goes down again, and I'm committing, I'd like my filesystem to not be in a corrupt state.

It doesn't solve the problem - imagine the git repo is on a server and I PUSH from a local repo, and while assimilating the commit in the server, the energy goes down again. The same problem but this time without human presence. — Luis Masuelli, May 15 '14 at 22:51

score 10 · Answer 1 · edited May 23 '17 at 12:40

10

I don't know if there is a way to make Git commits survive perfectly timed power failures, but you might be able to fix your repository.

Git objects are supposed to be immutable, so all your older commits should still be valid. According to this answer, you can change the hash in .git/refs/heads/<branch-name> to change the head of the branch you were working on to the previous commit(you can see them in .git/logs/HEAD).

The comment to that answer says that this method "Still leaves repository in broken state, but this allows to recover it". I'm did not test this(no idea how to replicate your situation), but I assume the recovery is done via git gc, that will delete the corrupted commit.

edited May 23 '17 at 12:40

Community

1

answered May 15 '14 at 22:59

Idan Arye

12,077

6

"no idea how to replicate your situation" Pull the plug? ;) – yannis May 16 '14 at 01:03
3

@YannisRizos Maybe if this was SVN, but Git commits are too fast for human timing abilities... – Idan Arye May 16 '14 at 01:21
5

Git is open source, right? So to reproduce the problem, build the code, set a breakpoint before data are saved to disk - and then pull the plug. – John Saunders May 16 '14 at 20:53

score 10 · Answer 2 · answered May 16 '14 at 20:13

Git's storage system isn't transactional, so there's definitely a chance that a hardware problem could leave things in an inconsistent state. On the other hand, Git is also really fast so you'd have to be really unlucky to be hit by problems of the “power failure” kind (systematic problems with the disk are something else). The speed comes in part precisely from the fact that it isn't transactional; transactions are really quite expensive since they have to wait for confirmation back from the disk that it has written the data. (Databases do all sorts of things to try to conceal this cost, but ultimately they still pay the price. Some of the competitor DVCSes are transactional, and yes, they're quite a bit slower on the same hardware as git.)

In the worst case — a total catastrophic disk failure (which I've seen happen) — the only way for any DVCS to recover is to use the fact that it is distributed. If you've pushed your changes up until very recently to another system and they've shared them to many different hosts, recovery is just a matter of using one of these other places as an artefact source, a place to pull your branches from (even if just temporarily). Like that you'll pull the history of the interesting branches back in and you'll be able to get working again very quickly; anything that wipes out lots of distributed copies of your repo at once is either a disaster of the kind where you don't worry about programming afterwards (think major meteorite impact) or is outright enemy action. (Try not to make such enemies…) This is in total contrast to non-distributed systems, where losing the central server that hosts everything is a fatal blow.

If you don't push your branches, don't keep backups and don't a UPS, you've only got yourself to blame if hardware problems wipe it all. You've got the tools… — Donal Fellows, May 16 '14 at 20:14
Is not any way or plugin to ensure git operations are transactional? If such option -or plugin- existed, I wouldn't care about they (i.e. operations) being slower. Fortunately I have no hardware problems. Unfortunately the energy goes off frequently (saying once per two weeks gets it's own presence) in this building, and I'm coding the whole time and have not UPS. Fortunately I remoted (cloned/pushed) it to github. Unfortunately I lose some time when having to roll everything back and replicate the project again. — Luis Masuelli, May 16 '14 at 22:15
Frequent power outages while working means you're really increasing the risk of filesystem corruption, if not damaging hardware. I would wonder why you're not entertaining the idea of getting a UPS, which would solve most of the described issues you're having and would probably be the best answer. — Bart Silverstrim, May 23 '14 at 14:08

How robust are Git commits when the power goes out?

2 Answers2