4

For background info, I just picked up a Xiaomi Mi 9 Global version and am running the latest weekly Xiaomi.eu Pure MIUI ROM on it (if it matters, the latest Xiaomi.eu weeklies for the Mi 9 are Android 10 based).

I've started taking full backups using TWRP and I noticed something strange. For reference these backups have compression disabled and no encryption to try and rule out any variables.

When I check the sha2 hashes of the various partitions, I've noticed that all of the various partitions (boot, recovery, modem, efs, etc) have the same sha2 hash between backups (meaning the images are exactly the same). However even just from one night to the next morning, or even from one boot to the next as I tested today, the hashes of the vendor and system partition backups are different. I don't have any custom system modules or kernels installed that could be modifying those partitions, though I am rooted with Magisk.

I used adb shell to browse around the system and vendor partitions and all of the date stamps of the folders and files are either "1969-12-31" (basically meaning the date is 0 since that's the beginning of "unix time if you take into account my time zone) or "2019-09-06" which is the probably the date that the ROM or it's base ROM was created (the original Xiaomi ROM they made the Xiaomi.eu version from). I even used the find command from an adb shell to search both the system and vendor partitions for any updated files (any file with a date newer than "2019-09-06") and there are none.

So my question is this: if I haven't flashed a new ROM or installed anything in the system partition, why would two backups be different?

I would think that only the data partition would change as I use the phone. I thought that since the system and vendor images are the ROM itself, they should never change unless I change ROMs. I was under the impression that even system settings are stored on the data partition because performing a factory reset essentially just wipes that partition...so I don't see why the system and especially vendor partitions should ever change unless I specifically flash a different ROM or update the ROM which would obviously write to those partitions (or at minimum the system partition)

Are there some kind of auto-generated files or folders in the system and vendor partitions that change slightly on ever boot or something?

I would just like to understand this better so I know whether for example it's ok to just do regular backups of the data partition and only back up system and vendor when I do a new ROM update.

Hopefully someone with more Android ROM experience can shed some light on this.

Ben Baron
  • 143
  • 4

1 Answers1

2

Answer to your question is dm-verity:

The dm-verity feature lets you look at a block device, the underlying storage layer of the file system, and determine if it matches its expected configuration. It does this using a cryptographic hash tree. For every block (typically 4k), there is a SHA256 hash.

So /system and /vendor partitions are always mounted read-only because mounting R/W causes changed hash values of the partition/filesystem (at least some inodes belonging to filesystem structure may change even if you don't touch a file), thus breaking the security feature. To make any changes to them, we need to disable dm-verity which must be your case.

TWRP creates backups in two ways. boot, recovery etc. are just dumped (dd) as a whole, so their hashes don't change because those aren't mountable filesystems. /system and /vendor are either dumped as images to retain dm-verity as discussed above. Or otherwise a tar archive is created to keep the backup size minimum. In second case even the filesystem is unchanged, a new archive file is created every time you do the backup. If the files aren't changed and are read/written by the archiving utility in the same order, using same options/parameters every time, then created archives should be identical. But even a minor factor e.g. compression may cause a different hash value every time.

So you must have dm-verity disabled, and therefore you will keep on getting changed hashes. But the contents don't change (if you don't change yourself or Magisk/GApps don't put some addon.d script), so a regular backup of both partitions isn't required. In fact only /data backup is enough, boot, /system, /vendor are already part of flashable zips.

Irfan Latif
  • 20,353
  • 3
  • 70
  • 213
  • Just to add a slight addition, I believe what's happening is that TWRP on my device still uses image backups even though my rom has dm-verity disabled, however, since the partitions are presumably mounted r/w due to dm-verity being off, they get slight changes each time they're mounted (tiny filesystem or journaling changes) which then still causes their hashes to be different even though they are not tar files. – Ben Baron Sep 13 '19 at 01:41
  • In any case, as you said in your other comment, there are no actual file changes and so new backups don't need to be made unless you manually modify those partitions like by installing something to /system in TWRP or a Magisk module, etc, or updating the rom. – Ben Baron Sep 13 '19 at 01:41
  • Actually even weirder, I tried moving something to /system and I had to remount it r/w which would indicate that it is being mounted read-only, but I guess the system itself my mount it first r/w then later lock it to read-only and that's why it's getting modified between backups. I can't think of any other explanation. – Ben Baron Sep 13 '19 at 03:36
  • 1
    @BenBaron Android by-default mounts /system read-only as R/W is no way an option b/c of VB. But when we flash Magisk and TWRP, Android is no more the single entity on the device to play with /system. SAR requires /system to be mounted by Magisk first before Android's init does so. Also note TWRP's warning: This device uses dm-verity!. – Irfan Latif Sep 13 '19 at 05:36
  • Ahh that's a good point, I forgot about Magisk, it does indeed write to /system. However, regarding dm-verity that link is for the Note 4, I have the Mi 9, though from what I can tell the stock rom does indeed also have dm-verity enabled, however the custom Xiaomi.eu rom I'm running does not. TWRP doesn't seem to know though and just assumes it's enabled and takes system images instead of regular system backups. – Ben Baron Sep 13 '19 at 16:29
  • 1
    @BenBaron the link is just for reference, I couldn't find Mi9 on official page of TWRP. You will find the same warning on hundreds of other devices too, and is in fact applicable to every device shipped with dm-verity enabled (devices "MUST support Verified Boot for device integrity"). Custom ROMs almost always have to disable security features like dm-verity because those are usually userdebug builds. Some disable SELinux too in beta phases. – Irfan Latif Sep 13 '19 at 16:40