Warning! Use at your own risk!
The sense and especially added security of all of this may be disputed. Find some notes about security in the end.
tl;dr:
It took me some days to find a solution to get this:
[ 2.251328] init: init first stage started!
[ 2.251561] init: Switching root to '/first_stage_ramdisk'
[ 2.252614] init: Using Android DT directory /proc/device-tree/firmware/android/
[ 2.603025] init: [libfs_mgr]Enabling dm-verity for system (mode 0)
[ 2.603181] init: [libfs_mgr]Verity table: updated block device from '/dev/block/platform/soc/c0c4000.sdhci/by-name/system' to '/dev/block/platform/soc/c0c4000.sdhci/by-name/system_b'
[ 2.603195] init: [libfs_mgr]Verity table: updated block device from '/dev/block/platform/soc/c0c4000.sdhci/by-name/system' to '/dev/block/platform/soc/c0c4000.sdhci/by-name/system_b'
[ 2.753873] device-mapper: verity-fec: 259:33: FEC 0: corrected 22 errors
[ 2.830804] device-mapper: verity-fec: 259:33: FEC 4096: corrected 7 errors
[ 2.831080] init: [libfs_mgr]superblock s_max_mnt_count:65535,/dev/block/dm-0
[ 2.854916] device-mapper: verity-fec: 259:33: FEC 0: corrected 22 errors
[ 2.878336] device-mapper: verity-fec: 259:33: FEC 4096: corrected 7 errors
[ 3.001281] device-mapper: verity-fec: 259:33: FEC 1052672: corrected 18 errors
[ 3.001825] EXT4-fs (dm-0): mounted filesystem without journal. Opts: barrier=1
[ 3.001947] init: [libfs_mgr]__mount(source=/dev/block/dm-0,target=/system,type=ext4)=0: Success
[ 3.002215] init: Switching root to '/system'
[ 3.139199] device-mapper: verity-fec: 259:33: FEC 3133440: corrected 12 errors
[ 3.272389] init: [libfs_mgr]Enabling dm-verity for vendor (mode 0)
[ 3.272662] init: [libfs_mgr]Verity table: updated block device from '/dev/block/platform/soc/c0c4000.sdhci/by-name/oem' to '/dev/block/platform/soc/c0c4000.sdhci/by-name/oem_b'
[ 3.272679] init: [libfs_mgr]Verity table: updated block device from '/dev/block/platform/soc/c0c4000.sdhci/by-name/oem' to '/dev/block/platform/soc/c0c4000.sdhci/by-name/oem_b'
[ 3.276744] init: [libfs_mgr]superblock s_max_mnt_count:65535,/dev/block/dm-1
[ 3.279830] EXT4-fs (dm-1): mounting with "discard" option, but the device does not support discard
[ 3.279845] EXT4-fs (dm-1): mounted filesystem without journal. Opts: barrier=1,discard
[ 3.279935] init: [libfs_mgr]__mount(source=/dev/block/dm-1,target=/vendor,type=ext4)=0: Success
Build LOS 17.1 with the reverted commit from the initial question
Place verity_key in ramdisk/first_stage_ramdisk folder
Append the following cmdline options, this is the android testing key:
BOARD_KERNEL_CMDLINE += root=/dev/dm-0 dm=\"system none ro,0 1 android-verity Android:\#7e4333f9bba00adfe0ede979e28ed1920492b40f /dev/block/by-name/system_b\" androidboot.slot_suffix=_b rootwait androidboot.force_normal_boot=1
Add system partition to the DTB fstab and include verify option
There are more caveats here than I thought. I will also try to include all words I was searching for in this text.
Key findings regarding booting and dm-verity about the payton
- It seems to be a legacy SAR (system-as-root) device (most important read!). The kernel mounts the root/system filesystem itself, no ramdisk or ramdisk partition involved. Typical cmdline arguments for the kernel are root=/dev/block/mmcblk0p6x, where x is 5 or 6 for the system A/B partition and init=/init.
- The original motorola files do not contain a fstab in the ramdisk
- There is no system partition defined in the DTB in the motorola files
- Therefore the bootloader seems to verify the boot image, most likely with some certificates baked into the bootloader? (strings -a on the bootloader hints at this). Then it sets the appropriate start partition and maybe the command line for verity verification, more on this later.
- vendor is called oem on motorola.
- Like mentioned the payton is missing fastboot flash avb_custom_key, so no new shiny Android Verified Boot (AVB) 2.0 with vbmeta.img. Most likely because it was initially shipped with Android 7.0 and nobody cared updating the structures.
Some things about building LineageOS 17.1
- Build LOS like written in their help
- LOS seems to also boot legacy SAR, except when you enter recovery mode.
- Using the instructions below will give you something like Android Verified Boot 1.0
- The build process prepares the following things for dm-verity integration: Creating boot.img, system.img and vendor.img, adds dm-verity metadata to the end of system.img and vendor.img and signs boot.img. vendor is verified in the default LOS configuration by adding it to fstab and additionally creating a DTB fstab entry for it.
The general build process
Codebase is impressively huge: 190 GB without ccache
Revert the mentioned commit in dm-android-verity.c
Use the following Kernel cmdline.
This circumevenes A/B and slotselect mechanism for selecting the proper system_a or system_b. Without bootloader support I do not know if it is possible to ever reenable A/B.
Also this configuration always skips recovery, even if you try to boot to recovery from bootloader.
This did not work with androidboot.veritykeyid, I do not know why (yet).
You have to escape " and # here!
device/motorola/sdm660-common/BoardConfigCommon.mk:
BOARD_KERNEL_CMDLINE += root=/dev/dm-0 rootwait
BOARD_KERNEL_CMDLINE += dm=\"system none ro,0 1 android-verity Android:\#7e4333f9bba00adfe0ede979e28ed1920492b40f
/dev/block/by-name/system_b\"
BOARD_KERNEL_CMDLINE += androidboot.slot_suffix=_b
BOARD_KERNEL_CMDLINE += androidboot.force_normal_boot=1
BOARD_BUILD_SYSTEM_ROOT_IMAGE := false
- without force_normal_boot a "normal" boot goes to recovery.
kernel/motorola/msm8998/arch/arm/boot/dts/qcom/sdm630.dtsi:
system {
compatible = "android,system";
dev = "/dev/block/platform/soc/c0c4000.sdhci/by-name/system";
type = "ext4";
mnt_flags = "ro,barrier=1";
fsmgr_flags = "wait,verify,slotselect"; };
Maybe you can also leave out the DTB change (mounting partitions with DTB will be obsolete in Android 11) and use a fstab file (somewhere, maybe in first_stage_ramdisk).
make sure verity is enabled in ./make/target/product/verity.mk
PRODUCT_SUPPORTS_BOOT_SIGNER := true
PRODUCT_SUPPORTS_VERITY := true
PRODUCT_SUPPORTS_VERITY_FEC := true
- When changing the verity key, do not forget to add it to the kernel in kernel/motorola/msm8998/certs and check the certificates in the running system in /proc/keys
- You can get the last (failed) boot messages in /sys/fs/pstore/console-ramoops
- The resulting system behaves like 2SI ramdisk SAR
Main Question: Does this add security to the device?
- You can lock the bootloader afterwards with fastboot flashing lock
- The boot partition has to be verified manually with the key ids shown while booting (at least I think it is some signatures of the signed boot.img)
- Flashing partitions from bootloader does not work
- Entering recovery also does not work
- fastboot boot file.img gives an error
- fastboot flashing unlock clears userdata, this is good!
- In case anything goes wrong with the running system, you have to clear your device.
- Updating just got a little bit harder without changes to the updater because the A/B partition is hardcoded
- Root needs to be enabled for updates (at least ADB root), so this will stay a userdebug build.
- Changes to /system can be executed without messing the dm-verity, just sign the changed system.
- dm-verity is in eio mode here, I actually did not try to set it to enforcing.
Open questions
- Can you change the kernel cmdline from bootloader?
- Maybe you can blankflash (fastboot oem blankflash) the device, I do not know what happens in this case.
Have fun!
