3

So I am using adb to copy all of my Android files to my Windows PC with this command: adb pull -a /mnt/sdcard C:\backup

And when it's done it says all x amount of files have been pulled, I can confirm that the number of pulled files matches the amount of files that reside inside the folder of my Android storage but when I check how many files are actually in the C:\backup folder, I see about 200 missing or in other words adb skipped about 200 files and did not copy them over to my C:\backup folder.

I understand that this could be due to path limit or due to some files containing characters in their filenames that Windows' NTFS file system just doesn't allow.

How can I see what files are actually being skipped? Is there a switch in adb or some trick?

Andrew T.
  • 15,988
  • 10
  • 74
  • 123
computationalprince
  • 45
  • 1
  • 5
  • 1
    You can instead use tar on device to create a tar archive with all files and stream that to your PC. May be tar works more reliable: https://android.stackexchange.com/a/194854/2241 If you device is not rooted remove the su part and adapt the path. – Robert May 23 '20 at 11:49
  • @Robert I used adb exec-out 'tar c /storage/emulated/0/123.txt' > 123.tar but it just creates a corrupted tar archive. – computationalprince May 23 '20 at 12:35
  • @alecxs /mnt/sdcard/ is internal storage and it is also the same as /storage/emulated/0 and no, I am not trying to pull things from a MicroSD, I am trying to pull all the files in the internal storage's root directory. – computationalprince May 23 '20 at 12:39
  • The tar archive was corrupted (what type of corruption)? May be you are facing a totally different problem with a broken USB cable and/or port. What adb version do you use the latest from Android SDK? – Robert May 23 '20 at 13:29
  • @Robert No matter what file or how many files I add to the tar archive with that command it always outputs a 90 byte archive that 7z can't open because it says "Is not archive". I am using the latest adb, I literally downloaded it today. Tried multiple USB cables, same issue. – computationalprince May 23 '20 at 15:37
  • Seems like tar has some problems with the absolute path (it then outputs some text which destroys the archive) also you used the wrong quotes. This works: adb exec-out "cd /sdcard && tar c *" > sdcard.tar – Robert May 23 '20 at 16:42
  • * does not match all file names use . instead – alecxs May 23 '20 at 18:24

1 Answers1

6

Just pulling all files via adb to a Windows computer has multiple problems:

  1. Android/Linux allows multiple files in one directory that just differ in their case.
  2. The file system on Linux allows characters in file and directory names that are disallowed on Windows.
  3. Some file names are completely disallowed on Windows which are just regular file names on Linux/Android.
  4. There is a path limit as you have already mentioned that may cause trouble.

Hence, the best solution to create a full backup of the SD card is to pack all files into one tar archive as all file system limitations of Windows can be bypassed using this method.

Via adb, you create a full backup of the SD card using the following command:

adb exec-out "cd /storage/emulated/0 && tar c * -" > sdcard.tar

This command creates the tar archive on the device and directly streams the data of the tar archive through adb to your PC and saves it there as sdcard.tar.

WARNING: The created file sdcard.tar will be corrupt if tar prints any warning or error message to stdout (this will be invisible to the user executing the command). Therefore, it is crucial to perform an integrity check on the created tar file if the tar structures and the contained files are correct. For more details, see this answer to ADB pull stops after first error.

Andrew T.
  • 15,988
  • 10
  • 74
  • 123
Robert
  • 20,025
  • 6
  • 47
  • 66
  • @alexs You are correct 1 is not possible but 2 and 3 are valid. Filenames with backslash are possible and reserved names like aux and com1 are also possible just as an example. – Robert May 23 '20 at 18:39
  • oh no how stupid from sdcardfs. thought behavior is same as vfat – alecxs May 23 '20 at 18:48
  • I wish this worked, but I simply get a corrupted tar archive no matter what. – computationalprince May 23 '20 at 19:01
  • @computationalprince But the tar size should be correct, yes? Use an hex editor to check if the file starts with an error message. – Robert May 23 '20 at 19:13
  • @Robert I just noticed that tar doesn't exist in /system/bin folder but I have tar installed in Termux but I can't execute it in adb because I don't have permission. – computationalprince May 24 '20 at 10:38
  • @computationalprince An Android without tar installed, ok that explains a lot. What device do you use (manufacturer, model and Android version)? In termux you should be able to copy busybox/tar to /data/local or /data/local/tmp. From there you can execute it from adb. – Robert May 24 '20 at 10:56
  • @Robert My phone is a Huawei G8 with Android 5.1.1 and I don't have permission on /data/local nor /data/local/tmp althought I can cd into both of them. – computationalprince May 24 '20 at 11:04
  • 2
    @computationalprince Right, access to that folders is only possible via adb. Hence if you really want a warking tar you have to copy the busybox binary to sdcard in Termux. Them in adb copy it from sdcard to /data/local/tmp and restore the executable flag via chmod. Then you can execute tar via /data/local/tmp/busybox tar. – Robert May 24 '20 at 11:08
  • 2
    @Robert You were right! I did everything you said and now when I run your command with the new tar path e.g. adb exec-out "cd /storage/emulated/0 && /data/local/tmp/busybox tar c * -" > sdcard.tar I get a healthy tar archive with all the files. Thank you a lot! – computationalprince May 24 '20 at 11:21