Thursday 25 June 2015

Password storage in Android M

While Android has received a number of security enhancements in the last few releases, the lockscreen (also know as the keyguard) and password storage have remained virtually unchanged since the 2.x days, save for adding multi-user support. Android M is finally changing this with official support for fingerprint authentication. While the code related to biometric support is currently unavailable, some of the new code responsible for password storage and user authentication is partially available in AOSP's master branch. Examining the runtime behaviour and files used by the current Android M preview reveals that some password storage changes have already been deployed. This post will briefly review how password storage has been implemented in pre-M Android versions, and then introduce the changes brought about by Android M.

Keyguard unlock methods

Stock Android provides three keyguard unlock methods: pattern, PIN and password (Face Unlock has been rebranded to 'Trusted face' and moved to the proprietary Smart Lock extension, part of Google Play Services). The pattern unlock is the original Android unlock method, while PIN and password (which are essentially equivalent under the hood) were added in version 2.2. The following sections will discuss how credentials are registered, stored and verified for the pattern and PIN/password unlock methods.

Pattern unlock

Android's pattern unlock is entered by joining at least four points on a 3�3 matrix (some custom ROMs allow a bigger matrix). Each point can be used only once (crossed points are disregarded) and the maximum number of points is nine. The pattern is internally converted to a byte sequence, with each point represented by its index, where 0 is top left and 8 is bottom right. Thus the pattern is similar to a PIN with a minimum of four and maximum of nine digits which uses only nine distinct digits (0 to 8). However, because points cannot be repeated, the number of variations in an unlock pattern is considerably lower compared to those of a nine-digit PIN. As pattern unlock is the original and initially sole unlock method supported by Android, a fair amount of research has been done about it's (in)security. It has been shown that patterns can be guessed quite reliably using the so called smudge attack, and that the total number of possible combinations is less than 400 thousand, with only 1624 combinations for 4-dot (the default) patterns.


Android stores an unsalted SHA-1 hash of the unlock pattern in /data/system/gesture.key or /data/system/users/<user ID>/gesture.key on multi-user devices. It may look like this for the 'Z' pattern shown in the screenshot above.

$ od -tx1 gesture.key
0000000 6a 06 2b 9b 34 52 e3 66 40 71 81 a1 bf 92 ea 73
0000020 e9 ed 4c 48

Because the hash is unsalted, it is easy to precompute the hashes of all possible combinations and recover the original pattern instantaneously. As the number of combinations is fairly small, no special indexing or file format optimizations are required for the hash table, and the grep and xxd commands are all you need to recover the pattern once you have the gesture.key file.

$ grep `xxd -p gesture.key` pattern_hashes.txt
00010204060708, 6a062b9b3452e366407181a1bf92ea73e9ed4c48

PIN/password unlock

The PIN/password unlock method also relies on a stored hash of the user's credential, however it also uses a 64-bit random, per-user salt. The salt is stored in the locksettings.db SQLite database, along with other settings related to the lockscreen. The password hash is kept in the /data/system/password.key file, which contains a concatenation of the password's SHA-1 and MD5 hash values. The file's contents may look like this:

$ cat password.key && echo
2E704465DB8C3CBFF085D8A5135A6F3CA32D5A2CA4A628AE48E22443250C30A3E1449BD0

Note that the hashes are not nested, but their values are simply concatenated, so if you were to bruteforce the password, you only need to attack the weaker hash -- MD5. Another helpful fact is that in order to enable password auditing, Android stores details about the current PIN/password's format in the device_policies.xml file, which might look like this:

<policies setup-complete="true">
...
<active-password length="6" letters="0" lowercase="0" nonletter="6"
numeric="6" quality="196608" symbols="0" uppercase="0">
</active-password>
</policies>


If you were able to obtain the password.key file, chances are that you would also have the device_policies.xml file. This file gives you enough information to narrow down the search space considerably when recovering the password by specifying a mask or password rules. For example, we can easily recover the following 6-digit pin using John the Ripper (JtR) in about a second by specifying the ?d?d?d?d?d?d mask and using the 'dynamic' MD5 hash format (hashcat has a dedicated Android PIN hash mode), as shown below . An 8-character (?l?l?l?l?l?l?l?l), lower case only password takes a couple of hours on the same hardware.

$ cat lockscreen.txt
user:$dynamic_1$A4A628AE48E22443250C30A3E1449BD0$327d5ce3f570d2eb

$ ./john --mask=?d?d?d?d?d?d lockscreen.txt
Loaded 1 password hash (dynamic_1 [md5($p.$s) (joomla) 128/128 AVX 480x4x3])
Will run 8 OpenMP threads
Press 'q' or Ctrl-C to abort, almost any other key for status
456987 (user)
1g 0:00:00:00 DONE 6.250g/s 4953Kp/s 4953Kc/s 4953KC/s 234687..575297

Android's lockscreen password can be easily reset by simply deleting the gesture.key and password.key files, so you might be wondering what is the point in trying to bruteforce it. As discussed in previous posts, the lockscreen password is used to derive keys that protect the keystore (if not hardware-backed), VPN profile passwords, backups, as well as the disk encryption key, so it might be valuable if trying to extract data from any of these services. And of course, the chance that a particular user is using the same pattern, PIN or password on all of their devices is quite high.

Gatekeeper password storage

We briefly introduced Android M's gatekeeper daemon in the keystore redesign post in relation to per-key authorization tokens. It turns out the gatekeeper does much more than that and is also responsible for registering (called 'enrolling') and verifying user passwords. Enrolling turns a plaintext password into a so called 'password handle', which is an opaque, implementation-dependent byte string. The password handle can then be stored on disk and used to check whether a user-supplied password matches the currently registered handle. While the gatekeeper HAL does not specify the format of password handles, the default software implementation uses the following format:

typedef uint64_t secure_id_t;
typedef uint64_t salt_t;

static const uint8_t HANDLE_VERSION = 2;
struct __attribute__ ((__packed__)) password_handle_t {
// fields included in signature
uint8_t version;
secure_id_t user_id;
uint64_t flags;

// fields not included in signature
salt_t salt;
uint8_t signature[32];

bool hardware_backed;
};

Here secure_id_t is randomly generated, 64-bit secure user ID, which is persisted in the /data/misc/gatekeeper directory in a file named after the user's Android user ID (*not* Linux UID; 0 for the primary user). The signature format is left to the implementation, but AOSP's commit log reveals that it is most probably scrypt for the current default implementation. Other gatekeeper implementations might opt to use a hardware-protected symmetric or asymmetric key to produce a 'real' signature (or HMAC).

Neither the HAL, nor the currently available AOSP source code specifies where password handles are to be stored, but looking through the /data/system directory reveals the following files, one of which happens to be the same size as the password_handle_t structure. This implies that it likely contains a serialized password_handle_t instance.

# ls -l /data/system/*key
-rw------- system system 57 2015-06-24 10:24 gatekeeper.gesture.key
-rw------- system system 0 2015-06-24 10:24 gatekeeper.password.key

That's quite a few assumptions though, so time to verify them by parsing the gatekeeper.gesture.key file and checking if the signature field matches the scrypt value of our lockscreen pattern (00010204060708 in binary representation). We can do so with the following Python code:

$ cat m-pass-hash.py
...
N = 16384;
r = 8;
p = 1;

f = open('gatekeeper.gesture.key', 'rb')
blob = f.read()

s = struct.Struct('<'+'17s 8s 32s')
(meta, salt, signature) = s.unpack_from(blob)
password = binascii.unhexlify('00010204060708');
to_hash = meta
to_hash += password
hash = scrypt.hash(to_hash, salt, N, r, p)

print 'signature %s' % signature.encode('hex')
print 'Hash: %s' % hash[0:32].encode('hex')
print 'Equal: %s' % (hash[0:32] == signature)

$./m-pass-hash.py
signature: 3d1a20985dec4bd937e5040aadb465fc75542c71f617ad090ca1c0f96950a4b8
Hash: 3d1a20985dec4bd937e5040aadb465fc75542c71f617ad090ca1c0f96950a4b8
Equal: True

The program output above leads us to believe that the 'signature' stored in the password handle file is indeed the scrypt value of the blob's version, the 64-bit secure user ID, and the blob's flags field, concatenated with the plaintext pattern value. The scrypt hash value is calculated using the stored 64-bit salt and the scrypt parameters N=16384, r=8, p=1. Password handles for PINs or passwords are calculated in the same way, using the PIN/password string value as input.

With this new hashing scheme patterns and passwords are treated in the same way, and thus patterns are no longer easier to bruteforce. That said, with the help of the device_policies.xml file which gives us the length of the pattern and a pre-computed pattern table, one can drastically reduce the number of patterns to try, as most users are likely to use 4-6 step patterns (about 35,000 total combinations) .

Because Androd M's password hashing scheme doesn't directly use the plaintext password when calculating the scrypt value, optimized password recovery tools such as hashcat or JtR cannot be used directly to evaluate bruteforce cost. It is however fairly easy to build our own tool in order to check how a simple PIN holds against a brute force attack, assuming both the device_policies.xml and gatekeeper.password.key files have been obtained. As can be seen below, a simple Python script that tries all PINs from 0000 to 9999 in order takes about 10 minutes, when run on the same hardware as our previous JtR example (a 6-digit PIN would take about 17 hours with the same program). Compare this to less than a second for bruteforcing a 6-digit PIN for Android 5.1 (and earlier), and it is pretty obvious that the new hashing scheme Android M introduces greatly improves password storage security, even for simple PINs. Of course, as we mentioned earlier, the gatekeeper daemon is part of Android's HAL, so vendors are free to employ even more (or less...) secure gatekeeper implementations.

$ time ./m-pass-hash.py gatekeeper.password.key 4
Trying 0000...
Trying 0001...
Trying 0002...

...
Trying 9997...
Trying 9998...
Trying 9999...
Found PIN: 9999

real 9m46.118s
user 9m6.804s
sys 0m39.107s

Framework API

Android M is still in preview, so framework APIs are hardly stable, but we'll show the gatekeeper's AIDL interface for completeness. In the current preview release it is called IGateKeeperService and look likes this:

interface android.service.gatekeeper.IGateKeeperService {

void clearSecureUserId(int uid);

byte[] enroll(int uid, byte[] currentPasswordHandle,
byte[] currentPassword, byte[] desiredPassword);

long getSecureUserId(int uid);

boolean verify(int uid, byte[] enrolledPasswordHandle, byte[] providedPassword);

byte[] verifyChallenge(int uid, long challenge,
byte[] enrolledPasswordHandle, byte[] providedPassword);
}

As you can see, the interface provides methods for generating/getting and clearing the secure user ID for a particular user, as well as the enroll(), verify() and verifyChallenge() methods whose parameters closely match the lower level HAL interface. To verify that there is a live service that implements this interface, we can try to call the getSecureUserId() method using the service command line utility like so:

$ service call android.service.gatekeeper.IGateKeeperService 4 i32 0
Result: Parcel(00000000 ee555c25 ea679e08 '....%\U...g.')

This returns a Binder Parcel with the primary user's (user ID 0) secure user ID, which matches the value stored in /data/misc/gatekeeper/0 shown below (stored in network byte order).

# od -tx1 /data/misc/gatekeeper/0
37777776644 25 5c 55 ee 08 9e 67 ea
37777776644

The actual storage of password hashes (handles) is carried out by the LockSettingsService (interface ILockSettings), as in previous versions. The service has been extended to support the new gatekeeper password handle format, as well as to migrate legacy hashes to the new format. It is easy to verify this by calling the checkPassword(String password, int userId) method which returns true if the password matches:

# service call lock_settings 11 s16 1234 i32 0
Result: Parcel(00000000 00000000 '........')
# service call lock_settings 11 s16 9999 i32 0
Result: Parcel(00000000 00000001 '........')

Summary

Android M introduces a new system service -- gatekeeper, which is responsible for converting plain text passwords to opaque binary blobs (called password handles) which can be safely stored on disk. The gatekeeper is part of Android's HAL, so it can be modified to take advantage of the device's native security features, such as secure storage or TEE, without modifying the core platform. The default implementation shipped with the current Android M preview release uses scrypt to hash unlock patterns, PINs or passwords, and provides much better protection against bruteforceing than the previously used single-round MD5 and SHA-1 hashes. 

Friday 12 June 2015

Decrypting Android M adopted storage

One of the new features Android M introduces is adoptable storage. This feature allows external storage devices such as SD cards or USB drives to be 'adopted' and used in the same manner as internal storage. What this means in practice is that both apps and their private data can be moved to the adopted storage device. In other words, this is another take on everyone's (except for widget authors...) favorite 2010 feature -- AppsOnSD. There are, of course, a few differences, the major one being that while AppsOnSD (just like app Android 4.1 app encryption) creates per-app encrypted containers, adoptable storage encrypts the whole device. This short post will look at how adoptable storage encryption is implemented, and show how to decrypt and use adopted drives on any Linux machine.

Adopting an USB drive

In order to enable adoptable storage for devices connected via USB you need to execute the following command in the Android shell (presumably, this is not needed if your device has an internal SD card slot; however there are no such devices that run Android M at present):

$ adb shell sm set-force-adoptable true

Now, if you connect a USB drive to the device's micro USB slot (you can also use an USB OTG cable), Android will give you an option to set it up as 'internal' storage, which requires reformatting and encryption. 'Portable' storage is formatted using VFAT, as before.


After the drive is formatted, it shows up under Device storage in the Storage screen of system Settings. You can now migrate media and application data to the newly added drive, but it appears that there is no option in the system UI that allows you to move applications (APKs).


Adopted devices are mounted via Linux's device-mapper under /mnt/expand/ as can be seen below, and can be directly accessed only by system apps.

$ mount
rootfs / rootfs ro,seclabel,relatime 0 0
...
/dev/block/dm-1 /mnt/expand/a16653c3-... ext4 rw,dirsync,seclabel,nosuid,nodev,noatime 0 0
/dev/block/dm-2 /mnt/expand/0fd7f1a0-... ext4 rw,dirsync,seclabel,nosuid,nodev,noatime 0 0

You can safely eject an adopted drive by tapping on it in the Storage screen, and the choosing Eject from the overflow menu. Android will show a persistent notification that prompts you to reinsert the device once it's removed. Alternatively, you also can 'forget' the drive, which removes it from the system, and should presumably delete the associated encryption key (which doesn't seem to be the case in the current preview build).

Inspecting the drive

Once you've ejected the drive, you can connect it to any Linux box in order to inspect it. Somewhat surprisingly, the drive will be automatically mounted on most modern Linux distributions, which suggests that there is at least one readable partition. If you look at the partition table with fdisk or a similar tool, you may see something like this:

# fdisk /dev/sdb
Disk /dev/sdb: 7811 MB, 7811891200 bytes, 15257600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: gpt


# Start End Size Type Name
1 2048 34815 16M unknown android_meta
2 34816 15257566 7.3G unknown android_expand

As you can see, there is a tiny android_meta partition, but the bulk of the device has been assigned to the android_expand partition. Unfortunately, the full source code of Android M is not available, so we cannot be sure how exactly this partition table is created, or what the contents of each partition is. However, we know that most of Android's storage management functionality is implemented in the vold system daemon, so we check if there is any mention of android_expand inside vold with the following command:

$ strings vold|grep -i expand
--change-name=0:android_expand
%s/expand_%s.key
/mnt/expand/%s

Here expand_%s.key suspiciously looks like a key filename template, and we already know that adopted drives are encrypted, so our next step is to look for any similar files in the device's /data partition (you'll need a custom recovery or root access for this). Unsurprisingly, there is a matching file in /data/misc/vold which looks like this:

# ls /data/misc/vold
bench
expand_8838e738a18746b6e435bb0d04c15ccd.key

# ls -l expand_8838e738a18746b6e435bb0d04c15ccd.key

-rw------- 1 root root 16 expand_8838e738a18746b6e435bb0d04c15ccd.key


# od -t x1 expand_8838e738a18746b6e435bb0d04c15ccd.key
0000000 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
0000020

Decrypting the drive

That's exactly 16 bytes, enough for a 128-bit key. As we know, Android's FDE implementation uses an AES 128-bit key, so it's a good bet that adoptable storage uses a similar (or identical) implementation. Looking at the start and end of our android_expand partition doesn't reveal any readable info, nor is it similar to Android's crypto footer, or LUKS's header. Therefore, we need to guess the encryption mode and/or any related parameters. Looking once again at Android's FDE implementation (which is based on the dm-crypt target of Linux's device-mapper), we see that the encryption mode used is aes-cbc-essiv:sha256. After consulting dm-crypt's mapping table reference, we see that the remaining parameters we need are the IV offset and the starting offset of encrypted data. Since the IV offset is usually zero, and most probably the entire android_expand partition (offset 0) is encrypted, the command we need to map the encrypted partition becomes the following:

# dmsetup create crypt1 --table "0 `blockdev --getsize /dev/sdb2` crypt \
aes-cbc-essiv:sha256 00010203040506070809010a0b0c0d0e0f 0 /dev/sdb2 0"

It completes with error, so we can now try to mount the mapped device, again guessing that the file system is most probably ext4 (or you can inspect the mapped device and find the superblock first, if you want to be extra diligent).

# mount -t ext4 /dev/mapper/crypt1 /mnt/1/
# cd /mnt/1
# find ./ -type d
./
./lost+found
./app
./user
./media
./local
./local/tmp
./misc
./misc/vold
./misc/vold/bench

This reveals a very familiar Android /data layout, and you should see any media and app data you've moved to the adopted device. If you copy any files to the mounted device, they should be visible when you mount the drive again in Android.

Storage manager commands

Back in Android, you can use the sm command (probably short for 'storage manager') we showed in the first section to list disks and volumes as shown below:

$ sm list-disks
disk:8,16
disk:8,0

$ sm list-volumes
emulated:8,2 unmounted null
private mounted null
private:8,18 mounted 0fd7f1a0-2d27-48f9-8702-a484cb894a92
emulated:8,18 mounted null
emulated unmounted null
private:8,2 mounted a16653c3-6f5e-455c-bb03-70c8a74b109e

If you have root access, you can also partition, format, mount, unmount and forget disks/volumes. The full list of supported commands is shown in the following listing.

$ sm
usage: sm list-disks
sm list-volumes [public|private|emulated|all]
sm has-adoptable
sm get-primary-storage-uuid
sm set-force-adoptable [true|false]

sm partition DISK [public|private|mixed] [ratio]
sm mount VOLUME
sm unmount VOLUME
sm format VOLUME

sm forget [UUID|all]

Most features are also available from the system UI, but sm allows you to customize the ratio of the android_meta and android_expand partitions, as well as to create 'mixed' volumes.

Summary

Android M allows for adoptable storage, which is implemented similarly to internal storage FDE -- using dm-crypt with a per-volume, static 128-bit AES key, stored in /data/misc/vold/. Once the key is extracted from the device, adopted storage can be mounted and read/written on any Linux machine. Adoptable storage encryption is done purely in software (at least in the current preview build), so its performance is likely comparable to encrypted internal storage on devices that don't support hardware-accelerated FDE.

Friday 5 June 2015

Keystore redesign in Android M

Android M has been announced, and the first preview builds and documentation are now available. The most visible security-related change is, of course, runtime permissions, which impacts almost all applications, and may require significant app redesign in some cases. Permissions are getting more than enough coverage, so this post will look into a less obvious, but still quite significant security change in Android M -- the redesigned keystore (credential storage) and related APIs. (The Android keystore has been somewhat of a recurring topic on this blog, so you might want to check older posts for some perspective.)

New keystore APIs

Android M officially introduces several new keystore features into the framework API, but the underlying work to support them has been going on for quite a while in the AOSP master branch. The most visible new feature is support for generating and using symmetric keys that are protected by the system keystore. Storing symmetric keys has been possible in previous versions too, but required using private (hidden) keystore APIs, and was thus not guaranteed to be portable across versions. Android M introduces a keystore-backed symmetric KeyGenerator, and adds support for the KeyStore.SecretKeyEntry JCA class, which allows storing and retrieving symmetric keys via the standard java.security.KeyStore JCA API. To support this, Android-specific key parameter classes and associated builders have been added to the Android SDK.

Here's how generating and retrieving a 256-bit AES key looks when using the new M APIs:

// key generation
KeyGenParameterSpec.Builder builder = new KeyGenParameterSpec.Builder("key1",
KeyProperties.PURPOSE_ENCRYPT | KeyProperties.PURPOSE_DECRYPT);
KeyGenParameterSpec keySpec = builder
.setKeySize(256)
.setBlockModes("CBC")
.setEncryptionPaddings("PKCS7Padding")
.setRandomizedEncryptionRequired(true)
.setUserAuthenticationRequired(true)
.setUserAuthenticationValidityDurationSeconds(5 * 60)
.build();
KeyGenerator kg = KeyGenerator.getInstance("AES", "AndroidKeyStore");
kg.init(keySpec);
SecretKey key = kg.generateKey();

// key retrieval
KeyStore ks = KeyStore.getInstance("AndroidKeyStore");
ks.load(null);

KeyStore.SecretKeyEntry entry = (KeyStore.SecretKeyEntry)ks.getEntry("key1", null);
key = entry.getSecretKey();

This is pretty standard JCA code, and is in fact very similar to how asymmetric keys (RSA and ECDSA) are handled in previous Android versions. What is new here is, that there are a lot more parameters you can set when generating (or importing a key). Along with basic properties such as key size and alias, you can now specify the supported key usage (encryption/decryption or signing/verification), block mode, padding, etc. Those properties are stored along with the key, and the system will disallow key usage which doesn't match the key's attributes. This allows insecure key usage patterns (ECB mode, or constant IV for CBC mode, for example) to be explicitly forbidden, as well as constraining certain keys to a particular purpose, which is important in a multi-key cryptosystem or protocol. Key validity period (separate for encryption/signing and decryption/verification) can also be specified.

Another major new feature is requiring use authentication before allowing a particular key to be used, and specifying the authentication validity period. Thus, a key that protects sensitive data, can require user authentication on each use (e.g., decryption), while a different key may require only a single authentication per session (say, every 10 minutes).


The newly introduced key properties are available for both symmetric and asymmetric keys. An interesting detail is that apparently trying to use a key is now the official way (Cf. the Confirm Credential sample and related video) to check whether a user has authenticated within a given time period. This quite a roundabout way to verify user presence, especially if you app doesn't make use of cryptography in the first place. The newly introduced FingerprintManager authentication APIs also make use of cryptographic objects, so this may be part of a larger picture, which we have yet to see.

Keystore and keymaster implementation

On a high level, key generation and storage work the same as in previous versions: the system keystore daemon provides an AIDL interface, which framework classes and system services use to generate and manage keys. The keystore AIDL has gained some new, more generic methods, as well support for a 'fallback' implementation but is mostly unchanged.

The keymaster HAL and its reference implementation have, however, been completely redesigned. The 'old' keymaster HAL is retained for backward compatibility as version 0.3, while the Android M version has been bumped to 1.0, and offers a completely different interface. The new interface allows for setting fine-grained key properties (also called 'key characteristics' internally), and supports breaking up cryptographic operations that manipulate data of unknown or large size into multiple steps using the familiar begin/update/finish pattern. Key properties are stored as a series of tags along with the key, and form an authorization set when combined. AOSP includes a pure software reference keymaster implementation which implements cryptographic operations using OpenSSL and encrypts key blobs using a provided master key. Let's take a more detailed look at how the software implementations handles key blobs.

Key blobs

Keymaster v1.0 key blobs are wrapped inside keystore blobs, which are in turn stored as files in /data/misc/keystore/user_X, as before (where X is the Android user ID, starting with 0 for the primary user). Keymaster blobs are variable size and employ a tag-length-value (TLV) format internally. They include a version byte, a nonce, encrypted key material, a tag for authenticating the encrypted key, as well as two authorization sets (enforced and unenforced), which contain the key's properties. Key material is encrypted using AES in OCB mode, which automatically authenticates the cipher text and produces an authentication tag upon completion. Each key blob is encrypted with a dedicated key encryption key (KEK), which is derived by hashing a binary tag representing the key's root of trust (hardware or software), concatenated with they key's authorization sets. Finally, the resulting hash value is encrypted with the master key to derive the blob's KEK. The current software implementation deliberately uses a 128-bit AES zero key, and employs a constant, all-zero nonce for all keys. It is expected that the final implementation will either use a hardware-backed master-key, or be completely TEE-based, and thus not directly accessible from Android.

The current format is quite easy to decrypt, and while this will likely change in the final M version, in the mean time you can decrypt keymaster v1.0 blobs using the keystore-decryptor tool. The program also supports key blobs generated by previous Android versions, and will try to parse (but not decrypt) encrypted RSA blobs on Qualcomm devices. Note that the tool may not work on devices that use custom key blob formats or otherwise customize the keystore implementation. keystore-decryptor takes as input the keystore's .masterkey file, the key blob to decrypt, and a PIN/password, which is the same as the device's lockscreen credential. Here's a sample invocation:

$ java -jar ksdecryptor-all.jar .masterkey 10092_USRPKEY_ec_key4 1234
master key: d6c70396df7bfdd8b47913485dc0a885

EC key:
s: 22c18a15163ad13f3bbeace52c361150 (254)
params: 1.2.840.10045.3.1.7
key size: 256
key algorithm: EC
authorizations:

Hidden tags:
tag=900002C0 TAG_KM_BYTES bytes: 5357 (2)

Enforced tags:

Unenforced tags:
tag=20000001 TAG_KM_ENUM_REP 00000003
tag=60000191 TAG_KM_DATE 000002DDFEB9EAF0: Sun Nov 24 11:10:25 JST 2069
tag=10000002 TAG_KM_ENUM 00000003
tag=30000003 TAG_KM_INT 00000100
tag=700001F4 TAG_KM_BOOL 1
tag=20000005 TAG_KM_ENUM_REP 00000000
tag=20000006 TAG_KM_ENUM_REP 00000001
tag=700001F7 TAG_KM_BOOL 1
tag=600002BD TAG_KM_DATE FFFFFFFFBD84BAF0: Fri Dec 19 11:10:25 JST 1969
tag=100002BE TAG_KM_ENUM 00000000

Per-key authorization

As discussed in the 'New keystore APIs' section, the setUserAuthenticationRequired() method of the key parameter builder allows you to require that the user authenticates before they are authorized to use a certain key (not unlike iOS's Keychain). While this is not a new concept (system-wide credentials in Android 4.x require access to be granted per-key), the interesting part is how it is implemented in Android M. The system keystore service now holds an authentication token table, and a key operation is only authorized if the table contains a matching token. Tokens include an HMAC and thus can provide a strong guarantee that a user has actually authenticated at a given time, using a particular authentication method.

Authentication tokens are now part of Android's HAL, and currently support two authentication methods: password and fingerprint. Here's how tokens are  defined:

typedef enum {
HW_AUTH_NONE = 0,
HW_AUTH_PASSWORD = 1 << 0,
HW_AUTH_FINGERPRINT = 1 << 1,
HW_AUTH_ANY = UINT32_MAX,
} hw_authenticator_type_t;

typedef struct __attribute__((__packed__)) {
uint8_t version; // Current version is 0
uint64_t challenge;
uint64_t user_id; // secure user ID, not Android user ID
uint64_t authenticator_id; // secure authenticator ID
uint32_t authenticator_type; // hw_authenticator_type_t, in network order
uint64_t timestamp; // in network order
uint8_t hmac[32];
} hw_auth_token_t;

Tokens are generated by a newly introduced system component, called the gatekeeper. The gatekeeper issues a token after it verifies the user-entered password against a previously enrolled one. Unfortunately, the current AOSP master branch does not include the actual code that creates these tokens, but there is a base class which shows how a typical gatekeeper might be implemented: it computes an HMAC over the all fields of the hw_auth_token_t structure up to hmac using a dedicated key, and stores it in the hmac field. The serialized hw_auth_token_t structure then serves as an authentication token, and can be passed to other components that need to verify if the user is authenticated. Management of the token generation key is implementation-dependent, but it is expected that it is securely stored, and inaccessible to other system applications. In the final gatekeeper implementation the HMAC key will likely be backed by hardware, and the gatekeeper module could execute entirely within the TEE, and thus be inaccessible to Android. The low-level gatekeeper interface is part of Android M's HAL and is defined in hardware/gatekeeper.h.

As can be expected, the current Android M builds do indeed include a gatekeeper binary, which is declared as follows in init.rc:

...
service gatekeeperd /system/bin/gatekeeperd /data/misc/gatekeeper
class main
user system
...

While the framework code that makes use of the gatekeepr daemon is not yet available, it is expected that the Android M keyguard (lockscreen) implementation interacts with the gatekeeper in order to obtain a token upon user authentication, and passes it to the system's keystore service via its addAuthToken() method. The fingerprint authentication module (possibly an alternative keyguard implementation) likely works in the same way, but compares fingerprint scans against a previously enrolled fingerprint template instead of passwords.

Summary

Android M includes a redesigned keystore implementation which allows for fine-grained key usage control, and supports per-key authorization. The new keystore supports both symmetric and asymmetric keys, which are stored on disk as key blobs. Key blobs include encrypted key material, as well as a set of key tags, forming an authorization set. Key material is encrypted with a per-blob KEK, derived from the key's properties and a common master key. The final keystore implementation is expected to use a hardware-backed master key, and run entirely within the confines of the TEE. 

Android M also includes a new system component, called the gatekeeper, which can issue signed tokens to attest that a particular user has authenticated at a particular time. The gatekeeper has been integrated with the current PIN, pattern or password-based lockscreen, and is expected to integrate with fingerprint-based authentication in the final Android M version on supported devices. 

Thursday 7 May 2015

Hardware-accelerated disk encryption in Android 5.1

In a previous post we looked at disk encryption enhancements introduced in Android 5.0. That article was written based on the Lollipop preview release, before the platform source code was available, and while the post got most of the details about hardware-backed key protection right (the official documentation has since been released), it appears that it was overly optimistic in expecting that high-end Lollipop devices will ship with hardware-accelerated disk encryption. Android 5.0 did come with disk encryption enabled by default (at least on Nexus devices), but FDE also brought some performance problems, and many Android enthusiasts rushed to disable it. While slower disk access generally doesn't affect perceived performance when using a particular app, longer load times can add up and result in slower switching between apps, as well as longer boot times. In order to improve performance without sacrificing device security Android 5.1 integrated support for hardware-accelerated disk encryption on devices that provide dedicated cryptographic hardware, such as the Nexus 6. Unfortunately, this feature ended up disabled in the current Android release, but hopefully will be turned back on in a future release.

This post will look into the implementation of hardware-backed disk encryption on the Nexus 6, show how it improves performance, and finally describe some of the problems of the current implementation.

Kernel crypto

As previously discussed, Android's FDE implementation is based on the dm-crypt device-mapper target. As such, it performs cryptographic operations via the interfaces provided by the Linux kernel crypto API. The kernel crypto API defines a standard, extensible interface to ciphers and other data transformations implemented in the kernel (or as loadable modules). The API supports symmetric ciphers, AEAD ciphers, message digests and random number generators, collectively referred to as 'transformations'. All transformations have a name and a priority, as well as additional properties that describe their block size, supported key sizes, and so on.  For example, a desktop Linux system you may support the following:

$ cat /proc/crypto
...
name : aes
driver : aes-generic
module : kernel
priority : 100
refcnt : 1
selftest : passed
type : cipher
blocksize : 16
min keysize : 16
max keysize : 32
...
name : aes
driver : aes-aesni
module : kernel
priority : 300
refcnt : 1
selftest : passed
type : cipher
blocksize : 16
min keysize : 16
max keysize : 32

name : aes
driver : aes-asm
module : kernel
priority : 200
refcnt : 1
selftest : passed
type : cipher
blocksize : 16
min keysize : 16
max keysize : 32
...

Here we see three different implementations of the aes transformation, all built into the kernel, but with different priorities. When creating an instance of a particular transformation clients of the crypto API only specify its name and the kernel automatically returns the one with the highest priority. In this particular example, the aes-aesni implementation (which takes advantage of the AES-NI instruction set available on recent x86 CPUs) will be returned. New implementations can be added using the crypto_register_alg() and crypto_register_algs() functions.

The API provides single-block ciphers and hashes, which can be combined in order to provide higher-level cryptographic constructs via 'templates'. For example, AES in CBC mode is specified with the cbc(aes) template. Templates can be nested in order to request composite transformations that include more than one cryptographic primitive.

The API defines synchronous and asynchronous versions of cryptographic operations. Asynchronous operations return immediately and deliver their result via a callback, while synchronous operations block until the result is available. The crypto API also provides a user space interface via a dedicated socket type, AF_ALG.

Accelerating dm-crypt

dm-crypt parses the cipher specification (aes-cbc-essiv:sha256 in stock Android) passed as part of its mapping table and instantiates the corresponding transforms via the kernel crypto API. Thus in order to have dm-crypt use hardware acceleration, one has to either register a hardware-backed AES implementation with a high priority (which may affect other kernel services), or use a unique AES transformation name and change the mapping table accordingly.

Pretty much all SoC's used in current Android devices come with some sort of AES-capable hardware, usually in order to implement efficient DRM. OMAP devices provide ecb(aes), cbc(aes), and ctr(aes) implementations (in omap-aes.c) backed by the OMAP Crypto Engine; Tegra devices provide ecb(aes), cbc(aes), and ofb(aes) (in tegra-aes.c) backed by NVIDIA's bitstream engine. ARMv8 devices offer an AES implementation which takes advantage of the the dedicated aese, aesd, and aesmc instructions of the CPU. If the hardware-backed AES transformations available on these devices have higher priority than the corresponding software implementations, dm-crypt will automatically use them and take advantage of any acceleration (offloading to dedicated hardware/co-processor) they provide.

Qualcomm crypto engine

Recent (and probably older, too) Qualcomm Snapdragon SoC include a dedicated cryptographic module which provides hardware acceleration for commonly used algorithms such as AES and SHA-256. While publicly released details are quite scarce, the Snapdragon 805 and 810 SoC's have been FIPS 140-2 certified and certification documents offer some insight into the implementation and supported features. 

The cryptographic hardware in the 805 is officially called the 'Crypto 5 Core' and provides hardware implementations of DES, 3DES and AES in various modes (ECB, CBC, etc.), authenticated encryption (AEAD), SHA-1 and SHA-256, HMAC, a hardware-seeded random number generator, as well as support for mobile communication algorithms like Kasumi and snow-3G. 

The services provided by the crypto core are integrated into the Linux kernel in the form of several drivers: qce50 (QTI crypto engine), qcrypto (kernel crypto API driver), and qcedev (for user-space applications). qcrypto and qcedev both depend on qce50, but provide different interfaces. The actual crypto hardware can be accessed either only from user space or kernel space at the same time, therefore the documentation recommends that only one of the interfaces be enabled. Here's the driver structure diagram from the kernel documentation:


The qcrypto driver registers the following transformations with the kernel crypto API:

$ grep -B1 -A2 qcrypto  /proc/crypto|grep -v kernel
name : rfc4309(ccm(aes))
driver : qcrypto-rfc4309-aes-ccm
priority : 300
--
name : ccm(aes)
driver : qcrypto-aes-ccm
priority : 300
--
name : hmac(sha256)
driver : qcrypto-hmac-sha256
priority : 300
--
name : hmac(sha1)
driver : qcrypto-hmac-sha1
priority : 300
--
name : authenc(hmac(sha1),cbc(des3_ede))
driver : qcrypto-aead-hmac-sha1-cbc-3des
priority : 300
--
name : authenc(hmac(sha1),cbc(des))
driver : qcrypto-aead-hmac-sha1-cbc-des
priority : 300
--
name : authenc(hmac(sha1),cbc(aes))
driver : qcrypto-aead-hmac-sha1-cbc-aes
priority : 300
--
name : qcom-sha256
driver : qcrypto-sha256
priority : 300
--
name : qcom-sha1
driver : qcrypto-sha1
priority : 300
--
name : qcom-xts(aes)
driver : qcrypto-xts-aes
priority : 300
--
name : qcom-cbc(des3_ede)
driver : qcrypto-cbc-3des
priority : 300
--
name : qcom-ecb(des3_ede)
driver : qcrypto-ecb-3des
priority : 300
--
name : qcom-cbc(des)
driver : qcrypto-cbc-des
priority : 300
--
name : qcom-ecb(des)
driver : qcrypto-ecb-des
priority : 300
--
name : qcom-ctr(aes)
driver : qcrypto-ctr-aes
priority : 300
--
name : qcom-cbc(aes)
driver : qcrypto-cbc-aes
priority : 300
--
name : qcom-ecb(aes)
driver : qcrypto-ecb-aes
priority : 300

As you can see some of them are registered with a generic transformation name (e.g., hmac(sha1)), while some have the qcom- prefix. Whether to use the generic or driver-specific name is controlled by the device tree configuration. The interesting algorithm in the list above is qcom-xts(aes). Unlike CBC and CTR, the XTS cipher mode is not a generic chaining mode, but has been specifically developed for the purposes of block-based disk encryption. XTS works on wide blocks which map nicely to disk sectors (or blocks) and efficiently generates a 'tweak' key, different for each encrypted block by using the sector number and offset into the sector as variable inputs. Compared to AES-CBC-ESSIV XTS is more complex to implement, but less malleable (even though it is not an authenticated cipher), and is thus preferable.

The Linux kernel crypto API does support XTS, so technically dm-crypt could take advantage of the hardware-backed AES-XTS implementation in the Qualcomm CE without modifications. However, dm-crypt is designed to operate on 512-byte sectors and if used as is with the Qualcomm CE would result in many small requests to the cryptographic hardware, which is rather inefficient. Instead of trying to modify or tune dm-crypt, Qualcomm added a new device-mapper target for use with its SoC's: dm-req-crypt.

Introducing dm-req-crypt

dm-req-crypt works with encryption requests of up to 512KB and sends asynchronous encryption/decryption requests to the Snapdragon cryptographic module via the kernel crypto API interface, implemented by the qcrypto driver. Without going into the intricacies of kernel programming, here are the most important calls it uses to encrypt disk blocks:

...
tfm = crypto_alloc_ablkcipher("qcom-xts(aes)", 0, 0);
req = ablkcipher_request_alloc(tfm, GFP_KERNEL);
ablkcipher_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG,
req_crypt_cipher_complete, &result);
crypto_ablkcipher_clear_flags(tfm, ~0);
crypto_ablkcipher_setkey(tfm, NULL, KEY_SIZE_XTS);

memset(IV, 0, AES_XTS_IV_LEN);
memcpy(IV, &clone->__sector, sizeof(sector_t));

ablkcipher_request_set_crypt(req, req_sg_in, req_sg_out,
total_bytes_in_req, (void *) IV);

rc = crypto_ablkcipher_encrypt(req);
...

This code first requests an asynchronous implementation of the qcom-xts(aes) transform, sets the encryption key, then allocates and sets up a request structure, and finally starts the encryption operation by calling the crypto_ablkcipher_encrypt() function.  The important bit here is that the input and output buffers (scatterlists) req_sg_in and req_sg_out can hold up to 1024 sectors, whereas the dm-crypt always encrypts a single sector at a time. Another important detail is that encryption key passed to the AES-XTS transformation object via crypto_ablkcipher_setkey() is actually NULL. We'll address this later in our discussion of Android 5.1's FDE implementation.

Integrating dm-req-crypt

As with dm-crypt, disk encryption and mounting is handled by the cryptfs module of the vold daemon. Because most of the heavy lifting is done by the device mapper kernel module, changing vold to support dm-req-crypt is fairly straightforward. The type of disk encryption stored in the crypto footer structure is changed to aes-xts, and the device mapper target used to create a DM device is changed from crypt (which maps to the dm-crypt driver) to req-crypt. These changes are triggered at build time by setting the CONFIG_HW_DISK_ENCRYPTION macro.

The disk encryption key passed to the kernel (also called 'master key') is generated, encrypted and stored exactly in the same way as with dm-crypt (see the diagram at the end of this post for details). When an encrypted device is booted, the PIN or password entered by the user is run through scrypt, then signed with a hardware-bound RSA key, the result is run through scrypt again to derive the key encryption key (KEK) and IV, which are in turn used to decrypt the master key stored in the crypto footer. The master key is then passed to the device mapper driver as part of the mapping table via an ioctl() call. However, the dm-req-crypt implementation completely ignores the passed cipher string, encryption key and IV offset, and only uses the device path and start sector parameters. As we saw in the previous section, the key passed to the kernel crypto API is also NULL, so where does the actual disk encryption key come from?

Key management

The key management implementation is unfortunately proprietary and depends on the Qualcomm Secure Execution Environment (QSEE, also used to implemented the hardware-backed keystore). That said, the glue code that integrates it with vold, as well as the kernel driver are open source, so we can get a fairly good idea of how the system works. The disk encryption key is set, updated and cleared using the cryptfs_hw glue library. This library merely loads several functions from the proprietary libQSEEComAPI.so library using dlopen() and provides wrappers around them. For example, the disk encryption key is set by calling set_hw_device_encryption_key(), which in turn calls QSEECom_update_key_user_info() from the proprietary library. This function send commands to the secure OS via the qseecom kernel driver which is visible to user space as the /dev/qseecom device.

Generating a disk encryption key causes the qseecom driver to request loading a trusted app in the secure OS, and then sends the QSEOS_GENERATE_KEY command, which kicks off key generation. Generated keys appear to be stored on the ssd ('secure storage device'?) partition which points to /dev/block/mmcblk0p34 on the Nexus 6. After the key is generated, it is loaded into the hardware crypto engine using the QSEOS_SET_KEY command and can henceforth be used for encryption or decryption.

Using HW-accelerated FDE

As discussed in the 'Integrating dm-req-crypt' section, two things are needed to enable hardware-accelerated disk encryption: a vold binary with dm-req-crypt support and the libcryptfs_hw library. And, of course, all of the proprietary bits and pieces that make up the QSEE need to be in place. Thus it is easier to start with a stock 5.1 image, rather than build one from AOSP, because some of the required proprietary binaries seem to be missing from the officially released tarballs. Once everything is in place, encryption works exactly as before: if the fstab.shamu file includes the forceencrypt flag, the device will be encrypted on first boot, otherwise you need to kick off encryption from Settings->Security->Encrypt phone. One thing to note is that there is no way to transition a dm-crypt encrypted partition to dm-req-crypt, so if the device is already encrypted, you need to wipe the userdata partition first. After the encryption completes, the crypto footer (in the metadata partition on the N6) will look like this:

Android FDE crypto footer
-------------------------
Magic : 0xD0B5B1C4
Major Version : 1
Minor Version : 3
Footer Size : 2320 bytes
Flags : 0x00000020
Key Size : 128 bits
Failed Decrypts : 0
Crypto Type : aes-xts
Encrypted Key : CC43B0AE14BF27BAFE4709A102A96140
Salt : 1BB69D5DE1132F15D024E65370C29F33
KDF : scrypt+keymaster
N_factor : 15 (N=32768)
r_factor : 3 (r=8)
p_factor : 1 (p=2)
crypt type : PIN
FS size : 55615232
encrypted upto : 0
hash first block : 000000000000000000000000000000...
scrypted IK : 8B6DDC19F047331740B31B0F41E4EC5F...

The important bit here is the crypto type which is set to aes-xts. Because the actual disk encryption key is manged by the crypto engine, all other parameters (encrypted key, salt, etc.) are only used when verifying the user PIN or password. On boot, vold checks the value of the crypto type, and if set to aes-xts, loads the disk encryption key using the cryptfs_hw library, and then initializes the dm-req-crypt device mapper target. From there, the system simply mounts the created dm-0 device as /data, and all reads and writes are decrypted/encrypted transparently.

Performance

As can be expected, hardware-backed disk encryption performs better than software-based dm-crypt implementation. The screenshots below show the actual numbers, as measured by the AndEBenchPro application ('low-tech' dd read/write results are similar).

No FDESoftware FDEHardware FDE

As you can see, while disk access when using hardware-backed disk encryption is still about 40% slower than on an unencrypted device, random and sequential reads are almost two times faster compared to the software implementation (when reading 256KB blocks of data: 46.3MB/s vs. 25.1MB/s). So why isn't hardware-backed FDE enabled on current Nexus 6 builds?

Stability problems

Unfortunately, while the current implementation performs pretty well, there are still some problems, especially when the device is in sleep mode. If the device is in sleep mode for a relatively long period of time, read errors can occur, and the userdata partition may be mounted as read only (which wreaks havoc with the system's content providers); the device may even power off. While a reboot seems to fix the issue, if the the userdata was mounted read-only, the SQLite databases storing system configuration and accounts may get corrupted, which in some cases can only be fixed by a factory reset. Thus, hardware-accelerated disk encryption is unfortunately currently not quite suitable for daily use on the Nexus 6.

The OnePlus One (which has a Snapdragon 801 SoC), running CyanogenOS 12 also includes a dm-req-crypt-based FDE implementation which is enabled out of the box (disk encryption has to be triggered manually though). The FDE implementation one the OnePlus One seems to be quite stable, with comparable performance (50MB/s random read), so hopefully the problem on the Nexus 6 is a software one and can be resolved with a kernel update.

Summary

Disk encryption on Android can be accelerated by adding a kernel crypto API driver which takes advantage of the SoC's cryptographic hardware. This allows block encryption to be offloaded from the main CPU(s), and improves disk access times. Devices based on recent Qualcomm Snapdragon SoC's such as the Nexus 6 and the OnePlus One can take advantage of the SoC's crypto core module using the qcedev and qcrypto kernel drivers. A dedicated disk encryption device mapper target, dm-req-crypt, which batches encryption requests in order to increase throughput is also supported. Additionally, disk encryption keys are managed through a TEE secure app, and thus are not accessible by the Android OS, including the kernel. When using hardware-accelerated FDE disk access  is almost two times faster compared to the software-based dm-crypt implementation, but unfortunately there are some major stability problems on the Nexus 6. Hopefully those will be fixed in the next Android release, and hardware-accelerated disk encryption will be enabled out of the box.

Wednesday 24 December 2014

Dissecting Lollipop's Smart Lock

Android 5.0 (Lollipop) has been out for a while now, and most of its new features have been introduced, benchmarked, or complained about extensively. The new release also includes a number of of security enhancements, of which disk encryption has gotten probably the most media attention. Smart Lock (originally announced at Google I/O 2014), which allows bypassing the device lockscreen when certain environmental conditions are met, is probably the most user-visible new security feature. As such, it has also been discussed and blogged about extensively. However, because Smart Lock is a proprietary feature incorporated in Google Play Services, not many details about its implementation or security level are available. This post will look into the Android framework extensions that Smart Lock is build upon, show how to use them to create your own unlock method, and finally briefly discuss its Play Services implementation.

Trust agents

Smart Lock is build upon a new Lollipop feature called trust agents. To quote from the framework documentation, a trust agent is a 'service that notifies the system about whether it believes the environment of the device to be trusted.'  The exact meaning of 'trusted' is up to the trust agent to define. When a trust agent believes it can trust the current environment, it notifies the system via a callback, and the system decides how to relax the security configuration of the device.  In the current Android incarnation, being in a trusted environment grants the user the ability to bypass the lockscreen.

Trust is granted per user, so each user's trust agents can be configured differently. Additionally, trust can be granted for a certain period of time, and the system automatically reverts to an untrusted state when that period expires. Device administrators can set the maximum trust period trust agents are allowed to set, or disable trust agents altogether. 

Trust agent API

Trust agents are Android services which extend the TrustAgentService base class (not available in the public SDK). The base class provides methods for enabling the trust agent (setManagingTrust()), granting and revoking trust (grant/revokeTrust()), as well as a number of callback methods, as shown below.

public class TrustAgentService extends Service {

public void onUnlockAttempt(boolean successful) {
}

public void onTrustTimeout() {
}

private void onError(String msg) {
Slog.v(TAG, "Remote exception while " + msg);
}

public boolean onSetTrustAgentFeaturesEnabled(Bundle options) {
return false;
}

public final void grantTrust(
final CharSequence message,
final long durationMs, final boolean initiatedByUser) {
//...
}

public final void revokeTrust() {
//...
}

public final void setManagingTrust(boolean managingTrust) {
//...
}

@Override
public final IBinder onBind(Intent intent) {
return new TrustAgentServiceWrapper();
}


//...
}

To be picked up by the system, a trust agent needs to be declared in AndroidManifest.xml with an intent filter for the android.service.trust.TrustAgentService action and require the BIND_TRUST_AGENT permission, as shown below. This ensures that only the system can bind to the trust agent, as the BIND_TRUST_AGENT permission requires the platform signature. A Binder API, which allows calling the agent from other processes, is provided by the TrustAgentService base class. 

<manifest ... >

<uses-permission android:name="android.permission.CONTROL_KEYGUARD" />
<uses-permission android:name="android.permission.PROVIDE_TRUST_AGENT" />

<application ...>
<service android:exported="true"
android:label="@string/app_name"
android:name=".GhettoTrustAgent"
android:permission="android.permission.BIND_TRUST_AGENT">
<intent-filter>
<action android:name="android.service.trust.TrustAgentService"/>
<category android:name="android.intent.category.DEFAULT"/>
</intent-filter>

<meta-data android:name="android.service.trust.trustagent"
android:resource="@xml/ghetto_trust_agent"/>
</service>
...
</application>
</manifest>

The system Settings app scans app packages that match the intent filter shown above, checks if they hold the PROVIDE_TRUST_AGENT signature permission (defined in the android package) and shows them in the Trust agents screen (Settings->Security->Trust agents) if all required conditions are met. Currently only a single trust agent is supported, so only the first matched package is shown. Additionally, if the manifest declaration contains a <meta-data> tag that points to an XML resource that defines a settings activity (see below for an example), a menu entry that opens the settings activity is injected into the Security settings screen.

<trust-agent xmlns:android="http://schemas.android.com/apk/res/android"
android:title="Ghetto Unlock"
android:summary="A bunch of unlock triggers"
android:settingsActivity=".GhettoTrustAgentSettings" />


Here's how the Trusted agents screen might look like when a system app that declares a trusted agent is installed.

Trust agents are inactive by default (unless part of the system image), and are activated when the user toggles the switch in the screen above. Active agents are ultimately managed by the system TrustManagerService which also keeps a log of trust-related events. You can get the current trust state and dump the even log using the dumpsys command as shown below.

$ adb shell dumpsys trust
Trust manager state:
User "Owner" (id=0, flags=0x13) (current): trusted=0, trustManaged=1
Enabled agents:
org.nick.ghettounlock/.GhettoTrustAgent
bound=1, connected=1, managingTrust=1, trusted=0
Events:
#0 12-24 10:42:01.915 TrustTimeout: agent=GhettoTrustAgent
#1 12-24 10:42:01.915 TrustTimeout: agent=GhettoTrustAgent
#2 12-24 10:42:01.915 TrustTimeout: agent=GhettoTrustAgent
...

Granting trust

Once a trust agent is installed, a trust grant can be triggered by any observable environment event, or directly by the user (for example, by via an authentication challenge). An often requested, but not particularly secure (unless using a WPA2 profile that authenticates WiFi access points), unlock trigger is connecting to a 'home' WiFi AP. This feature can be easily implemented using a broadcast receiver that reacts to android.net.wifi.STATE_CHANGE (see sample app; based on the sample in AOSP). Once a 'trusted' SSID is detected, the receiver only needs to call the grantTrust() method of the trust agent service. This can be achieved in a number of ways, but if both the service and the receiver are in the same package, a straightforward way is to use a LocalBroadcastManager (part of the support library) to send a local broadcast, as shown below.

static void sendGrantTrust(Context context,
String message,
long durationMs,
boolean initiatedByUser) {
Intent intent = new Intent(ACTION_GRANT_TRUST);
intent.putExtra(EXTRA_MESSAGE, message);
intent.putExtra(EXTRA_DURATION, durationMs);
intent.putExtra(EXTRA_INITIATED_BY_USER, initiatedByUser);
LocalBroadcastManager.getInstance(context).sendBroadcast(intent);
}


// in the receiver
@Override
public void onReceive(Context context, Intent intent) {
if (WifiManager.NETWORK_STATE_CHANGED_ACTION.equals(intent.getAction())) {
WifiInfo wifiInfo = (WifiInfo) intent
.getParcelableExtra(WifiManager.EXTRA_WIFI_INFO);

// ...
if (secureSsid.equals(wifiInfo.getSSID())) {
GhettoTrustAgent.sendGrantTrust(context, "GhettoTrustAgent::WiFi",
TRUST_DURATION_5MINS, false);
}
}
}


This will call the TrustAgentServiceCallback installed by the system lockscreen and effectively set a per-user trusted flag. If the flag is true, the lockscreen implementation allows the keyguard to be dismissed without authentication. Once the trust timeout expires, the user must enter their pattern, PIN or password in order to dismiss the keyguard. The current trust state is displayed at the bottom of the keyguard as a padlock icon: when unlocked, the current environment is trusted; when locked, explicit authentication is required. The user can also manually lock the device by pressing the padlock, even if an active trust agent currently has trust.

NFC unlock

As discussed in a previous post, implementing NFC unlock in previous Android versions was possible, but required some modifications to the system NFCService, because the NFC controller was not polled while the lockscreen is displayed. In order to make implementing NFC unlock possible, Lollipop introduces several hooks into the NFCService, which allow NFC polling on the lockscreen. If a matching tag is discovered, a reference to a live Tag object is passed to interested parties. Let's look into the how this is implementation in a bit more detail.

The NFCAdapter class has a couple of new (hidden) methods that allow adding and removing an NFC unlock handler (addNfcUnlockHandler() and removeNfcUnlockHandler(), respectively). An NFC unlock handler is an implementation of the NfcUnlockHandler interface shown below.

interface NfcUnlockHandler {
public boolean onUnlockAttempted(Tag tag);
}

When registering an unlock handler you must specify not only the NfcUnlockHandler object, but also a list of NFC technologies that should be polled for at the lockscreen. Calling the addNfcUnlockHandler() method requires the WRITE_SECURE_SETTINGS signature permission.

Multiple unlock handlers can be registered and are tried in turn until one of them returns true from onUnlockAttempted(). This terminates the NFC unlock sequence, but doesn't actually dismiss the keyguard. In order to unlock the device, an NFC unlock handler should work with a trust agent in order to grant trust. Judging from NFCService's commit log, this appears to be a fairly recent development: initially, the Settings app included functionality to register trusted tags, which would automatically unlock the device (based on the tag's UID), but this functionality was removed in favour of trust agents.

Unlock handlers can authenticate the scanned NFC tag in a variety of ways, depending on the tag's technology. For passive tags that contain fixed data, authentication typically relies either on the tag's unique ID, or on some shared secret written to the tag. For active tags that can execute code, it can be anything from an OTP to full-blown multi-step mutual authentication. However, because NFC communication is not very fast, and most tags have limited processing power, a simple protocol with few roundtrips is preferable. A simple implementation that requires the tag to sign a random value with its RSA private key, and then verifies the signature using the corresponding public key is included in the sample application. For signature verification to work, the trust agent needs to be initialized with the tag's public key, which in this case is imported via the trust agent's settings activity shown below.

Smart Lock

'Smart Lock' is just the marketing name for the GoogleTrustAgent which is included in Google Play Services (com.google.android.gms package), as can be seen from the dumpsys output below.

$ adb shell dumpsys trust
Trust manager state:
User "Owner" (id=0, flags=0x13) (current): trusted=1, trustManaged=1
Enabled agents:
com.google.android.gms/.auth.trustagent.GoogleTrustAgent
bound=1, connected=1, managingTrust=1, trusted=1
message=""



This trust agent offers several trust triggers: trusted devices, trusted places and a trusted face. Trusted face is just a rebranding of the face unlock method found in previous versions. It uses the same proprietary image recognition technology, but is significantly more usable, because, when enabled, the keyguard continuously scans for a matching face instead of requiring you to stay still while it takes and process your picture. The security level provided also remains the same -- fairly low, as the trusted face setup screen warns. Trusted places is based on the geofencing technology, which has been available in Google Play services for a while. Trusted places use the 'Home' and 'Work' locations associated with your Google account to make setup easier, and also allows for registering a custom place based on the current location or any coordinates selectable via Google Maps. As a helpful popup warns, accuracy cannot be guaranteed, and the trusted place range can be up to 100 meters. In practice, the device can remain unlocked for a while even when this distance is exceeded.

Trusted devices supports two different types of devices at the time of this writing: Bluetooth and NFC. The Bluetooth option allows the Android device to remain unlocked while a paired Bluetooth device is in range. This features relies on Bluetooth's built-in security mechanism, and as such its security depends on the paired device. Newer devices, such as Android Wear watches or the Pebble watch, support Secure Simple Pairing (Security Mode 4), which uses Elliptic Curve Diffie-Hellman (ECDH) in order to generate a shared link key. During the paring process, these devices display a 6-digit number based on a hash of both devices' public keys in order to provide device authentication and protect against MiTM attacks (a feature called numeric comparison). However, older wearables (such as the Meta Watch), Bluetooth earphones, and others are also supported. These previous-generation devices only support Standard Pairing, which generates authentication keys based on the device's physical address and a 4-digit PIN, which is usually fixed and set to a well-know value such as '0000' or '1234'. Such devices can be easily impersonated.

Google's Smart Lock implementation requires a persistent connection to a trusted device, and trust is revoked once this connection is broken (Update: apparently a trusted connection can be established without a key on Android < 5.1 ). However, as the introductory screen (see below) warns, Bluetooth range is highly variable and may extend up to 100 meters. Thus while the 'keep device unlocked while connected to trusted watch on wrist' use case makes a lot of sense, in practice the Android device may remain unlocked even when the trusted Bluetooth device (wearable, etc.) is in another room.


As discussed earlier, an NFC trusted device can be quite flexible, and has the advantage that, unlike Bluetooth, proximity is well defined (typically not more than 10 centimeters). While Google's Smart Lock seems to support an active NFC device (internally referred to as the 'Precious tag'), no such device has been publicly announced yet. If the Precious is not found, Google's NFC-based trust agent falls back to UID-based authentication by saving the hash of the scanned tag's UID (tag registration screen shown below). For the popular NFC-A tags (most MIFARE variants) this UID is 4 or 7 bytes long (10-byte UIDs are also theoretically supported). While using the UID for authentication is a fairly wide-spread practice, it was originally intended for anti-collision alone, and not for authentication. 4-byte UIDs are not necessarily unique and may collide even on 'official' NXP tags. While the specification requires 7-byte IDs to be both unique (even across different manufacturers) and read-only, cards with a rewritable UID do exists, so cloning a MIFARE trusted tag is quite possible. Tags can also be emulated with a programmable device such as the Proxmark III. Therefore, the security level provided by UID-based authentication is not that high.

Summary

Android 5.0 (Lollipop) introduces a new trust framework based on trust agents, which can notify the system when the device is in a trusted environment. As the system lockscreen now listens for trust events, it can change its behaviour based on the trust state of the current user. This makes it easy to augment or replace the traditional pattern/PIN/password user authentication methods by installing  trust agents. Trust agent functionality is currently only available to system applications, and Lollipop can only support a single active trust agent. Google Play Services provides several trust triggers (trustlets) under the name 'Smart Lock' via its trust agent. While they can greatly improve device usability, none of the currently available Smart Lock methods are particularly precise or secure, so they should be used with care.

Thursday 23 October 2014

Android Security Internals is out

Some six months after the first early access chapters were announced, my book has now officially been released. While the final ebook PDF has been available for a few weeks, you can now get all ebook formats (PDF, Mobi and ePub) directly from the publisher, No Starch Press. Print books are also ready and should start shipping tomorrow (Oct 24th). You can use the code UNDERTHEHOOD when checking out for a 30% discount in the next few days. The book will also be available from O'ReillyAmazon and other retailers in the coming weeks.

This book would not have been possible without the efforts of Bill Pollock and Alison Law from No Starch, who edited, refined and produced my raw writings. +Kenny Root  reviewed all chapters and caught some embarrassing mistakes, all that are left are mine alone. Jorrit �Chainfire� Jongma reviewed my coverage of SuperSU and Jon �jcase� Sawyer contributed the foreword. Once again, a big thanks to everyone involved!

About the book

The book's purpose and structure have not changed considerably since it was first announced. It walks you through Android's security architecture, starting from the bottom up. It starts with fundamental concepts such as Binder, permissions and code signing, and goes on to describe more specific topics such as cryptographic providers, account management and device administration. The book includes excerpts from core native daemons and platform services, as well as some application-level code samples, so some familiarity with Linux and Android programming is assumed (but not absolutely required). 

Android versions covered

The book covers Android 4.4, based on the source code publicly released through AOSP. Android's master branch is also referenced a few times, because master changes are usually a good indicator of the direction future releases will take. Vendor modifications or extensions to Android, as well as  device-specific features are not discussed.

The first developer preview of Android 5.0 (Lollipop, then known only as 'Android L') was announced shortly after the first draft of this book was finished. This first preview L release included some new security features, such as improvements to full-disk encryption and device administration, but not all planned features were available (for example, Smart Lock was missing). The final Lollipop developer preview (released last week) added those missing features and finalized the public API. The source code for Lollipop is however not yet available, and trying to write an 'internals' book without it would either result in incomplete or speculative coverage, or would turn into an (rather though) exercise in reverse engineering. That is why I've chosen not to cover Android 5.0 in the book at all and it is exclusively focused on Android 4.4 (KitKat).

Lollipop is a major release, and as such would require reworking most of the chapters and, of course, adding a lot of new content. This could happen in an updated version of the book at some point. Not to worry though, some of the more interesting new security features will probably get covered right here, on the blog,  first.

With that out of the way, here is the extended table of contents. You can find the full table of contents on the book's official page.

Update: Chapter 1 is now also freely available on No Starch's site.

Table of contents

 Chapter 1: Android�s Security Model
  • Android�s Architecture
  • Android�s Security Model
Chapter 2: Permissions
  • The Nature of Permissions
  • Requesting Permissions
  • Permission Management
  • Permission Protection Levels
  • Permission Assignment
  • Permission Enforcement
  • System Permissions
  • Shared User ID
  • Custom Permissions
  • Public and Private Components
  • Activity and Service Permissions
  • Broadcast Permissions
  • Content Provider Permissions
  • Pending Intents
Chapter 3: Package Management
  • Android Application Package Format
  • Code signing
  • APK Install Process
  • Package Verification
Chapter 4: User Management
  • Multi-User Support Overview
  • Types of Users
  • User Management
  • User Metadata
  • Per-User Application Management
  • External Storage
  • Other Multi-User Features
Chapter 5: Cryptographic Providers
  • JCA Provider Architecture
  • JCA Engine Classes
  • Android JCA Providers
  • Using a Custom Provider
Chapter 6: Network Security and PKI
  • PKI and SSL Overview
  • JSSE Introduction
  • Android JSSE Implementation
Chapter 7: Credential Storage
  • VPN and Wi-Fi EAP Credentials
  • Credential Storage Implementation
  • Public APIs
Chapter 8: Online Account Management
  • Android Account Management Overview
  • Account Management Implementation
  • Google Accounts Support
Chapter 9: Enterprise Security
  • Device Administration
  • VPN Support
  • Wi-Fi EAP
Chapter 10: Device Security
  • Controlling OS Boot-Up and Installation
  • Verified Boot
  • Disk Encryption
  • Screen Security
  • Secure USB Debugging
  • Android Backup
Chapter 11: NFC and Secure Elements
  • NFC Overview
  • Android NFC Support
  • Secure Elements
  • Software Card Emulation
Chapter 12: SElinux
  • SELinux Introduction
  • Android Implementation
  • Android 4.4 SELinux Policy
Chapter 13: System Updates and Root Access
  • Bootloader
  • Recovery
  • Root Access
  • Root Access on Production Builds