It's been over a month since we announced the first beta for our new open source project, CMS Airship. We've been hard at work since then, both at a code level and at a protocol design level, and we're nearing the release of the second beta (in semantic versioning: 0.2.0). Before we continue the next steps, let's take a look at some of the cryptography features currently implemented. (Yes, we're using libsodium. No need to panic.)

Although some of what we cover will be complicated, most of these details will be abstracted away from end users. (Only engineers who want to get their feet wet in our protocol designs and implementations need to know this stuff.)

Let's dive in, shall we?

How CMS Airship Will Store Passwords

In our experience, most peoples' experience with cryptography begins and ends with storing their users' passwords, so that seems a natural starting point for our examination.

Most of the cryptography features in Airship are provided by our libsodium wrapper, Halite. Password storage is no exception here.

• CMS Airship's password storage protocol: Password hash then authenticated encryption
  • Argon2i with OPSLIMIT_INTERACTIVE and MEMLIMIT_INTERACTIVE
  • HKDF-BLAKE2b (with a 256-bit salt) for key splitting
  • Xsalsa20 for encryption
  • Keyed BLAKE2b hash for a MAC
  • ...in an Encrypt-then-MAC construction

Local cryptography keys (including the one used to encrypt passwords) are generated on-demand, from your operating system's CSPRNG, when you first access the CMS Airship installer. If you already uploaded any keys before you first run the installer, those specific keys will not be replaced. Otherwise, everyone's Airship gets its own set of encryption keys.

Security Benefits of this Approach

Let's assume you install a community extension in the future that contains a SQL injection vulnerability that gives an attacker access to your database. For most applications, this means pivoting to filesystem access, which is game over, so let's further assume the webserver and the database are on separate bare-metal servers.

In the hardware-separated scenario: The encryption keys are stored in the webserver's filesystem, and the password hashes are encrypted in the database. Without access to the 256-bit encryption key, an attacker still cannot begin to attempt to crack the password hashes.

Even if the attacker can obtain access to the encryption key, they still have to attack the Argon2i password hashes.

What Do Other Content Management Systems Do?

• WordPress uses a variant of PHPass, which will use bcrypt in a non-timing-safe manner and fallback to multi-pass MD5.
  • If you're running an unsupported PHP version doesn't support bcrypt, it tries DESCrypt.
  • If your ancient PHP setup doesn't support DESCrypt either, you get multi-pass MD5 instead.
  • Update: No, it turns out that WordPress only uses multi-pass MD5.
• Drupal uses a variant of PHPass, which opts for SHA512Crypt (morally equivalent to PBKDF2, which is acceptable but not ideal).
• Joomla correctly implements PHP's password hashing API.

Some other CMS platforms do as well as Joomla, many others do worse than WordPress. We published an in-depth investigation of the security features of other open source CMS platforms elsewhere. Let's move on.

Continuum - Digitally Signed Automatic Updates

Goal: When a security update is available for our platform, everyone should have it installed within an hour (unless the user elects to turn updates off or change the frequency).

By having a narrow window of patch-available to patch-installed, we can run our project without relying on lengthy bug embargoes to protect your systems. This in turn allows us multiple benefits:

• Hardly anyone will be caught running vulnerable systems 7 hours after a patch is available.
• We get to enjoy a culture of openness with other security researchers (without demanding any of them to take a vow of silence). Even full disclosure of a 0day vulnerability won't harm anyone if the patch is released before an attacker gets their hands on the exploit code.
• Everyone who develops extensions for CMS Airship can avoid the quagmire of legacy bug support.

Naturally, we decided it would be a good idea to also offer this capability to anyone who wants to make and share their own Airship extensions.

There's one complication, however: An automatic update mechanism by itself is not an adequate security mechanism: What happens to everyone's Airship if the update was compromised?

If you recall Defuse Security's Triangle of Secure Code Delivery, there are three properties that a code delivery mechanism must satisfy in order to be secure:

1. Reproducible builds.
2. Userbase consistency verification.
3. Cryptographic signatures.

Continuum is our automatic update system which takes care of updating CMS Airship itself, as well as any Cabins, Gadgets, or Motifs installed. Every hour (in the default configuration), Continuum will check the update server for any updates to Airship or any of the types of extensions listed above. If any updates are available, Continuum will:

1. Download the update file (typically a PHP Archive-- except Motifs which are ZIP Archives) and its digital signature.
2. Verify the signature using a valid signing key for the supplier of the extension.
3. If the signature matches, apply the update. If it doesn't, log a scary error message and abort.

Update files (and the digital signatures) can come from two places. Most developers will use barge to create, build, sign, and release their CMS Airship extensions. Paragon Initiative Enterprises will use hangar to build and sign updates from an airgapped computer. In both cases, there are two kinds of signing keys used:

• Master keys, whose job is to sign and revoke general-purpose signing keys and should be kept offline.
• Signing keys, which are suitable for everyday use.

All master and signing key-pairs are intended for use with the Ed25519 digital signature algorithm, and are derived (using Argon2i) from a password and 128-bit random salt. Master keys use libsodium's OPSLIMIT_SENSITIVE and MEMLIMIT_SENSITIVE constants; signing keys use OPSLIMIT_MODERATE and MEMLIMIT_MODERATE. (This will probably turn out to be overkill, but unlike password verification, there's no risk of DoS since it has to be triggered locally.)

Strictly speaking, the Ed25519 signature we use is calculated over a keyed 512-bit BLAKE2b hash of the file's contents (with the Ed25519 public key used as the BLAKE2b key) rather than a plain Ed25519 signature. The motivation for this decision was to allow users to sign a large file on a system with very low RAM without sacrificing security. Considering it will take $2^{256}$ hashed messages before the first hash collision occurs (at 50% probability), we consider this acceptable for feeding into a signature algorithm that targets the $2^{128}$ security level.

When you create a new supplier account, the first key you create must be a master key, and is the only key that will be accepted without being signed by an existing master key. You may have as many concurrent master and signing keys as you need.

In short, Continuum takes care of cryptographic signatures by ensuring every update is signed by one of the keys associated with the trusted supplier for that particular update. But how do we know which key to trust?

Keyggdrasil - The Public Key World Tree

If I had to summarize Keyggdrasil in one sentence, it would be:

Keyggdrasil makes sure everyone sees the same public keys, prevents anyone from rewriting history, and makes targeted attacks noisy.

At its core, Keyggdrasil is a very humble BLAKE2b-based Merkle tree of every key transaction since its inception. Any time a supplier creates or revokes a key, this is recorded in the channel's copy of Keyggdrasil and propagated throughout the network.

Your Airship will, quite frequently, check for key updates with the update server (dubbed the Channel). Both key creation and key revocation events are recorded (publicly) in this append-only data structure. A simplified overview of the process is:

1. Your Airship asks the Channel for key updates since $lastKnownMerkleRoot, with a challenge nonce to include in its response (which is Ed25519-signed and the public key for the Channel is pinned to the client).
   • If there are any, the Channel responds with any new key updates (and each update's respective Merkle root). Proceed to step two.
   • If there aren't, the Channel responds with a "no updates" message containing the current timestamp (to mitigate silent denial-of-updates and replay attacks should a challenge-nonce collision occur).
2. Your Airship will validate the signature of every key update with the Channel's public key (to make sure they're genuine).
3. Next, the Merkle root of all of the updates is recalculated, compared with the one received from the Channel. If they match, we move onto peer verification.
4. Your Airship will shuffle the peers you've explicitly trusted (either at install time or through the Bridge) and then begin querying each one until any of these conditions are met (where $N$ is the number of peers you trust):
   • If $\ln N$ random peers respond with the same Merkle root, it is accepted as valid.
   • If any random peer responds with a different Merkle root, then the peer verification fails.
   • If $e\ln N$ random peers fail to respond (due to network errors), then the peer verification fails.
5. If peer verification failed, pop off the last update and try step 4 again until we get a match or run out of updates. (Not updating is preferable to accepting malicious/false updates.) Otherwise, proceed to step 6.
6. Store the key updates locally, then iterate through the updates and do this for each:
   • If it's a new master key for a new supplier, simply store it. (TOFU)
   • If it's a new key for an existing supplier, verify that it was signed by one of the master keys on file for that supplier, then add it to that supplier.
   • if it's a key revocation for an existing supplier, verify that it was signed by one of the (other) master keys on file for that supplier, then remove it from that supplier.

Or, as an illustration:

Security Benefits of this Approach

There are a number of small infrastructure attacks that become much more difficult to pull off with the way Keyggdrasil is implemented.

• Compromising the Channel is, by itself, almost useless.
  • Airship and Extension updates must be signed by the appropriate Ed25519 key.
  • All key updates (except the first one) must be signed by that supplier's master key. You can't just generate a master key for a supplier.
  • Even if you steal one supplier's master key, you can't direct a silent, targeted attack to a specific Airship because they will verify the Merkle root with their peers before they accept the update.
  • The Channel is never reliably informed of which other Airships any particular Airship trusts as a peer notary.
  • In order to pull off any attack, you have to announce your presence.
• Compromising a subset of Airships, without the Channel, doesn't help an attacker move laterally.
  • You can't reliably know which other Airships your target trusts.
  • Some Airships will only update over Tor, thus adding severe misery to traffic/timing analysis efforts.
  • Channels can only attest for what the Channel provides, or block updates.
  • Even if an attacker controls a very small number of the peers you trust, and launches a DDoS attack against every other Airship in the universe, if too many peer verification queries fail due to network errors the update is aborted. Attackers lose again.

The only way to silently and reliably perform a targeted attack against any one Airship is to:

1. Have the master key for one of the suppliers they trust.
2. Compromise the Channel.
3. Identify and compromise every one of their peer notaries.

An attacker capable of pulling off all three would be just as successful attacking one directly. We like to think of this property of herd immunity: In order to compromise the target you want, you effectively have to attack everyone.

In sum, Keyggdrasil solves the userbase consistency verification problem by ensuring everyone sees the same public key updates.

What Do Other Content Management Systems Do?

As far as we're aware, the only popular CMS that supports automatic updates is WordPress, which does not perform any digital signature verification (and therefore has no need for a system to manage public keys). Thus, if you compromise the WordPress update server, you can compromise 25% of the websites on the Internet. WordPress core developer, Dion Hulse, was investigating solutions for this problem. So far, they haven't been implemented.

Click this link to learn more about the (in)security of WordPress, Drupal, and Joomla.

Reproducible Builds

Our solution for the third leg of the triangle of secure code delivery is to, when applicable, include the git commit hash in the metadata for each digitally signed deliverable (PHP Archive for most cases, ZIP archives for Motifs) and store it in Keyggdrasil. With our previous work (Pharaoh), it should be easy for anyone to rebuild and verify that there are no differences between what was downloaded and what was reproduced.

We aren't automating this process yet. Because of the userbase consistency verification property of Keyggdrasil, if anyone verifies that a package is reproducible from it source code, then everyone enjoys the security guarantees.

Tying it All Together

• Continuum ensures that our automatic updates are digitally signed so that the trust lies with the developer, not the update server.
• Keyggdrasil ensures that everyone sees the same public keys (so Continuum can do its job effectively) without a certificate authority system. (When it comes to your signing keys, you are your own CA.)
• With these two components in place, we can reliably allow developers to build and sign their own third-party Airship extensions (that they control, not us).
• Builds are easily reproduced from their source code, and the git commit hash is included in both Keyggdrasil and in the package metadata that the developer signs.

Consequently, CMS Airship is a solution to the problem of secure code delivery.

Anticipated Questions

Why don't you ensure the userbase consistency of package updates themselves, rather than just the public keys?

The main concern here is disk space usage. Public keys are small (thank you, elliptic curve cryptography), while every version of every Cabin, Gadget, and Motif ever released would quickly become prohibitively huge for most low-end servers to store. We can explore having a separate notary mechanism for Continuum in addition to Keyggdrasil, but right now we feel it's costs far outweighs its benefits.

UPDATE: We now also store metadata about each release (including a BLAKE2b checksum of the deliverable, version identifier, etc.) in Keyggdrasil. This includes core updates.

Why happens if I lose all my keys? Can't Paragon Initiative Enterprises recover my supplier account?

We can do so only by issuing a new CMS Airship core update with an update trigger to revoke all existing keys and create a new master key for a given supplier, bypassing Keyggdrasil in the process. This is the nuclear option and we will not do so without substantial public discussion on the circumstances before and afterwards. Should we fail to do hold a public discussion prior to any emergency key replacement updates: Assume the worst and act accordingly.

What happens if a practical quantum computer is developed?

As soon as we're certain of the benefits (and reliability of the implementations), we will migrate a future version of this protocol to use post-quantum cryptography.

Want to Get Involved?

Secure code delivery is a hard problem, and we intend to make it as easy and painless as possible so our users don't have to worry about it. If you'd like to help, all of our projects are open source on our Github Organization. Feel free to dive in and contribute however you see fit.

If you represent a company wants to use CMS Airship to solve business needs and would like to purchase a commercial license or support contract, please get in touch at your earliest convenience.

If you're facing security problems, big or small, consider hiring our team to perform a code audit or develop innovative and secure solutions.