You Shouldn’t Trust Your Cloud Vendors

Trusting cloud vendors increases your attack surface.
I've built a lot of stuff over the course of my career. Most of it has used one or more hosted services from any number of providers. Those services come with security claims that I can't verify. 
Current best practices don't really address this issue, either; if I set up a new service following those best practices, I only store hashed, salted, and peppered passwords (with Argon or bcrypt). I validate all user input and sanitize all outputs - including my log messages. I double-check all the permissions on my storage buckets. All of these practices are necessary, but not sufficient; they assume the security of the underlying service based on the (questionable) principle that the provider is a trusted entity.
Even trusted providers are fallible. We all make mistakes, the employees at all our vendors do too. Just scroll through https://krebsonsecurity.com. Security is a hard problem, and human error compounds with difficult engineering problems at every vendor to increase your attack surface. So the real question here is, "how do you protect yourself from your vendor's mistakes?"
For something like a managed queue or pub/sub, you can encrypt the messages you pass through it: easy. Same for a database, you would just make sure that everything that goes in is securely encrypted. But where are those encryption keys coming from? The natural solution would be to use your vendor's Key Management System (KMS), but then you'd have to secure that too.
Securing a KMS system is not an easy task - you need to solve two problems: (1) the only people who can get keys are people you explicitly authorize, and (2) the keys haven't been tampered with or otherwise delivered from a box-in-the-middle. If your inner AWS guru said "IAM" you're almost right. But the point here is to protect yourself from potential mistakes *at your vendor*.
Almost by definition, you can't do that with products you get *from your vendor*. It all starts with what is known as "the root of trust". For cryptography, the root of trust is typically where encryption keys are derived.
Your vendor's KMS system has a root of trust that is based on servers under their control. You don't have access to these servers because that would only make them less secure. These servers hold onto master keys that can be used in two ways:
1. You send your plaintext data to your vendor (who promises not to peek), and their service will perform the cryptography for you using a master key and send it back.
2. Or, your client can use envelope encryption, which means that your client generates a data key that is used for cryptography. Then that key is encrypted by sending it to your vendor (who promises they won't store that key anywhere). The encrypted data key is then stored/sent alongside your encrypted data.
How can you bring the root of trust back under your control in a system like this? 
Since the KMS allows you to import master keys from your own key management infrastructure, you should build a key deriver that stores a master secret on your infrastructure under your control, and uses a Key Derivation Function (KDF) to generate rotate-able symmetric keys. This brings the root of trust back into your control. Now I can start solving the authorization and tampering problems.
To solve the authorization problem, you'd probably set up or choose a Certificate Authority (CA). You'd use this CA to generate certificates for all your authorized clients. Then, you'd have your clients issue signed key derivation requests to your key deriver service. They would sign their requests and staple the certificate to the requests so that the key deriver can verify it with the CA. Then, the key deriver would export the key to the KMS system, and send the key ID into your system so the clients can use the appropriate key.
To solve the tampering problem, you would like to have some way of checking that the key your clients received is legitimate.
In an ideal world, your key deriver would have a certificate as well, and it would sign all the keys it delivers so the clients can verify those signatures and ensure the keys haven't been tampered with. Since you can't get access to the key once it's been imported into the KMS, you just have to trust that any data you send into the KMS service is encrypted with the key you exported.
You were not able to solve this problem within the guardrails of your vendor's KMS - you would have to deliver your own keys to get this property.
For those of you keeping track, you basically built an entire KMS solution on top of the existing KMS solution. Just because you wanted to protect yourself from potential mistakes at your vendor. If you have to go to such lengths, why are you using them in the first place? And indeed, in order to solve the tampering problem, you can't use their service anyway!
At Peacemakr, we reject the idea that you need to trust *us* to use our service. That service I talked about before? That's more or less what we built, but better. We built Key Derivers that run in your trusted infrastructure. Our open-source SDKs allow you to specify a CA to generate X.509 certificates your Key Derivers can use to validate requests. All keys are derived, encrypted, and signed in your Key Deriver. The Peacemakr service (which is unable to decrypt any keys) delivers those encrypted keys directly to your clients, who check the signatures and decrypt the keys locally. All your data stays under your control - we never want to see it. Our service is designed to be secure even in the case of a malicious active employee; nobody at Peacemakr can impersonate one of your clients or inject bad keys into your system because all the steps are secured with well-known and well tested protocols. You can follow the chain of trust from the client to the root of trust in the Key Deriver, all under your control.
I don't trust my vendors, and you shouldn't either. Use Peacemakr and you don't have to. Discover peace with Peacemakr.

Collapse