AWS SIGv4 and SIGv4A
In the time it takes to read this sentence, the AWS Identity and Access Management (IAM) service will handle several billion requests. Pretty close to every one of those requests is authenticated using the AWS SIGv4 protocol, before IAM authorization policy is applied to check if the request is allowed or not.
I didn’t have a hand in designing the AWS SIGv4 protocol so hopefully it’s not too biased to say that I dearly love how AWS SIGv4 works. SIGv4 rigorously authenticates every request individually and it’s reassuringly simple. It uses fast cryptography based on the HMAC algorithm to strongly authenticate every request. Customer’s secrets never appear in these requests directly, and there are neat properties built in that mean every request is finely scoped down and signed with a token that is valid only for a given AWS service, in a given AWS region, on a given day. All of this is checked and verified at the authentication stage, before AWS tries to authorize a request.
Starting last week, as part of S3 Multi-Region Access Points, we’re using a new version of AWS SIGv4, called SIGv4A, which is nearly identical to regular SIGv4 but goes even further and uses public-private keys and asymmetric cryptographic signatures for every request. These asymmetric signatures help us solve an interesting challenge. S3 Multi-Region Access Points give customers a flavor of S3 that utilizes several regions. The same authenticated request has to work whether it arrives in eu-west-1 or us-west-1, with no changes. But at the same time, AWS customers also expect strong isolation between AWS regions and one of our most important security measures to back up that separation is never to have the same key material in two regions.
SIGv4A delivers a new kind of defense in depth. In short, with SIGv4A customers get a private key that AWS services don’t know. AWS services only know the corresponding public key. When a customer’s SIGv4A request is served and logged to CloudTrail, customers can be mathematically confident that only someone with the private key could have made that request.
The AWS SIGv4 per-request design
When clients connect to AWS services, they use the HTTPS protocol over TLS/SSL. There’s nothing special about this TLS connection. It’s important that clients verify the AWS service’s certificate – but that’s routine, there’s no authentication of the client or caller happening at the TLS level.
Instead, AWS SIGv4 authenticates each request individually. By signing each request, the client has to intentionally process and sign each specific transaction. This helps prevent attacks that involve smuggling or injecting requests into connections, and it facilitates sophisticated isolation techniques. For example, some applications choose to keep their secret key for signing requests in a secure enclave. Instead of using that key to authorize a connection without any control over how that connection will be used, the application in the enclave can scrutinize each transaction and decide whether to sign it or not. It’s like the difference between trusting someone to borrow your checkbook and autopen for a while, and being able to scrutinize individual checks and sign them individually.
How AWS SIGv4 requests are signed
When an AWS customer creates an IAM user, they get an access key ID, and a cryptographically strong secret access key from the IAM service, which together identify and authenticate that user. Instead of using these secret keys directly as bearer tokens, SIGv4 uses a simple four step process to sign requests.
The first step is canonicalize the request and any headers we want to sign into a string. That’s so the sender and the recipient can all agree about what is to be signed in the first-place. The second step is to add the signature algorithm, time, and credential scope. The signature algorithm and time are included here to prevent downgrade and replay attacks, but credential scope is where things get really interesting. In SIGv4, the clients don’t even use their secret keys to sign the requests, instead they use credentials that are finely scoped down in important ways. Each scoped credential is valid only for a request made using a particular version of the signature protocol, to a particular AWS service, in a particular AWS region, on a particular day.
You can see this in action with the third step in the SIGv4 process, when clients cryptographically derive those scoped credentials using the HMAC algorithm. The reason for this scoping is to enforce boundaries between AWS regions and AWS services. The IAM authentication process runs on AWS service hosts, for example the front-ends handling S3 requests, and in order to authenticate a customer request, those hosts need to know the signing credential. These hosts have many layers of security and access control, and by design AWS service team members do not have access to these signing credentials, but we must always consider defense in depth against potential issues. If the signing credentials were ever exposed, we have processes to revoke and invalidate them, but the credential scoping also means that these credentials simply can’t be used to access other AWS services, other AWS regions, and the credentials are time-limited. Impact would not be allowed to spread.
There aren’t even copies of the regional credentials outside of the region they are for, and on the AWS side, knowledge of a customer’s actual secret key is limited to an isolated core component of the AWS IAM service.
The final step of the SIGv4 process is for clients to use the scoped credential to sign the request, again using the HMAC algorithm, and then add that signature to the request as a header. We add it as a header, rather than putting in the URL itself, because URLs often end up in logs and so that users can’t be tricked into clicking a URL that is a fully-signed AWS request. Pre-signed URLs are great for web-content, but not APIs. Even still, if a signature does end up being logged or recorded somewhere it’s not catastrophic. SIGv4 signatures don’t disclose the secret and can’t be used for anything except to make that exact request within a very short period of time. You could make a signature public but no-one can figure out your secret from it.
AWS customers don’t have to think about any of this. The AWS SDKs, CLI, and the AWS Console all perform this signing automatically “under the hood.”
What’s new with SIGv4A
As we’ve seen, SIGv4 request signing credentials are scoped to a single AWS service, in a single AWS region, on a particular day. One of our principles of data sovereignty at AWS is that we don’t move or share data between AWS regions unless a customer intended it. It isn’t good enough for us to simply say this, we want robust mechanisms that prevent it.
S3 Multi-Region Access Points are a new S3 feature where customers can use S3 in multiple regions, with seamless routing and failover between regions. This means that the same SIGv4 request from an S3 client needs to work whether it goes to S3 in us-east-1, or eu-west-1, to pick the same two regions from earlier as our example.
Rather than come up with a new “global” scoped credential, which would not preserve our AWS Region isolation or data sovereignty principles, we’re adding support for asymmetric signatures based on public-private key cryptography. SIGv4A goes through a similar scoped credentials derivation process as SIGv4, except that it derives a credential that is valid for a single service, on a single day, and then it adds another step to derive a public-private ECDSA key pair from that scoped credential.
Since deriving this key pair requires a customer’s secret key, only a customer and the isolated core of the AWS IAM service can derive it. AWS Regions and Services get only the public key component; when S3 receives an SIGv4A request, it uses this public key to authenticate the customer request. Crucially, asymmetric public keys can’t be used to sign requests. There’s no credential on the service hosts that could be used to make requests to a different service or a different region. This is how customers can be mathematically certain that a SIGv4A request must come from a client who has the scoped private key.
Asymmetric cryptography is more computationally expensive than the HMAC algorithm that’s used as the final step of SIGv4, but AWS handles the more expensive verification part and we’ve been working steadily on improvements and optimizations to ensure that requests still get very high performance. We think the benefits are well worth it.
I’ll be sharing more at AWS re:Invent about SIGv4A and also more about other techniques we use to provide strong isolation around customer data, keys, and how we implement other controls that are designed to keep customers secure and isolated even in the event of potential issues in AWS services themselves.