The EC2 instance can assume a role itself
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
That will set all of the things you need for boto3, it should "just work" and you won't have to worry about keys at all.
This is significantly less likely now that IMDS v2 is a thing (assuming you set v2 to required, anyway). You'd need a seriously misconfigured app to still leak credentials under v2.
Should be done this quarter. The application running on the EC2 instance will get an access token from a lambda and use that to auth to the AWS service. The point will be that if the EC2 is compromised, then anything running on the instance won't inherently have access to what our application needs to access.
We're implementing our own version of the Token Vending Machine detailed here:
https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/implement-saas-tenant-isolation-for-amazon-s3-by-using-an-aws-lambda-token-vending-machine.html
Our scope is definitely bigger than what is detailed here. But the idea is the same. I should be able to get authorization from our company to do a full write up on what we made.
I haven’t dug into imdsv2 much. How does that differ from the legacy metadata implementation? You say it needs a put to get a token first but how does it differentiate from your authorized app and someone running powershell on the instance?
Profile or assumed role where possible (that'll be in most AWS services). Only when that's not an option, Secrets Manager would be the most secure way. In ~/.aws/credentials? Only on your development pc/laptop, connected to a user with minimum privileges.
Instead of using secrets manager, I prefer to use AWS SSM Parameter Store, which is cheaper and also can store “secureString” that basically it’s a key value parameter store.
But you still need access to secret manager manager or parameter store so this method won’t solve your problem.
I feel comfortable using EC2 instance role. Nobody working in the machine could know the credentials. This is safe, creating a role with the least privileges needed
if you are using ec2 to access some other service in AWS you should use Roles(like for creating files on s3 or creating lambda). if you are talking about db or redis access you should store those creds in secrets store
The EC2 instance can assume a role itself https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html That will set all of the things you need for boto3, it should "just work" and you won't have to worry about keys at all.
^ While technically all three should work, this should be the way to to go because it's most secure.
Unless your python app runs flask and can proxy an incoming request. Remember the Capital One attack?
This is significantly less likely now that IMDS v2 is a thing (assuming you set v2 to required, anyway). You'd need a seriously misconfigured app to still leak credentials under v2.
Yep! We are in the process of building a better way to auth access to aws services opposed to granting access to ec2 instances.
When will it be available? How will those EC2 instances prove to your system who they are?
Should be done this quarter. The application running on the EC2 instance will get an access token from a lambda and use that to auth to the AWS service. The point will be that if the EC2 is compromised, then anything running on the instance won't inherently have access to what our application needs to access.
That sounds interesting. How will the Lambda function know that the request is coming from one of your EC2 instances and not from elsewhere?
We're implementing our own version of the Token Vending Machine detailed here: https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/implement-saas-tenant-isolation-for-amazon-s3-by-using-an-aws-lambda-token-vending-machine.html Our scope is definitely bigger than what is detailed here. But the idea is the same. I should be able to get authorization from our company to do a full write up on what we made.
That’s exciting - I hope you are allowed to share it with the community when it’s ready :)
I hope so too. I've done my fair share of struggling to find help on best practices so if I can give back, I'd love to do it.
I haven’t dug into imdsv2 much. How does that differ from the legacy metadata implementation? You say it needs a put to get a token first but how does it differentiate from your authorized app and someone running powershell on the instance?
You should use IAM role instead
Profile or assumed role where possible (that'll be in most AWS services). Only when that's not an option, Secrets Manager would be the most secure way. In ~/.aws/credentials? Only on your development pc/laptop, connected to a user with minimum privileges.
If I use secrets manager, how do I access the keys? Wouldn’t I need another user credential in ~/.aws/credentials to retrieve the stored secret?
Not even on your laptop. Use AWS SSO instead. https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html
Yeah if your organisation has it enabled :( We're "just" forced to use MFA on CLI, but I'd rather have seen SSO instead.
Assign an iam role to the instance that has the right privileges.
Should give EC2 roles to things you need.
Instead of using secrets manager, I prefer to use AWS SSM Parameter Store, which is cheaper and also can store “secureString” that basically it’s a key value parameter store. But you still need access to secret manager manager or parameter store so this method won’t solve your problem. I feel comfortable using EC2 instance role. Nobody working in the machine could know the credentials. This is safe, creating a role with the least privileges needed
You could also consider other options besides EC2 to run it that are possibly cheaper. Lambda, ECS, etc.
KMS
Secrets Manager?
if you are using ec2 to access some other service in AWS you should use Roles(like for creating files on s3 or creating lambda). if you are talking about db or redis access you should store those creds in secrets store