AWS
Tutorials
Limiting CloudQuery access to AWS accounts
When using CloudQuery to sync data from AWS, a common practice is to use a read-only IAM role. This approach gives CloudQuery access to your resources without posing a security risk of unwanted writes. However, sometimes, you will want more granular control, limiting CloudQuery's access to only the specific resources defined in their sync configurations. Previously, obtaining the necessary permissions for this level of control was a manual, cumbersome process. That’s why we now provide an API endpoint that simplifies this process, allowing you to automate the collection of required permissions based on your chosen tables.
How to limit access to selected tables only #
When the only tables you want to sync are from the EC2 and EKS services, you could use a role with permissions limited to those services, not broad read access across the entire AWS account. You can go through the plugin documentation and see the permissions for each table, or you could use CloudQuery API to do that automatically.
The API endpoint that provides the required information is the GetPluginVersionTable. All you need to know is the plugin name, plugin version, and the table name. Here’s an example request to get permissions for the
aws_ec2_instances table and extract the permissions using jq:curl -s https://api.cloudquery.io/plugins/cloudquery/source/aws/versions/v32.1.1/tables/aws_ec2_instances | jq -r '.permissions_needed.[]'Output:
ec2:DescribeInstancesThe only thing remaining is to list the tables from your AWS sync config,call the API endpoint for each table, and then extract the
permissions_needed field from the JSON response.Suppose you want to sync the following AWS tables using the CloudQuery AWS plugin version
v32.1.1:aws_ec2_instancesaws_eks_clustersaws_ec2_hosts
Using the API endpoint mentioned above, you can retrieve the permissions for each table with an API call similar to the following (replace
TABLE_NAME_GOES_HERE with the actual table name):curl -s https://api.cloudquery.io/plugins/cloudquery/source/aws/versions/v32.1.1/tables/TABLE_NAME_GOES_HERE | jq -r '.permissions_needed.[]'Calling the above API calls will return the following permissions needed for each table:
ec2:DescribeInstanceseks:DescribeClustereks:ListClustersec2:DescribeHosts
Based on the results, a minimal IAM policy can be created to grant access to the tables from the example above only with the following permissions:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": ["ec2:DescribeInstances", "ec2:DescribeHosts"],
"Effect": "Allow",
"Resource": "*"
},
{
"Action": ["eks:DescribeCluster", "eks:ListClusters"],
"Effect": "Allow",
"Resource": "*"
}
]
}Putting it all together #
Here's a full shell script that generates a JSON IAM policy based on the defined tables:
# Define the tables to check
tables="aws_ec2_instances aws_eks_clusters aws_ec2_hosts"
# Get the latest version of the aws plugin
latest_version=$(curl -s https://api.cloudquery.io/plugins/cloudquery/source/aws | jq -r ".latest_version")
# Get the permissions needed for each table
permissions=$(echo $tables | xargs -n 1 -I {} curl -s https://api.cloudquery.io/plugins/cloudquery/source/aws/versions/$latest_version/tables/{} | jq '.permissions_needed.[]' | sort -u)
# Print the sorted permissions as JSON IAM policy
echo "{
\"Version\": \"2012-10-17\",
\"Statement\": [
{
\"Action\": [$(echo $permissions | tr ' ' ',')],
\"Effect\": \"Allow\",
\"Resource\": \"*\"
}
]
}"The output of the above:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:DescribeHosts",
"ec2:DescribeInstances",
"eks:DescribeCluster",
"eks:ListClusters"
],
"Effect": "Allow",
"Resource": "*"
}
]
}Wrap Up #
By using CloudQuery’s new API endpoint, you can easily automate the creation of the least privileged IAM roles, eliminate manual errors in mapping tables to their required permissions, and streamline the setup of secure and granular CloudQuery syncs.
CloudQuery simplifies cloud asset discovery and governance, making it easy to audit, monitor, and manage cloud environments at scale. If you haven’t already, get started with CloudQuery today to enhance your cloud visibility.
Have questions or need help? Join the CloudQuery Developer Community to connect with other users, share insights, and get support. Let us know how your team is tackling cloud security by engaging with us on LinkedIn or X.
Written by Michal Brutvan
Michal is CloudQuery's senior product manager and has responsibility for new features and CloudQuery's product roadmap. He has had a wealth of product ownership roles and prior to that, worked as a software engineer.
