New
Join our webinar! Building a customizable and extensible cloud asset inventory at scale
AWS
Tutorials

How to Leverage AWS RDS Processor Features for Pricing Comparison Using CloudQuery

Ron Shemesh

Ron Shemesh

Overview #

Understanding the cost implications of your instance configurations is crucial to managing your AWS resources, particularly Amazon Relational Database Service (RDS) instances. By leveraging the processor_features field from CloudQuery's AWS plugin, you can gain insights into your AWS RDS instances' core count and threads per core. This information can be used to compare the hourly pricing of different instance types with the same vCPU configuration. This approach helps you make informed decisions to optimize costs while maintaining optimal performance.
Before diving into leveraging the processor_features, let's first understand the aws_rds_instances table that is generated by our AWS source plugin. The aws_rds_instances table contains various details about your RDS instances, including the processor_features field, which provides information such as the number of cores (coreCount) and threads per core (threadsPerCore).
Note: This post assumes that you have already read the comprehensive guide on using the AWS Pricing API from CloudQuery and that you are familiar with how to use the AWS pricing plugin. Ensure that you have configured your AWS Pricing Plugin to pull data from the necessary AWS pricing tables, including: awspricing_service_products and awspricing_service_terms.
With CloudQuery, you can easily export your AWS data to a PostgreSQL database (or any other data destination), enabling comprehensive analysis and visualization. Ensure that the aws_rds_instances table is included in your AWS source configuration. This post will be using our most popular data destination, Postgres, for all our example queries.

Extracting Processor Features #

To start, you need to check the RDS instances you have and select a specific one by its ARN (Amazon Resource Name). The following query will list your RDS instances:
SELECT * FROM aws_rds_instances;
For demonstration purposes, we'll assume a simplified result set. In reality, you'll get more columns, but this example focuses on key attributes:
_cq_sync_time_cq_idaccount_idregionarnprocessor_featurestagsactivity_stream_kinesis_stream_nameactivity_stream_kms_key_idactivity_stream_modeactivity_stream_policy_statusdb_instance_arn
d290f1ee-6c54-4b01-90e6-d701748f0851123456789012arn:aws:rds:us-west-2:123456789012:dbus-west-2arn:aws:rds:us-west-2:123456789012:db{"ProcessorFeatures": [{"Name": "coreCount", "Value": "4"}, {"Name": "threadsPerCore", "Value": "2"}]}trueexample-kinesis-streamexample-kms-key-idasyncenabledarn:aws:rds:us-west-2:123456789012:db

Finding Hourly Costs #

To find the hourly costs for a specific RDS instance, you will use its ARN in the following query. This query compares the specific values of the chosen instance and checks the price of other node types with the same vCPU. This query uses the following formula to calculate your vCPU :
coreCount * threadsPerCore = vCPU
This calculation allows you to match the vCPU values across different tables for comparison. Note that this calculation has already been performed in the query below.
WITH vcpu_calculation AS (
    SELECT
        arn,
        region,
        CAST(pf->>'Value' AS INTEGER) AS core_count,
        CAST(tf->>'Value' AS INTEGER) AS threads_per_core,
        CAST(pf->>'Value' AS INTEGER) * CAST(tf->>'Value' AS INTEGER) AS vcpu
    FROM
        aws_rds_instances,
        jsonb_array_elements(processor_features->'ProcessorFeatures') AS pf,
        jsonb_array_elements(processor_features->'ProcessorFeatures') AS tf
    WHERE
        processor_features IS NOT NULL
        AND pf->>'Name' = 'coreCount'
        AND tf->>'Name' = 'threadsPerCore'
        AND arn = 'arn:aws:rds:us-west-2:123456789012:db:example-db-instance'  -- Note: Change the value of `arn:aws:rds:us-west-2:123456789012:db:example-db-instance' to the ARN of the RDS instance you want to check.
),
rds_pricing_info AS (
    SELECT
        p.sku,
        p.attributes->>'instanceType' AS instance_type,
        p.attributes->>'vcpu' AS vcpu,
        p.attributes->>'regionCode' AS region,
        p.attributes->>'memory' AS memory,
        p.attributes->>'storage' AS storage,
        p.attributes->>'networkPerformance' AS network_performance,
        p.attributes->>'operation' AS operation,
        p.attributes->>'usagetype' AS usagetype,
        p.attributes->>'engineCode' AS engine_code,
        p.attributes->>'databaseEngine' AS database_engine,
        p.attributes->>'deploymentOption' AS deployment_option,
        p.attributes->>'normalizationSizeFactor' AS normalization_size_factor
    FROM
        awspricing_service_products AS p
    JOIN
        vcpu_calculation AS v ON v.vcpu = CAST(p.attributes->>'vcpu' AS FLOAT)
        AND v.region = p.attributes->>'regionCode'
    WHERE
        p.attributes->>'servicecode' = 'AmazonRDS'
),
pricing_details AS (
    SELECT
        p.sku,
        p.instance_type,
        p.vcpu,
        p.region,
        p.memory,
        p.storage,
        p.network_performance,
        p.operation,
        p.usagetype,
        p.engine_code,
        p.database_engine,
        p.deployment_option,
        p.normalization_size_factor,
        st.type,
        st.effective_date,
        jsonb_array_elements(st.price_dimensions) AS price_dimension
    FROM
        awspricing_service_terms AS st
    JOIN
        rds_pricing_info AS p ON p.sku = st.sku
)
SELECT
    pd.sku,
    pd.instance_type,
    pd.vcpu,
    pd.region,
    pd.memory,
    pd.storage,
    pd.network_performance,
    pd.operation,
    pd.usagetype,
    pd.engine_code,
    pd.database_engine,
    pd.deployment_option,
    pd.normalization_size_factor,
    pd.price_dimension->'pricePerUnit'->>'USD' AS price_per_unit_usd,
    pd.price_dimension->>'description' AS price_description,
    pd.price_dimension->>'unit' AS unit
FROM
    pricing_details AS pd
WHERE
    pd.price_dimension->>'unit' = 'Hrs'
    AND pd.type = 'OnDemand'
ORDER BY
    pd.instance_type,
    pd.price_dimension->'pricePerUnit'->>'USD';
This query calculates the virtual CPU (vCPU) for a specific RDS instance based on the provided ARN. It retrieves the coreCount and threadsPerCore values from the processor_features field and calculates the vCPU. The query then joins this data with the AWS pricing information to find the hourly costs for similar vCPU configurations in other instance types. Let's break it down:
  1. Calculate vCPU for a specific RDS instance:
    • The vcpu_calculation CTE (Common Table Expression) extracts the coreCount and threadsPerCore values from the processor_features jsonb array and calculates the vCPU for the specific RDS instance identified by the provided ARN.
    • It uses the jsonb_array_elements function to flatten the processor_features array twice, once for coreCount and once for threadsPerCore.
    • The WHERE clause filters to ensure that processor_features is not null and matches the specific ARN of the RDS instance.
  2. Join RDS vCPU data with AWS pricing information:
    • The rds_pricing_info CTE joins the vcpu_calculation CTE with the awspricing_service_products table based on the vCPU count and region.
    • It filters for RDS service codes to ensure only relevant data is retrieved.
    • The CTE selects various attributes related to the instance type, region, and vCPU.
  3. Expand price dimensions:
    • The pricing_details CTE joins the rds_pricing_info CTE with the awspricing_service_terms table based on the SKU.
    • It flattens the price_dimensions array from the awspricing_service_terms table, allowing us to work with individual price dimensions.
  4. Final Selection and Filtering:
    • The main SELECT statement retrieves detailed pricing information by joining the pricing_details CTE with the awspricing_service_products table.
    • The WHERE clause filters for pricing units in hours and ensures that the pricing type is OnDemand.
    • The results are ordered by instance type and price per unit.

Detailed Explanation of Specific Clauses #

  • RDS Service Codes: The servicecode for Amazon RDS is used to identify the specific service (in this case, Amazon RDS) within the AWS pricing data. This ensures that only pricing information related to RDS instances is retrieved.
  • Pricing Type is OnDemand: On-Demand pricing refers to a pay-as-you-go model where you are charged for compute capacity by the hour or second with no long-term commitments or upfront payments. This ensures that the query retrieves only On-Demand pricing information, which is relevant for users who want to know the cost of running an instance without any reserved or spot pricing considerations.
  • Pricing Unit is in Hours: This ensures that the retrieved pricing information is based on an hourly rate, which is useful for calculating the cost of running an RDS instance on a per-hour basis.

Example Query Results #

Here is an example of the query results:
skuinstance_typevcpuregionmemorystoragenetwork_performanceoperationusagetypeengine_codedatabase_enginedeployment_optionnormalization_size_factorprice_per_unit_usdprice_descriptionunit
H5UN6EVK42GGQQDCdb.c6gd.2xlarge8us-west-216 GiB1 x 474 NVMe SSDUp to 10 GbpsCreateDBInstance:0014USW2-Multi-AZClusterUsage.c6gd.2xl14PostgreSQLMulti-AZ (readable standbys)481.9080000000$ 1.908 per RDS db.c6gd.2xlarge Multi-AZ (readable standbys) instance hour (or partial hour) running PostgreSQLHrs
KA22DCAQY4EDYEUVdb.c6gd.2xlarge8us-west-216 GiB1 x 474 NVMe SSDUp to 10 GbpsCreateDBInstance:0002USW2-Multi-AZClusterUsage.c6gd.2xl2MySQLMulti-AZ (readable standbys)481.9080000000$ 1.908 per RDS db.c6gd.2xlarge Multi-AZ (readable standbys) instance hour (or partial hour) running MySQLHrs
QZXPVWX3WDFXWQQDdb.m2.4xlarge8us-west-268.4 GiB2 x 840HighCreateDBInstance:0014USW2-Multi-AZUsage.m2.4xlarge14PostgreSQLMulti-AZ642.7800000000$2.78 per RDS db.m2.4xlarge Multi-AZ instance hour (or partial hour) running PostgreSQLHrs
The results show various instance types with the same vCPU configuration, allowing you to compare their hourly costs directly. Here’s a breakdown of what each column represents:
  • sku: The Stock Keeping Unit (SKU) identifying the specific product.
  • instance_type: The type of the instance (e.g., db.c6gd.2xlarge, db.m2.4xlarge).
  • vcpu: The number of virtual CPUs.
  • region: The AWS region where the instance is located.
  • memory: The amount of memory allocated to the instance.
  • storage: The storage configuration of the instance.
  • network_performance: The network performance level of the instance.
  • operation: The specific operation or lifecycle stage (e.g., CreateDBInstance).
  • usagetype: The usage type, which often includes the billing category.
  • engine_code: The code representing the database engine.
  • database_engine: The type of database engine running on the instance (e.g., PostgreSQL, MySQL).
  • deployment_option: The deployment configuration (e.g., Single-AZ, Multi-AZ).
  • normalization_size_factor: A factor used for normalizing instance sizes.
  • price_per_unit_usd: The hourly cost of running the instance.
  • price_description: A detailed description of the pricing.
  • unit: The unit of time for the pricing, typically hours (Hrs).

How to Use This Information #

This information lets you make informed decisions about which instance types offer the best value for your specific requirements. Here are some ways you can use this data:
  • Cost Optimization: Compare the hourly costs of different instance types with the same vCPU to find more cost-effective options.
  • Performance Analysis: Evaluate other features such as memory, storage, and network performance to ensure that you choose instances that meet your performance requirements.
  • Deployment Decisions: Consider the deployment options (e.g., Single-AZ vs. Multi-AZ) and how they affect both cost and availability.

Summary #

By using CloudQuery to export your AWS data to a PostgreSQL database, you can efficiently check the cost implications of different RDS instance configurations. This approach allows you to compare pricing, optimize your resource allocation, and make better decisions for your AWS environment.
If you want to gain immediate insights into your AWS environment, try CloudQuery for free or connect with our engineering team for further assistance.
Want help getting started? Join the CloudQuery community to connect with other users and experts, or message our team directly here if you have any questions.
Ron Shemesh

Written by Ron Shemesh

Ron is a Senior Software Engineer at CloudQuery and has a background in data science and now works as a senior software engineer at CloudQuery. He has experience working in Python, SQL, React, Java and R. At CloudQuery, Ron has worked on our range of integrations and several projects foundational to platform performance. He loves taking on a challenge and using it to improve his skills.

Turn cloud chaos into clarity

Find out how CloudQuery can help you get clarity from a chaotic cloud environment with a personalized conversation and demo.

Join our mailing list

Subscribe to our newsletter to make sure you don't miss any updates.

Legal

© 2024 CloudQuery, Inc. All rights reserved.

We use tracking cookies to understand how you use the product and help us improve it. Please accept cookies to help us improve. You can always opt out later via the link in the footer.