CloudQuery Product Updates #6

Product News

Yevgeny Pats

•

Hey everyone! Our monthly round-up of CloudQuery product and roadmap updates is here!

Apache Arrow #

We were heads down adopting Apache Arrow as our in-memory data format!

TLDR:

Support for more than 30 types for any source plugin including repeated and nested types like lists, structs, maps.
Database source plugins (with CDC) will be able to sync their schema accurately using Apache Arrow type system.
Any database that already supports Arrow format natively like ClickHouse, DuckDB will be able to enjoy enhanced performance.

Sources #

New Sources #

Welcoming 4 new partner and community plugins!

New Official plugins:

OracleDB
Firestore Thanks to @BubbaJoe for initial contribution!

Community Contributions to Security Policies:

Azure CIS Thanks to aruneko for community contribution!

New Resources #

AWS: 18 New tables:
- aws_config_config_rule_compliance_details
- aws_config_remediation_configurations
- aws_acmpca_certificate_authorities
- aws_cloudformation_stack_templates
- aws_cloudformation_template_summaries
- aws_cloudwatchlogs_log_group_data_protection_policies
- aws_config_config_rule_compliance_details
- aws_config_remediation_configurations
- aws_config_configuration_aggregators
- aws_config_delivery_channels
- aws_config_retention_configurations
- aws_costexplorer_cost_30d
- aws_costexplorer_cost_forecast_30d
- aws_efs_access_points
- aws_networkfirewall_firewall_policies
- aws_networkfirewall_rule_groups
- aws_securityhub_enabled_standards
- aws_securityhub_hubs
Azure: 39 New tables:
- azure_consumption_billing_*
- azure_consumption_subscription_*
- azure_dns_record_sets
- azure_network_private_link_services
- azure_network_watcher_flow_logs
- azure_security_sub_assessments
- azure_sql_server_advanced_threat_protection_settings
- azure_sql_server_firewall_rules
- azure_storage_queue_acl
GCP: New Table:
- gcp_dns_resource_record_sets
Datadog: 3 Tables:
- datadog_rum_events
- datadog_slo_corrections
- datadog_slos
Digital Ocean: 2 Tables:
- digitalocean_load_balancers
- digitalocean_regions
GitLab: 2 Tables:
- gitlab_group_billable_members
- gitlab_project_members
Oracle: 5 Tables:
- oracle_compute_volume_attachments
- oracle_virtualnetwork_assigned_public_ips
- oracle_virtualnetwork_ephemeral_public_ips
- oracle_virtualnetwork_private_ips
- oracle_virtualnetwork_reserved_public_ips
Snyk: 2 Tables:
- snyk_groups
- snyk_group_members
Stripe: 2 Tables:
- stripe_checkout_session_line_items
- stripe_payment_link_line_items

Policies #

Thanks to @aruneko for contribution to our Azure CIS Policies!

Other notable blogs and use-cases #

Engineering #

Security #

AWS Identity Center (formerly known as AWS SSO): A Guide to Privilege Escalation and Identity and Access Management

CloudQuery