This is another issue of CloudQuery’s monthly newsletter featuring cloud security and data engineering tips, tools, tutorials, and a changelog of all the new updates rolled out this month in CloudQuery.

How To Run Efficient Embedded ELT CloudQuery Workflows Inside Dagster Learn to run CloudQuery, an open-source high-performance ELT framework, with Dagster orchestrator. You’ll leverage Dagster’s resource management and scheduling capabilities to run CloudQuery locally, avoiding additional cloud costs. This is possible due to CloudQuery’s design, which allows it to run as single binaries without third-party dependencies.

How to Build a CSPM with Grafana and CloudQuery Learn to build an extensible Cloud Security Posture Management (CSPM) solution using Grafana and CloudQuery. This tutorial guides you through setting up a local development environment with Docker Compose, integrating CloudQuery, PostgreSQL, dbt, and Grafana. You’ll create customizable dashboards for security compliance and monitoring, gaining practical experience in enhancing cloud security for both local and production environments.

Unmasking Adversary Cloud Defense Evasion Strategies: Modify Cloud Compute Infrastructure Part 2 To prevent abuse against the use of Snapshot Creation, Instance Creation, and Instance Deletion features within cloud environments, security teams and cyber-defenders must ensure that proper monitoring and logging services are enabled across all cloud providers they utilize.

Tactical Cloud Audit Log Analysis with DuckDB Using DuckDB to query Cloud Provider audit logs when you don't have a SIEM available.

Container security fundamentals part 6: seccomp A look at how seccomp is used in Linux and container systems.

AWS has new tables with reserved instances: ElasticSearch, MemoryDB, OpenSearch, RedShift. We have also added tables for AWS Health and RDS.

Azure has got tables with storage account access keys and key rotation policies.

GCP now supports tables with data from the Security Center.

Finally, new tables were added for secret and code-scanning alerts to our GitHub plugin.

These are just major highlights. To see all the changes described in detail, see the individual plugin changelogs in Hub.

We have added support for the overwrite-delete-stale write mode in the BigQuery plugin.

All destinations now support “Sync summary”. After syncing, a table named cloudquery_sync_summaries is created and includes the number of resources synced, the number of errors, and details about the plugins (both source and destination).

We have published the following plugins in preview:

Orca: Sync data about your assets, alerts, or attack paths related to your infrastructure Wiz: Sync issues, vulnerability, and cloud infrastructure findings from Wiz. tempo.io: Sync data about time spent working on issues in Jira (using Tempo.io app). BambooHR: Sync employees and time off requests, and connect them with data from other plugins to find owners of your resources, or employees on call that should handle an incident.

Note: The preview means the plugins are free to use but their schema may change in the next major version before they officially reach the “Generally Available” stage.

Ready to get started with CloudQuery? You can try out CloudQuery locally with our quick start guide or explore the CloudQuery Platform (currently in beta) for a more scalable solution.

Got feedback or suggestions? Join the CloudQuery community to connect with other users and experts, or message our team directly here if you have any questions.