CloudQuery News

Introducing the new OpenAI Source Integration

• September 5, 2024

We have heard from our community maintaining security and compliance in the fast-evolving AI landscape is a challenge. Today, we are excited to introduce our new OpenAI Source Integration designed to help organizations integrate compliance and audit data from OpenAI.

What does the OpenAI integration do? #

This new integration gets information about your OpenAI projects and their users allowing you to audit access of individual users to your OpenAI projects. Connecting the user project access information with other integrations, such as our BambooHR Source Integration, will help you make sure that their access is removed when the employees leave, for example.

How the OpenAI Source Plugin Works #

Here’s a step-by-step guide on how to configure the OpenAI Source Plugin and utilize the data collected from OpenAI.

Obtain Your API Key: Start by getting the administration API key from the OpenAI platform by following these instructions.
Install CloudQuery CLI: Download and install the CloudQuery CLI on your machine by following the steps on our download page.
Set Up the Sync Configuration: Create a CloudQuery configuration file by running the following init command:

cloudquery init --source openai --destination postgresql

Update the Configuration File: Open the generated openai_to_postgresql.yaml file in your favorite text editor. Replace the placeholders ${OPENAI_ADMIN_API_KEY} with your API key from step 1, and ${POSTGRESQL_CONNECTION_STRING} with your PostgreSQL connection string.
Sync with CloudQuery: Log in using cloudquery login, then run the command below to begin syncing:

cloudquery sync openai_to_postgresql.yaml

For more details on this process, refer to the OpenAI Source Integration documentation.

Note that these examples assume the data was synced using the OpenAI to PostgreSQL configuration.

Querying Synced Data #

Now, let’s explore how to use SQL to query the synced data.

To find out which projects a specific team member has access to, use the following query:

select distinct project_name
from openai_project_users
where email = '[email protected]';

The new CloudQuery OpenAI integration also syncs the audit log that can be used to review all login activities. To view when each user last logged in, execute:

select actor, type, effective_at
from openai_audit_logs
where type = 'login.succeeded'
and effective_at = (
    select max(effective_at)
    from openai_audit_logs as al
    where al.actor->'session'->'user'->>'email' = openai_audit_logs.actor->'session'->'user'->>'email'
    and al.type = 'login.succeeded'
);

This can help identify any unexpected sign-ins to the OpenAI platform.

To find invites sent to external members, run:

select *
from openai_invites
where email not ilike '%@example.com';

This query filters out invites sent to external email addresses.

With the OpenAI Source Plugin, you can play around with the data and fine-tune the queries in any format that's easiest for you. Once the sync is complete, you'll have all the data in a destination such as Postgres, MySQL, BigQuery, or any other supported destination. These destinations are specifically designed for efficient data exploration and querying, making them a great fit for handling data from OpenAI.

Get Started Today #

You can also try out CloudQuery locally with our quick start guide or explore the CloudQuery Platform for a more scalable solution.

The OpenAI Source Plugin is available now.

Got feedback or suggestions? Join the CloudQuery Community to connect with other users and experts.

Written by Alex Savanovich

Alex is a Senior Software Engineer at CloudQuery, specializing in cloud data infrastructure and application development, with a focus on building scalable solutions for multi-cloud environments.

On this page