# BigQuery Setup Source: https://support.detrics.io/bigquery/bigquery-setup Create a destination and connect Detrics to your BigQuery project **Last updated:** May 18, 2026 A **destination** is your BigQuery connection, it tells Detrics which GCP project and dataset to write data to. You set it up once, then create as many transfers as you need pointing to it. Detrics offers two setup modes, chosen on the first screen of destination creation. **This guide covers Standard Setup**, it grants Detrics the **BigQuery User** role at the project level and is the simplest path for most teams. Security-critical organizations and shared GCP projects can instead use **[Least-Privilege Access](/bigquery/least-privilege-access)**, which scopes Detrics to a single dataset and prevents it from seeing anything else in your project, see that guide for its extra steps. The setup mode **can't be changed** after the destination is created. ## Prerequisites Before creating a destination, you need: * A Google Cloud Platform (GCP) account * A GCP project with the **BigQuery API** enabled * Permission to manage IAM roles in the project (to grant Detrics access) If you don't have a GCP project yet, create one at [console.cloud.google.com](https://console.cloud.google.com). BigQuery is automatically available in every GCP project, just make sure the BigQuery API is enabled in **APIs & Services**. ## Creating a Destination 1. In the Detrics web app, go to [**Data Warehouse → Destinations**](https://app.detrics.io/data-warehouse/destinations) 2. Click [**New Destination**](https://app.detrics.io/data-warehouse/destinations/new) 3. Fill in the details: ### Destination Name A label for your reference (e.g., "Production Warehouse", "Client Analytics", "Dev/Testing"). This is only visible in Detrics, not in BigQuery. ### GCP Project ID The project ID from your Google Cloud Console. You can find this on the [GCP Dashboard](https://console.cloud.google.com/home/dashboard) or in the project selector dropdown. It looks like `my-company-analytics` or `project-id-123456`. Use the **Project ID**, not the Project Name or Project Number. The Project ID is the lowercase identifier, usually visible under the project name in the GCP console. ### Dataset Name The BigQuery dataset where Detrics will create tables. Use lowercase letters, numbers, and underscores (e.g., `marketing_data`, `warehouse_prod`). If the dataset doesn't exist, Detrics will create it automatically. If it already exists, Detrics will add tables to it, existing tables are not affected. ### Data Location The geographic region where your data is physically stored. This is permanent and cannot be changed after the dataset is created. | Location | Type | Description | | ------------------------------------- | ------------- | ------------------------------------------- | | `US` | Multi-region | Data stored across multiple US data centers | | `EU` | Multi-region | Data stored across multiple EU data centers | | `us-east1`, `us-central1`, etc. | Single region | Data stored in a specific US region | | `europe-west1`, `europe-west2`, etc. | Single region | Data stored in a specific EU region | | `asia-east1`, `asia-southeast1`, etc. | Single region | Data stored in a specific Asia region | For most users, **US** or **EU** multi-region is the best choice. Multi-region gives you higher availability and doesn't restrict which regions can query the data. Choose a specific region only if you have data residency requirements. ### Timezone The timezone used by all transfers on this destination. The timezone is used for: * Interpreting schedule times (e.g., "6:00 AM" in which timezone) * Date boundary calculations ## Granting Access to Detrics After creating the destination, Detrics displays a **service account email** that needs access to your GCP project. This service account is what Detrics uses to create datasets, tables, and load data. ### Required IAM Roles Grant the Detrics service account these two roles: | Role | Purpose | | ----------------- | ------------------------------------------------------------------------------ | | **BigQuery User** | Create datasets, create tables, insert data, run queries, and manage load jobs | ### How to Grant Access 1. Go to [Google Cloud Console → IAM & Admin → IAM](https://console.cloud.google.com/iam-admin/iam) 2. Make sure you've selected the correct project in the top dropdown 3. Click **Grant Access** 4. In the **New principals** field, paste the Detrics service account email shown on your destination page 5. In the **Select a role** dropdown, search for and select **BigQuery User** 6. Click **Save** For security-conscious organizations or shared GCP projects, see [Least-Privilege Access](/bigquery/least-privilege-access) for a setup that scopes Detrics to a single dataset and prevents it from seeing any other data in your project. ### Sending Instructions to a Teammate If you don't have GCP admin access, click **Send Instructions** on the destination page. This generates an email with: * The exact service account email to add * Step-by-step instructions for granting access * The specific roles needed You can send this to your GCP administrator or DevOps team. ## Testing the Connection After granting access, test the connection: 1. Go to your destination's detail page 2. Click **Test Connection** 3. Detrics runs a series of checks: * **Can access the project**: Verifies the service account has basic access * **Can create a dataset**: Checks dataset creation permissions (or verifies the existing dataset) * **Can create tables**: Verifies table creation within the dataset * **Can insert data**: Confirms data loading works * **Can query data**: Validates read access If any check fails, you'll see a specific error message. The most common issue is missing IAM roles, see [Troubleshooting](/bigquery/troubleshooting) for solutions. Once the test passes, your destination is ready. The status badge changes to **Connected**. ## Managing Destinations ### Editing a Destination You can update the destination's name and timezone at any time. The GCP project ID, dataset name, and data location cannot be changed after creation (these are BigQuery constraints). ### Multiple Destinations You can create multiple destinations for different purposes: * **Production vs. staging**: Test changes in a staging dataset before pointing transfers at production * **Regional separation**: Keep EU client data in an EU dataset and US client data in a US dataset * **Team separation**: Different teams or departments with their own datasets ### Sharing a Destination You can share a destination with other members of your Detrics workspace. Shared users can create transfers using the destination but cannot modify the destination settings. Go to the destination detail page and click **Share** to manage access. ### Deleting a Destination Deleting a destination in Detrics does **not** delete the BigQuery dataset or any data. It only removes the connection from Detrics. Any transfers using the destination will stop syncing. # Chunking Source: https://support.detrics.io/bigquery/chunking How Detrics splits large syncs into smaller pieces to avoid timeouts and preserve progress **Last updated:** May 14, 2026 When a sync would be too large to fetch in a single API call, many accounts, long date ranges, or detailed entity levels like individual ads, Detrics automatically splits it into smaller pieces called **chunks**. Each chunk fetches a subset of the data, persists it to BigQuery as soon as it finishes, and then the next chunk starts. This is purely an execution detail. Your BigQuery tables end up with exactly the same data either way; chunking just changes how Detrics gets there. ## Why Chunking Exists Marketing platforms impose limits on how much data a single API request can return, and even when they don't, very large requests are slow and fragile. Without chunking, a single timeout or transient error would lose the entire sync's progress. Chunking solves three problems at once: * **Timeouts**: Each chunk runs in its own request, well under the platform's time limits * **Memory**: Detrics never holds an entire historical backfill in memory at once * **Recovery**: If a sync is interrupted, completed chunks are preserved; only the remaining chunks need to be retried on the next run ## How Chunks Are Split Chunking happens along two axes: * **Time range**: Long date ranges are split into segments (typically by calendar month) * **Account batches**: Many accounts are processed in smaller groups The exact chunk size depends on the platform and the **entity level** of the table. Denser data needs smaller chunks: | Example | Chunk Size | | -------------------------------- | ----------------------------------- | | Meta Ads, ad-level metrics | \~1 month × 5 accounts per chunk | | Meta Ads, campaign-level metrics | \~12 months × 25 accounts per chunk | | Google Ads, keyword-level | Smaller chunks (high cardinality) | | Google Ads, account summary | Larger chunks (low cardinality) | If a table's data fits comfortably in a single request, for instance, a short date range with few accounts, **no chunking is performed**, and the run executes as a single fetch. This is normal and expected for small or recently-created tables. ## Chunk Order Chunks are executed in **recent-first** order: the most recent time windows are fetched before the older ones. This means fresh data lands in BigQuery first, so even if a long initial backfill is paused or interrupted, you still have usable recent data to query. ## What You See in the UI In the **Run History** of a transfer, each table row shows `X/Y chunks` when chunking was used (e.g., `4/4 chunks` means all 4 chunks completed successfully, `12/13 chunks` means one chunk failed or is still retrying). If you don't see a chunk count, the table was fetched in a single request, no chunking was needed for that run. ## Chunking and Resyncs When you trigger a manual resync or when an adaptive retry runs, Detrics replays only the chunks that didn't succeed in the original run. Successful chunks are not refetched, which keeps resyncs fast and minimizes API quota usage. See [Resync and Recovery](/bigquery/resync-and-recovery) for details. # Column Naming Source: https://support.detrics.io/bigquery/column-naming How Detrics normalizes column names to snake_case across all platforms **Last updated:** March 18, 2026 ## The Problem Every marketing platform uses its own naming convention for fields: ``` Google Ads: campaign.name, ad_group.name, metrics.clicks Meta Ads: campaign_name, adset_name, clicks GA4: sessionSourceMedium, totalUsers Pinterest: CAMPAIGN_NAME, TOTAL_IMPRESSION Shopify: orderCount, grossSales ``` Writing SQL across platforms means constantly remembering which convention each platform uses, and dealing with dots, camelCase, and inconsistent capitalization. ## The Solution Detrics normalizes all column names to **snake\_case** before loading data into BigQuery. This means every platform's fields follow the same convention, and you can write clean cross-platform SQL without worrying about naming differences. ## Normalization Rules Detrics applies these rules in order: | Rule | Before | After | | ------------------------------------- | ----------------- | ----------------- | | Dots become underscores | `campaign.name` | `campaign_name` | | camelCase splits | `campaignName` | `campaign_name` | | ALL\_CAPS lowercased | `CAMPAIGN_NAME` | `campaign_name` | | Acronyms lowered | `CTR` | `ctr` | | Letter-digit boundaries | `videoViewsP25` | `video_views_p25` | | Consecutive underscores collapsed | `campaign__name` | `campaign_name` | | Leading/trailing underscores stripped | `_campaign_name_` | `campaign_name` | | Digit-leading names prefixed | `7day_click` | `_7day_click` | ## Examples ### Before and After | Platform | Original Field | BigQuery Column | | ---------- | -------------------------------------- | -------------------------------------- | | Google Ads | `campaign.name` | `campaign_name` | | Google Ads | `metrics.cost_micros` | `metrics_cost_micros` | | Meta Ads | `campaign_name` | `campaign_name` | | GA4 | `sessionSourceMedium` | `session_source_medium` | | GA4 | `totalUsers` | `total_users` | | Pinterest | `CAMPAIGN_NAME` | `campaign_name` | | Pinterest | `TOTAL_IMPRESSION` | `total_impression` | | Shopify | `grossSales` | `gross_sales` | | Shopify | `orderCount` | `order_count` | | TikTok | `campaign_name` | `campaign_name` | | Meta Ads | `offsite_conversion.fb_pixel_purchase` | `offsite_conversion_fb_pixel_purchase` | ### Cross-Platform SQL With normalized column names, cross-platform queries become straightforward: ```sql theme={null} -- Compare spend across Meta Ads and Google Ads SELECT 'Meta Ads' as platform, campaign_name, SUM(spend) as total_spend FROM `project.dataset.meta_ads_campaigns` WHERE date >= '2026-03-01' GROUP BY campaign_name UNION ALL SELECT 'Google Ads' as platform, campaign_name, SUM(spend) as total_spend FROM `project.dataset.google_ads_campaigns` WHERE date >= '2026-03-01' GROUP BY campaign_name ORDER BY total_spend DESC ``` ## System Columns System columns added by Detrics always use the `_detrics_` prefix and are already in snake\_case: * `_detrics_account_id` * `_detrics_row_id` * `_detrics_sync_id` * `_detrics_synced_at` These are never affected by normalization rules and are consistent across all tables. ## Type Mapping In addition to name normalization, Detrics maps platform data types to BigQuery-native types: | Detrics Type | BigQuery Type | | -------------- | ------------- | | string | STRING | | float, float64 | FLOAT64 | | integer, int64 | INT64 | | date | DATE | | datetime | DATETIME | | timestamp | TIMESTAMP | | boolean | BOOL | | percent | FLOAT64 | | currency | FLOAT64 | | number | FLOAT64 | # Connect to Claude Source: https://support.detrics.io/bigquery/connect-to-claude Connect your Detrics BigQuery warehouse to Claude and ask questions about all your marketing data in plain language **Last updated:** August 10, 2026 Once your marketing data lives in BigQuery, you can connect it to **Claude** and ask questions about it in plain language. No SQL, no exports. Claude reads the real rows in your warehouse and answers across every platform, account, and country at once. It is easy to set up, and you only do it once. ## How to connect In your GCP project (with the BigQuery API enabled), go to **Google Auth Platform → Clients → Create client** and choose **Web application**. * Authorized JavaScript origins: `https://claude.ai` * Authorized redirect URI: `https://claude.ai/api/mcp/auth_callback` Copy the **Client ID** and **Client Secret**. Open Claude's connector directory, select **Google Cloud BigQuery**, and paste the Client ID and Client Secret. Sign in with a Google account that has access to your BigQuery project and approve. Claude is now connected to your warehouse and ready to answer. That's the whole setup. From here, just ask. ## What you can do * Ask questions in plain language across all your platforms, accounts, and countries. * Let Claude write and run the SQL for you, then explain the result. * Walk through multi-step analysis, where each answer builds on the last. * Draft dashboards, reports, and summaries straight from live data. * Investigate trends, anomalies, and "why did this change" questions in one conversation. ## Manage your warehouse from Claude Beyond querying, Claude can now build and run your warehouse for you through the Detrics MCP server. Create destinations, build table groups, schedule and run transfers, and query your data, all in natural language. * **Create destinations.** Set up a BigQuery destination with guided GCP setup, connection tests, and sharing. * **Build table groups.** Create table groups and tables from pre-built presets, with safe schema updates. * **Schedule and run transfers.** Create, pause, resume, and trigger syncs, with per-table sync settings. Full reference of the MCP tools that let Claude manage your BigQuery warehouse. ## Why this matters ### The problem with scattered data When your numbers live across dozens of spreadsheets and platform dashboards, a few things break: 1. **No single source of truth.** The same metric shows different values depending on which file you open. 2. **Misaligned teams.** Marketing, BI, and finance each work from their own copy and reach different conclusions. 3. **Incomplete reports.** Pulling everything together by hand is slow, so reports end up partial and you lose the big picture. 4. **Slow decisions.** By the time the data is gathered and cleaned, the moment to act has passed. 5. **Unreliable AI answers.** An AI with no grounded data guesses and hallucinates, which leads to bad calls dressed up as confident ones. ### The solution: one centralized warehouse Detrics syncs every platform and account into a single BigQuery warehouse. The data arrives normalized, deduplicated, and refreshed on the schedule you set. That gives every team and every tool one consistent, current source to work from. ### Why connecting it to Claude is powerful A centralized warehouse is the foundation. Claude on top of it is the leverage: * **Grounded answers.** Claude reads your actual rows, so its reasoning is anchored to real numbers instead of guesses. * **Chain of reasoning.** It can break a hard question into steps, query each one, and combine the results into an answer you can trust. * **Dashboards on demand.** Describe the view you want and Claude builds the query and the breakdown for you. * **Real-time decisions.** Ask a question and get the answer now, against data that is already up to date. ### Why Detrics The hard part is not Claude. It is getting clean, unified, current data into one place. Detrics handles the platform authentication, the API maintenance, the column normalization, the deduplication, and the scheduled refresh. Claude always queries data that is complete and trustworthy, so the answers are too. New to the Data Warehouse? Start here to get your data into BigQuery, then come back to connect Claude. # Creating Transfers Source: https://support.detrics.io/bigquery/creating-transfers Connect a table group to a destination with a schedule and start syncing data **Last updated:** March 18, 2026 A **transfer** is the core of the Detrics Data Warehouse pipeline. It binds together: * A **data source** (your platform connection and accounts) * A **table group** (what data to fetch) * A **destination** (where to put it) * A **schedule** (when to sync) ## Creating a Transfer 1. Go to [**Data Warehouse → Transfers**](https://app.detrics.io/data-warehouse/transfers) 2. Click [**New Transfer**](https://app.detrics.io/data-warehouse/transfers/new) 3. Follow the steps below ### Step 1: Select Data Source Choose the platform connection to use. This is the same OAuth connection you set up in Detrics for Google Sheets or Looker Studio, no need to reconnect. ### Step 2: Select Accounts Choose which platform accounts to sync: * **All accounts**: Syncs every account available under the connection. If new accounts are added to the connection later, they're automatically included * **Selected accounts**: Pick specific accounts from the list For platforms with a single account per connection (Shopify, Stripe, WooCommerce, Klaviyo, TiendaNube, HubSpot, TikTok Organic), account selection is skipped, there's only one account to sync. **All accounts** is recommended for most use cases. Each row in BigQuery includes `_detrics_account_id`, so you can always filter to specific accounts in your SQL queries. This way, new accounts are automatically picked up without modifying the transfer. ### Step 3: Select Table Group Choose an existing table group or create a new one. The table group must match the platform you selected in Step 1 (e.g., a Meta Ads table group for a Meta Ads connection). ### Step 4: Choose a Sync Strategy First, choose a **sync strategy** that sets the initial sync mode for all tables in the transfer: | Strategy | What It Does | Best For | | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | | **Replicate Source** (recommended) | Applies smart defaults based on each table's configuration, time-series tables with daily aggregation get **Incremental**, snapshot and non-daily tables get **Full Refresh** | Most use cases. Detrics picks the right mode per table automatically | | **Full Refresh All** | Sets every table to **Full Refresh** mode | When you want a clean, complete replacement of all data on every sync | | **Append Full Snapshots** | Sets every table to **Full Append** mode | When you want to track how all data changes over time by keeping every snapshot | **Replicate Source** is recommended for most transfers. It uses platform-specific logic to assign the optimal sync mode to each table, for example, daily ad performance tables get Incremental with a platform-appropriate refresh window, while campaign settings tables get Full Refresh. #### Customize per-table sync modes After selecting a strategy, you can override the sync mode for individual tables. The three sync modes are: * **Incremental**: Best for daily time-series data (spend, impressions, clicks). Deduplicates by date using a refresh window * **Full Refresh**: Best for snapshot/reference data (campaign settings, product catalogs). Replaces the table on each sync * **Full Append**: Best for tracking changes over time (metric evolution, audit trails). Appends without removing For tables assigned Incremental mode, you'll also configure the **refresh window** (how many days to re-fetch on each sync). See [Sync Modes](/bigquery/sync-modes) for guidance on choosing the right mode. ### Step 5: Select Destination Choose which BigQuery destination to write data to. The destination must have a **Connected** status (pass the connection test). ### Step 6: Configure Schedule Set how often the transfer runs: | Setting | Options | | ---------------- | ------------------------------------------------- | | **Frequency** | Every 1h, 2h, 3h, 6h, 12h, Daily, Weekly, Monthly | | **Hour of day** | 0-23 (for daily, weekly, monthly frequencies) | | **Day of week** | Sun-Sat (for weekly frequency) | | **Day of month** | 1-31 (for monthly frequency) | | **Timezone** | Set on the destination | See [Scheduling & Refresh Windows](/bigquery/scheduling-and-refresh-windows) for guidance on choosing a frequency. ### Step 7: Additional Settings * **Transfer name**: A descriptive name (e.g., "Meta Ads - Daily Campaign Metrics") * **Table name prefix**: Optional prefix added to all BigQuery table names in this transfer (e.g., `meta_ads_` makes tables like `meta_ads_campaign_performance`). If not set, a default prefix based on the platform is used ### Step 8: Create Click **Create Transfer**. Detrics immediately starts the **initial sync**, which fetches historical data based on each table's configured historical sync range. ## After Creation ### Initial Sync The first sync fetches historical data. Depending on the number of tables, accounts, and historical range, this can take from a few minutes to over an hour for very large syncs. You can monitor progress in real time on the transfer detail page: * Which table is currently syncing * How many rows have been loaded * Progress through chunks (for large syncs) ### Subsequent Syncs After the initial sync, the transfer runs on schedule: * **Incremental tables** fetch only the refresh window (e.g., last 3 days) * **Full Refresh tables** re-fetch the full dataset * **Full Append tables** append the full dataset ## Transfer Status | Status | Meaning | | ----------- | ---------------------------------------------------------------------------------- | | **Active** | Transfer is running on schedule | | **Paused** | Transfer is paused, no syncs will run until resumed | | **Pausing** | A pause was requested but a sync is currently running, will pause after completion | | **Error** | Transfer encountered a system error, check the detail page for specifics | ### Pausing and Resuming * **Pause**: Stops scheduled syncs. Data in BigQuery is preserved as-is * **Resume**: Restarts the schedule. The next sync runs at the next scheduled time (or immediately if the scheduled time has passed) ### Auto-Pause If a transfer fails **3 consecutive times**, it automatically pauses to prevent unnecessary API calls and error noise. You'll receive an alert (if configured). Check the error details, fix the issue (e.g., re-authenticate the connection, fix BigQuery permissions), then resume. ## Organizing with Folders You can organize transfers into folders: * Create folders from the transfer list page * Drag transfers between folders * Nest folders for hierarchical organization This is purely for organization in the Detrics UI, folders don't affect how transfers run or where data lands in BigQuery. ## Sharing Transfers Share a transfer with other workspace members: * **View only**: Can see the transfer status and run history * **Edit**: Can modify settings, trigger syncs, and pause/resume Go to the transfer detail page and click **Share** to manage access. ## Alerts Configure email alerts per transfer: | Alert Type | When It Fires | | -------------------- | ------------------------------------------------------------------ | | **On failure** | When a sync fails (any table) | | **On success** | When a sync completes successfully | | **On schema change** | When BigQuery table schema is modified (new columns added/removed) | Add one or more email addresses to receive alerts. This is useful for monitoring transfers you don't check daily. # Custom Fields Source: https://support.detrics.io/bigquery/custom-fields Discover and sync platform-specific custom dimensions, conversions, and properties **Last updated:** March 18, 2026 ## What Are Custom Fields? Most platforms offer a standard set of metrics and dimensions that every account has access to (e.g., `spend`, `impressions`, `campaign_name`). But some platforms also support **custom fields**, user-defined metrics or dimensions specific to individual accounts. Examples of custom fields: * **Google Analytics 4**: Custom dimensions and custom metrics you've defined in your GA4 property * **Meta Ads**: Custom conversions, custom events, and offline event sets * **Klaviyo**: Custom conversion metrics tied to your flows and campaigns * **HubSpot**: Custom properties defined on your contacts, companies, deals, and other CRM objects Detrics can discover these fields per account and sync them alongside standard fields in your BigQuery tables. ## How Custom Field Discovery Works 1. When editing a table, click **Load Custom Fields** in the field picker 2. Select an account from the dropdown, custom fields are discovered per account since each account may have different custom fields 3. Detrics queries the platform API to discover available custom fields 4. Custom fields appear in the field picker alongside standard fields, marked with a special badge 5. Select the custom fields you want to include in your table The account you select for discovery is only used to find available fields, it doesn't affect which accounts the transfer syncs. When the transfer runs, Detrics syncs all selected accounts (or all accounts, depending on your setting). ## Multi-Account Handling When a transfer syncs multiple accounts, some accounts may have custom fields that others don't. Detrics handles this gracefully: * If Account A has custom dimension `user_segment` but Account B doesn't, the column still exists in BigQuery * Rows from Account A will have values for `user_segment` * Rows from Account B will have `NULL` for that column * BigQuery handles sparse columns natively, no performance impact This means you don't need to worry about field compatibility across accounts. Include whatever custom fields you need, and Detrics will populate them where available. ## Supported Platforms ### Google Analytics 4 GA4 supports custom dimensions and custom metrics that you define in your property settings. These appear in Detrics with their configured parameter name. Custom dimensions commonly used: * User-scoped dimensions (e.g., `user_type`, `membership_level`) * Event-scoped dimensions (e.g., `content_category`, `product_brand`) Custom metrics commonly used: * Event count metrics (e.g., `custom_purchase_value`, `video_completions`) ### Meta Ads Meta Ads supports custom conversions that track specific actions on your website or app. These appear as additional conversion metrics in the field picker. Examples: * Custom conversion events mapped to specific pixel events * Offline conversion sets ### Klaviyo Klaviyo supports custom conversion metrics tied to your flows and campaigns. These track specific revenue or engagement actions. ## Custom Fields and Column Naming Custom fields follow the same [column naming rules](/bigquery/column-naming) as standard fields. Field names are normalized to snake\_case in BigQuery. For example, a GA4 custom dimension called `UserSegment` becomes `user_segment` in your BigQuery table. # FAQ Source: https://support.detrics.io/bigquery/faq Frequently asked questions about Detrics Data Warehouse **Last updated:** March 18, 2026 ## General Currently, Detrics supports **Google BigQuery** as a destination. BigQuery is the most widely used cloud data warehouse for marketing analytics, and Detrics is optimized for deep integration with it. With the **Standard Setup**, no, when you create a destination and specify a dataset name, Detrics creates the dataset automatically in your GCP project (if it doesn't already exist). You just need to make sure the Detrics service account has the right permissions. With [**Least-Privilege Access**](/bigquery/least-privilege-access), you do pre-create the dataset yourself, since Detrics is not granted project-wide dataset-creation rights. Every platform supported by Detrics can be synced to BigQuery. See the [Fields Catalog](/fields-catalog) for the full list of platforms and the metrics and dimensions available for each. Yes. The Data Warehouse uses the same OAuth connections you've set up for Google Sheets or Looker Studio. No need to re-authenticate your platforms. The Data Warehouse is available on select plans. Check [Pricing & Plans](/pricing) for current details. ## Data & Syncing You choose the schedule when creating a transfer. Options range from **every hour** to **monthly**. You can also trigger a manual sync at any time. You can configure the historical sync range per table: 1 month, 3 months, 6 months, 12 months, 24 months, or all time. The available history depends on what the platform API provides, some platforms limit historical data access. If a sync fails, the transfer shows an error status with details about what went wrong (e.g., expired token, BigQuery permission issue, platform API error). Your existing data in BigQuery is not affected, failed syncs don't delete or corrupt previously loaded data. After 3 consecutive failures, the transfer automatically pauses to prevent unnecessary API calls. Yes. When creating a transfer, you can choose **All accounts** (syncs every account under that connection) or **Selected accounts** (pick specific ones). Each account's data is tagged with `_detrics_account_id` so you can filter by account in your SQL queries. It depends on the sync mode: * **Incremental**: Detrics deletes rows within the refresh window for the synced accounts, then inserts fresh data. This prevents duplicates while preserving data outside the window. * **Full Refresh**: The entire table is replaced on each sync, so duplicates are impossible. * **Full Append**: Rows are appended without deduplication. If you re-sync the same date range, you will get duplicate rows. This mode is designed for use cases where you want to track snapshots over time. Nothing. Your data in BigQuery stays exactly as it is. Pausing a transfer simply stops scheduled syncs. When you resume, the next sync picks up where it left off based on the refresh window. **No.** Detrics will never modify, delete, or overwrite a table it didn't create. Before writing to any table, Detrics checks for the presence of its four [system columns](/bigquery/system-columns) (`_detrics_account_id`, `_detrics_row_id`, `_detrics_sync_id`, `_detrics_synced_at`). If a table exists but doesn't have these columns, Detrics treats it as a user-created table and **refuses to write to it**, the sync will fail with a clear error message asking you to rename the existing table or change the Detrics table prefix. This means it's completely safe to use a shared dataset that contains both Detrics-managed tables and your own tables, views, or materialized queries. Detrics will only ever touch tables that have its system columns. Detrics will detect the conflict and **block the sync** rather than overwrite your data. You'll see a `TABLE_NOT_MANAGED` error explaining that the table exists but wasn't created by Detrics. To resolve the conflict, you have two options: * **Rename your existing table** in BigQuery (e.g., add a `_backup` suffix), then re-run the sync * **Change the table prefix in Detrics**: each transfer has a configurable prefix (defaults to the platform name, e.g., `facebook_ads_`, `ga4_`). Set a custom prefix to avoid the collision See [Troubleshooting → Table Not Managed by Detrics](/bigquery/troubleshooting#table-not-managed-by-detrics) for more details. Yes. You can create multiple transfers using the same table group, each pointing to a different destination. This is useful for maintaining separate production and staging datasets. ## BigQuery & Costs No. BigQuery costs are billed directly by Google through your GCP project. Detrics handles the data pipeline; Google bills you for storage and query compute. For most marketing use cases, BigQuery costs are minimal, often under \$10/month. **Incremental** is significantly cheaper for ongoing syncs. It only touches a small window of data (e.g., the last 3 days), which means lower query and storage costs compared to Full Refresh (which replaces the entire table each time). For a table with 12 months of history, an incremental sync with a 3-day window processes roughly 0.8% of the data compared to a full refresh. No. BigQuery does not charge for loading data via the streaming or load APIs. You're only charged for storage and queries you run against the data. Yes. When creating a destination, you choose the **data location**, either a multi-region (US, EU) or a specific region (e.g., `us-east1`, `europe-west1`). This is permanent for that dataset and cannot be changed after creation. ## Tables & Fields Yes. You can edit a table at any time to add new metrics/dimensions or remove existing ones. On the next sync, Detrics will migrate the BigQuery table schema, adding new columns and preserving existing data. Removed fields will no longer receive new data but existing values are preserved. Premade table groups are presets with pre-configured metrics and dimensions for common use cases. They come in three tiers: * **Light**: Essential metrics only, minimal table count * **Standard**: Recommended for most use cases, balanced coverage * **Enhanced**: Comprehensive coverage with detailed breakdowns You can customize any premade group after creating it. All column names are normalized to **snake\_case**. This means `campaignName` becomes `campaign_name`, `campaign.name` becomes `campaign_name`, and `CTR` becomes `ctr`. This ensures consistent naming across all platforms so you can write clean SQL without worrying about each platform's conventions. See [Column Naming](/bigquery/column-naming) for the full rules. Some platforms support user-defined fields that aren't part of the standard catalog. For example, GA4 custom dimensions, Meta Ads custom conversions, or Klaviyo custom metrics. Detrics can discover these fields per account and sync them alongside standard fields. See [Custom Fields](/bigquery/custom-fields) for details. ## Transfers & Scheduling The new schedule takes effect on the next scheduled run. If you change from daily to hourly, the transfer will start running hourly from the next scheduled time. Yes. On any active transfer's detail page, click the **Sync Now** button. This triggers an immediate run without affecting the regular schedule. A resync re-fetches data from the platform, optionally clearing existing data first. Use it when: * You added new fields to a table and want historical data for those fields * You suspect data quality issues and want a clean reload * You changed filters and want to re-fetch with the new criteria See [Resync & Recovery](/bigquery/resync-and-recovery) for details. The timezone is set on the destination. All transfers using that destination follow its timezone. Schedules are interpreted in the destination's timezone, so "Daily at 6:00 AM" means 6:00 AM in that timezone, even when DST changes happen. Detrics handles DST transitions automatically. # Filter Overrides Source: https://support.detrics.io/bigquery/filter-overrides Customize table filters per transfer without duplicating table definitions **Last updated:** March 18, 2026 ## What Are Filter Overrides? Filter overrides let you **reuse the same table group** across multiple transfers with different filtering criteria, without duplicating the table definitions. ### The Problem Suppose you have a "Campaign Performance" table with metrics like spend, impressions, and conversions. You want three transfers: * One for **Brand** campaigns (campaign name contains "Brand") * One for **Performance** campaigns (campaign name contains "Perf") * One that syncs **all campaigns** (no filter) Without filter overrides, you'd need to create three nearly identical table groups that differ only in their filters. ### The Solution With filter overrides, you create **one table group** with no filters (or with default filters), then override the filters at the transfer level: ``` Table Group: "Campaign Performance" └── Table: campaign_metrics (no filter) Transfer 1: "Brand Campaigns" └── Override: campaign_name CONTAINS "Brand" Transfer 2: "Performance Campaigns" └── Override: campaign_name CONTAINS "Perf" Transfer 3: "All Campaigns" └── No override (uses table's default — no filter) ``` All three transfers use the same table group but sync different subsets of data. ## How It Works * Filter overrides are set per **table** per **transfer**, you can override filters for some tables in a group while leaving others at their defaults * An override **replaces** the table's filters entirely (it doesn't merge with existing filters) * If no override is set, the table's own filters are used * Overrides do not modify the original table definition, other transfers using the same table group are unaffected ## Setting Up Filter Overrides 1. Go to a transfer's detail page 2. Navigate to the **Tables** tab 3. Click on a table to see its current filter configuration 4. Click **Override Filters** to set transfer-specific filters 5. Define your custom filter criteria using the same operators available in table configuration The transfer detail page shows which tables are using default filters and which have overrides applied, marked with a distinct badge. ## Use Cases ### Multi-Brand Agency An agency managing campaigns for multiple brands under one ad account: * Table group defines the metrics and dimensions * Each transfer filters by brand name * Each transfer points to a different destination (one BigQuery dataset per brand) ### Campaign Type Segmentation A marketing team that wants separate tables for different campaign objectives: * One transfer for prospecting campaigns * One transfer for retargeting campaigns * One transfer for all campaigns (for totals) ### Regional Segmentation A global team syncing the same metrics but filtered by country or region: * Transfer for LATAM markets * Transfer for US market * Transfer for EU markets ## Removing an Override To remove an override and revert to the table's default filters, go to the table's override settings and click **Remove Override**. The next sync will use the table's original filter configuration. # Glossary Source: https://support.detrics.io/bigquery/glossary Key terms and definitions for Detrics Data Warehouse **Last updated:** March 18, 2026 ## Core Terms ### Destination A connection to a BigQuery dataset where your synced data is stored. A destination includes a GCP project ID, dataset name, data location (region), and timezone. You can have multiple destinations for different use cases (e.g., production vs. staging). ### Table Group A collection of related tables for a single platform. For example, a "Meta Ads - Performance" table group might contain tables for campaign-level metrics, ad-level metrics, and audience breakdowns. Table groups can be created from **premade presets** or built from scratch. ### Table An individual data definition within a table group. Each table specifies: * Which **metrics** and **dimensions** to fetch * The **time aggregation** (daily, weekly, monthly, or total) * Any **filters** to narrow the data * The **historical sync range** (how far back to fetch on the first sync) A table maps directly to one BigQuery table. The **sync mode** for each table is assigned when creating a transfer, see [Transfer](#transfer). ### Transfer The scheduled pipeline that connects a table group to a destination. A transfer defines: * Which **data source** (platform connection) to use * Which **accounts** to sync (all or selected) * The **sync mode** for each table (incremental, full refresh, or full append) * The **schedule** (how often to sync) * The **destination** (where to put the data) One transfer syncs one platform's data. To sync Meta Ads and Google Ads, you create two transfers. ### Transfer Run A single execution of a transfer. Each run fetches data, processes it according to the sync mode assigned to each table, and loads it into BigQuery. Runs can be triggered by the schedule, manually, or as part of a resync. ### Sync Mode How data is written to and maintained in a BigQuery table. Detrics supports three sync modes: * **Incremental**: Append and deduplicate by date * **Full Refresh**: Replace the entire table * **Full Append**: Append without deduplication See [Sync Modes](/bigquery/sync-modes) for full details. ### Refresh Window For **Incremental** sync mode only. The number of days Detrics re-fetches and deduplicates on each sync. For example, a 3-day refresh window means each sync fetches the last 3 days and replaces those rows in BigQuery. This handles late-arriving data and attribution adjustments. ### Historical Sync Range How far back Detrics fetches data on the **first sync** of a transfer. Options range from 1 month to all time. After the initial sync, only the refresh window (incremental) or full dataset (full refresh/append) is fetched on each run. *** ## BigQuery Terms ### Dataset A BigQuery container that holds tables. When you create a destination, you specify a dataset name. If it doesn't exist, Detrics creates it in your GCP project. ### Data Location The geographic region where your BigQuery data is physically stored. Options include multi-region locations (US, EU) or specific regions (us-east1, europe-west1, etc.). Once set, a dataset's location cannot be changed. ### Service Account A Google Cloud identity that Detrics uses to access your BigQuery project. You grant this service account the **BigQuery User** role so Detrics can create datasets, tables, and load data. *** ## Data Terms ### Metrics Quantitative values that can be summed or averaged. Examples: `spend`, `impressions`, `clicks`, `conversions`, `revenue`, `sessions`. ### Dimensions Categorical or descriptive values used to group and filter data. Examples: `campaign_name`, `date`, `ad_name`, `country`, `device`, `product_title`. ### Time Aggregation How data is grouped over time within a table: * **Daily**: One row per day per dimension combination * **Weekly**: One row per ISO week * **Monthly**: One row per calendar month * **Hourly**: One row per hour (where supported by the platform) * **Total**: No time breakdown; aggregated across the entire date range ### Filters Conditions applied to narrow the data fetched from a platform. For example, filtering campaigns where `campaign_name CONTAINS "Brand"` to sync only branded campaigns. Filters can be set at the table level or overridden per transfer. ### Filter Override A transfer-level setting that replaces a table's default filters with custom filters for that specific transfer. Useful when you want to reuse the same table definition across multiple transfers with different filtering criteria. ### Custom Fields Platform-specific fields that aren't part of the standard field catalog. Examples include GA4 custom dimensions, Meta Ads custom conversions, and Klaviyo custom metrics. These are discovered per account and synced alongside standard fields. *** ## System Terms ### System Columns Columns automatically added by Detrics to every BigQuery table: * `_detrics_account_id`: The platform account ID that owns each row * `_detrics_row_id`: A content hash for deduplication * `_detrics_sync_id`: The ID of the transfer run that created the row * `_detrics_synced_at`: Timestamp of when the row was loaded See [System Columns](/bigquery/system-columns) for details. ### Sync Log A metadata table (`_detrics_sync_log`) automatically created in your BigQuery dataset. Records one row per table per transfer run, including status, row count, duration, and error details. Useful for monitoring and auditing. ### Resync A manual operation that re-fetches data for a transfer. Two options: * **Resync (retain data)**: Re-fetch the configured date range without deleting existing data outside that range * **Resync (remove data)**: Delete all existing data for the transfer's accounts, then re-fetch from scratch ### Initial Sync The first run of a new transfer. Fetches historical data based on each table's configured historical sync range. Subsequent runs use the refresh window (incremental) or re-fetch the full dataset (full refresh/append). ### Chunk For large syncs, Detrics splits the work into smaller pieces called chunks, by time range, by account batch, or both. Chunking prevents timeouts and allows progress tracking. If a sync is interrupted, completed chunks are preserved. ### Image Persistence An optional feature that downloads images from temporary platform URLs (e.g., ad creative thumbnails) and stores them in a Google Cloud Storage bucket. This gives you permanent URLs that don't expire, unlike the originals which typically expire within 24 hours. # How Detrics Syncs Work Source: https://support.detrics.io/bigquery/how-detrics-syncs-work Understand the Detrics data pipeline from platform APIs to your BigQuery tables **Last updated:** March 18, 2026 ## The Detrics Approach Detrics follows an **ETL (Extract, Transform, Load)** model for data warehousing. Unlike tools that dump raw API responses into your warehouse and ask you to clean them up, Detrics gives you full control over what gets extracted and how it's structured before it lands in BigQuery. You decide: * **Which metrics and dimensions** to include in each table * **Which filters** to apply at the source * **How to aggregate over time** (daily, weekly, monthly, or total) * **Which sync strategy** to use for each table This means every table in your warehouse contains exactly the data you asked for, clean, normalized, and ready to query. ## The Sync Pipeline When a transfer runs, whether on schedule or triggered manually, Detrics executes a five-stage pipeline: ### Stage 1: Configuration Resolution Detrics loads the transfer's configuration: which tables to sync, which accounts, which filters (including any per-transfer filter overrides), the sync mode assigned to each table, and refresh window settings. ### Stage 2: Data Extraction For each table in the transfer, Detrics calls the platform's API to fetch the requested metrics and dimensions. The date range depends on the sync mode and whether this is the initial sync or a subsequent run: **Initial sync** (first run after creating a transfer): * All sync modes fetch the full **historical sync range** configured on each table (e.g., last 3 months, last 12 months, or all time). This backfills your BigQuery tables with historical data. **Subsequent syncs:** * **Incremental**: Fetches only the last N days (the refresh window), deduplicates, and preserves everything outside the window * **Full Refresh**: Fetches the full configured date range and replaces the table * **Full Append**: Fetches the full configured date range and appends a new snapshot For large syncs (many accounts, long date ranges, or detailed entity levels like individual ads), Detrics automatically **chunks** the work, splitting by time range and account batches to prevent timeouts and manage memory. ### Stage 3: Transformation Before loading into BigQuery, Detrics normalizes the data: * **Column naming**: All field names are converted to `snake_case` for consistent SQL across platforms. See [Column Naming](/bigquery/column-naming) * **System columns**: Four columns are added to every row: `_detrics_account_id`, `_detrics_row_id`, `_detrics_sync_id`, and `_detrics_synced_at`. See [System Columns](/bigquery/system-columns) * **Type mapping**: Platform data types are mapped to BigQuery-native types (STRING, FLOAT64, INT64, DATE, TIMESTAMP, BOOL) * **Row identity**: A SHA-256 hash is computed from each row's content for deduplication tracking ### Stage 4: Schema Migration If this is the first sync for a table, Detrics creates the BigQuery table with the appropriate schema. On subsequent syncs, if you've added or removed fields, Detrics migrates the schema, adding new columns and preserving existing data. ### Stage 5: Data Loading Data is loaded into BigQuery according to the sync mode assigned to each table in the transfer: | Sync Mode | What Happens | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Incremental** | Delete rows in the refresh window for the synced accounts, then insert fresh data. This is done atomically, if the insert fails, the delete is rolled back | | **Full Refresh** | Replace the entire table with the new data in one atomic operation | | **Full Append** | Append new rows to the existing table without modifying or removing anything | After loading, Detrics records the results in the [sync log](/bigquery/system-columns#sync-log) metadata table and updates the transfer's status. ## Platform Focus Detrics is built specifically for the platforms used by modern digital marketing teams. This focus allows for deeper integration than general-purpose ETL tools: ### Advertising Platforms Meta Ads, Google Ads, TikTok Ads, LinkedIn Ads, X Ads, Pinterest Ads, Bing Ads, Mercado Ads, Google DV360, with support for attribution windows, conversion tracking, and ad-level creative data. ### Analytics & SEO Google Analytics 4 and Google Search Console, with support for custom dimensions and property-level configuration. ### Social Media Instagram Insights, Facebook Insights, Facebook Public Data, TikTok Organic, organic performance metrics alongside paid data. ### Email Marketing Klaviyo and Mailchimp, campaign performance, flow analytics, and subscriber metrics. ### CRM HubSpot, contacts, deals, companies, and other entities with custom properties. ### E-commerce Shopify, TiendaNube, WooCommerce, Google Merchant Center, orders, products, inventory, and storefront data. ### Payments Stripe, charges, subscriptions, and payment analytics. ## Chunked Execution Large syncs are automatically split into smaller chunks to ensure reliability. Chunking happens along two axes: * **Time range**: Long date ranges are split into monthly segments * **Account batches**: Many accounts are processed in smaller groups The chunk strategy varies by platform and entity level. For example, Meta Ads at the ad level uses smaller chunks (1 month × 5 accounts) because ad-level data is denser, while campaign-level data uses larger chunks (12 months × 25 accounts). Each completed chunk is persisted immediately. If a sync is interrupted (timeout, transient error), completed chunks are preserved, only the remaining chunks need to be retried. ## What Detrics Adds to Your Data Every table in your warehouse includes four system columns managed by Detrics: | Column | Description | | --------------------- | -------------------------------------------------------------------------- | | `_detrics_account_id` | The platform account ID that owns each row (e.g., your Meta ad account ID) | | `_detrics_row_id` | A content hash for deduplication tracking | | `_detrics_sync_id` | The transfer run ID that created the row | | `_detrics_synced_at` | Timestamp of when the row was loaded into BigQuery | These columns let you filter by account, track data freshness, and join with the sync log for auditing. See [System Columns](/bigquery/system-columns) for details. # Image Persistence Source: https://support.detrics.io/bigquery/image-persistence Store permanent copies of ad creative images from platforms with expiring URLs **Last updated:** March 18, 2026 ## The Problem with Ephemeral URLs Some marketing platforms return **temporary URLs** for images and creative assets. Meta Ads, TikTok, and Instagram generate CDN-signed URLs that typically expire within 24 hours. This means: * Image URLs in your BigQuery data stop working after a day * If you use these URLs in dashboards or reports, the images break * Every API call returns a new URL for the same image, even if the content hasn't changed ## How Image Persistence Works Image Persistence is an optional feature that gives you **permanent URLs** for ad creative images. When enabled on a destination: 1. After a transfer sync completes, Detrics identifies all image URL fields in the synced data 2. Each image is downloaded from the platform's CDN 3. The image is compressed to an optimized WebP format (approximately 400px wide, \~14KB per image) 4. A content hash is computed, if the same image already exists in your bucket, it's not re-uploaded 5. The image is uploaded to a Google Cloud Storage (GCS) bucket in your project 6. The temporary URL in your BigQuery table is replaced with a permanent `https://storage.googleapis.com/...` URL ### What Gets Processed Currently, image persistence supports these fields from **Meta Ads**: * `ad_image_url`: The ad's main creative image * `ad_image`: Image references embedded in creative data Support for additional platforms (TikTok, Instagram) is planned. ## Enabling Image Persistence 1. Go to [**Data Warehouse → Destinations**](https://app.detrics.io/data-warehouse/destinations) 2. Open the destination where you want to enable image persistence 3. Click **Enable Image Bucket** 4. Detrics will create a GCS bucket in your project named `detrics-images-{workspace-id}` 5. Grant the Detrics service account **Storage Object Admin** role on the bucket (or project-wide) The Detrics service account needs the **Storage Object Admin** IAM role to upload images to GCS. This is in addition to the BigQuery roles already granted. The steps above describe the **Standard Setup**. Under **[Least-Privilege Access](/bigquery/least-privilege-access)** you pre-create the bucket yourself and grant a narrower **Storage Object User** role scoped to that bucket, see that guide for the exact steps. ## Storage Costs Image persistence is designed to be cost-effective: * Images are compressed to WebP format (\~14KB per image) * Content-based deduplication means the same image is stored only once, even if it appears in multiple rows or across accounts * Typical monthly cost for a large account: under \$1/month for storage ## How It Affects Deduplication Image URL fields are marked as **ephemeral** in Detrics. This means: * URL changes don't affect the `_detrics_row_id` hash * When a platform returns a new temporary URL for the same image, Detrics doesn't count it as a row change * This prevents unnecessary row updates in Incremental sync mode This behavior applies regardless of whether Image Persistence is enabled. Image Persistence adds the permanent URL replacement on top of the dedup exclusion. ## Disabling Image Persistence You can disable Image Persistence on a destination at any time. Existing images in the GCS bucket are preserved, and their URLs in BigQuery continue to work. New syncs will write the platform's ephemeral URLs instead of permanent ones. # Interactive Setup Demo Source: https://support.detrics.io/bigquery/interactive-demo Click through the full BigQuery setup flow at your own pace