Add Google BigQuery to external database connections
74
Views
0
Comments
New
Data & Integrations

We adopted Google BigQuery as the central database for storing, processing, and analyzing large volumes of information. Currently, the integration between OutSystems and BigQuery depends on custom solutions (via APIs or external drivers), which leads to:

  • Increased development and maintenance effort;

  • Risks related to data security and governance;

  • Loss of efficiency in delivering critical business projects.

A native integration is essential to ensure alignment with market best practices, simplify development workflows, and enable the scalability of ongoing digital solutions.

Proposed Technical Requirements

Authentication and Authorization

  • Support for OAuth 2.0 (end-user consent via Google Identity).

  • Support for Service Accounts (server-to-server integrations, preferably with temporary credentials).

IAM Permission Management

  • Compatibility with the minimum required roles (roles/bigquery.dataViewer, roles/bigquery.user, roles/bigquery.jobUser).

Connection Layer

  • Consumption of official APIs (REST/gRPC).

  • Alternatively, support for official JDBC/ODBC drivers.

Query Execution

  • Synchronous and asynchronous queries (jobs).

  • Result pagination handling.

  • Monitoring of running jobs with jobId return.

Data Type Compatibility

  • Support for primitive types (STRING, INT64, FLOAT64, NUMERIC, TIMESTAMP).

  • Complex structures (STRUCT, ARRAY) with JSON fallback when necessary.

Network and Security

  • Connections via HTTPS/TLS 1.2+.

  • Support for Private Google Access and VPC Service Controls.

Limits and Quotas Handling

  • Retry with exponential backoff for transient errors.

  • Configurable concurrency control.

Monitoring and Auditing

  • Logging of integration events.

  • Compatibility with Cloud Audit Logs.

Expected Impact

For Equatorial, the availability of this functionality means:

  • Reduced effort and development time in critical integrations;

  • Increased security in accessing sensitive data;

  • Improved performance in analytical queries;

  • Assurance of compliance with established governance and regulatory practices.


Our ask is for a native connection, same way we have in the Connections tab in the ODC Portal, to connect with the Google BigQuery database.