Datasource APIs
Create a datasource
Connect a warehouse or database to Databrain with its credentials.
Databrain tests the connection, saves the datasource and reads its schema. The call returns when the schema is cached.
curl --request POST https://api.usedatabrain.com/api/v2/datasource \ --header "Authorization: Bearer $DATABRAIN_SERVICE_TOKEN" \ --header "Content-Type: application/json" \ --data '{ "datasourceType": "postgres", "credentials": { "name": "Sales warehouse", "host": "db.example.com", "port": 5432, "username": "databrain_reader", "password": "<password>", "database": "analytics", "schema": "public", "sslMode": "require" } }'Auth
A service token. See Authentication.
Limits
- 10 requests in 5 minutes per organization, shared with update, sync and schema discovery. Over the limit you get
429with a plain-text body. - The host must be a public address. Databrain cloud can refuse loopback, private, link-local and metadata addresses.
ApiTokenAuthorizationBearer <token>A service token or a data app API token.
Body
One of the types in the table below, in lower case.
snowflakepostgrescockroachdbredshiftmysqlmongodbclickhousesinglestorebigquerydatabrickselasticsearchopensearchmssqlawss3csvfireboltathenatrinoConnection fields for the type. credentials.name is the datasource name and must be unique in your organization.
52 fields in credentials
The datasource name, unique in your organization.
For postgres, mysql, mongodb, clickhouse, singlestore, snowflake, trino. See Credentials by type.
For postgres, mysql, mongodb, clickhouse, singlestore, mssql, trino. See Credentials by type.
For postgres, snowflake, trino. See Credentials by type.
For postgres, mysql, mongodb, clickhouse, singlestore, mssql, snowflake, elasticsearch, opensearch, trino. See Credentials by type.
For postgres, singlestore, mssql, snowflake, athena, firebolt. See Credentials by type.
For postgres, snowflake, firebolt, trino. See Credentials by type.
For postgres. See Credentials by type.
For postgres, mssql, trino. See Credentials by type.
For postgres, mssql, trino. See Credentials by type.
For postgres, mssql, trino. See Credentials by type.
For postgres, mssql, trino. See Credentials by type.
For mysql, mongodb, clickhouse, singlestore, mssql. See Credentials by type.
For mssql. See Credentials by type.
For mssql. See Credentials by type.
For mssql. See Credentials by type.
For snowflake. See Credentials by type.
For snowflake. See Credentials by type.
For snowflake, databricks. See Credentials by type.
For snowflake. See Credentials by type.
For snowflake. See Credentials by type.
For bigquery. See Credentials by type.
For bigquery. See Credentials by type.
For bigquery. See Credentials by type.
For bigquery. See Credentials by type.
For databricks. See Credentials by type.
For databricks. See Credentials by type.
For databricks. See Credentials by type.
For databricks. See Credentials by type.
For elasticsearch, opensearch. See Credentials by type.
elastic-cloudopen-cloudself-managedFor elasticsearch, opensearch. See Credentials by type.
For elasticsearch, opensearch. See Credentials by type.
For elasticsearch, opensearch. See Credentials by type.
For elasticsearch, opensearch. See Credentials by type.
For awss3. See Credentials by type.
For awss3. See Credentials by type.
For athena. See Credentials by type.
For firebolt. See Credentials by type.
For firebolt. See Credentials by type.
For firebolt. See Credentials by type.
For firebolt. See Credentials by type.
For trino. See Credentials by type.
databricks with OAuth service principal, and microsoft fabric: the app's client id.
microsoft fabric: the Microsoft Entra tenant id. Can be empty.
elasticsearch and opensearch, self-managed only: true connects without a username and password.
falseAWS region of the bucket. Required for awss3 and athena.
AWS access key id. Required for awss3 and athena.
AWS secret access key. Required for awss3 and athena.
athena only. Optional.
Tenancy for this datasource.
6 fields in tenancySettings
TABLE (a tenant column) or DATABASE (a schema or database per tenant)
Required for TABLE. Schema of the table that lists your tenants.
Required for TABLE. Table that lists your tenants.
Required for TABLE. Column with the tenant name.
Required for TABLE. Column with the tenant id, the value you send as clientId
Unknown fields are refused, in the body and in credentials.
Credentials by type
Every type also needs name. R = required, O = optional.
datasourceType | Fields |
|---|---|
postgres | host R, port R, username R, password R, database R, schema R, sslMode O (true, false, disable, prefer, require, verify-ca or verify-full), SSH fields O |
redshift, cockroachdb | As postgres, but sslMode is true or false |
mysql, mongodb, clickhouse | host R, port R, user R, password R |
singlestore | host R, port R, user R, password R, database R |
mssql | server R, port R, user R, password R, database O, isDisableDatabase O, readOnlyIntent O, SSH fields O |
snowflake | host R, username R, role R, warehouse R, database R, schema R, credentials R (username/password or Key-pair authentication), password R for username/password, privateKey R for key pair, privateKeyPass O |
bigquery | credentials_json R (the service account JSON as a string), project_id R, dataset_location R, dataset_id O |
databricks | serverHostname R, httpPath R, credentials O (Personal access token, the default, or OAuth service principal), token R for a personal access token, clientId and clientSecret R for a service principal, queryTags O (key:value,key:value, up to 10 pairs) |
elasticsearch, opensearch | server_type R (elastic-cloud, open-cloud or self-managed), cloud_id R for a cloud type, server_url R for self-managed, username and password R unless disableAuth is true (self-managed only), isIgnoreVerifyCert O, isIgnoreSsl O |
awss3 | bucketName R, s3Region R, s3AccessKeyId R, s3SecretAccessKey R, datasetPath O, tableLevel O (File or Folder) |
athena | database R, outputBucket R, s3Region R, s3AccessKeyId R, s3SecretAccessKey R |
firebolt | client_id R, client_secret R, account R, database R, engineName R, schema O |
trino | host R, port R, catalog R, schema R, username R, password R, SSH fields O |
csv | name only |
port is a whole number from 1 to 65535.
SSH fields: sshTunnel (enable or disable). With enable, also send sshHost, sshPort, sshUsername and sshPrivateKey.
Microsoft Fabric cannot be created with this endpoint. Connect it in the app.
Databricks OAuth service principal must be turned on for your organization. If it is not, you get 403 DATABRICKS_OAUTH_NOT_ENABLED.
Response
{ "name": "Sales warehouse", "warnings": [ { "code": "TENANCY_CLIENT_NAME_NOT_UNIQUE", "message": "120 client ids share 118 names in company_name. Pick a name column with one name per client, or a table with one row per client.", "keyCount": 120, "nameCount": 118 } ]}| Field | Type | Description |
|---|---|---|
name | string | The datasource name |
warnings | object[] | Only with TABLE tenancy. TENANCY_CLIENT_TABLE_EMPTY: the tenant table has no rows. TENANCY_CLIENT_NAME_NOT_UNIQUE: two tenant ids share a name |
The response has no id. To get it, list datasources.
Errors
| Status | error.code | Why |
|---|---|---|
| 400 | INVALID_REQUEST_BODY | A field is missing, unknown or wrong. The message names it, for example Invalid credentials for postgres: "host" is required. Also a refused host, or an unsupported type |
| 400 | DATASOURCE_NAME_ERROR | The name is already used |
| 400 | CREDENTIAL_TEST_FAILED | Databrain could not connect. hint says what to check. retryable says if a retry can help |
| 400 | CREATE_DATASOURCE_FAILED | The datasource could not be saved |
| 400 | SCHEMA_CACHE_FAILED | The datasource was saved, but its schema could not be read. Do not create it again: run sync |
| 403 | DATABRICKS_OAUTH_NOT_ENABLED | Databricks OAuth is not on for your organization |
| 429 | Rate limit |
For errors that every endpoint returns, see Authentication.
Next, create a datamart on the datasource.
curl -X POST 'https://api.usedatabrain.com/api/v2/datasource' \ -H 'Content-Type: application/json' \ -d '{ "datasourceType": "postgres", "credentials": { "name": "Sales warehouse", "host": "db.example.com", "port": 5432, "username": "databrain_reader", "password": "<password>", "database": "analytics", "schema": "public", "sslMode": "require" }}'{ "name": "Sales warehouse", "warnings": [ { "code": "TENANCY_CLIENT_NAME_NOT_UNIQUE", "message": "120 client ids share 118 names in company_name. Pick a name column with one name per client, or a table with one row per client.", "keyCount": 120, "nameCount": 118 } ]}