Integration · Warehouse

A SQL warehouse target, one token away.

Mapped rows are written through the SQL Statement Execution API against a named warehouse, in independent batches of 500, with catalog and schema set explicitly.

POST /v1/gatewaykind: databricks

How it works

What actually happens on a sync run

The Databricks destination talks to POST https://<host>/api/2.0/sql/statements against a named SQL warehouse, authenticated with an access token. Catalog and schema are set explicitly rather than inherited from a session, so the same connector behaves identically whoever runs it, and identifiers are validated before a statement is built.

  1. Step 1

    Authenticate

    A bearer access token, read from the encrypted connector record.
  2. Step 2

    Bind the context

    Warehouse id, catalog and schema travel with the statement instead of relying on a session default.
  3. Step 3

    Send batched statements

    500 rows per statement, each batch independent.
  4. Step 4

    Report per batch

    Failed indices named; total failure is a 502 with the platform’s own message.

What you get

Built for files that keep arriving

Explicit context

Catalog and schema travel with the statement

Nothing depends on a session default, so the connector behaves the same on every run and for every caller.

How · catalog, schema and warehouse_id are sent per statement. The host is validated and normalized at save time.

Write

Independent batches of 500

A partial failure is reported as one, with the failing batch indices named.

How · written_rows, batches, failed_batches; total failure is 502 destination_write_failed carrying the platform’s error text.

Schema

Read the table, do not restate it

A gateway call maps onto the destination’s own columns; a column you name that does not exist fails before anything is written.

How · GET /v1/connectors/{id}/schema — metadata only. Mismatch is 422 schema_destination_mismatch in pre-flight.

Secrets

The credential never travels in a request body

You reference a connector by id. Keys, tokens and service-account JSON are encrypted at rest and read only by the code that makes the call.

How · normalizeSecretField() folds every provider spelling — private_key, token, secret_access_key, account_key, service_account_json — into one auth_value field, which is KEK-envelope-encrypted before the row is written. A GET masks it to a 4-character hint. If encryption fails the field is dropped rather than stored in plaintext.

Configuration

The connector record, field by field

KeyRequiredWhat it is
hostRequiredWorkspace host. Validated as a host, with any scheme or trailing slash stripped.
warehouse_idRequiredThe SQL warehouse to execute on.
auth_valuesecretRequiredThe access token. token is accepted as an alias and folded into this field before storage.
tableOptionalTarget table, optionally dotted. Overridable per call with destination.table.
catalogOptionalCatalog context for the statement.
schemaOptionalSchema context for the statement.

In code

A file, a token, a populated table.

Save the workspace host, warehouse and token once. Every later call names the connector by id and nothing else.

  • POST/v1/connectorsSave the connector. The secret is encrypted before it reaches Postgres.session
  • GET/v1/connectors/{id}/schemaRead the target’s own columns — metadata only, never row data.bearer
  • POST/v1/gatewayAny input in, this destination populated, a delivery report out.bearer
  • POST/v1/connectors/{id}/rotate-secretReplace the credential in place; the old one becomes unrecoverable.session
  • Scope the token to what it needs. AdaptivMapr uses it to insert mapped rows into the configured table — nothing else in the run requires more.
  • The host is validated at save time, so a scheme or a trailing slash in a pasted value does not become a broken statement later.
  • dry_run reports the batch count without executing.
POST /v1/connectors
{
  "kind": "databricks",
  "name": "Lakehouse — claims",
  "config": {
    "host": "dbc-1a2b3c4d-5e6f.cloud.databricks.com",
    "warehouse_id": "abc123def456",
    "catalog": "main",
    "schema": "raw",
    "table": "claims_line_items",
    "token": "dapi…"
  }
}
curl
curl https://api.adaptivmapr.com/v1/gateway \
  -H "Authorization: Bearer $MAPR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": { "url": "https://acme.example.com/exports/claims.csv" },
    "destination": { "connector_id": "con_c081…" }
  }'
response
{
  "schema_id": "claims_line_items_v1",
  "source": "destination",
  "row_count": 12000,
  "destination": {
    "connector_id": "con_c081…",
    "kind": "databricks",
    "table": "claims_line_items",
    "schema_source": "destination",
    "protocol": "databricks",
    "written_rows": 12000,
    "batches": 24,
    "failed_batches": []
  }
}
→ 12 000 rows in 24 statements · catalog + schema bound per statement · 0 failed batches

Limits & failure modes

What it refuses, and what it tells you

CodeWhenWhat to do
400 config_invalidHost, warehouse id or token is missing, or the host is not a valid host.Each is named individually so you fix the right field.
502 destination_write_failedEvery batch was rejected.Returned with the platform’s own message rather than a generic failure.
422 schema_destination_mismatchA supplied schema names a column the table does not have.Pre-flight catches it before the first statement.

Incremental sync

Not applicable: this is a destination. To pull FROM a lakehouse on a cadence, expose the query over HTTPS and use the SQL-over-HTTP source with a {{since}} watermark.

PHI & residency

A lakehouse delivery runs under the workspace’s routing policy, not around it. PHI routing is a separate axis from the data mode: it sends X-PHI and X-Region to phi-cloud so a regulated run lands on an in-region, BAA-eligible model, it costs +20% on the whole charge, and it is locked until the workspace accepts the BAA in Settings → Security & Data. An explicit PHI ask without an acceptance is 403 agreement_required, never a silent downgrade. A standard run keeps the workspace’s region pin — the region decides where compute may run, and the sandbox refuses a region-less run.

What it costs

Billed on the same prepaid wallet

Moving bytes is not a line item. A sync that pulls a file and a destination write that lands the rows are both part of one map, and the map is what the wallet sees. There is no free tier and no subscription — top up from $10, a balance shared across the phi-cloud suite.

ChargeRateNotes
Every map$0.001A flat per-map fee — a few tokens — charged even when the run was fully deterministic or hit the layout cache and used no AI at all.
AI, only when it ranat cost × 2Layer-5 cleanup, any-to-any convert and structural reshape bill the phi-cloud tokens actually consumed. Bring your own model key and it is × 0.5.
PHI / enterprise routing+20%Multiplies the whole charge, flat fee included — and only when the run genuinely got that routing. Locked until the workspace accepts the BAA in-app.

Questions

Before you wire it up

The SQL Statement Execution API — POST https://<host>/api/2.0/sql/statements — against the warehouse id you configure, with catalog and schema bound per statement rather than inherited from a session. Rows are sent in independent batches of 500.
In the connector record, envelope-encrypted with a key-encryption key before the row reaches Postgres, held service-role-only, and masked to a four-character hint on read. It is never accepted in a request body — a call names the connector by id — and /rotate-secret replaces it in place with no recovery window.
Yes. destination.dry_run runs the whole pipeline — parse, map, validate — and reports what would be written, including the batch count, without executing a statement. It is the safe first call against a production table.
Not as a scheduled connector. It is a write destination whose schema can be read for mapping. For a scheduled pull, expose the query behind an HTTPS endpoint and use the SQL-over-HTTP source, which supports an incremental {{since}} watermark.

Verified against lib/warehouseSql.ts · lib/destinations.ts · lib/introspect.ts · app/api/v1/connectors/route.ts

Databricks is a trademark of Databricks, Inc. Named here to describe interoperability only — no affiliation, endorsement or partnership is claimed.

Ready when you are

Point it at Databricks. Get your schema back.

Start with a $10 prepaid wallet. Every map draws a few tokens; in schema-only mode only headers and a few sample rows ever leave you.

$10 minimum to start · pay only for what you map · PHI under BAA coverage