VEZA DEVELOPERS

Appearance

OAA Push API ​

Methods for working with Custom Data Providers and Sources

This document provides a basic overview of the API requests for creating and updating an OAA data source. These steps and API calls can be adapted for your client or programming language of choice. You can also use the oaaclient Python module to handle Veza authentication, register the data source, and push the populated template.

Overview ​

While registering sources and pushing authorization metadata with Open Authorization API is relatively straightforward, it is important to understand how Veza organizes custom providers and data sources as endpoints:

  • You will first register a new custom application provider with a POST request to /api/providers/custom (specifying the app name and template).
    • The template determines the application type of the provider's custom data sources (identity_provider or application).
  • Each custom provider can have one or more data sources (such as different instances or domains), generated using Create Datasource.
    • The populated template can describe additional resources and sub-resources, such as individual databases, repositories, or views.
  • You can push authorization metadata for each custom data source.
  • All custom data sources are shown on the Configuration > Apps & Data Sources menu, and can be retrieved using List Custom Provider Datasources.

You should typically name the provider based on the generic application provider (such as GitHub) and the data source after the top-level instance (such as GitHub - Organization). See the Core Concepts for more information about parsing and identifying entities using metadata from the source application.

Authentication ​

Your requests will need to include a Veza API key. Provide it as the bearer auth token in the header of each request, for example:

bash
curl -X GET "https://$VEZA_URL/api/v1/providers/custom" \
-H "authorization: Bearer $API_KEY"

Follow best practices for managing API keys: Do not save credentials in connector code. Consider using a secrets manager to securely store API keys and make them available at run time.

First Run ​

To add a custom application, you will need to:

  • Create a new custom provider and data source.
  • push the entity and authorization data in a JSON payload.

Create a provider using the "Custom Application" template ​

Use Create Custom Provider to register a new top-level custom provider:

bash
curl -X POST "https://$VEZA_URL/api/v1/providers/custom" \
-d '{"name":"DemoApp","custom_template":"application"}'

The response will return the Provider's ID, which you will need to create and manage the data sources:

json
{
  "value": {
    "id": "a6ef8d8d-d17b-4491-a67a-635ad70f1ba9",
    "name": "DemoApp",
    "custom_template": "application",
    "state": "ENABLED",
    "application_types": [],
    "resource_types": [],
    "idp_types": [],
    "schema_definition_json": "e30="
  }
}
  • Name the provider generically after the application or SaaS provider. Use the same Provider for all data sources for that application.
  • If you are creating providers dynamically, your OAA integration should check if the provider and data source exists before creating a new one.

Create a Data Source for the Provider ​

Each provider needs at least one data source. Create one with Create Custom Provider Datasource

bash
curl -X POST "https://${VEZA_URL}/api/v1/providers/custom/${PROVIDER_ID}/datasources" \
-H 'accept: application/json' \
-H 'authorization: Bearer '${API_KEY} \
-d '{"id": "<PROVIDER ID>", "name":"DemoApp - Documentation Datasource"}'

The response will include the data source ID:

json
{
  "value": {
    "id": "1bd31da0-64ee-4dfe-82c9-cb9f0f2fc369",
    "name": "DemoApp - Documentation Datasource"
  }
}
  • Datasources should be unique to the data collected by an OAA integration instance. For example, if an application has a "prod" and "dev" instance, creating a datasource for each will enable independent updates to each environment.
  • Name the data source uniquely based on the application instance to discover. Try to include the hostname or organization name in the data source. For example, don't use "GitHub" use "GitHub - Acme Inc" or "Portal - prod.corp.com"
  • Note that a provider id is required in both the request path and body.

Push data source metadata ​

Once the data source and provider are active, publish the payload with Push Custom Provider Datasource. The request body must include the Provider ID and Data Source ID.

json_data must contain the populated OAA template as a single JSON string (escaping any unsafe characters such as ").

json
{
  "id": "532f6fe3-189f-4576-afdf-8913088961e4",
  "compression_type": "none",
  "data_source_id": "b6a32af6-b854-47e1-8325-e5984f78bb4d",
  "json_data": "{\"name\":\"CustomApp\",\"application_type\"...}"
}
bash
curl -X POST "https://$VEZA_URL/api/v1/providers/custom/$PROVIDER_ID/datasources/$DATASOURCE_ID:push" \
-H 'accept: application/json' \
-H 'authorization: Bearer '$API_KEY \
--compressed --data-binary @body.json

Get Custom Providers ​

bash
curl -X GET "https://$VEZA_URL/api/v1/providers/custom" \
-H 'accept: application/json' \
-H 'authorization: Bearer '$API_KEY

Get Custom Data Sources ​

bash
curl -X GET "https://$VEZA_URL/api/v1/providers/custom/$PROVIDER_ID" \
-H 'accept: application/json' \
-H 'authorization: Bearer '$API_KEY

Update Custom Data Source ​

Specifying the Provider ID and Data Source ID, perform the same post operation used for the initial push.

bash
curl -X POST "https://$VEZA_URL/api/v1/providers/custom/$PROVIDER_ID/datasources/$DATASOURCE_ID:push" \
-H 'accept: application/json' \
-H 'authorization: Bearer '$API_KEY \
--compressed --data-binary @payload.json

To update an existing data source, use the operations List Provider and List Datasources operations to get the provider and data source IDs.