Using CluedIn GraphQL API to automate UI actions

As a Master Data Management system, CluedIn lets users work with data through a UI and low-code tools. You can complete the full data lifecycle, from ingestion through modeling, transformation, cleaning, enrichment, deduplication, and export, without writing a line of code.

Some work is still easier to automate. A small script can spare business users repetitive UI tasks, especially when you need to keep development, staging, and production instances in sync through a CI/CD pipeline.

In this article, we'll build basic automation without knowing the API in advance. Our starting point is that the CluedIn UI communicates with the server through a GraphQL API.

Two GraphQL API endpoints

CluedIn provides two GraphQL API endpoints:

• /api/api/graphql: The endpoint used to query data.
• /graphql: The endpoint used for UI interactions.

Authentication

CluedIn uses JSON Web Tokens (JWT) for authorization.

There are two kinds of tokens in CluedIn: API Tokens and Access Tokens.

API Tokens are used to extract data from CluedIn or ingest data into it. You can create or copy one from Administration -> API Tokens.

Access Tokens are generated when you log in to CluedIn as a user.

You need an API Token to use the /api/api/graphql endpoint.

The /graphql endpoint only accepts Access Tokens, so you must log in with your email and password to obtain one.

The article Using Postman to interact with the CluedIn API explains how to obtain an Access Token with Postman.

For data queries with the Python SDK, see Search Entities with CluedIn Python SDK.

Exploring GraphQL API

We'll automate an action normally performed in the UI by using the CluedIn GraphQL API and CluedIn Python SDK.

Suppose you need to synchronize Vocabularies from another system. You could create each one in the CluedIn UI, but this is a good task to automate.

Start by creating a test Vocabulary in the browser and watching the API requests. You can use the Network tab or GraphQL Network Inspector. I find GraphQL Network Inspector easier.

I'll show both approaches. First, I create a test CluedIn Vocabulary while the Network tab is open:

Among the GraphQL requests, I can see a call to the createVocabulary mutation:

GraphQL Network Inspector makes the call easier to find:

Either way, we can now copy the GraphQL query:

mutation createVocabulary($vocabulary: InputVocabulary) {
  management {
    id
    createVocabulary(vocabulary: $vocabulary) {
      ...Vocabulary
      __typename
    }
    __typename
  }
}

fragment Vocabulary on Vocabulary {
  vocabularyId
  vocabularyName
  keyPrefix
  isCluedInCore
  entityTypeConfiguration {
    icon
    entityType
    displayName
    __typename
  }
  isDynamic
  isProvider
  isActive
  grouping
  createdAt
  providerId
  description
  connector {
    id
    name
    about
    icon
    __typename
  }
  __typename
}

The request also includes these variables:

{
  "vocabulary": {
    "vocabularyName": "Test",
    "entityTypeConfiguration": {
      "new": false,
      "icon": "Profile",
      "entityType": "/IMDb/Name",
      "displayName": "IMDb Name"
    },
    "providerId": "",
    "keyPrefix": "test",
    "description": ""
  }
}

Python SDK

We can now use the CluedIn Python SDK to create a Vocabulary.

I start by creating a file with my CluedIn credentials. You can provide them through environment variables instead:

{
  "domain": "172.167.52.102.sslip.io",
  "org_name": "foobar",
  "user_email": "admin@foobar.com",
  "user_password": "mysecretpassword"
}

Then I install the CluedIn Python SDK:

%pip install cluedin

Next I log in to CluedIn to get an Access Token:

import cluedin

ctx = cluedin.Context.from_json_file('cluedin.json')
ctx.get_token()

The ctx object now contains the Access Token and can be used to create a Vocabulary:

def create_vocabulary(ctx, name, prefix):
  query = """
mutation createVocabulary($vocabulary: InputVocabulary) {
  management {
    id
    createVocabulary(vocabulary: $vocabulary) {
      ...Vocabulary
    }
  }
}
fragment Vocabulary on Vocabulary {
  vocabularyId
  vocabularyName
  keyPrefix
  entityTypeConfiguration {
    icon
    entityType
    displayName
  }
  providerId
  description
}
"""

  variables = {
    "vocabulary": {
      "vocabularyName": name,
      "entityTypeConfiguration": {
        "new": False,
        "icon": "Profile",
        "entityType": "/IMDb/Name",
        "displayName": "IMDb Name"
      },
      "providerId": "",
      "keyPrefix": prefix,
      "description": ""
    }
  }

  return cluedin.gql.org_gql(ctx, query, variables)

Call the function:

print(create_vocabulary(ctx, 'Foo Bar', 'foo.bar'))

Output:

{
  "data": {
    "management": {
      "id": "management",
      "createVocabulary": {
        "vocabularyId": "e4528e92-f4ad-406a-aeab-756acc20fd01",
        "vocabularyName": "Foo Bar",
        "keyPrefix": "foo.bar",
        "entityTypeConfiguration": {
          "icon": "Profile",
          "entityType": "/IMDb/Name",
          "displayName": "IMDb Name"
        },
        "providerId": null,
        "description": ""
      }
    }
  }
}

The new Vocabulary appears in the UI:

I've used the same approach in real projects, for example to:

• create missing Vocabularies and keys for a given Entity;
• explore a CluedIn Entity's Data Parts to troubleshoot overmerging.