CluedIn Python SDK 3.0.0

CluedIn Python SDK 3.0.0 no longer supports older, deprecated Python versions. It requires Python 3.10 or newer.

The more practical addition is support for CluedIn ingestion endpoints. It simplifies a common ingestion pattern and uses sensible defaults. You can pass a list, a generator, or a data frame, and the SDK will split the data into batches and return the response for each one:

import cluedin

# TODO: define API_TOKEN and ENDPOINT_URL

ctx = cluedin.Context.from_jwt(API_TOKEN)

# 1. ingest a simple list
data = [
  { 'id': 1, 'name': 'foo' },
  { 'id': 2, 'name': 'bar' }
  # HINT: you can have millions of records here
]

for processed_batch in cluedin.ingestion.post(ctx, ENDPOINT_URL, data):
  	# HINT: cluedin.ingestion.post splits data into batches
    #	and posts a batch after a batch
    #   the next batch will not be posted if you break the loop
    # HINT: the processed_batch contains records posted in the batch
    #   and the ingestion endpoint's response, including the receiptId
	print(processed_batch['response'])
    
# 2. stream data with generators
def stream_data():
	for i in range(1_000_000):
    	yield { 'id': i, 'email': f'user{i}@cluedin.com' }

for processed_batch in cluedin.ingestion.post(ctx, ENDPOINT_URL, stream_data()):
	print(processed_batch['response'])
    
# 3. pandas
import pandas as pd

# TODO: define a DataFrame df

for processed_batch in cluedin.ingestion.post(ctx, ENDPOINT_URL, df.to_dict('records')):
	print(processed_batch['response'])

What's Changed

Full Changelog: https://github.com/romaklimenko/cluedin/compare/2.6.0...3.0.0