Reference

Welcome to the Reference page for our API Documentation which will give you brief introduction on APIs used in our platform. You will also get an idea on Authentication part.

API Reference

API Guide

It is a comprehensive guide that provides developers with all the necessary information to understand and interact with an API effectively. It includes details on endpoints, parameters, and usage examples, helping developers integrate and use the API seamlessly in their applications. It includes sample code in various programming languages. This can be valuable for developers to quickly get started with making API calls. The interactive UI(Elements UI) allows to experiment with the API in real-time, seeing both request and response examples interactively.

Refer the link - API Guide

API Authentication

Platform API supports 2 mechanisms for API Authentication. These are described in the below section. With these authentication mechanisms, you will be able to use the API functionalities.

Authentication with API Key

To authenticate your request, creation of an <api_key> is needed, as explained in the next section.

Generate API Key using Web UI

The <api_key> can be generated from our Web UI. The <api_key> can be copied from the user profile section as shown in the image by providing a <Token name>.

Naming the api key Screenshot

Please copy <api key> as shown below. Copying the api Token key Screenshot

Generate API Key using API

The other method is through sending API request to create the <api_key>. The API details are as follows -

POST https://api.lab45.ai/v1.1/apikey
Authorization: Bearer {{access_token}}
Content-Type: application/json

        {
            "name":"api_key"
        }

The API uses HTTP Post method. The <access_token> is the one which you generate from Authentication with JWT Token section which is explained above. The user can provide any “name” in the request body for the <api_key>. The response generated provides the <api_key> value. The usage of <api_key> can be seen in Completion API section.

Managing your API Key

The <api_key> is valid for 30 days only. User can have only one <api_key> at a time. There will be an option to the user to see the existing <api_key> and delete the <api_key>. If an active <api_key> already exists for the user, the creation <api_key> will return an error. To create a new user <api_key>, you should delete the existing <api_key> if available. To use the <api_key>, you need to copy it somewhere safe and use it in the client code you are developing. Whenever the <api_key> is created, existing group of the user will be captured. On user group change you must create a new <api_key>.

Authentication with JWT Token

To access the API which creates <api_key> for the user to get authenticated, you will need to get the JWT token first which is the <access_token>. The <access_token> is valid only for short period of time. The way <access_token> is passed is shown below -

Authorization: Bearer {{access_token}}

The token is added as a Bearer token in the HTTP request header whenever the API request is made.

API Functionalities

There are 5 different APIs integrated with our platform. Each of it is explained in detail below.

Chat AI API

The Chat AI API is a powerful tool that enables developers to seamlessly integrate conversational AI capabilities into their applications. Our API provides access to a variety of pre-trained, closed-source AI models that excel at tasks such as text generation, question answering, and conversation generation.

Developers can directly use the Chat AI API functionality via our intuitive Web UI. Additionally, those interested in exploring its capabilities can refer to the Skill Documentation for detailed instructions and examples. To access the Chat AI API, the endpoint maps {skill_id} to the completion.

Document Chat AI API

Document Chat AI is an advanced technology that leverages natural language processing (NLP) and conversational AI to deliver a comprehensive understanding of documents. It employs the Retrieval-Augmented Generation (RAG) concept, where uploaded documents are embedded and stored in a vector database. Based on user queries, the system retrieves relevant information and generates context-aware answers from the provided documents.The API supports a wide range of file types, including PDFs, HTML files, PowerPoint presentations (PPT), images, text files, DOCX files, and emails.

Developers can directly use the Chat AI API functionality via our intuitive Web UI. Comprehensive details about the API, including request samples, are available in the Skill Documentation. To access the Document Chat AI API, the endpoint maps {skill_id} to doc_completion.

Agent AI API

The Agent AI APIs provide a framework for enabling interactive, agent-driven conversations via APIs. This system allows the creation of custom AI agents, each leveraging a diverse range of integrated models, document-based RAG (Retrieval-Augmented Generation), and a growing catalog of pre-defined tools.

With this API framework, multiple AI agents can be configured with unique, prompt-based instructions and grouped together to form a Team. The Team concept employs an orchestrator to facilitate group chats between agents, enabling them to collaborate and tackle complex tasks by utilizing the specialized capabilities of each agent.

The platform’s library of pre-defined tools allows agents to connect with external APIs, retrieve real-time data, and generate outputs such as images and presentations, making it a versatile solution for a wide array of applications.

Developers can use the Agent AI API functionality directly from our Web UI. Additionally, those interested in exploring its capabilities can refer to the Agent Chat Session Documentation for detailed instructions and examples. To use the Agent Chat AI API, the first step is to create the agents. Creation of Agents can be done using Agents API.

There are other operations that Agents support. They are - Get the Agents, Update the Agents and Delete the Agents.

Structured AI API

The Structured AI API is designed to enable users to upload data in structured file formats such as Excel, CSV files. Once the data is uploaded, users can query the document with specific questions. The API leverages advanced AI models to analyze the structured data and provide contextually relevant answers based on the content of the uploaded file.

By parsing the data within tables, rows, columns, and cells, the API extracts insights and answers user queries with precision. This functionality is ideal for working with spreadsheets and structured datasets, where users may need to quickly retrieve and analyze information from large datasets.

The details for the API along with the request samples can be found in the Skill Documentation. For Structure AI API, the {skill_id} would be structured_ai.

Web Query API

The Web Query API is a powerful tool that integrates with the Bing Search API to retrieve relevant content from the web and generate precise answers based on that information. By leveraging Bing’s vast search capabilities, the Web Query API provides a dynamic way to access, process, and synthesize information from the wealth of online resources available.

This API is especially useful for obtaining up-to-date information, as it taps into Bing’s real-time search results, ensuring that users receive the most current and accurate answers. Whether users are seeking the latest news, research papers, product details, or other web-based content, the Web Query API can seamlessly integrate this data into applications, chatbots, or other AI-driven systems.

The API works by taking a user’s query and utilizing the Bing Search engine to identify relevant web pages, articles, and resources. It then analyzes the retrieved content to generate concise, context-aware responses, making it ideal for answering questions that require the latest information.

The details for the API along with the request samples can be found in the Skill Documentation. For Web Query API, the {skill_id} would be web_completion.

Lab45 AI Platform Autogen extension

This package introduces an extension to the Autogen 0.4 agents framework. Following Lab45 AI Platform specific extensions have been added:

A custom Lab45AIPlatform model client class which internally routes all LLM and tool based calls to platform’s backend APIs.
A catalog of tools available with the platform, which can be consumed by this extended library
Platform’s RAG capability has been made available using customs tools.
Multi-modal support (limited to images)

With this extended library, users can use most of the features that Autogen offers, all the while leveraging the the Lab45 AI Platform’s backend APIs, models and tools.

Note: To run this code, make sure Python 3.11 is set up in your environment.

Installation

Python Environment Setup

This guide explains how to create and use a Python 3.11.0 virtual environment, which is supported for running sample API requests.

Set up Python using pyenv:

Install a compatible Python version (3.11.0 recommended) and Set it as your local Python version.

pyenv local 3.11.0

Make sure pyenv is properly initialized in your shell

eval "$(pyenv init --path)" eval "$(pyenv init -)"

Verify the Python version

python --version

If ‘python’ command is still not found, you can use the full path:

~/.pyenv/versions/3.11.0/bin/python --version

Create and activate a virtual environment:

Create a virtual environment

python -m venv .venv

On macOS/Linux:

source .venv/bin/activate

On Windows:

.venv\Scripts\activate

Use the package manager pip to install the package.

pip install lab45_autogen_extension-0.1.0-py3-none-any.whl

Model Client Usage

In order to use this model client class with existing Autogen functionalities, the users post installation of this extended library, need to initialize the Lab45AIPlatformCompletionClient class with the right LAB45AIPLATFORM_URL and LAB45AIPLATFORM_API_KEY (either passed as arguments or as environment variables).

os.environ["LAB45AIPLATFORM_URL"] = "https://api.lab45.ai/v1.1/"
os.environ["LAB45AIPLATFORM_API_KEY"] = "your_api_key_here"  # Platform API key or Bearer token

This model client class can be imported from the lab45aiplatform_autogen_extension module and the object can then be added to the model_client field of any agent or team created using Autogen.

from autogen_extension.lab45aiplatform_autogen_extension import Lab45AIPlatformCompletionClient

client = Lab45AIPlatformCompletionClient(
        model_name='gpt-4'
    )

agent = AssistantAgent(
        name="assistant",
        model_client=client,
        system_message="Your name is AutogenAssist, you are a general purpose AI assistant which can help with user queries"
    )

Tools Usage

The lab45_autogen_extension library allows users to make use of various different type of tools:

The catalog of tools available in the platform can be accessed using lab45_autogen_extension.custom_tools. The imagetools.Lab45AIPlatformText2ImageTool can take in either dalle or stablediffusion as options for image models.

from lab45_autogen_extension.custom_tools.imagetools import Lab45AIPlatformText2ImageTool
image_tool = LangChainToolAdapter(Lab45AIPlatformText2ImageTool('dalle'))

List of platform tools available: Lab45AIPlatformText2ImageTool, Lab45AIPlatformBingSearchTool, Lab45AIPlatformSlideGenerationTool

Any Langchain tool can be passed as a tool.

from langchain_experimental.tools.python.tool import PythonAstREPLTool
tool = LangChainToolAdapter(PythonAstREPLTool(locals={"df": df}))

Any custom function can be passed as a tool to the agent. The Autogen function: FunctionTool can be used for this purpose.

def arxiv_search(query: str, max_results: int = 1) -> str:
    pass
arxiv_search_tool = FunctionTool(arxiv_search, description="Search Arxiv for papers related to a given topic, including abstracts")

RAG/Doc-completion Usage:

The platform’s RAG feature has been extended to this library in the form of a tool: Lab45AIPlatformRAGTool. The files and documents will have to be pre-indexed using the dataset and doc_completion endpoints. The dataset_id along with message_history, if required will be passed to this tool and finally doc_completion query endpoint be called to fetch responses based on documents indexed.

from lab45_autogen_extension.custom_tools.ragtool import Lab45AIPlatformRAGTool
rag_tool = LangChainToolAdapter(Lab45AIPlatformRAGTool(dataset_id="<dataset_id>", model_name="gpt-4o", top_k=30))

Lab45 AI Platform SharePoint Scan Support

Effortlessly access files directly from SharePoint with Microsoft Power Automate for secure and automated data retrieval. Seamlessly integrate extracted content into the workflow, eliminating manual downloads and boosting efficiency. With support for multiple file formats, the solution ensures flexible data handling and smooth integration into the existing processes. Robust authentication safeguards access, ensuring only authorized users can retrieve files with confidence.

With this extended feature, users can leverage sharepoint_completion as a skill to create and update datasets by ingesting files directly from their SharePoint site URL. This is done in combination with Microsoft Power Automate, enabling seamless data integration and automation.

Installation and Pre-Requisites

This guide outlines the steps to set up Microsoft Power Automate to fetch the workflow URL required for the SharePoint Scan feature.

Pre-Requisites

Before proceeding, ensure you have the following:

An active Microsoft Power Automate account.
Download the provided workflow ZIP file from the following link to import it as a package. Zip file for importing the Power Automate Workflow

Setup Steps

Sign in to Power Automate with your Microsoft account.
Click on Connections tab under more and verify if SharePoint and Office 365 Users is signed in. If not, reconnect by signing in with your Microsoft Account.
Once verified go and click on my flows on the left panel.
On the open panel, select import and click on Import Package(Legacy).
Now upload the downloaded workflow zip file mentioned in the pre-requisites.
From the review package, click on update from the first column for choose your import options. A window should pop-up to the right, choose create as new from the dropdown and Save it.
Now again review package, click on Select during Import from the second column for Related Resources. A window should pop-up to the right, choose yourself as the user from the resource list name to assign the user and Save it.
Once the user is assigned, click on Import and wait for it to successfully create the flow.
Navigate to my flows from the left panel and refresh the page to load the imported power automate workflow.
Turn on the imported workflow from the workflow list appearing in the configuration button beside the workflow name.
Once the above step is completed, go inside the workflow and click on edit to view the workflow.
Once the workflow opens select the manual box from the workflow. A pop-up should appear to the left.
Copy the URL found under the HTTP URL block while creating or updating the dataset. This URL will be used later to utilize the SharePoint scan feature. Etc.

Limitations & Known Issues

Agents Custom Transition through FSM: Limited support for bi-directional or cyclic transitions, a known issue occurs when the group chat manager prematurely terminates a conversation if it detects that a query has already been addressed.
Image-Based Completion: Accuracy may vary depending on image quality, especially for fine details or closely spaced data points (e.g., line graphs). The performance is influenced by the capability of the image model in use.
Vision-Based Summary to Enhance RAG: Does not support nested tables or tables split across multiple pages.