API Enterprise SDK
Video Bot

Video Bot Enterprise SDK

The Video Bot SDK enables you to integrate video transcription, notes generation, summaries, and vector Q&A capabilities directly into your software platform. With the Video Bot, you can parse video file streams, convert them to indexed audio, extract transcribed content, and query the video content dynamically.

Technical Overview

The Video Bot pipeline extracts the audio track from your input video containers, runs ASR (speech-to-text), utilizes LLM summarizers for notes and executive overviews, and embeds the text transcripts into your local vector database.

Key Endpoints

1. Upload Video & Start Processing (POST /videoBotPostData)

Uploads a video file and triggers the automated transcription and summarization process.

  • Request: Multipart Form-Data

    • file_path: Binary file (e.g. MP4 or MOV file).
    • prompt: String command configuration structure.
    • modeljsonStructure: String model specifications JSON.
    • user_id: String user identifier.

  • Response:

    {
  "message": "File uploaded and processed successfully",
  "combinedResponse": {
    "transcribed_text": "...transcribed speech...",
    "Notes": "...key bulleted takeaways...",
    "Summary": "...high level summary...",
    "Session_Details": {
      "Session ID": "session-uuid-1234",
      "text_file_path": "path/to/extracted/results.txt"
    }
  }
}

2. Generate Vector Database Index (POST /videobotVectorApi)

Creates a vector index on the extracted transcript to support RAG query lookups.

  • Request Payload:

    • prompt: String prompt config containing key mappings.
    • filename: Session ID.
    • modeljsonStructure: Config structure defining embedding provider models.

  • Response:

    {
  "message": "Vector DB generated successfully",
  "details": { ... }
}

3. Query Video Content (POST /videobotChatApi)

Queries the vector database index of a video.

  • Request Payload:

    • prompt: String natural language question.
    • uuId: Session ID.
    • userMessage: String conversational user question.

  • Response:

    {
  "answer": "According to the video, the product launches in Q4."
}

Code Example: Integrating Video SDK

Below is a Python script demonstrating how to connect to a local hosted Video Bot server to upload, index, and query a video file:

import requests
 
API_URL = "http://localhost:5001/api"
 
# 1. Upload and Process Video File
with open("./tutorial.mp4", "rb") as f:
    files = {"file_path": f}
    data = {
        "user_id": "user_dev_01",
        "modeljsonStructure": '{"temperature": 0.2}',
        "prompt": "Analyze tutorial video"
    }
    response = requests.post(f"{API_URL}/videoBotPostData", files=files, data=data)
 
res_data = response.json()
session_id = res_data["combinedResponse"]["Session_Details"]["Session ID"]
print("Transcribed Speech:", res_data["combinedResponse"]["transcribed_text"])
 
# 2. Generate Vector Index
vector_res = requests.post(f"{API_URL}/videobotVectorApi", json={
    "filename": session_id,
    "prompt": "Create vector index",
    "modeljsonStructure": '{"embedding_model": "text-embedding-3"}'
})
print("Vector Index Creation:", vector_res.json()["message"])
 
# 3. Chat Query
chat_res = requests.post(f"{API_URL}/videobotChatApi", json={
    "uuId": session_id,
    "userMessage": "What are the first steps to install the library?"
})
print("Video Answer:", chat_res.json()["answer"])
Voice BotYouTube Bot