raga-ai-hub /RagaAI-Catalyst

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

★ 16,190⑂ 3,570👁 37⑃ 210● 55▤ 19
PythonApache-2.0Created 2024-08-26catalyst.raga.ai/
Download: 2.2.42.2.4 ↓2.2.42025-06-23
+0
Daily
16,190
stars
3,570
forks
37
watchers
210
prs
55
issues
19
releases

README

RagaAI Catalyst  !GitHub release (latest by date) !GitHub stars  !Issues 

RagaAI Catalyst is a comprehensive platform designed to enhance the management and optimization of LLM projects. It offers a wide range of features, including project management, dataset management, evaluation management, trace management, prompt management, synthetic data generation, and guardrail management. These functionalities enable you to efficiently evaluate, and safeguard your LLM applications.

 Table of Contents

- RagaAI Catalyst
  - Installation
  - Configuration
  - Usage
    - Project Management
    - Dataset Management
    - Evaluation Management
    - Trace Management
    - Agentic Tracing
    - Prompt Management
    - Synthetic Data Generation
    - Guardrail Management
    - Red-teaming

 Installation

To install RagaAI Catalyst, you can use pip:

bash
pip install ragaai-catalyst


 Configuration

Before using RagaAI Catalyst, you need to set up your credentials. You can do this by setting environment variables or passing them directly to the RagaAICatalyst class:

python
from ragaai_catalyst import RagaAICatalyst

catalyst = RagaAICatalyst(
    access_key="YOUR_ACCESS_KEY",
    secret_key="YOUR_SECRET_KEY",
    base_url="BASE_URL"
)

you'll need to generate authentication credentials:

1. Navigate to your profile settings
2. Select "Authenticate" 
3. Click "Generate New Key" to create your access and secret keys

!How to generate authentication keys

Note: Authetication to RagaAICatalyst is necessary to perform any operations below.


 Usage

 Project Management

Create and manage projects using RagaAI Catalyst:

python
 Create a project
project = catalyst.create_project(
    project_name="Test-RAG-App-1",
    usecase="Chatbot"
)

 Get project usecases
catalyst.project_use_cases()

 List projects
projects = catalyst.list_projects()
print(projects)

!Projects

 Dataset Management
Manage datasets efficiently for your projects:

py
from ragaai_catalyst import Dataset

 Initialize Dataset management for a specific project
dataset_manager = Dataset(project_name="project_name")

 List existing datasets
datasets = dataset_manager.list_datasets()
print("Existing Datasets:", datasets)

 Create a dataset from CSV
dataset_manager.create_from_csv(
    csv_path='path/to/your.csv',
    dataset_name='MyDataset',
    schema_mapping={'column1': 'schema_element1', 'column2': 'schema_element2'}
)

 Get project schema mapping
dataset_manager.get_schema_mapping()


!Dataset

For more detailed information on Dataset Management, including CSV schema handling and advanced usage, please refer to the Dataset Management documentation.


 Evaluation

Create and manage metric evaluation of your RAG application:

python
from ragaai_catalyst import Evaluation

 Create an experiment
evaluation = Evaluation(
    project_name="Test-RAG-App-1",
    dataset_name="MyDataset",
)

 Get list of available metrics
evaluation.list_metrics()

 Add metrics to the experiment
schema_mapping={
 

Top issues

Discussions

Latest release

2.2.4 v2.2.4 2025-06-23
Changes

- Bug-fix: Updation of external_id, metadata not working as expected while updating dataset_name
- Bug-fix: logs getting missed in load testing using Locust
- Bug-fix: Error in SDG while generating dataset
- Feat: Update project name in tracer on runtime

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON