Bodhi is a tool that allows you to run language models locally on your computer using open-source technology. It offers a chat interface with markdown support, prioritizes privacy by processing data locally, and provides model management through HuggingFace. It features API compatibility, local processing for improved privacy, and enhanced performance with llama.cpp and GPU acceleration.
Features
Built-in Chat UI
Intuitive chat interface with full markdown and settings for seamless user interactions.
Privacy First
Run everything locally on your machine with complete data control, ensuring privacy.
Model Management
One-click downloads from HuggingFace, making it easy to manage and deploy models.
API Compatibility
Drop-in replacement for OpenAI APIs, allowing you to use your existing code and tools seamlessly.
Local Processing
Run models on your hardware for enhanced privacy and control, without relying on cloud-based solutions.
High Performance
Optimized inference with llama.cpp and GPU acceleration for fast and efficient processing.
Three-Panel Design
Bodhi App's UI features a clean three-panel design that includes the Chat History Panel, Main Chat Panel, and Settings Panel, allowing users to manage past conversations, interact with the AI assistant, and configure various parameters.
Real-time Streaming
The Main Chat Panel offers real-time streaming of AI responses, allowing users to see responses as they are generated, enhancing interaction speed and fluidity.
Markdown and Syntax Highlighting
Responses in the Main Chat Panel support markdown, making content rich and readable with syntax highlight for code blocks.
Adjustable AI Parameters
In the Settings Panel, users can adjust AI behavior with parameters like Temperature, Top P, Seed, Max Tokens, and Stop Words to customize creativity, response length, and control flow.
Security Notice and API Token Privacy
API tokens are stored in plain text on your browser; users must remove them after use for security. The history is stored locally, ensuring privacy.
Collapsible Panels
Both the Chat History and Settings Panels are collapsible, allowing users to maximize workspace when needed by hiding unused sections.
Asynchronous Processing
Download requests are handled in the background, allowing you to continue using the app while the download occurs. You can monitor the status of downloads on the Download Models page.
Idempotency
If a download request for a model already exists based on repository, filename, and snapshot, the system returns the existing request rather than creating a duplicate.
Error Reporting
If an error occurs, such as the file already existing, the system notifies you with an error message to indicate the issue.
Download Monitoring
On the Downloads page, you can view a list of all download requests with details such as repository, filename, status, and timestamp. You can expand entries with errors to view more details.
Interactive API Documentation with Swagger UI
Allows users to explore, understand, and integrate with RESTful APIs through an interactive graphical interface. Enables browsing of detailed API specifications, endpoint descriptions, and data schemas, as well as testing APIs with real-time requests.
Continuous Update of API Endpoints
Ensures that all public and authenticated endpoints are kept up to date as new features are added, providing users with the latest and most accurate API information.
Authentication Methods
Provides information on available authentication methods, such as browser session and API token-based authentication, facilitating secure API interactions.
Quick and Direct Access to Documentation
Users can directly access interactive API documentation via the app's menu or through a specific URL, ensuring easy access to comprehensive API resources.
Automatic OpenAPI Specification Generation
Automatically generates the OpenAPI specification from the backend, ensuring that the documentation is always up-to-date with the current state of the codebase.
Chat UI
A comprehensive guide to using Bodhi App's Chat Interface for seamless communication and interaction.
Model Downloads
Download model files from HuggingFace repositories into Bodhi's local storage for use within the app.
Model Aliases
Configure and manage your model alias configurations in the Bodhi App to streamline model usage.
Model Files
View and manage the model files downloaded from HuggingFace, ensuring accessibility and organization.
App Settings
Configure and manage application settings directly from the UI in Bodhi App to customize user experience.
API Tokens
Manage API tokens for secure external access to Bodhi App, allowing integration with other services.
API Reference
API documentation and integration guide for Bodhi App, assisting developers in connecting with the platform.
OpenAI Compatible Chat and Model APIs
Integrate seamlessly with any tool that accepts OpenAI's API endpoints. This feature makes it easy to connect with various AI platforms.
Ollama Compatible Chat and Model APIs
Offer endpoints, providing flexibility and integration options within the Ollama ecosystem. This allows users to work within Ollama's AI framework.
Advanced Security & Authentication
Provides a flexible security policy with modes like authenticated mode with role-based access control, and non-authenticated mode for easy setup.
Built-in Chat UI
Ready-to-use chat UI with markdown support and multi-conversation management. It offers customization and real-time feedback for seamless integration with external chat fronts.
Model Alias
Easily create alias configurations for inference by specifying parameters and settings, allowing seamless switching without restarts.
Model Files Management
Manage AI model files within the HuggingFace ecosystem, downloading models into your local folder for inference, which saves disk space and bandwidth.
Robust API Access & Developer Tools
Comprehensive API documentation and developer tools are provided, supporting generating API tokens and integrating Bodhi App into external applications.
Future-Ready and Cross-Platform
Designed for desktop, mobile, and web, with plans to expand AI API integrations and enhance authorization flows.
Guided Onboarding & Ongoing Support
Guided journey through installation and continuous support for smooth user experience and technical assistance.