1.6 KiB
AGENTS.md
This file provides guidance to agents when working with code in this repository.
Project Overview
This project is an AI gateway that routes requests to various backend AI models. It provides a unified API that is compatible with the OpenAI API format.
Technology Stack
- Language: Go
- Web Framework: Gin
- ORM: GORM
- Database: SQLite
How to Run
To run the project, use the following command:
go run main.go
The server will start on port 8080.
Architecture
The project is divided into the following packages:
api: Contains the API handlers that process incoming requests.router: Implements the logic for selecting the appropriate backend model for a given request.db: Handles the database initialization and migrations.models: Defines the database schema.middleware: Contains the authentication middleware.logger: Provides asynchronous logging of requests to the database.
Authentication
The gateway uses API key authentication. The API key must be provided in the Authorization header as a Bearer token.
Routing
The gateway uses a virtual model system to route requests. Each virtual model can be associated with multiple backend models. When a request is received for a virtual model, the gateway selects the backend model with the highest priority that can accommodate the request's token count.
Logging
All requests are logged to the RequestLog table in the database. Logging is performed asynchronously to avoid impacting request processing time.