47 lines
1.6 KiB
Markdown
47 lines
1.6 KiB
Markdown
# AGENTS.md
|
|
|
|
This file provides guidance to agents when working with code in this repository.
|
|
|
|
## Project Overview
|
|
|
|
This project is an AI gateway that routes requests to various backend AI models. It provides a unified API that is compatible with the OpenAI API format.
|
|
|
|
### Technology Stack
|
|
|
|
* **Language:** Go
|
|
* **Web Framework:** Gin
|
|
* **ORM:** GORM
|
|
* **Database:** SQLite
|
|
|
|
### How to Run
|
|
|
|
To run the project, use the following command:
|
|
|
|
```bash
|
|
go run main.go
|
|
```
|
|
|
|
The server will start on port `8080`.
|
|
|
|
### Architecture
|
|
|
|
The project is divided into the following packages:
|
|
|
|
* `api`: Contains the API handlers that process incoming requests.
|
|
* `router`: Implements the logic for selecting the appropriate backend model for a given request.
|
|
* `db`: Handles the database initialization and migrations.
|
|
* `models`: Defines the database schema.
|
|
* `middleware`: Contains the authentication middleware.
|
|
* `logger`: Provides asynchronous logging of requests to the database.
|
|
|
|
### Authentication
|
|
|
|
The gateway uses API key authentication. The API key must be provided in the `Authorization` header as a Bearer token.
|
|
|
|
### Routing
|
|
|
|
The gateway uses a virtual model system to route requests. Each virtual model can be associated with multiple backend models. When a request is received for a virtual model, the gateway selects the backend model with the highest priority that can accommodate the request's token count.
|
|
|
|
### Logging
|
|
|
|
All requests are logged to the `RequestLog` table in the database. Logging is performed asynchronously to avoid impacting request processing time. |