AI Assistant

Built-in AI that understands your API

Bytedocs includes a built-in AI assistant that has complete context of your API and can answer questions, provide code examples, and help debug integration issues.

Supported AI Providers

Bytedocs supports multiple AI providers:

ProviderModelsBest For
OpenAIGPT-4, GPT-4o-mini, GPT-3.5-turboBest overall quality
Google Geminigemini-1.5-flash, gemini-1.5-proFast responses, free tier
Anthropic Claudeclaude-3-sonnet, claude-3-haikuTechnical accuracy
OpenRouter100+ modelsAccess to all models

Setup

1. Get API Key

Choose your provider and get an API key:

2. Configure

Environment Variables:

BYTEDOCS_AI_ENABLED=true
BYTEDOCS_AI_PROVIDER=openai
BYTEDOCS_AI_API_KEY=sk-...
BYTEDOCS_AI_MODEL=gpt-4o-mini

Go Configuration:

config := &core.Config{
    AIConfig: &ai.AIConfig{
        Enabled:  true,
        Provider: "openai",  // or: gemini, claude, openrouter
        APIKey:   os.Getenv("OPENAI_API_KEY"),
        Features: ai.AIFeatures{
            ChatEnabled:         true,
            Model:               "gpt-4o-mini",
            MaxTokens:           1000,
            MaxCompletionTokens: 500,
            Temperature:         0.7,
        },
    },
}

Laravel Configuration:

// config/bytedocs.php
'ai' => [
    'enabled' => env('BYTEDOCS_AI_ENABLED', true),
    'provider' => env('BYTEDOCS_AI_PROVIDER', 'openai'),
    'api_key' => env('BYTEDOCS_AI_API_KEY'),
    'features' => [
        'chat_enabled' => true,
        'model' => env('BYTEDOCS_AI_MODEL', 'gpt-4o-mini'),
        'max_tokens' => 1000,
        'temperature' => 0.7,
    ],
],

3. Use the Assistant

Visit /docs and click the chat icon in the bottom-right corner.

How It Works

Complete API Context

When you ask a question, the AI receives:

  1. Full OpenAPI Specification: All endpoints, schemas, parameters
  2. Validation Rules: Required fields, data types, constraints
  3. Example Values: Real example requests and responses
  4. Your Question: What you're asking about

Example Context

=== API SPECIFICATION ===
API Title: My API
Version: 1.0.0

=== ENDPOINTS ===
GET /users/{id}
  Parameters:
    - id (path, integer, required)
  Response 200: User object
    - id: integer
    - name: string (required)
    - email: string (required, format: email)

POST /users
  Request Body:
    - name: string (required)
    - email: string (required)
  Response 201: User object

=== STRICT INSTRUCTIONS ===
- ONLY answer questions related to this API
- Use the provided specification as source of truth
- Provide code examples when helpful

What You Can Ask

API Usage Questions

"How do I create a user?"

To create a user, send a POST request to /users with JSON body:

{
  "name": "John Doe",
  "email": "john@example.com"
}

Example with cURL:
curl -X POST http://localhost:8080/users \
  -H "Content-Type: application/json" \
  -d '{"name":"John Doe","email":"john@example.com"}'

Example with JavaScript:
fetch('http://localhost:8080/users', {
  method: 'POST',
  headers: {'Content-Type': 'application/json'},
  body: JSON.stringify({
    name: 'John Doe',
    email: 'john@example.com'
  })
})

Integration Help

"How do I authenticate with this API?"

This API uses Bearer token authentication.

Add the Authorization header to your requests:
Authorization: Bearer YOUR_TOKEN_HERE

Example:
curl -H "Authorization: Bearer abc123" \
  http://localhost:8080/users

Code Examples

"Show me Python code to get all users"

import requests

response = requests.get('http://localhost:8080/users')
users = response.json()

for user in users:
    print(f"{user['name']} - {user['email']}")

Debugging Help

"Why am I getting a 400 error when creating a user?"

A 400 error typically means validation failed. Check that:

1. name is provided (required)
2. email is provided (required)
3. email is a valid email format

Make sure your request includes both fields:
{
  "name": "John Doe",
  "email": "john@example.com"
}

Endpoint Discovery

"What endpoints are available for user management?"

Available user endpoints:

1. GET /users - List all users
2. GET /users/{id} - Get specific user
3. POST /users - Create new user
4. PUT /users/{id} - Update user
5. DELETE /users/{id} - Delete user

Model Selection Guide

  • Cost: $0.15/1M input tokens, $0.60/1M output tokens
  • Speed: Fast
  • Quality: Excellent for API questions
  • Best for: Most use cases

GPT-4

  • Cost: $30/1M input tokens, $60/1M output tokens
  • Speed: Moderate
  • Quality: Best available
  • Best for: Complex queries, code generation

Gemini 1.5 Flash (Free Tier Available)

  • Cost: Free tier: 15 RPM, paid: $0.07/1M tokens
  • Speed: Very fast
  • Quality: Good
  • Best for: High volume, budget-conscious

Claude 3 Haiku

  • Cost: $0.25/1M input tokens, $1.25/1M output tokens
  • Speed: Very fast
  • Quality: Good technical accuracy
  • Best for: Technical documentation

OpenRouter

  • Cost: Varies by model
  • Speed: Varies
  • Quality: Access to 100+ models
  • Best for: Flexibility, trying different models

Configuration Options

Temperature

Controls response randomness:

Temperature: 0.0  // Deterministic, same answer every time
Temperature: 0.7  // Balanced (recommended)
Temperature: 1.0  // More creative/varied

Max Tokens

Limits response length:

MaxTokens: 500   // Short responses
MaxTokens: 1000  // Medium (recommended)
MaxTokens: 2000  // Long, detailed responses

Max Completion Tokens

Limits only the AI's response (excluding prompt):

MaxCompletionTokens: 500  // Shorter answers, lower cost

Cost Management

Estimate Costs

Approximate costs per question:

ModelCost per Question
GPT-4o-mini$0.001 - $0.002
GPT-4$0.05 - $0.10
Gemini FlashFree (15 RPM)
Claude Haiku$0.002 - $0.004

Optimization Tips

  1. Use smaller models for simple questions (GPT-4o-mini, Gemini Flash)
  2. Limit max tokens to reduce costs
  3. Use free tiers when available (Gemini)
  4. Cache common questions (future feature)
  5. Monitor usage via provider dashboards

Security & Privacy

What Gets Sent

Sent to AI:

  • Your question
  • API specification (routes, schemas)
  • Validation rules
  • Example values

NOT Sent:

  • Actual API data
  • User information
  • Secrets or credentials
  • Handler implementation code
  • Database queries

Best Practices

  1. Use environment variables for API keys
  2. Restrict documentation access if using sensitive API schemas
  3. Monitor AI provider usage for unexpected activity
  4. Consider self-hosted models for highly sensitive APIs

Troubleshooting

AI Not Responding

Check:

  1. API key is correct
  2. Provider is reachable
  3. Rate limits not exceeded
  4. Model name is correct

Debug:

# Test your API key
curl https://api.openai.com/v1/models \
  -H "Authorization: Bearer $OPENAI_API_KEY"

"Rate limit exceeded"

Solution:

  • Wait a few minutes
  • Upgrade your plan
  • Switch to different provider
  • Reduce max_tokens

Poor Quality Responses

Try:

  1. Use a better model (GPT-4 vs GPT-4o-mini)
  2. Adjust temperature (lower for factual, higher for creative)
  3. Rephrase your question
  4. Provide more context in your question

"Cannot answer non-API questions"

Expected Behavior: The AI is instructed to only answer API-related questions.

To override (if needed):

// This would require code modification
// Not recommended for production

API Endpoint

The AI chat is accessible via:

POST /docs/chat
Content-Type: application/json

{
  "message": "How do I create a user?"
}

Response:

{
  "response": "To create a user, send a POST request to /users...",
  "usage": {
    "prompt_tokens": 1234,
    "completion_tokens": 567,
    "total_tokens": 1801
  },
  "model": "gpt-4o-mini"
}

What's Next?