Managing LLM Models

This guide covers the full lifecycle of LLM models in Open Chat Studio: adding new models, deprecating models that are being phased out, and fully deleting models. See the user documentation for how deprecation and removal are handled automatically from the end user's perspective.

Adding a Model

Step 1: Define parameters (if needed)

If the new model has a distinct set of configurable parameters (e.g. a different reasoning effort range), add a new parameters class in apps/service_providers/llm_service/model_parameters.py:

class GPT55Parameters(LLMModelParamBase):
    # reasoning.effort: none, low, medium (default), high, xhigh
    effort: GPT52ReasoningEffortParameter = Field(
        title="Reasoning Effort",
        default=GPT52ReasoningEffortParameter.MEDIUM,
        json_schema_extra=UiSchema(widget=Widgets.select, enum_labels=GPT52ReasoningEffortParameter.labels),
    )
    # ... other fields

If an existing parameter class fits, reuse it directly.

Step 2: Register the model

In apps/service_providers/llm_service/default_models.py, add the model to the relevant provider list(s). The token_limit should reflect the model's context window:

DEFAULT_LLM_PROVIDER_MODELS = {
    "openai": [
        # ...
        Model("gpt-5.5", 1050000, parameters=GPT55Parameters),
    ],
}

Import the parameters class at the top of the file if you created a new one.

Step 3: Create a Django migration

from django.db import migrations

from apps.service_providers.migration_utils import llm_model_migration


class Migration(migrations.Migration):
    dependencies = [
        ("service_providers", "0052_previous_migration"),
    ]

    operations = [
        # Seeds the new model into LlmProviderModel for all teams
        llm_model_migration(),
    ]

Step 4: Remove llm_model_migration() from previous migrations

llm_model_migration() re-syncs the entire model list each time it runs. It should only run once per deployment — in the newest migration. Strip it from all earlier migrations that still call it:

grep -rl "llm_model_migration\|notify_deprecated_models\|remove_deprecated_models" \
    apps/service_providers/migrations/

For each migration in that list (except your new one), remove the call and its import, leaving operations = [] if nothing else remains.

Why? Data migrations like notify_deprecated_models and remove_deprecated_models would otherwise do the same work repeatedly on every deploy. The notifications themselves are announced once per team per model, so a slip here no longer spams teams — but the redundant scans still cost time on every migrate.

Step 5: Seed pricing for the model

Registering a model does not give it a price. Add pricing rows for the new model to the seed file apps/cost_tracking/seed_data/llm_pricing.json, then apply them with load_ai_pricing via the load_pricing_data() helper. This can go in the same migration that seeds the model (Step 3):

from django.db import migrations

from apps.cost_tracking.migration_utils import load_pricing_data
from apps.service_providers.migration_utils import llm_model_migration


class Migration(migrations.Migration):
    dependencies = [
        ("service_providers", "0061_previous_migration"),
        ("cost_tracking", "0001_initial"),  # required so the PricingRule table exists
    ]

    operations = [
        llm_model_migration(),
        load_pricing_data(),
    ]

load_ai_pricing is idempotent and supersedes on change, so the same step applies to pricing changes for existing models — edit the prices in the seed JSON and add a load_pricing_data() migration. Unlike llm_model_migration(), it is safe to leave earlier load_pricing_data() calls in place; each run only reseeds the current JSON. You can also run it locally after editing the seed file:

python manage.py load_ai_pricing

Deprecating a Model

Deprecation marks a model as unsupported while keeping it accessible. Deprecated models show a warning in the UI. If a replacement is specified, users are notified so they can migrate before the model is eventually deleted.

Step 1: Update Model Definitions

In apps/service_providers/llm_service/default_models.py, set deprecated=True on the model. Optionally set replacement to the name of the recommended successor model — this is included in the notification to affected teams:

DEFAULT_LLM_PROVIDER_MODELS = {
    "openai": [
        Model("gpt-4", k(8), deprecated=True, replacement="gpt-4o"),
        # ... other models
    ],
}

Step 2: Create Django Migration

from django.db import migrations

from apps.data_migrations.utils.migrations import RunDataMigration
from apps.service_providers.migration_utils import llm_model_migration


class Migration(migrations.Migration):
    dependencies = [
        ("service_providers", "0040_previous_migration"),
    ]

    operations = [
        # Update model list (marks deprecated models in DB)
        llm_model_migration(),
        # notify affected teams about the deprecation and recommended replacement
        RunDataMigration("notify_deprecated_models", command_options={"force": True}),
    ]

Step 3: Remove these operations from all previous migrations

Same rule as when adding — llm_model_migration() and data migrations should only run once per deployment. Find and strip them from earlier migrations:

grep -rl "llm_model_migration\|notify_deprecated_models\|remove_deprecated_models" \
    apps/service_providers/migrations/

Deleting a Model

Deletion fully removes a model from the platform. References are either auto-migrated to a replacement or cleared, and affected teams are notified.

Step 1: Update Model Definitions

In apps/service_providers/llm_service/default_models.py:

1. Remove the model from DEFAULT_LLM_PROVIDER_MODELS:

   DEFAULT_LLM_PROVIDER_MODELS = {
       "openai": [
           # Model("gpt-4", k(8), deprecated=True),  # Remove this line
           Model("gpt-4o", 128000),
           # ... other models
       ],
   }
   

2. Add the model to DELETED_MODELS. Optionally include a replacement model name to auto-migrate references instead of clearing them:

   DELETED_MODELS = [
       # Without replacement: references to this model are set to None
       ("openai", "gpt-4"),
       # With replacement: references are updated to point to the replacement model
       ("anthropic", "claude-2.0", "claude-3-5-sonnet-latest"),
   ]
   

Step 2: Create Django Migration

from django.db import migrations

from apps.data_migrations.utils.migrations import RunDataMigration
from apps.service_providers.migration_utils import llm_model_migration


class Migration(migrations.Migration):
    dependencies = [
        ("service_providers", "0040_previous_migration"),
    ]

    operations = [
        # Update model list
        llm_model_migration(),
        # Clean up references and notify teams
        RunDataMigration("remove_deprecated_models", command_options={"force": True}),
    ]

Step 3: Remove these operations from all previous migrations

Same as for deprecation — strip llm_model_migration() and data migrations from all earlier migrations except your new one.

Auto-migrating Usages to a Replacement Model

When deleting a model, you can specify a replacement to automatically update all references to point to a different model. This avoids leaving users with broken or null model references.

Add a third element to the DELETED_MODELS tuple:

DELETED_MODELS = [
    ("openai", "gpt-4", "gpt-4o"),  # All references updated to gpt-4o
]

When a replacement is specified: - Pipeline node references are updated to use the replacement model - Direct FK references (assistants, analyses, etc.) are updated to the replacement - The deletion notification tells teams which replacement was applied

When no replacement is specified: - Pipeline node references are set to None - Teams are still notified that references were cleared

User Notifications

Both the deprecation notification and the deletion command use the OCS notifications system (see notifications guide) rather than admin emails. Notifications are sent to team members with the service_providers.change_llmprovidermodel permission.

Notifications include: - Which model was deprecated/deleted - The replacement model (if any) - Counts of affected chatbots, pipelines, and assistants

Slug conventions

Each model is announced once per team

notify_deprecated_models re-scans every model marked deprecated=True on each run, not just the ones your migration deprecated. So a release that deprecates one model would otherwise re-announce every model deprecated before it — a team that had already read the earlier notice would see it marked unread again and get a second email.

Both model notifications therefore pass once_per_event_type=True to create_notification, which skips the notification if a member of the team has already been notified about that slug + model name. Nothing extra is needed in your migration; keep passing force=True.

The check is keyed on the recipients (EventUser), not on the EventType, because the event type is created even when the run reaches nobody — if no team member held service_providers.change_llmprovidermodel at the time, or every eligible member was on Do Not Disturb, the next run still announces it.

The consequence to be aware of: a team is told about a given model exactly once. If you need to re-announce the same model (say the recommended replacement changed), change the event_data so the identifier differs, or clear the team's existing EventType for it.

Maintenance: Clearing Old Models from DELETED_MODELS

The DELETED_MODELS list should be cleaned up periodically to avoid accumulating stale entries.

When to Remove Models

It is safe to remove models from DELETED_MODELS when both conditions are met:

1. The remove_deprecated_models command has run successfully in all environments (development, staging, production)
2. The models have been in the main branch for more than 1 month

Why Wait 1 Month?

• Ensures all environments have executed the migration
• Gives time for any rollback scenarios
• Allows thorough testing across deployment cycles
• Accounts for environments that deploy less frequently

How to Clean Up

1. Verify migration has run everywhere:

   # Check if migration is applied
   python manage.py custom_migrations list --name remove_deprecated_models
   

2. Check git history to confirm models have been in main for 1+ month:

   # Find when models were added to DELETED_MODELS
   git log -p --all -S 'DELETED_MODELS' -- apps/service_providers/llm_service/default_models.py
   

3. Remove old entries from DELETED_MODELS:

   DELETED_MODELS = [
       # Keep recent additions (< 1 month in main)
       ("openai", "gpt-4"),
       # Remove these - added 2 months ago, migrated everywhere
       # ("azure", "gpt-35-turbo"),
       # ("groq", "llama3-70b-8192"),
   ]
   

4. Commit the cleanup:

   git add apps/service_providers/llm_service/default_models.py
   git commit -m "chore: clean up DELETED_MODELS list"
   

Important Notes

• Do not remove models that are still in active migrations
• Do not remove models if any environment hasn't deployed the migration yet
• This cleanup is purely for code hygiene; it doesn't affect functionality once migrations have run