Managing LLM Models

This guide covers the full lifecycle of LLM models in Open Chat Studio: adding new models, deprecating models that are being phased out, and fully deleting models.

Adding a Model

Step 1: Define parameters (if needed)

If the new model has a distinct set of configurable parameters (e.g. a different reasoning effort range), add a new parameters class in apps/service_providers/llm_service/model_parameters.py:

class GPT55Parameters(LLMModelParamBase):
    # reasoning.effort: none, low, medium (default), high, xhigh
    effort: GPT52ReasoningEffortParameter = Field(
        title="Reasoning Effort",
        default=GPT52ReasoningEffortParameter.MEDIUM,
        json_schema_extra=UiSchema(widget=Widgets.select, enum_labels=GPT52ReasoningEffortParameter.labels),
    )
    # ... other fields

If an existing parameter class fits, reuse it directly.

Step 2: Register the model

In apps/service_providers/llm_service/default_models.py, add the model to the relevant provider list(s). The token_limit should reflect the model's context window:

DEFAULT_LLM_PROVIDER_MODELS = {
    "openai": [
        # ...
        Model("gpt-5.5", 1050000, parameters=GPT55Parameters),
    ],
}

Import the parameters class at the top of the file if you created a new one.

Step 3: Create a Django migration

from django.db import migrations

from apps.service_providers.migration_utils import llm_model_migration


class Migration(migrations.Migration):
    dependencies = [
        ("service_providers", "0052_previous_migration"),
    ]

    operations = [
        # Seeds the new model into LlmProviderModel for all teams
        llm_model_migration(),
    ]

Step 4: Remove llm_model_migration() from previous migrations

llm_model_migration() re-syncs the entire model list each time it runs. It should only run once per deployment — in the newest migration. Strip it from all earlier migrations that still call it:

grep -rl "llm_model_migration\|notify_deprecated_models\|remove_deprecated_models" \
    apps/service_providers/migrations/

For each migration in that list (except your new one), remove the call and its import, leaving operations = [] if nothing else remains.

Why? Data migrations like notify_deprecated_models and remove_deprecated_models would fire multiple times, sending duplicate notifications to teams.

Deprecating a Model

Deprecation marks a model as unsupported while keeping it accessible. Deprecated models show a warning in the UI. If a replacement is specified, users are notified so they can migrate before the model is eventually deleted.

Step 1: Update Model Definitions

In apps/service_providers/llm_service/default_models.py, set deprecated=True on the model. Optionally set replacement to the name of the recommended successor model — this is included in the notification to affected teams:

DEFAULT_LLM_PROVIDER_MODELS = {
    "openai": [
        Model("gpt-4", k(8), deprecated=True, replacement="gpt-4o"),
        # ... other models
    ],
}

Step 2: Create Django Migration

from django.db import migrations

from apps.data_migrations.utils.migrations import RunDataMigration
from apps.service_providers.migration_utils import llm_model_migration


class Migration(migrations.Migration):
    dependencies = [
        ("service_providers", "0040_previous_migration"),
    ]

    operations = [
        # Update model list (marks deprecated models in DB)
        llm_model_migration(),

        # notify affected teams about the deprecation and recommended replacement
        RunDataMigration("notify_deprecated_models", command_options={"force": True}),
    ]

Step 3: Remove these operations from all previous migrations

Same rule as when adding — llm_model_migration() and data migrations should only run once per deployment. Find and strip them from earlier migrations:

grep -rl "llm_model_migration\|notify_deprecated_models\|remove_deprecated_models" \
    apps/service_providers/migrations/

Deleting a Model

Deletion fully removes a model from the platform. References are either auto-migrated to a replacement or cleared, and affected teams are notified.

Step 1: Update Model Definitions

In apps/service_providers/llm_service/default_models.py:

1. Remove the model from DEFAULT_LLM_PROVIDER_MODELS:

   DEFAULT_LLM_PROVIDER_MODELS = {
       "openai": [
           # Model("gpt-4", k(8), deprecated=True),  # Remove this line
           Model("gpt-4o", 128000),
           # ... other models
       ],
   }
   

2. Add the model to DELETED_MODELS. Optionally include a replacement model name to auto-migrate references instead of clearing them:

   DELETED_MODELS = [
       # Without replacement: references to this model are set to None
       ("openai", "gpt-4"),
   
       # With replacement: references are updated to point to the replacement model
       ("anthropic", "claude-2.0", "claude-3-5-sonnet-latest"),
   ]
   

Step 2: Create Django Migration

from django.db import migrations

from apps.data_migrations.utils.migrations import RunDataMigration
from apps.service_providers.migration_utils import llm_model_migration


class Migration(migrations.Migration):
    dependencies = [
        ("service_providers", "0040_previous_migration"),
    ]

    operations = [
        # Update model list
        llm_model_migration(),

        # Clean up references and notify teams
        RunDataMigration("remove_deprecated_models", command_options={"force": True}),
    ]

Step 3: Remove these operations from all previous migrations

Same as for deprecation — strip llm_model_migration() and data migrations from all earlier migrations except your new one.

Auto-migrating Usages to a Replacement Model

When deleting a model, you can specify a replacement to automatically update all references to point to a different model. This avoids leaving users with broken or null model references.

Add a third element to the DELETED_MODELS tuple:

DELETED_MODELS = [
    ("openai", "gpt-4", "gpt-4o"),  # All references updated to gpt-4o
]

When a replacement is specified: - Pipeline node references are updated to use the replacement model - Direct FK references (assistants, analyses, etc.) are updated to the replacement - The deletion notification tells teams which replacement was applied

When no replacement is specified: - Pipeline node references are set to None - Teams are still notified that references were cleared

User Notifications

Both the deprecation notification and the deletion command use the OCS notifications system (see notifications guide) rather than admin emails. Notifications are sent to team members with the service_providers.change_llmprovidermodel permission.

Notifications include: - Which model was deprecated/deleted - The replacement model (if any) - Counts of affected chatbots, pipelines, and assistants

Slug conventions

Maintenance: Clearing Old Models from DELETED_MODELS

The DELETED_MODELS list should be cleaned up periodically to avoid accumulating stale entries.

When to Remove Models

It is safe to remove models from DELETED_MODELS when both conditions are met:

1. The remove_deprecated_models command has run successfully in all environments (development, staging, production)
2. The models have been in the main branch for more than 1 month

Why Wait 1 Month?

• Ensures all environments have executed the migration
• Gives time for any rollback scenarios
• Allows thorough testing across deployment cycles
• Accounts for environments that deploy less frequently

How to Clean Up

1. Verify migration has run everywhere:

   # Check if migration is applied
   python manage.py custom_migrations list --name remove_deprecated_models
   

2. Check git history to confirm models have been in main for 1+ month:

   # Find when models were added to DELETED_MODELS
   git log -p --all -S 'DELETED_MODELS' -- apps/service_providers/llm_service/default_models.py
   

3. Remove old entries from DELETED_MODELS:

   DELETED_MODELS = [
       # Keep recent additions (< 1 month in main)
       ("openai", "gpt-4"),
   
       # Remove these - added 2 months ago, migrated everywhere
       # ("azure", "gpt-35-turbo"),
       # ("groq", "llama3-70b-8192"),
   ]
   

4. Commit the cleanup:

   git add apps/service_providers/llm_service/default_models.py
   git commit -m "chore: clean up DELETED_MODELS list"
   

Important Notes

• Do not remove models that are still in active migrations
• Do not remove models if any environment hasn't deployed the migration yet
• This cleanup is purely for code hygiene; it doesn't affect functionality once migrations have run