Routers and Routing strategies

Note

This section currently pertains only to the Django ORM as the other ORM libraries that ship with this package currently don't provide support for routing to multiple databases at the ORM level in a thread-safe manner.

Since the Django ORM supports routing to multiple databases at the ORM level using a concept called database routers, there are a few scenarios to consider with respect to defining custom routers and the different routing strategies that are possible.

A router in Django when implemented, is responsible for providing one or more of the following functionalities:

• Provides a database for every write operation made via the ORM.
• Provides a database for every read operation made via the ORM.
• Tells whether relations between objects should be allowed or not.
• Tells which models should be migrated onto a particular database.

Scenario 1: Single DB replicated across tenants

Consider the following example of a conventionally built single-tenant Django app.

# settings.py file
import os

DATABASES = {
    "default": {
        "NAME": os.environ["PG_DATABASE"],
        "USER": os.environ.get("PG_USER"),
        "PASSWORD": os.environ.get("PG_PASSWORD"),
        "HOST": os.environ["PG_HOST"],
        "PORT": os.environ["PG_PORT"],
        "OPTIONS": {...}
    }
}

# KV store settings for the above template
{
    "some_tenant_a_prefix_default": {
        "NAME": "some_value",
        "USER": "some_value",
        "PASSWORD": "some_value",
        "HOST": "some_value",
        "PORT": "some_value",      
    },
    "some_tenant_b_prefix_default": {
        "NAME": "some_value",
        "USER": "some_value",
        "PASSWORD": "some_value",
        "HOST": "some_value",
        "PORT": "some_value",      
    }
}

From the above example, we infer that the default key in the DATABASES setting is actually a template and the actual configuration for this template, for each tenant, is defined in the KV store.

Now for this scenario, the expected behavior of the router is as follows:

• All writes should be sent to the respective tenant's DB with alias of the form some_tenant_prefix_default.
• All reads should be sent to the respective tenant's DB with alias of the form some_tenant_prefix_default.
• All relations between objects should be allowed within a particular tenant's DB.
• All models present in the app should be migrated to all tenant DBs'.

The above behavior is exactly what the app will get when it plugs-in the DjangoOrmRouter class that ships with the package.

# settings.py

...
DATABASE_ROUTERS = ['tenant_router.orm_backends.django_orm.router.DjangoOrmRouter']
...

Scenario 2: Central / Tenant-specific DBs

This is a typical scenario in a multi-tenant cloud environment wherein the web app interacts with both a tenant-specific DB and a central DB, which is common across tenants. The diagram below illustrates this case.

// diag goes here.

The configuration in settings.py could be as follows:

import os
...

DATABASES = {
    # template alias
    "default": {
        "NAME": os.environ["PG_DATABASE"],
        "USER": os.environ.get("PG_USER"),
        "PASSWORD": os.environ.get("PG_PASSWORD"),
        "HOST": os.environ["PG_HOST"],
        "PORT": os.environ["PG_PORT"],
        "OPTIONS": {...}
    },
    # reserved key
    "central_db": {
        "NAME": os.environ["PG_DATABASE"],
        "USER": os.environ.get("PG_USER"),
        "PASSWORD": os.environ.get("PG_PASSWORD"),
        "HOST": os.environ["PG_HOST"],
        "PORT": os.environ["PG_PORT"],
        "OPTIONS": {...}
    }
}

# KV store settings for the above template
{
    "some_tenant_a_prefix_default": {
        "NAME": "some_value",
        "USER": "some_value",
        "PASSWORD": "some_value",
        "HOST": "some_value",
        "PORT": "some_value",      
    },
    "some_tenant_b_prefix_default": {
        "NAME": "some_value",
        "USER": "some_value",
        "PASSWORD": "some_value",
        "HOST": "some_value",
        "PORT": "some_value",      
    }
}

From the above example, we infer that the default key is treated as a template alias for which the actual configuration comes from the KV store, for each tenant. However the central_db key is treated as a reserved alias as its configuration comes from the env and not the KV store since it remains the same across all tenants.

In such a case, the expected behavior of the router is as follows:

• All writes should be sent either to a tenant-specific DB or the central DB depending on some condition.
• All reads should be sent either to a tenant-specific DB or the central DB depending on some condition.
• Relations between objects should be allowed within the same DB (either tenant-specific / central)
• A migration strategy which decides whether a model should be migrated on to the central DB or tenant-specific DB.

In order to achieve the above, the following modifications should be made:

1. Action:
   Provide a router for each reserved alias defined and stack it on top of the DjangoOrmRouter class in the DATABASE_ROUTERS setting.
   Purpose:
   Since all routers defined for each reserved alias would be consulted in order before falling back to the DjangoOrmRouter, it ensures that custom routing/migration logic for each reserved alias would be tried and executed before executing the routing/migration logic for template aliases.
   
   

2. Action:
   Provide keys which should be treated as reserved aliases to the manager class.
   Purpose:
   This helps in isolating the template aliases from the reserved ones so as to power up a few public APIs of the manager class and also for other internal purposes.
   
   

3. Action:
   A custom migration strategy callable should be plugged into the DjangoOrmRouter class. This would be called from within the allow_migrate method whose return value will be the value that this callable returns.
   Purpose:
   This callable would decide whether a particular model gets migrated onto DBs' defined as part of the template aliases.

Example:

# custom router for the 'central_db' reserved alias
class CustomRouter:

    def db_for_read(self, model, **hints):
        if some_condition:
            return "central_db"
        return None

    def db_for_write(self, model, **hints):
        if some_condition:
            return "central_db"

        return None

    def allow_relation(self, *args, **kwargs):
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        if some_condition:
            return True
        else:
            return False
        return None


# sample migrate_strategy_callable
def migrate_strategy(
        router, db, app_label, model_name=None, **hints
):
    if some_condition:
        return True
    else:
        return False

    return None


# modified settings
DATABASE_ROUTERS = [
    'path.to.CustomRouter',
    'tenant_router.orm_backends.django_orm.router.DjangoOrmRouter'
]

TENANT_ROUTER_ORM_SETTINGS = {
    'django_orm': {
        'MANAGER': 'tenant_router.orm_backends.django_orm.manager.DjangoOrmManager',
        'SETTINGS_KEY': 'DATABASES',
        'OPTIONS': {
            # marking the 'central_db' key as a reserved alias
            'RESERVED_CONN_ALIASES': {'central_db'},
            # plugging in the callable which defines the custom migrate strategy
            'ROUTER_OPTS': {
                'MIGRATE_STRATEGY': 'path.to.migrate_strategy_callable'
            }
        }
    }
}