Multi-tenancy at scale

[Fibs7000]

In the docs you guys mention that to achieve multitenancy one should create a schema and a own instance of kratos for each tenant.

From what I read in the cloud docs you are using this approach in the cloud as well.

• could you tell how well this scales? (eg. if we are talking about 200k plus tenants in one cluster)
• How is the operational overhead?
• Do you have any directions how you implemented this in k8s? Have a loadbalancer which decides to which kratos instance the requests are routed?
• Any suggestions at managing the same schema and potential migrations for the tenants?

thanks in advance!

Fabio

[aeneasr]

Hello Fabio, the guidance you find in the docs will not scale to hundreds of thousands of tenants unless you invest significant amount of resources on memory, CPU, storage, and database administration, SRE, operations, migrations, ... It still will scale much better than Keycloak which is known to have serious issues when scaling to more than 100 tenants (keycloak/keycloak#11074), but it will be a challenge, like running 5000 Kubernetes clusters.

In our cloud system we have spent an almost ridiculous amount of engineering and planning to build a multi tenant system which scales to millions of tenants. Our core team (myself included) has been working on this since early 2019 under guidance from cloud veterans like Lloyd Taylor and Lee Atchison as well as others. None of that source base is open source and it never will be. Multi-tenant architectures using purely open source will always be limited by the things stated above (memory, cpu, db, ops, sre).

Having said that we are already in conversation with major cloud providers to offer Ory technology as a multi-tenant system through our Ory Network system. We can spin up thousands of tenants within milliseconds and by using simple API calls line POST /projects. If you’re interested in this, please contact us so that we can better understand your use case and partnership opportunities! Of course we wont charge the fees listed on the pricing page but instead will together figure out a model that is both economical as well as scalable for you.

Hope this helps!

[Fibs7000]

Thx for the fast reply.

I definitely can understand why you are keeping this closed source.

I will definitely reach out and am looking forward to a good cooperation!

Br

[edardev]

is this a closed source project? 🤔

[vinckr]

Hello @edardev.
Our projects Ory Kratos, Ory Keto, Ory Oathkeeper, and Ory Hydra are all open source under the Apache 2 License - as well a whole lot of other projects that you can find under the Ory GitHub (and an overview here).

Ory Network - which is what is talked about above as "multi tenant system" - provides a managed way to use Ory software and uses the Ory Open Source projects under the hood. Ory Network is not open source (except from the aforementioned core components).

[localhosted]

From the docs:

its data model does not support this due to data, scalability, and operational complexity

Seems like this was done intentionally to keep multi-tenancy expensive and commercial?

[vinckr]

@localhosted
The intention of Ory's open source projects, including Kratos, is not to make multi-tenancy expensive or commercial. In fact, the open source projects are licensed under the Apache 2 License, which allows for free use, modification, and distribution.
However, scaling multi-tenancy in a purely open source solution can be challenging due to resource limitations related to memory, CPU, storage, database administration, SRE, operations, migrations, etc.

If you run a multi-tenancy system you have to invest.
Either in building it yourself in terms of time, code, and knowledge or you get a subscription at Ory Network. All Ory software is free for everyone to use without charge and Ory Network is the way to support the project financially and help making it sustainable long-term.

[brunocleite]

Thanks for the explanations. Is it possible to explain why Kratos does not support multi-tenancy, and what is really meant by multi-tenancy? I also found myself confused with this statement on Kratos documentation.
Wouldn't it be possible to create a field on user data tenantId and create rules based on that?

[vinckr]

Hey @brunocleite

Wouldn't it be possible to create a field on user data tenantId and create rules based on that?

Yes that is possible sure.

Multi-tenancy in that context means you have several "tenants" who do not share the database, configuration, UI, etc.
If you have a use case where tenants share the UI and its fine that all identities are in the same database - then you can use a "tenantId" metadata field and separate like this.