How to handle multi-tenancy in Cumul.io

Multi-tenancy is the ability of software to serve multiple clients. This means that the platform is capable of keeping data of different clients separate. There are multiple ways to implement this and Cumul.io aims to provide a good solution for each of them. Since it’s always easier to reason about a concrete situation, let’s describe a simple scenario.

A burrito restaurant chain

Imagine a burrito restaurant chain. The chain produces burritos and each independent shop can become a distributor of their burritos. A distributor will buy burritos via the chain’s platform and resell them to clients. In essence, we are talking about a B2B2C strategy.

Example of a multi-tenant platform

That platform that the chain uses to sell its products contains valuable data. They are planning to use this data to provide insightful information to their distributors as an extra service:

  • How many burritos did a distributor sell this month?
  • Which burritos are most popular in his region and how they could optimise their price?
  • Which promotions are available and how would they influence their profit?

To provide this information in an understandable way, they need visualizations. Luckily, Cumul.io is the ideal platform to integrate dashboards in a multi-tenant platform. In this case, multi-tenancy is the ability of one integrated Cumul.io dashboard to serve multiple distributors. In other words, depending on the distributor that logs into the platform, the dashboard will show different data.

Example of how to handle multi-tenancy in Cumul.io

But how does Cumul.io integrate into multi-tenant platforms?

The Burrito Factory has integrated a dashboard in their platform. How do they make sure that the dashboard is populated depending on the client that is logged in?

It depends!

As is often the case in IT, the answer to this question starts with: “it depends”. In this case, it depends on how the Burrito Factory designed its data layer. Since the platform is a multi-tenant application, the data layer also has to be foreseen for multi-tenancy. There are multiple ways to do this:

Different ways to foresee your data layer for multi-tenancy

  • One database for each distributor

This is the most simple case, many applications start like this or are still designed like this.

  • a database for each distributor

For performance reasons, the database might be split up per client.

  • an API (in one of its many forms)

In an age of API-driven development, it is evident that a lot of companies expose an API instead of a database. However, each API is different. For example, the API might be smart enough to handle multi-tenancy. It might require you to pass the id of the distributor or a token and will then return only data. Another API might expose all data with one token and would push the responsibility to filter the data to the caller.

Ok, tell me how!

Luckily, Cumul.io has options for each scenario.

  • Token filters are the easiest solution to filter data that lives in a single database.
  • Token account overrides is an easy solution for a multi-database approach.
  • Or token metadata + a broker plugin: a broker plugin works in all possible scenarios. By using a plugin, you are in complete control to answer a specific query. A broker plugin receives all information that is necessary to answer a query (including any extra information you linked to tokens) and is then responsible to answer the question. Answering the query can be done by calling an API, executing a query on a database, querying multiple databases or a combination of all of those.

We will go over each of these solutions in detail. To make it easier to understand it is important to know how the integration flow works.

Secure integration of a dashboard

You can integrate Cumul.io dashboards into your application in a secure way using tokens.

 
...
Cumulio.addDashboard({ 
  dashboardId: '273b8bc6-0955-4a7b-b50e-84652a810c63', 
  container: '#my-dashboard-336', 
  key: '< the id of your authorization ', 
  token: '< the token of your authorization >'
});

To secure the dashboard, you need to insert a token. You can generate it by requesting a temporary token from our Core API:

 
let promise = client.create('authorization', {
  type: 'temporary',
  securables: [  ... ], 
  expiry: ...
})

So in essence, what a Cumul.io client has to do to integrate a dashboard securely is very simple. Retrieve a token, insert it in the dashboard html snippet and serve the snippet:


In case you want more information on the explanation above, there is also a video available on this topic.

The complete integration flow

When you request a token, there are a few token properties which clients can use to implement multi-tenancy. To be able to follow the explanation, we are going to explain in depth how the complete integration flow works, including what goes on behind the scene. Below is an animation that shows what actually happens in the background when loading a dashboard. This is all managed by Cumul.io so you don’t have to worry about it, but it will help to understand the multi-tenancy explanations below.

In the animation flow below, pay specific attention to step 2 where a token is generated and step 5. These are the steps that will change in the following explanations. We provided the possibility to go through the animations step by step by clicking on the buttons on the right.

Let’s get started

Enough background information, let’s go more in detail. Since every client is different, Cumul.io provides a lot of options to support multi-tenancy. We will discuss the most popular implementations.

Token filters

Instead of requesting a token which simply grants access to the dashboard, we will add filters to the token. In this case, the Burrito Factory has access to its distributor ids in their backend once they log in. Therefore, they can add a filter on the distributor id.

let promise = client.create('authorization', {
  type: 'temporary',
  securables: [  ... ], 
  expiry: ...
  filters: [
    {
      clause: 'where',
      origin: 'global',
      securable_id: '< dataset id >',
      column_id: '< client column id >',  
      expression: '? = ?',
      value: 'Fiesty Fiesta'
    }
  ]
})

When a dashboard loads, Cumul.io translates the filters to a filter syntax that your database understands. As a result, it will only return the rows that are relevant for that particular client. This becomes clear when we adapt the previous animation and highlight the differences in orange.


Before we continue, a small word on security: extra information such as a filter is linked to the token and is securely saved at Cumul.io. In other words, only the token ID and the key will pass through the frontend. The extra information such as filters, account overrides or metadata does not exist as far as the frontend is concerned.

Token account overrides

Using filters to filter on column id is less convenient when you have a database per client. Instead, you can override the account (host, user, password) in a token to switch between databases. You could also use this approach to change the user/password in a single database in case your database provides row-level security. Row-level security in this context means that the database can automatically filter out rows based on the credentials.

Before we continue, it is a good idea to specify what an account is in Cumul.io. An account contains the information that is necessary to access a certain plugin or database. So when you connect to a database or a plugin from within Cumul.io via the UI (as shown below), you are creating an account. If you already did that in Cumul.io, you will recognize the following UI:

Connect PostgreSQL database to Cumul.io

You could start by adding a default account via the UI, for example, a database with fake data. That way, you can go ahead and start dashboarding with the provided data. Once you are ready and you’ve embedded the dashboard with a token, you can switch to another account by placing an account override on the token.

In our example, we will override the host to switch to another database. When Cumul.io receives a request for data from a dashboard, there is always a token included in that request. Cumul.io will then notice that this token is linked to an account override. When the host is overridden, instead of launching this query to the default database on which the dashboard was built, Cumul.io will launch it to the database as specified by the override.

Implementing this is very similar to the previous one, we simply have to pass these extra properties as an account override to the token, Cumul.io will handle the rest. Again we have highlighted the differences.


Broker plugin

The most powerful approach is the plugin broker. Broker plugins are Cumul.io plugins which you use to apply extra business rules. Example business rules could be:

  • select a specific database;
  • filter the data before sending it to Cumul.io;
  • combine data from multiple sources.

A plugin sits in between Cumul.io and the data stores. Instead of sending a query to a database, Cumul.io will send a query with all token properties to the plugin.

Specifically for broker plugins, there is a token property called metadata. You’ll use this property to pass information from your backend to your plugin. When Cumul.io launches the query to your plugin, you will be able to use the information that you have stored in token metadata to decide how to handle the query. For example, you could use it to provide the plugin with the client id of the client that opened the dashboard.

How your plugin then handles the data call is completely in your hands! For example:

  • an API call with a token provided via metadata.
  • an SQL query on a database from which the host was provided in the metadata.


Broker plugin and base_url overrides

Before we started to talk about plugins, we have overridden the database host in the token. There is one more option that we have to discuss. What if you want to select a different plugin dynamically? This is an option which is rarely needed but can be very powerful.

  • If you want to deploy a plugin in the VPC of the client
  • if you want to deploy multi-region plugins to avoid latency

For example, imagine in our scenario that the burrito factory provides a piece of software but requires their distributors to host it themselves. In that case, the plugin might be a part of the software that will be deployed on the client’s infrastructure. Therefore, you’ll want to override the plugin URL instead of the database host. That is exactly what base_url override does!

If this is a solution that is of interest to you, then you probably have a complex setup and want to see how it works in a complex scenario. So in the final animation below, you’ll find a complex example of base_url overrides with a broker plugin for each region and client VPC within that region.


Conclusion

Cumul.io has a wide range of options to support multi-tenant integrations for each scenario. There are endless possibilities by combining token overrides with the plugin architecture. We have extensive documentation on both the Plugin API as integrations and token overrides:

In this article, we explain how to combine these features to make secure multi-tenant integrations. Future articles will explain how to write a pushdown plugin and provide an example multi-tenant plugin.

Happy hacking!

Add a Comment

Your email address will not be published. Required fields are marked *