Overview and concepts

Goal of Peliqan

Peliqan is an all-in-one data collaboration platform that provides direct data access to a wide range of data sources including databases, data warehouses and SaaS business applications. Peliqan is truly self-service for business teams, without the need for a data engineer.

Peliqan helps in a wide range of data-centric use cases:

  • Ad hoc analysis: asking ad-hoc data questions and finding an answer fast
  • Data visualization / BI
  • Data preparation & transformation, making data ready for e.g. analysis or machine learning
  • Data checks: e.g. find anomalies
  • Working with data: e.g. keeping track of things, manual verifications etc.
  • Building interactive data apps
  • Set up data syncs / writeback

In each of these use cases, Peliqan avoids the need of a data team or data engineers. Peliqan also avoids the need to setup data pipelines or perform complex tasks. Peliqan allows to start working on the data instantly, without waiting on IT or other specialists to take action.

Peliqan comes with features for 3 personas in a business team:

  • Business user: work with Peliqan’s spreadsheet interface to explore and work with data
  • Power user: write SQL statements on any data source and combine data from any source
  • Developer: use low-code Python scripting to build interactive apps, process data and implement writebacks

Direct data access, data virtualization

In Peliqan you add connections to data sources such as databases and SaaS business applications. Peliqan will fetch the data in real-time from the source where possible, each time you access data to view it or work with it. This concept is also known as “data virtualization”. In some cases, e.g. with SaaS sources, Peliqan will make a copy of the data in its built-in data warehouse.

Spreadsheet interfaces

Data can be viewed, edited, enriched and transformed in a spreadsheet interface. Peliqan eliminates the need to make exports of your data that you dump into e.g. Excel, because data exports are not a repeatable process to work with data.

SQL on anything

Power users can apply “SQL on anything” and write SQL queries that can be shared as new virtual tables to business users. When the business user opens the table to view the data, the query will be executed in real-time. For the business user, it’s just another table with all the capabilities of Peliqan such as editing, filtering, adding charts etc.

Data structure

Peliqan follows the structure of relational databases to organize data:

  • Connection
  • Database
  • Schema
  • Table
  • Columns

For example you can add a connection to a MySQL server. The connection uses certain credentials that have access to multiple databases on the sever. Each database can have schemas (e.g. “public”) and each schema can have tables. Each table has one or more columns.

For SaaS Sources, one virtual database is created for each source and the schema is “public”. Peliqan will create one table for each object type, for example for a CRM such as Hubspot you will see tables such as Companies, Contacts, Deals etc.

Tables

Every table in Peliqan can be accessed via the Spreadsheet interface, each table can be used in SQL queries and each table can be referenced and used in Data apps (see below).

For example you could write an SQL query to join 2 tables. The result can be used by business users in the spreadsheet interface. They could add a few columns to enrich the data. This table could then be used by someone else in a query or data app.

Following table types exist in Peliqan:

  • Internal tables: created by the user in Peliqan
  • External tables: tables from connections (MySQL, Postgres, Snowflake, SaaS sources etc.)
  • Queries: SQL SELECT queries written in the Peliqan SQL editor

Data edits in “virtual copies”

Users can edit data in the Spreadsheet interface in any table. The data edits are not actually applied to the source, instead they are stored as a list of “Changes” or transformations that are applied to the data each time the table is viewed or accessed.

Each table therefore appears to be a “virtual” copy of the source table, because it will stay up to date with the source, while users can still edit the data, for example make a correction, or even add columns and rows.

Data transformations

Data can easily be transformed in Peliqan. This is required to prepare data for usage by other people or teams, or to combine data from different sources. Data transformations are also applied to the table dynamically.

Each time a table is viewed or accessed, Peliqan will build up a query behind the scenes. This query is executed by Peliqan’s query engine in real-time. The query that Peliqan builds up is essentially a “SELECT * FROM table” query, to which all the data edits and transformations are added. The final query provides the result that is visible in Peliqan. For tables that reference other tables, for example a join query, Peliqan will prepend the necessary statements to the generated query in order to fetch the data from those other tables, again with all edits and transformations applied.

Data apps

Peliqan also allows the creation of interactive data apps such as custom data interfaces for internal teams. Peliqan offers a low-code Python scripting environment that has direct access to all tables in Peliqan. These scripts are used for a wide range of use cases including reporting, writebacks, machine learning, data quality etc.