Qlik Sense Relation With Big Data – Qlik ABDI & ODAG

In this article, we will discuss the relation of Qlik Sense with big data. Earlier, we have discussed the role of QlikView in Big Data. This is an extension of the same where we focus only on Qlik Sense and its new ABDI and ODAG functionalities for big data utilization. Firstly, we will discuss, what is Qlik Associative Big Data Index.

1. What is Qlik Associative Big Data Index?

You must be aware of the Qlik Associative Indexing Engine (QIX Engine) which functions as a core of all Qlik BI related tools especially Qlik Sense and QlikView. The associative indexing engine structures the data coming from different data sources and forms associations between them logically. This capability of Qlik expands to index and associate big data from varied sources. This new technology is termed as Qlik Associative Big Data Index engine (ABDI Engine).

It is a governed, high-performance, incremental and associative indexing engine which is released by Qlik in 2018 as a services-led offering (an EAP) and requires purchasing alongside the main Qlik Sense package. The key capabilities of the ABDI engine are:

• It can deploy with a big data storing environment such as Cloudera, Apache Spark, Hadoop databases etc.
• Highly scalable and can deploy across multiple nodes in a cluster. The functionality distributes across the nodes in both on-premise and cloud-based environment.
• A script in the data load editor generates automatically to access the data index through an ABDI connector.
• It supports data requesting and delivery from ODAG apps.
• The ABDI engine is capable of querying data directly from data sources such as data lakes, etc.
• The Qlik ABDI index is powered by Qlik Selection Language (QSL) which is a high-speed selection language used to extract data from the index.

2. On-Demand App Generation and Big Data Management

Qlik Sense provides a unique feature of the on-demand app generation (ODAG) for big data management. It is difficult to work with big data sources because of the enormous data volume it has. Therefore, through ODAG, users can select a section or subset of Big Data they require from the data sources and then work with it on the app that creates on-demand. The on-demand apps provide the users with aggregate views into the big data store from where they select and load a subset into Qlik Sense in-memory to perform analytical operations on it.

Now, let us understand how does the on-demand app generation work in Qlik Sense from a picture.

• Firstly, a user gets to access a selection app that provides aggregate views into the big data stores.
• Within these selections, apps are navigation links to one or more on-demand app templates. These templates serve as the base on which on-demand apps get to generate.
• You can set the properties of both template apps and on-demand apps according to the volume and type of data subset selected from the big data stores to create on-demand apps.
• Data into apps generated on-demand is loaded according to the selections made in the selection app. From there, data gets filter and goes into the generated apps.
• A selection app can link to multiple template apps and a template app can link to multiple selection apps.
• Both on-demand apps and template apps require special data load scripting which is done best by an experienced professional.

Thus, ODAG enables a Qlik Sense user to utilize and manage big data efficiently without cluttering the in-memory of the system.

3. Advantages of On-Demand Apps for Qlik Sense

Apps created on-demand are beneficial to the user in handling big data in many ways as discussed:

• You can update an app or create more apps using the same template as soon as the data gets an update at the source. This enables up-to-date analysis and drawing fresh insights from big data.
• You can easily select subsets of data from the data store and populate your app from different kinds of data set based on location, time period, customer segment etc.
• ODAG offers full visualization and analytical functionalities for the data subset loaded into the in-memory of Qlik Sense.
• You can perform IT-based governance and monitoring on the applications created on-demand by managing data volumes and making dimensional selections.
• You can access data from non-SQL data sources as well, such as Teradata Aster, SAP BEx, MapR, etc.
• Also, It allows sectional access and load script generation by customizable SQL.

4. Summary

Thus, Qlik Sense has the potential to handle and manage big data through the Associative Big Data Index and on-demand app generation. Furthermore, you can comment any query or give feedback in the comment section, below. We hope our explanation was helpful.