In this article, we will explore data management in the context of user interfaces. First, it’s important to differentiate between data and information. For our purposes, we’ll define data as the raw collection of records retrieved from a data source. Information, on the other hand, refers to the insights derived from this data, providing a deeper understanding of its subject matter.

To transform data into actionable information, there are three key steps:

1. Data: This is the raw collection of rows that provide details about the context and subject matter.
2. Perspective: This involves altering the way we view the data to extract specific insights.
3. Visualization: This is the process of displaying data and perspectives in a manner that allows for quick and easy comprehension.

Perspective

To better understand perspectives, let’s delve into the concept in more detail. The core idea of a perspective is to view data from various angles to extract different insights. Consider a cell phone as an example. Depending on the angle from which you view it, you gain different insights about the device. Looking at it from the bottom reveals the types of ports it has for peripheral devices. Viewing it from the front provides information about the screen space, brightness, and reflectivity. Examining it from the side gives you an understanding of possible external mechanical interactions.

The question is, how do we view data from different perspectives or angles? This can be achieved through:

• Sorting
• Filtering
• Grouping
• Aggregate functions

Let’s illustrate this with an example. Suppose we have a list of 1,000 work items that need to be performed. To understand how this work is distributed among different staff members, we can group the data by the staff member field and apply a count aggregate to the grouping. This simple data manipulation allows us to see who is doing the work and how many work items are assigned to each person.

If we then represent this data on a bar chart, with each bar representing a staff member and the height of the bar indicating the count of work orders, we can quickly identify if any one person is overloaded with work.

However, a count alone may not provide a complete picture, as not all work items are equal in duration. One person might have ten work items that each take one hour, while another person might have only two work items, but they total 20 hours.

To address this, we can create a second perspective by summing the duration of the work items instead of just counting them. This gives us a better view of not only how many items are assigned to each person but also the total expected duration of their tasks.

Visualization

Choosing the right visualization is crucial. A well-chosen visualization helps highlight relationships between different perspective items. For example, a bar chart is ideal for comparing counts, as the size of each bar visually represents the differences in quantity, making it easy to compare them at a glance.

However, if the goal is to understand when tasks need to be completed, a timeline is more effective. A timeline visually represents the relative timing and duration of tasks. If ten tasks, each requiring a full day, are scheduled for the same day, the timeline immediately shows the scheduling conflict, as it’s clear that one person cannot complete all tasks simultaneously.

This insight enables actionable decisions, such as rescheduling tasks or reassigning them to other team members who have available time.

The data manager

This class temporarily stores data in its original form but does not handle long-term storage. It serves as a client-side cache for data fetched from the server, intended for display to the user. The size of this data store can vary depending on the use case. Typically, it resides in memory (RAM), but it might be stored in IndexedDB instead. This can be beneficial for devices with limited RAM or when you want to persist data locally to avoid fetching it from the server again, thereby saving on costs and improving performance for subsequent fetches of the same data.

The data manager class is uniquely identified based on the subject or context of the data it stores. Using the process API, it is straightforward to create a new data manager for a specific set of records.

manager = await crs.call("data_manager", "register", {
    manager: "store",
    id_field: "id",
    type: "memory",
    records: records
})

This approach allows you to perform common actions on the data manager without being tightly coupled to a specific instance. However, you will need to specify the manager ID for each call, such as “store.”

The data manager operates on an event-based system, notifying listeners of any changes made to the data. This includes the removal or addition of rows, as well as updates to specific records. This event-based notification system ensures that any interested parties are aware of data changes. Since multiple visualizations may render the data, each one needs to be informed of changes to perform the necessary render updates.

await crs.call("data_manager", "on_change", {
    manager: "store",
    callback: async (args) => {
        changeArgs.push(args)
    }
});

The data manager can also track the dirty state of records, keeping note of updates made and identifying new records. However, it does not persist these changes to long-term storage. Instead, a custom process that will handle this. This process will query the data manager for changes and, after applying them to long-term storage, will notify the data manager to clear the dirty states.

Perspectives Manager

The perspectives manager functions similarly to the data manager, allowing multiple perspectives, each identified by a unique name. Each data manager can have several perspectives. Instead of duplicating the data from the data manager, perspectives contain either a collection of indexes or a data structure along with indexes indicating how to present the data.

For example, when sorting a collection, we only need an array of record indexes indicating the order in which the records should be rendered.

For grouping, we need a dictionary structure that represents the grouping information, with arrays of indexes at the leaf nodes of the structure.

These operations can be combined, resulting in a final data structure that is filtered, grouped, and sorted.

Aggregation is a distinct operation, performed on a subset of records and executed only when necessary. For instance, with multi-level grouping, aggregates may only need to be calculated for the records at the leaf nodes when they become visible. Thus, aggregation actions are explicitly performed only when the data needs to be displayed.

The perspective manager points to the data manager that has the original data. Lets call this data manager the source of truth.
The perspective manager is also a data manager as it contains the perspective data after perspective operations has completed.

A perspective can change.
For example you can add or remove filtering, sorting or grouping settings from the perspective. When this happen the perspective indexes that it stores are out of date with the perspective definition. When this happens the perspective needs to process the source of truth again, now applying the updated perspective definition. The result of this is stored in the perspective manager. A event is also fired to ensure that visualizations watching the perspective manager are updated.