1. Introduction
This essay looks into aspects of real-time data visualization. Past studies discuss/mention different methods of displaying data, primarily dashboard, which is a method/representation of data. Few studies provide guidelines on choosing/selecting the right metrics for practitioner and researchers (Abdelfattah, 35). This essay will evaluate the variety of dashboards and how to create an effective dashboard specifically for the real-time QARTOD test result. In order to do this, theories of graphics and examples of best practices will be introduced. This essay will also address potential challenges faced by the users and the designers of the dashboards, which help to suggest directions for further research.
2. QARTOD Test Results
As described in the background section, QARTOD stands for Quality Assurance/Quality Control of Real-time Oceanographic Data. It’s a program initiated by the U.S Integrated Ocean Observing System (IOOS) as a standard procedure for its partners. IOOS aims to provide high-quality real-time data for its users on a national scale by establishing definitive tests on 26 core variables (e.g., water level, wind data, etc.). For each variable, IOOS organized a written manual to identify the required tests and to direct procedures of running the test (“Manual for Real-Time Quality Control of Water Level Data,” 1). The test results on the quality of the data are presented and recorded by mining flags into the datasets that are received in real-time (“Manual for Real-Time Quality Control of Water Level Data,” 12). As shown in Figure 1, the flags are numbers that correspond to the different status of the data. No other visual representation is available (colors are used for explaining the concept, but are not used in practice). Therefore, to understand the status of the test, one has to be able to understand the meanings behind the numbers/flags. In addition, a user needs to either have knowledge in reading the numbers/flags among the lines of codes or have the ability to format and filter the codes to retrieve or highlight the numbers/flags. However, for GLOS, different stakeholders, who are involved in handling these test results, have different levels of technical efficiency on processing the flags. Most of them are not data scientists. Currently, only the individuals, who have knowledge in reading the codes/flags, has the ability to analyze the data and send out emails to alert the buoy owners if the test failed. This not only results severe lags in the delivery of information (emails are sent out on weekly basis), but also limits on the quality and accountability of the information (the content of the emails is normally one or two sentence explaining the situation) Therefore, a data visualization method on presenting these real-time test results in a digestible and accountable way for decision-making is needed.
Figure 1 Flags for real-time data. (“Manual for Real-Time Quality Control of Water Level Data”,13)
3. Dashboard Overview
Several scholarly articles mention dashboard as a solution for real-time data visualization. Dashboards provides real-time monitoring of dynamic data, making it an appropriate solution for displaying test results. So what is a dashboard? There is no consensus about the true definition of dashboard. The definition is in flux as the concept and application have evolved throughout history (Sarikaya, et al., 2). In the 1980s, the dashboard was acknowledged as a communication tool under the executive information system (EISs). It was not until the 2001 Enron scandal that it began to be pushed by the government to include monitoring abilities that track performance and accountability. For social sector and nonprofit organizations, funders began to require proof of accountability and evaluations on their work (Smith, 22). Here is when the dashboard evolved into an active data visualization tool for businesses and organizations. In 2006, the information thought leader Stephen Few described a dashboard as “a visual display of the most important information needed to achieve one or more objectives consolidated on a single screen so it can be monitored and understood at a glance.” (Few, 12) As dashboard developed further in 2017, he added that a dashboard is “a predominantly visual information display that people use to rapidly monitor current conditions that require a timely response to fulfill a specific role.” (Few) The evolution suggests that the concept of the dashboard has developed to better serve the purpose of visualizing real-time data for decision-making in recent years. Now is the perfect timing to adapt this system for use.
4. Dashboard Types and Best Choice for the Case
All dashboards can be grouped into three main categories by purpose: strategic purposes, analytical purposes, and operational purposes. Dashboards with strategic purposes focus on presentations of the organization’s performance. It refreshes on a monthly or quarterly basis (an example is shown as Figure 2). As for dashboards with analytical purpose, they mostly are equipped with interactive features for the users to filter information for exploration and analysis. The refresh cycle for this type of dashboard is flexible. It could be daily, weekly, monthly, quarterly, or yearly (Smith, 27). The last type of dashboard, the operational dashboard, is identified as the most suitable type for visualizing real-time data. Its operational purposes includes “formative, quality assurance, or safety purposes” (Smith, 27). This type of dashboard aims to keep track of the activities with constant flux and support for the instant decision-making process. Therefore, the dashboards are required to present the data that falls out of the preset parameters in a way that grabs the user’s attention immediately. The refresh cycle is in real-time (Smith, 27).
Figure 2 Strategic Dashboard (Smith, 28)
At a competition held by Few in 2012, the participants were asked to design a dashboard “to prevent or resolve problems and to help each student improve as much as possible” (Few). The winning design (Figure 3) included the functionality from both an analytical and an operational dashboard. Smith suggests that a useful dashboard can also be a combination of the categories outlined above (Smith, 27). For the GLOS case that this essay focused on, the organization can benefit from the same combination as the winning dashboard design in Figure 3. It can support real-time monitoring of test results, and a drill-down function can provide trends of a specific test result for analysis. This can help to satisfy needs from different stakeholders, e.g. scientists who are in charge of alerting the buoys owners if a test failed, or researchers who want to see the patterns of a specific buoy’s behaviors.
Figure 3 Operational-Analytical Dashboard (Smith, 30)
Sarikaya et al. suggested a more complex matrix for dissecting the types of dashboards and their uses (Figure 4). As shown in the diagram, instead of categorizing the dashboards by purpose, they are divided up into three different goals, decision-making, awareness and motivation/learning. Purpose becomes one aspect of the three goals. The main goals for GLOS are decision-making and awareness. Also, since we are focusing on operational dashboards, two of the models, operational decision-making model (labeled as Cluster 5) and Static Operational model (labeled as Cluster 3), will be analyzed (Sarikaya et al., 4-5).
Figure 4 Dashboard Types (Sarikaya et al., 5)
According to Sarikaya et al., the two models share similar functionalities, and the differences between the two make operational decision-making a better choice for GLOS to adapt. First, the static operational model (an example shown as Figure 5) allows little interactivity for the users. The format is mostly static, one-page presentation (Sarikaya et al., 5). For GLOS, some degree of interactive elements can benefit the effectiveness of the system, considering different stakeholders are involved. The same reason applies to another characteristic of the static operational model. Most of the graphics on the dashboards have low visualization literacy, which means simple visualization with lines and bar charts. This requires the users to have basic knowledge in context beforehand to understand the meaning of the graphics (Sarikaya et al., 5). This won’t apply to multiple stakeholders. What makes the operational decision-making model (an example is shown as Figure 6) outscore the static operational model is the benchmarks element that is carried by most of the dashboards in the cluster. Benchmarks are “indications of breaking user- or model- defined thresholds, providing the viewer with additional data context.” (Sarikaya et al., 5). An example can be the gauge graphics (Figure 7) that constantly presented in operational dashboards. However, it has been proven that the gauge graphics are ineffective in its use of space (Few, 125). A better solution is suggested in the latter part of this essay. Based on the analysis above, an operational decision-making dashboard is recommended for GLOS to display the real-time QARTOD test results.
Figure 5 (Left) Static Operational Dashboard (Sarikaya et al., 6)
Figure 6 (Right) Operational Decision-Making Dashboard (Sarikaya et al., 6)
Figure 7 Gauge Graphics (Sarikaya et al., 3)
5. Developing a Dashboard
The analysis above supports a general direction of the type of the dashboard that suits GLOS’s situation. The next crucial step is to determine the process of creating a dashboard. This will not only create a strong foundation for the dashboard but also present a clear idea of what happens next when the dashboard is developed. According to Smith, six steps should be included when developing a dashboard from scratch:“
layout of the screen view
building a dashboard using software of choice
populating the dashboard with baseline data
publishing the dashboard for use by stakeholders
refreshing the dashboard with new data, which is followed by publishing again at an agreed-upon interval; and eventually
evaluating and refining the dashboard.” (Smith, 29)
Dashboard tools, like software and templates, are available to business owners and organizations by large tech companies (BI, Tableau, etc.). However, Smith argues that organizations need to be mindful when using these templates because they might not be the most appropriate and relevant options for their specific use cases. She suggests that if not used properly “dashboards can degrade the signal and increase the noise, depending on the design, software, management, use, and context” (Smith, 24) Therefore, it would be crucial to introduce theories of the graphics to help organizations evaluate their dashboards.
Tufte devised two theories to evaluate the effectiveness of the graphics in 1983. One is called the data density index (DDI), which is “the size of the graphics in relation to the amount of data displayed.” (Tufte, 1983) The idea is that the bigger the index number (less data displayed on the same sized graphics), the lower the effectiveness of the graphic. This supports Few’s ideas that the gauge graphic is ineffective because it wastes much space. He invented a new type of graph, the bullet graph, in place of the gauges. The bullet graph is in a linear format and can be stacked together to save space (Figure 8). Another formula devised by Tufte is the dark-ink ratio, which is the “proportion of a graphic’s ink devoted to the non-redundant display of data information.” (Tufte, 93) Wainer identified grids and scales in the graphics as forms of redundant display. Except for their use of plotting the dots in the designing process, when shown to the users, the data disappeared in the grids (Wainer, 139). Other ideas of the lousy design of graphics that are not supported by theories, but highly applicable, are introduced in Wainer’s essay of How to Display Data Badly. Insights of good design are embedded in them for organizations to reference.
Figure 8 Bullet Graph vs. Gauge Graphs (Smith, 35)
6. Challenges and Future Research
Some of the challenges raised by Sarikaya, et al. are still present. For instance, the user interface for the end-users of the dashboards is not flexible enough. The user is looking for a more automatic organization of data tailored towards their use. In addition, only representing parts of the data may result in insufficient communication (Sarikaya et al., 6). This problem has further led to the social impact of concerns on “data democratization”: only a small number of people have control over the big data and decide on what should be presented to the users (Sarikaya et al., 7).
Another challenge observed is the lack of practice models of operational dashboards for environmental use. Most of the operational dashboards are developed by health organizations to track the patient’s status or the other way around, for the patient to report back to the doctors. However, some of the functionalities from the models developed for medical emergencies are highly applicable to alert and analyze environmental spikes. For scholars and researches, evaluating the functionalities of medical operational dashboards and how applicable they are for environmental operations can be an opportunity to encourage the development of dashboards in environmental fields.
7. Conclusion
This essay looks into the format of the QARTOD test result and provides analysis of the methodologies of visualizing real-time data. Note that a portion of the analysis on the types of the dashboards and reflections may not apply to every instance, as they are based on correctly, the real-time flagged data type. The aim of this essay is focusing on answering the question of how can GLOS visualize real-time QARTOD test results? An operational decision-making dashboard is recommended. Further research on what information should be present on the dashboard for real-time QARTOD test results is suggested as the next step for research.