Vision – SYMBIOTIK

Information overload is the by-product of information over-exposure and input abundance which, nowadays, has become an aggravated phenomenon. Simply put, when presented with a large array of options, users are at a higher risk of experiencing impeded cognitive functioning, which may reduce their capacity to perform effective decision making, derive meanings, and gain insights, especially in situation- and time-sensitive contexts. While using visualizations to mitigate this effect has been widely discussed as a possible countermeasure, the potential of information visualisation (InfoVis) systems has not yet unlocked their true potential to best assist the user in critical decision-making tasks.

InfoVis technologies provide a way to expand and enhance our innate cognitive abilities, and enable us to accomplish a range of tasks, from simple (e.g. company revenue assessment) to intractable (e.g. air-traffic control). However, depending on the timing and context, we may have a greater or lesser ability to make rapid and effective decisions, yet our ability to do so based on fast-flowing streams of data may be decisive. From emergency rooms and autonomous cars to operational command centres, a clear understanding, rapid assessment, and decisive actions based on the available information can make the difference between life and death. We believe artificial intelligence (AI) enhanced with context-aware and human-centric attributes will play a key role in leading the necessary advancements for next-generation InfoVis systems, and the SYMBIOTIK project will set the first workable precedent to achieve that goal.

Objectives

1. Support a natural dialogue between AI agents and humans following symbiosis principles

Technological challenge: Formulate a vocabulary based on a mechanistic understanding of important regulatory and utilitarian functions responsible for decision making and perception within the human body, upon which a low-level dialogue will occur between humans and AI agents. According to cognitive psychology, humans and AIs alike are characterised as “information processors” and perception is the first stage in information processing whereby sensory information from the environment is made available to them, hence leading to the desirable awareness property. In turn, cognitive reactions refer to processes by which information is manipulated (e.g. filtered, coded, compared) and which support adaptation, learning, and decision making.

Scientific breakthrough: InfoVis systems will be enhanced with neuroimaging methods that integrate cognitive models to adapt visualisations. It will provide AI agents with a unique class of awareness – implicit awareness of the environment and its inhabitants via the provision of an embedding space of cross-modal neurophysiological descriptors – and the capacity to interact with its environment, via intelligent adaptations that optimize user-derived rewards.

Market potential: The research output will provide a comprehensive, spatiotemporal characterization of the relevant cognitive phenomena and the means to innovate outside of the laboratory environment, using automatic, low-cost and scalable methods. Broader market adoption possibilities will emerge as a result of wider adoption of low-cost yet accurate wearable devices such as next-gen BCIs and mobile eye-tracking systems.

2. Develop human-centric perception analysis methods and tools for situation/time-sensitive tasks in InfoVis systems

Technological challenge: With the volume, variability, and velocity of today’s data streams, our ability to successfully respond to critical events in InfoVis depends on the dashboard controls we have available, and how quickly we can draw actionable insights from them. Human visual functions are extremely fast and efficient, whereas our cognitive processes (i.e. the act of thinking) are much slower and less efficient. SYMBIOTIK aims at placing human information processing under extensive scrutiny to measure aspects like attention (e.g. reaction times), reasoning (e.g. neurophysiological changes in user behavior) and effectiveness of decision making (e.g. empirical assessment of task completion).

Scientific breakthrough: Consider the social and collaborative aspects of user-AI interaction, where the objectives are often competitive in nature and usually affected by other humans or AI agents, which increases the uncertainties associated with decision making processes. When designing new data visualizations, we will be able to tap the strengths of our visual functions and reduce the disadvantages of our cognitive functions. Furthermore, by engineering AI agents with social and collaborative attributes, SYMBIOTIK will allow the orchestration of highly complex and collaborative ways of working at various scales.

Market potential: Altruistic agents may see the interests of other agents as objectives themselves, and therefore they will aim to come up with fair solutions that will benefit everyone. As such, it is also possible to consider varying levels of altruism, according to symbiotic environments. Utility functions are still an open problem; e.g. minimize thinking time, maximize user engagement, etc. Therefore this project will provide valuable new means to deal with these tradeoffs in practice.

3. Facilitate a novice-to-expert transition in InfoVis systems by means of context-aware self-adaptive graphical user interfaces

Technological challenge: Dashboard controls provide visual representations of non-physical and abstract concepts, such as statistical trends. However, for the non-expert human operator, it is unnatural to think in purely numerical and mathematical abstractions. Our brains are wired to recognize patterns and, as good as humans may be at this form of perception and cognition, some of the most important visual patterns may not be directly available for computation.

Scientific breakthrough: Self-adapting visualizations tailored to the cognitive level of the user, that evolve together with their increased expertise and learn from past experiences. Novice or expert users will not entrust completely decision-support systems if they cannot ascribe some form of awareness and true understanding to them. This breakthrough will ensure that they remain useful and relevant in future interactions, also supporting an effective and efficient adaptation and learning.