Social Media Data in an Augmented Reality System for Situation Awareness Support in Emergency Control Rooms

Abstract

During crisis situations, emergency operators require fast information access to achieve situation awareness and make the best possible decisions. Augmented reality could be used to visualize the wealth of user-generated content available on social media and enable context-adaptive functions for emergency operators. Although emergency operators agree that social media analytics will be important for their future work, it poses a challenge to filter and visualize large amounts of social media data. We conducted a goal-directed task analysis to identify the situation awareness requirements of emergency operators. By collecting tweets during two storms in Germany we evaluated the usefulness of Twitter data for achieving situation awareness and conducted interviews with emergency operators to derive filter strategies for social media data. We synthesized the results by discussing how the unique interface of augmented reality can be used to integrate social media data into emergency control rooms for situation awareness support.

1 Introduction

Crisis situations involve a high degree of uncertainty and the need for rapid, critical and potentially irreversible decisions (Rosenthal and Hart 1991). Emergency control rooms are facilities used during crisis situations to coordinate response and recovery actions, conduct strategic decision-making and manage resource allocation (Mignone and Davidson 2003). Among other sources, emergency operators obtain information through the calls they receive from citizens and decide about the resources required to support first responders at the scene (Militello et al. 2007). To make the best possible decisions about the allocation of resources, it is crucial for emergency operators to achieve a high level of situation awareness (Endsley and Garland 2000). This means they need to process sufficient information to develop an understanding of the present situation and project the future state of the situation (Endsley 1996). Especially in the beginning of a crisis situation, there is often a lack of information and emergency operators wish for access to further credible data sources (Tapia et al. 2013).

The adoption of smartphones equipped with multiple sensors and a constant Internet connection has increased the creation of incident reports in crisis situations and the volume of available information. In particular, social media posts from eyewitnesses are seen as a potentially valuable data source for emergency operators (Palen and Vieweg 2008; Palen et al. 2009; Sutton et al. 2008). Still, control rooms are hesitating to make use of social media analytics as it is challenging to filter relevant and credible information (Landwehr and Carley 2014). Moreover, social media data are complex since they are generated in an informal way, include many different data formats (such as text, video, audio) and are generated by unknown and diverse users and organizations (Stieglitz et al. 2018b). Due to the large amount of data available on social media, control rooms need filtering algorithms to categorize the data and visualize them in a meaningful way (Ludwig et al. 2015a; Stieglitz et al. 2018e). In addition, the integration of social media data into the information systems currently used in control rooms poses a further challenge that needs to be addressed.

Information systems for emergency management must be designed to reduce the operator’s workload and support the decision-making process in the dynamic work environment (Buck et al. 2006; Carver and Turoff 2007; Taylor 2004). The following quote illustrates the importance of information systems in this work environment: “A crisis management plan is only as good as the information system that supports it” (Housel et al. 2006). Over the years, the used systems have improved significantly through the lessons learned from retrospective analyses of real-world disasters (Perry 2003). However, crisis situations are also changing as they are not only becoming more complex in nature (Rosenthal et al. 2001) but their number is also “increasing in proportion to the growing complexity of the world” (Feng and Lee 2010). Therefore, it is important to constantly improve the used information systems to keep up with these new developments. Augmented reality is a technology that made major progress in the last decade and is now implemented in the industry (Palmarini et al. 2018). Recent results revealed that this technology can help operators to work faster and reduce their mental workload, for example, in procedural work where a fixed sequence of activities are carried out to achieve a specific result which is common for installation, assembly, and maintenance work (Braly et al. 2019). This reduction can lead to an increased situation awareness and in turn to better decision-making (Endsley et al. 2003). It still needs to be investigated to what extent these findings also apply for the operators working in emergency control rooms.

The goal of our research is to propose a conceptual augmented reality design for improving the situation awareness of emergency operators. First, we used the technique of goal-directed task analysis to analyze the situation awareness requirements of emergency operators. Second, we collected Twitter data during two heavy storms in Germany to discuss the usefulness of different types of tweets with emergency operators. Based on the results, we derived filter strategies that allow to identify the tweets that are perceived as most useful by emergency operators in crisis situations. Third, we applied the eight design principles for situation awareness (Endsley et al. 2003) to propose a conceptual augmented reality design for integrating the filtered Twitter data into the information systems of control rooms. Hence, we answer the following research questions with our study:

• RQ1: What are the situation awareness requirements for operators working in emergency control rooms?

• RQ2: Which types of Twitter data do emergency operators perceive as useful for achieving situation awareness?

• RQ3: How can augmented reality be used to visualize the types of Twitter data which are perceived as useful by emergency operators for achieving situation awareness?

The remainder of this article is structured as follows: In the next section, we provide the theoretical background on situation awareness. Subsequently, we summarize previous studies on social media analytics and augmented reality in emergency management. Then, we describe our two-step research approach consisting of a goal-directed task analysis and expert interviews. Afterwards, we present our findings and answer our research questions by discussing the identified situation awareness requirements, derived filter strategies, and conceptual augmented reality design for integrating Twitter data into control rooms and improving situation awareness of emergency operators. We conclude with a summary of our contribution, limitations, and suggestions for future research.

1.1 Research Background

1.1.1 Situation Awareness

The formal definition of situation awareness is “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” (Endsley and Connors 2008). This awareness usually focuses on the information needed for a specific task or goal and is therefore critical for effective decision-making (Endsley et al. 2003). Typically, the concept of situation awareness is applied to operational tasks such as driving a car or flying a plane (Endsley and Garland 2000). Operators working in emergency control rooms require individual situation awareness, team situation awareness and shared situation awareness which emphasizes the need to achieve “a shared understanding of that subset of information that is necessary for each of their goals” (Endsley et al. 2003). Based on the formal definition, situation awareness comprises of three different levels: 1. Perception of the elements in the environment, 2. Comprehension of the current situation, and 3. Projection of the future status (Valecha 2019). The first step towards achieving situation awareness solely represents the receipt of information in its raw form and does not comprise any interpretation of the data (Stanton et al. 2001). The second step towards achieving situation awareness is to understand what the perceived data means in relation to the goal and objectives (Endsley et al. 2003). This comprehension is based on schemata or knowledge structures that are stored in the long-term memory and are activated by recognized patterns in the perceived elements (Sarter and Woods 1991). The third level of situation awareness describes the ability to predict how perceived elements in the environment will impact the future (Stanton et al. 2001). This allows operators to proactively avoid undesired situations and respond faster which is why a lot of experts in different work environments devote a significant amount of their time to form this ability (Endsley et al. 2003). In combination, the three levels of situation awareness help to make well-founded decisions which will affect the current situation. The operator’s decision-making performance will change the state of the environment, resulting in new elements the operator can perceive, comprehend, and use to predict future events. Figure 1 shows additional factors that have an impact on the achievement of situation awareness.

Fig. 1

Model of situation awareness in dynamic decision-making (adapted from Endsley et al. 2003)

The operator’s expertise helps to develop mental models and schemata allowing the comprehension and projection of the future faster and with less effort (Salvendy 2012). This can lead to a level of automation in mental processing that can create a positive effect on situation awareness since it reduces the mental workload of the operator (Endsley et al. 2003). Furthermore, the operator may have preconceptions of what he or she should see or hear in a particular situation because of prior experiences, mental models, and instructions (Endsley and Garland 2000). The capability and user interface of an information system also influence the achievement of situation awareness. The presented information and provided functionalities can be beneficial for obtaining situation awareness. However, especially with many elements presented simultaneously, it is important to support the perception of the most important elements (Endsley et al. 2003). Additionally, the goals of the operator are essential to the development of situation awareness as for many operators their individual goals determine the elements of the environment that are perceived (Endsley et al. 2003). This active approach of perceiving elements is also known as goal-driven information processing (Casson 1983). Conversely, data-driven processing of data describes when elements are perceived unintentionally and independent of the operator’s goal (Endsley et al. 2003). Alternating between these two forms of processing is one of the most important mechanisms supporting situation awareness (Endsley et al. 2003). For instance, when an operator only focuses on a given goal and fails to consider cues that indicate that a new goal is more important, he or she might make bad decisions. At the same time, relying on data-driven processing is overly taxing and highly ineffective in an attempt to achieve a specific goal (Endsley and Garland 2000). Therefore, the right balance is essential for a good job performance. By understanding how an operator selects and uses goals, it is possible to design an information system that supports the perception of important elements in specific situations. Endsley et al. (2003) proposed eight design principles for creating interfaces that effectively support the achievement of situation awareness (Table 1).

Table 1 Design principles for situation awareness (Endsley et al. 2003)

1.1.2 Social Media Analytics in Emergency Management

The term social media describes “a group of Internet-based applications that build on the ideological and technological foundations of Web 2.0 and that allow the creation and exchange of user-generated content” (Kaplan and Haenlein 2010). Since everyone with a smartphone can share videos and pictures from the emergency scene, information can already be found on social media before authorities or traditional media publish their reports (Cameron et al. 2012; Crooks et al. 2013; Huang et al. 2015; Mirbabaie and Zapatka 2017; Stieglitz et al. 2018d), sometimes even before emergency responders are on site (Stieglitz et al. 2018a). The wealth of user-generated content on social media could therefore serve as an additional information source for emergency operators to achieve situation awareness. Previous research has highlighted the usefulness of social media analytics before, during, and after a disaster (Ahmed 2011; Ghosh et al. 2018, Houston et al. 2015; X. Li et al. 2019). This applies especially to the micro-blogging service Twitter, which allows users to communicate in real time by allowing them to post public messages of up to 280 characters including images, videos, and links. There are example cases displaying the responsiveness of Twitter, e.g. in the 2011 Christchurch Earthquake in New Zealand, where “the first tweet was within 30 s, the first photo was within 4 min” (McLean 2015). Especially at the beginning of a crisis situation, emergency managers perceive a lack of reliable information (Tapia et al. 2013). Extracting posts from eyewitnesses could therefore provide actionable information for the planning of emergency operations (Landwehr and Carley 2014) and close information gaps concerning the situation at emergency scenes (Young et al. 2013) which accelerates the response time and reduces risks to citizens (Luna and Pennock 2018; Stieglitz et al. 2018c).

Although social media analytics could contribute to achieving situation awareness, emergency management organizations are still hesitant to adopt social media (Plotnick and Hiltz 2016; Stieglitz et al. 2018c). Nowadays, many emergency management organizations maintain a presence on social media to reach a larger percentage of the population in times of crisis (Eismann et al. 2016). However, analyzing social media data is still perceived rather as an experiment than as common practice (Imran et al. 2015; Lazreg et al. 2018; McLean 2015). Previous research identified the questionable veracity of social media information, the large volume of available data, the variety of data formats, and the high velocity of information diffusion as the most important technological challenges inhibiting the adoption of social media analytics in crisis management (Stieglitz et al. 2018c). Despite these challenges, many emergency managers acknowledge the potential benefits of social media data and believe that social media analytics will become an important part of their work routine in the future (Reuter et al. 2015). Regarding the questionable veracity of information, addressing the credibility of social media information is still seen as one main priority for the future development of social media analytics (Hiltz et al. 2020). Due to the unpredictable nature of crisis situations, emergency managers are used to limited and vague information, and therefore could trust social media sources, especially in the beginning of a crisis (Palen et al. 2011). However, it still poses a challenge to filter potentially useful information from the large number of irrelevant social media posts (McLean 2015). To address these challenges, Ludwig et al. (2015a) emphasize the need for filtering algorithms allowing to separate useful and credible information from the wealth of irrelevant social media content in a timely manner.

In previous research, crowdsourcing and automated approaches based on machine learning were identified as promising concepts (Imran et al. 2013; Tapia et al. 2013). Although crowdsourcing approaches can benefit emergency operations, they reach their limit as soon as the humans fueling them reach theirs. Therefore, automated approaches might be appropriate to solve the problem of analyzing the high number of social media posts in addition to crowdsourcing approaches (Hiltz and Plotnick 2013). However, it is difficult to correctly classify tweets using machine learning approaches because of the short text format (C. Li et al. 2012). Despite the advances that were made, it has been argued that the proposed solutions must become more accurate (Tapia et al. 2013). The examples in Table 2 demonstrate that the development of social media analytics applications for emergency management is already advanced and offering a variety of approaches with different strengths, while all of them express a demand for improvement and more accuracy. Especially, the processing time can be very long, because of the volume of data and the need of human input for labels (Troudi et al. 2018) and some authors argue that natural language processing needs to be improved (Aupetit and Imran 2017; Luchetti et al. 2017). Also, the automatic classifications could be more detailed and more robust, which implies the need for more detailed test labels (Alam et al. 2020; X. Li et al. 2019). Moreover, Lazreg et al. (2018) argue that social media analytics has to focus on providing information answering the specific needs of individual emergency managers.

Table 2 Overview about state-of-the-art approaches for filtering social media data

1.1.3 Augmented Reality in Emergency Management

Recently, augmented reality has received attention in information systems research as the technology allows to present information in direct association with relevant elements in the physical world (Olshannikova et al. 2015). The unique interface of augmented reality is considered most useful when “the success [of a task] is increased or made more likely (...) through additional visual information being presented alongside the physical world” (Steffen et al. 2017). According to prior studies, the unique way of presenting information can improve the user’s analytical reasoning and decision-making abilities (Chandler et al. 2015; ElSayed et al. 2016). Augmented reality systems are characterized through a combination of real and virtual content (Azuma 1997). This distinguishes them from virtual reality systems, which allow the user to immerse completely in a virtual world (Milgram and Kishino 1994). In the context of our study, we focus on the potential of augmented reality as it is critical for emergency operators to continuously perceive their real environment while processing information. This is due to the fact that, similar to the emergency response forces in the field, it is essential for operators in emergency control rooms to cooperate and achieve shared situational awareness for a successful emergency management (Seppänen et al. 2013). Augmented reality experiences are realized on different devices such as head-up displays, head-mounted displays, virtual retina displays, smart glasses, and hand-held devices (Van Krevelen and Poelman 2010). In previous studies, participants reported the heavy weight and the limited field of view as the major disadvantages of wearable devices (Braly et al. 2019). Moreover, the usage of augmented reality can lead to attention tunneling because the user’s attention is drawn to highlighted objects (Dixon et al. 2013).

Nevertheless, there are already examples of augmented reality applications that have been used to support the development of situation awareness. For example, augmented reality systems have improved situation awareness when driving a vehicle (Park et al. 2015), riding a motorcycle (Jenkins and Young 2016), and driving automated cars (Lindemann et al. 2019). Similar results were also found in a study investigating the effects of wearable augmented reality on the operators of safety-critical systems. By providing real-time information through augmented reality, an improvement of the operator performance and situation awareness could be ascertained. These impacts are important because in a security-critical environment such as emergency management, they can prevent catastrophic loss of life or damage to property or the environment (Rowen et al. 2019). Additionally, Berkemeier et al. (2019) identified cross-industrial application areas and use cases for augmented reality systems based on the unique features of this technology. These identified application areas include communication, data visualization, text handling, and navigation which are also relevant for the work in emergency control rooms. Augmented reality could be beneficial in these areas by processing data to enable context-adaptive functions and information to enhance the operator’s workflow (Berkemeier et al. 2019). Further examples of augmented reality use in the field of emergency management can be found in the literature: For instance, the multi-agent system called Smart Augmented Field of Emergency combines augmented reality, wearable computing, and intelligent agents to support teams in emergency and rescue scenarios (Brunetti et al. 2015). Furthermore, the augmented reality application THEMIS-AR was developed to assist first responders in disaster relief operations (Nunes et al. 2019). The application was designed to gather information, offer advice and guidance on response priorities, and provide situation awareness information based on georeferenced pictures that could be shared among different users. In addition, Brandao and Pinho (2017) made suggestions on how augmented reality could be used to improve the situation awareness of dismounted operators who tend to deal with a large amount of volatile information. Luchetti et al. (2017) describe the advantages of applying augmented reality on the emergency scene with their tool Whistland as “making the information from social and sensor networks accessible for quick interpretation of geo-data to professionals and common users working on the scene of an emergency”. Hereby, Luchetti et al. (2017) highlight the capability of augmented reality to simplify visualization and improve situation awareness. In the case of Whistland, the focus lies on mobile applications for the emergency field operators. In summary, most studies so far focused on augmented reality to improve situation awareness in other contexts than emergency management or on supporting emergency response teams in the field. However, it still constitutes a research gap to examine how augmented reality can be used to increase situation awareness and decision-making in the control room work environment.

1.2 Research Approach

1.2.1 Expert Interviews and Goal-Directed Task Analysis

The goal-directed task analysis is a cognitive task analysis technique that allows to identify critical decisions and information requirements of operators in various application domains (Endsley 1993). This method focuses on the basic goals of the operator, major decisions that need to be made to accomplish these goals and the situation awareness requirements that are mandatory for decision-making (D. G. Jones et al. 2012). This method is suitable for exploring what operators ideally want to know, regardless of whether this information is already available (Endsley et al. 2003). The knowledge of the required information helps designers to find better ways for the presentation of situation awareness information which leads to an improved decision-making performance of operators (Paulusová and Paulus 2018). To obtain this knowledge, interviews should be conducted with subject matter experts who have extensive knowledge of the operator’s job (Rummukainen et al. 2015). The results are organized into hierarchical charts depicting the overall operator goal, the major goals, the sub goals, the decisions that need to be made for each sub goal, and the situation awareness requirements for each decision (Usher and Kaber 2000). The hierarchical chart aids in identifying the essential information that has to be provided by an information system to support the operator’s situation awareness.

Goals are higher-order objectives necessary for a successful job performance and have to be distinguished from tasks or information requirements. Every major goal and subgoal is numbered to define the branch in which it is located; however, the numbers do neither indicate the sequence nor priority of goals. In some cases, a sub goal may be relevant for more than one major goal. In these cases, the sub goal can be described in depth at one part of the hierarchical chart and referenced at another part of the hierarchical chart using callouts. Each major goal forms an individual branch in the hierarchical chart with its own sub goals, decisions and situation requirements. Decisions are presented as octagons and phrased as questions that have to be answered. The subsequent level in the individual major goal branch includes the situation awareness requirements which are presented as rectangles and describe the information needed to answer the questions. The situation awareness requirements can be listed in an intended stacked format according to the level of situation awareness they belong to. This ensures that all situation awareness levels are considered when designing an information system for the operator’s goals.

As part of the goal-directed task analysis, we conducted expert interviews with nine emergency operators working at fire departments throughout Germany (see Table 3), as these are the main responsible institutions in disaster control and civil protection activities (BBK 2020). The interviewees were selected due to their job experience to ensure that they were familiar with the goals, decisions, and situation awareness requirements in the control room work environment. Additionally, we selected operators with different roles in the operation center to consider different perspectives, and thus improve the depth of information obtained.

Table 3 Overview of respondents: first round of interviews

At the beginning, the participants were informed about the goal of the interview and their rights as participants. The interviewees gave their written consent to the recording of the interviews. In the first part of the interview, we asked the participants to introduce themselves and to talk about their tasks, job experience and position at the emergency operation center. We then explained the goal-directed task analysis technique to the interviewees and presented them a template for creating the primary goal hierarchy. In the second part of the interview, we asked the operators about the relevant goals to achieve a successful job performance and to describe their working steps in a typical emergency case chronologically. Afterwards, we presented a template for describing a major goal in detail to the participants. In the third part of the interview, we aimed to identify the decisions and situation awareness requirements related to the mentioned goals by repeatedly asking the following questions suggested by Endsley et al. (2003): What decisions do you have to make to achieve this goal? What do you need to know to make that decision? Which information are needed to make that decision? How do you use that information? What would you ideally like to know? How do people do this badly? What do they typically fail to consider? An overview about the structure and main questions asked in the expert interviews can be found in Table 4.

Table 4 Structure and main questions of the first round of expert interviews

After each interview, we created a transcript following the rules of Kuckartz et al. (2008) and used the provided information to create a preliminary version of the hierarchical chart. We subsequently discussed this chart with the interviewees. This way, they could add, remove or edit certain aspects of the hierarchical chart and therefore improve the results. This procedure is also recommended by (Endsley et al. 2003) since it allows a review of the goals, decisions and situation awareness requirements by another person who is familiar with the illustrated steps. We conducted interviews until we reached theoretical saturation, meaning that the last interview did not result in any changes to the hierarchical chart. In the last part of the interview, we asked the participants how satisfied they were with the information systems currently used in control rooms, how well these meet the needs of the operators, and how these could be improved. Overall, the conducted interviews took between 30 min and one hour.

1.3 Twitter Data Analysis and Expert Interviews

To examine which situation awareness requirements could be fulfilled by integrating social media data into the information systems of control rooms, we first collected tweets during the storms Dragi and Eberhard in Germany and then conducted a second round of interviews with emergency operators about the usefulness of the collected data. Storm front Dragi was a less severe event on 09 March 2019 followed by the main storm front Eberhard that crossed the middle of Germany from west to east on 10 March 2019. In combination, these storm fronts resulted in the second highest warning level for storms in Germany and caused 10,000 operations for the fire department in the federal state of North Rhine-Westphalia (Waz 2019). The news reported on several property damages and impacted victims of the storm, namely 27 injured and one dead person in North Rhine-Westphalia. We collected the storm-related tweets on 10 March 2019 using a self-programmed java tool connected to the Twitter Search API. The search terms were German equivalents for the keywords “storm”, “storm front” and “extreme weather”, from which a tweet had to contain at least one to be included in the dataset. We decided against using the names of the storms as keywords, because they have an ambiguous meaning in German, and therefore many irrelevant tweets would have been collected. In total, the data set contained 13,155 tweets of which 6843 were retweets. The goal of the subsequent analysis was to identify the tweets from different category types that were rated most relevant by users and select these as examples for the interviews with the emergency operators. In a first step, we excluded all retweets, replies, and comments since these include duplicate information or are difficult to interpret without reading the related original tweet. By removing these types of postings, the data set was reduced to 6312 original tweets. In a second step, we excluded all original tweets with less than ten retweets since previous research has suggested that the retweet count can be used as an indicator for relevancy (Starbird et al. 2012). Finally, the data set contained 172 original tweets representing 1.31% of the total data set and 2.72% of the original tweets.

The remaining 172 original tweets were manually categorized by data format, role of the communicator, and type of content. Regarding the data format, we distinguished between tweets that consisted of 1) text only, 2) text and image, 3) text and video / GIF-file, 4) text and URL, and 5) mixed data formats (e.g. text, image, and URL). For categorizing the communicator’s role, we combined the classification schemes from Stieglitz et al. (2017) and Olteanu et al. (2015) who identified the following communication roles during crisis situations: media organizations (e.g. news organizations, weather channels), governmental organizations (e.g. local or national administration, political parties, and public authorities), juristic persons (e.g. business organizations, non-governmental organizations), private persons (e.g. eyewitnesses, non-affected persons), and public persons (e.g. journalists, politicians, and celebrities). To categorize the type of content, we adopted the classification scheme from Heverin and Zach (2012) who distinguish between information, opinion, technology, emotion, and action tweets during crisis situations. Due to the large number of informative tweets in the dataset, we inductively generated subcategories for the category of information-related tweets (news, weather, traffic, and eyewitness reports). Once we completed the manual categorization, we selected the three most retweeted tweets for each category to present these to the interviewees as examples of realistic Twitter communication during a storm. An overview about the selected 21 example tweets can be found in Table 5.

Table 5 Overview of the selected example tweets

We then conducted semi-structured expert interviews with five emergency management officers working at fire departments throughout Germany (see Table 6). All interviewees had experience with the work in control rooms and were involved in the emergency response to a storm event. To consider different perspectives about the usefulness of social media data, we have selected interview partners working at institutions with various implementation levels of social media. While the fire department of one interviewee was not using social media at all, the remaining fire departments used Twitter and Facebook for crisis communication and implemented a manual screening of tweets into their work routine.

Table 6 Overview of respondents: second round of interviews

At the beginning, the respondents were asked to introduce themselves and describe the typical work processes in a control room during a crisis situation. The interviewees also described the information sources currently used in their work. Afterwards, we asked the interviewees to describe how the fire department responded to the storms Dragi and Eberhard to get an overview about the timeline from the view of an emergency agency. Then, we presented the representative sample of tweets from our Twitter dataset to the interviewees and asked them to rate the usefulness of these tweets providing them with a scale from 1 (not useful at all) to 5 (very useful). During the main part of the interviews, we asked the respondents to explain the reasons for their evaluation in detail. Thereby, we frequently used follow-up questions to obtain more detailed information about the usefulness of the different data formats, communicator’s roles, and content types. We ended the interviews with questions about the interviewees’ intention to use social media as part of their work in the future. An overview about the structure and main questions asked in the expert interviews can be found in Table 7.

Table 7 Structure and main questions of the second round of expert interviews

Overall, the interviews lasted between 60 and 90 min. Again, the interviews were transcribed according to the rules of Kuckartz et al. (2008). We then analyzed the interviews using the approach of deductive category assignment according to Mayring (2014). The categories and subcategories used for the manual classification of tweets were selected as predefined categories for the qualitative content analysis of the interviews. From the transcripts, we extracted the opinions expressed about the usefulness of the selected tweets. By applying nominal category systems (Mayring 2014), we further distinguished between positive, negative, and ambiguous statements of the interviewees regarding the different categories of tweets.

2 Results

2.1 Situation Awareness Requirements

The goal-directed task analysis based on the first round of expert interviews resulted in a hierarchical chart consisting of the primary goal hierarchy and two individual branches including decisions and situation awareness requirements. All interviewees declared “the saving of human and animal life as well as the protection of property and cultural assets” as the overall operator goal in their profession. As illustrated in Fig. 2, the two major goals are divided into “sending of emergency forces” and “support of emergency operations”. The detailed individual branches of the first and second major goal can be found in the electronic supplementary material.

Fig. 2

Primary goal hierarchy

Major Goal 1: Sending of Emergency Forces

The first major goal presents the process from receiving the call to sending the first responders. To determine the required emergency forces, the operator first has to identify the exact location (P09). Related to this, the second interviewee said: “In the case of an emergency call, the first priority is to verify the place of operation. The reason for this is that if the conversation breaks off in between, then we have at least a place where we can send the emergency services” (P02). The situation awareness requirements differ based on the location the operator tries to identify. For example, when the conversation indicates an emergency situation in a building, the operator tries to gain information about the street, apartment and building. However, when the caller describes a situation in a forest, this activates a different mental model and the operator’s approach on the identification of the location changes. There is also the possibility that the callers on the phone do not know where they are or that they unintentionally give misdirecting information (P01, P06). These are situations in which the operator must use the described features in the caller’s environment, such as bus stops or churches within sight. Another important situation awareness requirement regarding the location is the accessibility of the scene (P03). It is possible that a person is trapped in a car or in a building which requires additional rescue tools to be taken into account when selecting the emergency forces.

After gaining the required situation awareness information to identify the location and the accessibility, the operator needs to understand the situation to select the right forces. Similar to the information needed to determine the location, what the operator needs to know depends heavily on the nature of the emergency. For example, in case of fire the operators need information about the color of the smoke, the size and location of the fire, the surroundings, and the affected people to understand the current situation and predict the possible spreading of the fire (P01, P02). For an operation involving injured or sick people, the necessary information focuses more on the condition of the patient and the needed medical treatment (P03, P04). With the required information to locate the emergency, understand the situation and verify the responsibility, the operator is able to select a keyword according to which the system makes a suggestion regarding the needed emergency vehicles.

Furthermore, the operators need information to determine the necessity of additional special emergency forces. Operators often face situations where special tools or specially trained forces are required, for example, when a person is trapped or certain substances are needed to extinguish a fire. One interview partner said: “Special means or special vehicles or forces are always required when the case is different than the norm. It already starts when we have a person in the water” (P06). The same applies to the choice of additional authorities. If the operation involves situations the emergency forces are not trained for, the emergency operator may also involve other authorities trained for such situations. With all the above-mentioned situation awareness requirements, the operator can choose the emergency forces, special vehicles with the required tools, and authorities that are needed for the operation. The last step towards achieving the first major goal is to alert the selected emergency forces. However, this is an automated process and therefore no additional situation awareness requirements are needed.

Major Goal 2: Support of the Emergency Operation

The second major goal involves the support of the emergency operation and includes three subgoals. The first subgoal is achieved by sending additional emergency forces and contains four decisions that require information similar to the decision of the initial selection of the required emergency forces. The main difference is that within the second major goal, the situation awareness requirements for these decisions are mostly obtained from the first responders and not from the callers (P04, P05). However, it is also possible that this information may be received from other callers, although this case occurs less often. The obtained information helps the operator to understand the lack of emergency forces as well as the mental condition of first responders and bystanders to forecast future demand. With this projection, additional forces, special vehicles, other authorities and psychosocial care can be sent after the arrival of the first responders at the scene of emergency.

The sending of further forces or aids is not the only way in which an operator can support an emergency operation. It often happens that these situations involve injured or ill people that need to be stationed in a hospital (P01). Since the first responders are busy with the handling of the accident, the operators are responsible of finding a suitable hospital. To achieve this, operators are dependent on the information they receive from the first responders since they decide if a person needs medical treatment. There are different situation awareness requirements to identify the adequate clinic. The most important one is the availability of the required equipment for appropriate medical treatment. For example, if a person has suffered a head injury in an accident, the hospital ought to have a neurological department. Additionally, the operator and first responders need to consider the preference of the patients. If they designate a clinic to which they wish to be transferred to and that clinic has the appropriate equipment, the emergency services will try to comply with the request. This preference may be based on factors such as proximity to home or family and experience or pre-treatment in the facility, which are also aspects that the operator considers even if the patient does not have a preferred clinic. The support of the operation is not only provided by assisting the emergency forces, but also by hel** the person on the phone before the arrival of the forces. This help can include the instruction for medical treatment like cardiopulmonary resuscitation (P01). The situation awareness requirements to understand the need of this support on the phone is based on the operator’s comprehension of the situation and the information gained in the conversation. It is possible that the callers request guidance because they do not know how to behave or that the operator understands that the caller is overwhelmed by the situation and predicts the need for an instruction.

2.2 Usefulness of Tweets for Situation Awareness Support

The interviewed fire fighters preferred to keep social media analytics separate from the working field of the emergency operators. One fire fighter stated that, “it is not the task of the control center to access the Twitter channel” (P11). In their daily social media work, the interviewees network and extend their range, e.g. by connecting and subscribing to official local channels, such as the police, press agencies, and traffic companies. During a crisis, the social media channels are operated alongside the normal press work and four interviewees already use social media to monitor ongoing crisis situations. If they find relevant information, they pass it on to the emergency management and the control room. Most interviewees saw a lot of potential in the automated analysis of large amounts of social media data and described the analysis of Twitter data as useful and inevitable while still mentioning some concerns regarding the trustworthiness. Regarding the content of a tweet, the interviewees generally preferred detailed, concrete and up-to-date information about the situation. They also mentioned the importance of hashtags, since they are monitoring manually and use the official hashtags, which they either registered or distributed themselves, to find relevant information. Credibility of the source was also very important to emergency managers and was perceived ambiguously by them. Concerning the different communicator’s roles, official sources were viewed as trustworthy, e.g. transport services (Deutsche Bahn, the national railway operator) or governmental organizations (German Meteorological Service). In contrast, the interviewees were concerned with the credibility of private persons, while also stating that trustworthiness depended on the actual case and they take every hint seriously. Finally, most interview partners agreed, that the data format is not as important as the content itself. One person said: “Whether it is a picture, a text or a video is irrelevant” (P12). However, all interviewees rated the specification of an exact location as crucial for planning operations and sending task forces. Table 8 summarizes the types of tweets that were described as useful and trustworthy by the interviewees.

Table 8 Usefulness for situation awareness and trustworthiness of the categories

2.3 Conceptual Augmented Reality Design

The goal-directed task analysis is a technology-independent method that focuses on identifying the important goals, decisions, and situation awareness requirements of operators (D. G. Jones et al. 2012). A goal-directed task analysis can be used to determine what operators would ideally like to know to meet their goals, even if that information is not available with their currently used technology (Endsley et al. 2003). Therefore, the situation awareness requirements of the goal-directed task analysis can be used to explore possibilities of improving the presentation of information with any kind of technology (D. G. Jones et al. 2012). Based on previous literature, we identified augmented reality as a promising technology that might improve situation awareness in emergency control rooms (Brunetti et al. 2015; Luchetti et al. 2017; Nunes et al. 2019). In the following, we propose a conceptual augmented reality design considering the identified situation awareness requirements in the first round of expert interviews, the identified useful and trustworthy social media information identified in the second round of expert interviews, and the proposed situation awareness design principles from Endsley et al. (2003). We evaluated and improved the conceptual design based on the results of a workshop with two situation awareness and two augmented reality design experts. Table 9 provides an overview about the participants of the workshop.

Table 9 Overview of workshop participants

During the workshop, we demonstrated the design to the participants and asked them to discuss the following six questions: 1) How could the design enhance the situation awareness of an emergency operator? 2) How well does the design implement design principles that you are familiar with? 3) What are the innovative aspects of the design that distinguish it from other solutions in the literature or on the market? 4) What kind of technological challenges do you see in implementing this design? How could these challenges be solved? 5) What do you think is the added value of social media data for achieving situation awareness in control rooms? 6) What added value do you think augmented reality offers compared to the existing solutions in control rooms? In the following, we present our conceptual design and feedback from the situation awareness and augmented reality design experts.

The first design principle of organizing information around goals is already met by the currently used systems, as the control centers we visited used around 6–7 displays to present various software programs related to different goals. For example, there was one display that presented the information regarding the location of the emergency, one contained information gained from the emergency call, and another display was used to select the keyword and emergency vehicles (P01, P05). This division of visuals should also be implemented in augmented reality to fulfill the first design principle. However, augmented reality allows for more freedom when it comes to organizing information around goals: “If you simply have to drag and drop the information with your hands, you can be a lot faster with grou** the information and categorizing it and get a nice overview” (AR02). As the operators mentioned, the determination of the location is the most important situation awareness requirement to send the required emergency vehicles. Therefore, we are going to use the subgoal “Identification of the Location” as an example to show how an augmented reality interface could look like. The proposed interface includes a map and customizable filter options to display the types of tweets that were described as useful and trustworthy in the second round of interviews (see Fig. 3).

Fig. 3

Conceptual wireframe design for an augmented reality application in emergency control rooms

The map provides the operator with an overview of the area of the emergency scene including the different tweets depending on the type of crisis. The selection of the Twitter data that can support the operation could be accomplished by defining keywords for each type of crisis. The location could be derived by accessing the georeferencial data if provided by the user, the author location stated in the user profile, or by extracting mentioned locations from the tweet text. To differentiate the tweets from the marked place of the emergency, the tweets could be displayed as colored pins. The usage of customizable filter options is mandatory to assure that the operator using the system is not overwhelmed by the amount of crisis-related tweets. In our suggestion, the operator would be able to select different filter options by touching the document icon in the top right corner. By touching one of the pins representing a tweet, the operator could open a small window with a detailed view of the tweet. This view could contain information about the author and the tweet content. According to one of the situation awareness experts, “the design also considers the principles of Shneiderman’s information-seeking mantra very well by allowing operators to establish an overview first, then zoom and filter to get details on demand” (SA02). Pictures and videos of the scene that were posted on social media were perceived as very useful by some subject matter experts and could increase the operator’s situation awareness. One interviewee explained: “Sometimes it is a matter of seconds. To have a system that picks up videos and pictures and shares them with the emergency services would be very useful for the first responders who does not know anything before coming to the scene” (P07). The situation awareness experts confirmed the importance of pictures and videos because “verbal communication can’t always answer in a very good way what you as an emergency dispatcher want to know. For instance, if there is a fire. They want to know what the color of the smoke is. That is kind of hard for the caller to explain” (SA02). The detailed view of the tweet could further include different indicators for the relevance of the content and trustworthiness of the author.

The second principle recommends the direct presentation of the level 2 situation awareness requirements to assist with the comprehension. To support the “Identification of the location”, our user interface includes a map of the affected area based on the entered address of the emergency with information about the surroundings. This map could contain further level 2 information about traffic jams, conditions of the streets, access to the location, and the environment of the emergency. To increase their understanding of the situation the dispatchers already use Google Street View (P05, P06). The situation awareness experts also perceived that an additional 3D view of the map supports comprehension because, “if you get some videos and you cannot fully comprehend from which angle or perspective it was taken, you can get additional insights if you have an additional 3D perspective” (SA01). The other situation awareness expert added: “I think the 3D perspective can sometimes be overwhelming, so two dimensions are good for the overview and three dimensions on demand when you need additional information” (SA02). Especially the usage of 3D models was seen as useful. One emergency operator said: “I personally already use Google [...] that is already a kind of 3D thing. [...] But it has to be easier. Google is not that easy, it cannot be done just like that and for us it has to be fast and secure and that is not the case on the Internet with such open software” (P06). These 3D building models could present additional information regarding the accessibility of the scene or display the exact location like the floor where it burns in the case of a fire. For example, the windows of the room with the fire could be illustrated with the color red to make them salient while the entrance and emergency exits are presented in green. This way, the used colors also activate mental models and schemata of the emergency operators. However, this still represents a technical challenge as one of the augmented reality experts explained: “If you want to have exact information perhaps where the emergency exits are, then you need some other data sources. Perhaps integrate data from the building or city departments” (AR01).

The third principle suggests providing assistance for level 3 situation awareness projections. This can be achieved through displays that present changes in parameters over time. One of the augmented reality experts saw huge potential in using the 3D building models to display changes over time: “For example, if some parts of the building broke down and you have pictures from social media. I think this could be really difficult but also an interesting challenge for future research. Maybe there is a way of getting into the perspective of where the social media photo was taken. And then use the photo to lay it over the 3D perspective you have” (AR02). Currently, the control rooms use large monitors in the front of the room to display the changes of conditions in different areas (P01). For instance, one of these screens shows a weather map which presents possible storm incidents, climatic changes, and other variables. This representation of the weather parameters helps the dispatchers to recognize possible problems caused by storms at an early stage and prepare themselves accordingly. This representation already fulfills the third principle, however, augmented reality would allow the users to place this information as they like since the representation in the virtual environment does not depend on the physical equipment in the room. This possibility was also seen by one of the interview partners who said: “We do not use such a thing [augmented reality]. But I think that would be a good thing, because we are building a situation wall, for example, for major damage situations, in which a map is presented that shows the burning objects and the units that are present in an emergency. [...] To present such a situation wall digitally with simple means would be a dream come true” (P06).

The fourth design principle highlights that the support of global situation awareness is important. This can be achieved by giving the user an overview without restricting their access to support the selection of the highest priority goal. Augmented reality offers the opportunity to place a large number of virtual windows in a beneficial way, which ensures that the operators focus on the most important elements without creating an attention narrowing effect. In comparison to the currently used 6–7 monitors, augmented reality allows a larger overview which supports the global situation awareness. Therefore, excessive menuing and windowing with specific data in order to obtain information is not necessary. It is possible to provide the needed knowledge in different windows at the same time allowing the simultaneous perception without overloading the user with information. Therefore, the user interface should contain multiple elements, each providing the situation awareness information necessary for the respective decisions.

However, as described in the fifth principle, it is important to find a balance between the goal-driven and data-driven processing. An extreme tendency to one of the processing types is detrimental for the operator’s attention and affects the situation awareness in a negative way (Endsley et al. 2003). The design in augmented reality should provide an overview over the important elements without overloading the user. Too few elements, on the other hand, can cause the user to have attention narrowing since not every important information can be perceived. The above-mentioned combination of elements already considers the fourth principle and thereby supports the data-driven processing. To enhance the goal-driven processing, an augmented reality system should make the high-priority elements conspicuous to the user, for example, by using different colors or a specific arrangement in the virtual environment.

According to the sixth design principle, it is important to make critical cues for schema activation salient. As mentioned before, mental models and schemata are important to achieve high level situation awareness. First of all, it is important to identify the cues that activate the mental models. For instance, control centers and hospitals use the traffic light colors along with the colors blue and black to prioritize injured patients of mass casualties (P03, P04). Green indicates the lightly injured, yellow marks the badly injured, and red the severely injured (P03). The additional color blue marks the patients who cannot be treated and black symbolizes casualties (P06). Since the red marked ones have to be treated faster, this prioritization helps the operators to organize the treatment accordingly (P03). These colors can be used as critical cues to activate specific mental models and schemata. For example, the previously mentioned relevance or trustworthiness indicators for the tweets could also define different colors for the pins on the map allowing the operator to react faster on the more important information.

Another way of supporting the operator’s situation awareness is described with the seventh principle, which advises to take advantage of parallel processing. If it is possible, the system should provide not only visual, but also acoustic or haptic information. Since most information is gained through the emergency call, the auditory channel is already used in this work environment (P01, P02). To improve the situation awareness visually, the previously mentioned 3D building models could be utilized. Furthermore, one augmented reality expert highlighted the importance of a multimodal interaction approach: “When you use gestures in the air, you don’t have any feedback. This means the situation awareness could be reduced because the operators have to use mental capacities for just controlling the system” (AR01). Therefore, we suggest enabling operators to switch between hand gesture and eye gaze interaction with visual feedback. The eighth design principle emphasizes the careful use of information filtering. The reduction of information is often times associated with the thought that it would lead to a reduction of information overload and an improved situation awareness. But this reduction is more likely to inhibit the situation awareness of the operator by removing important information (Endsley et al. 2003). The recommended approach is to give the user the opportunity to decide what kind of information he or she wants to see. One of the situation awareness experts suggested that the design could not only offer customized filter options for different types of tweets but also for different data sources: “It could serve as a very good extensible base platform. When I was in contact with fire departments they were also concerned about drones. And you can extend your concept if you have some drones deployed and you click on a drone and see the live footage” (SA02).

3 Discussion

3.1 Situation Awareness Requirements

The first round of expert interviews and the goal-directed task analysis were conducted to answer our first research question and allowed us to identify the goals, decisions and situation awareness requirements that need to be considered when designing a situation awareness support system for emergency operators. With this method, we were able to determine “sending the emergency forces” and “supporting the emergency operation” as two major goals that need to be accomplished to achieve the overall operator goal which is “the saving of human and animal life as well as the protection of property and cultural assets”. The different subgoals and decisions to achieve the first major goal focus on the operator’s understanding of the situation and the selection of the appropriate emergency forces. To achieve the first major goal, the operator requires information about the location of an emergency, the accessibility of the emergency scene, information about what exactly happened at the scene, and information about the medical condition of affected people. We found that the detailed description of the emergency location constitutes the most important situation awareness requirement for emergency operators because this information enables them to send emergency forces even if all other relevant information is missing. It is especially critical that the emergency operators are mostly dependent on the information gained from the person who made the emergency call to make decisions about the sending of emergency forces. Thus, the integration of social media data into the control room work environment could be especially useful if the caller has only provided incomplete information. The second major goal focuses on sending additional emergency forces and aids to the scene, selecting an appropriate hospital for injured persons, and assisting callers with instructions on the phone. To achieve the second major goal, the operator requires information about the required number of emergency forces at the scene, the required knowledge, skills and authority to deal with extreme situations, the required special vehicles and tools, information about the mental condition of first responders and bystanders at the scene, information about the medical condition of affected persons, and information about the knowledge of the caller. The situation awareness requirements necessary to support the emergency operation are different from those required for the first major goal of sending emergency forces, since they are mostly obtained from the first responders at the scene.

These situation awareness requirements are essential to design a system that can support the operator’s situation awareness. To obtain these requirements, we applied the goal-directed task analysis technique that was first demonstrated in the aviation domain and afterwards tested in different work environments, for example, in industrial plants (Paulusová and Paulus 2018) or within the army brigade (Bolstad et al. 2012). In the emergency field, there were similar approaches to identify the user requirements in emergency management such as the applied cognitive task analysis used by Pfaff (2015) or the hierarchical task analysis used by De Leoni et al. (2007). Compared to these approaches, our method resulted in a more detailed and comprehensive view of the required information to achieve a good job performance as an emergency operator. For example, the hierarchical task analysis utilized by De Leoni et al. (2007) revealed the different goals required to achieve the overall operator goal without describing the different decisions that need to be made. Since goals can involve multiple decisions to be achieved, it is important to also include them in addition with the required situation awareness information to completely understand the requirements in a specific work environment. The conducted expert interviews in combination with the goal-directed task analysis allowed us to illustrate the decision-making process of the emergency operators in a very detailed way. However, there are also studies that utilized the goal-directed task analysis in the emergency field. For example, the goal-directed task analysis was applied to identify situation awareness requirements during a single event like the attack on the World Trade Center (Dawes and Cresswell 2004). Thus, the obtained situation awareness requirements are not universally applicable like the results of our comprehensive analysis. Furthermore, Humphrey (2009) created a hierarchical chart using the goal-directed task analysis which included every stage and step involved in an emergency situation. The major goals in this study start with the prevention/deterrence of a crisis and end with the last steps of the recovery/remediation after a crisis situation. A similar broad view on the field of emergency management was also created by R. E. T. Jones et al. (2010) in which they additionally included major goals like the detection of bio events. Compared to our results, the decisions and goals are in some occasions akin but since we focused on the specific role of emergency operators, we were able to identify a considerable larger number of situation awareness requirements. In summary, our results represent a novel view with unprecedented detail of the situation awareness requirements of operators working in control rooms.

With the obtained situation awareness requirements it is possible to compare the information needed with the information available through social media. This allows us to determine how social media data can be used to further support the operator’s situation awareness. For example, the geolocation of an eyewitness who posted a tweet about a certain crisis can provide useful information to achieve the subgoal “Identification of the location”. Since most users deactivate the sharing of their geolocation, this information could in many cases also be obtained through the post’s content by using text and image analysis or integrating data from other social networks like YouTube or Facebook. For example, pictures or videos posted from the emergency scene could obtain information regarding the accessibility of the scene and details about the location which the operator did not receive during the initial emergency call. This information could further enhance the understanding of the situation which would make the decision-making easier. Information contained in the post could also support the operator in determining what exactly happened at the emergency scene. For example, user generated tweets from an emergency scene could be extracted by existing text mining tools and presented in an apprehensive way, so that emergency operators could estimate the threats of a situation, how many civilians are involved or how accessible the location is. Furthermore, traffic information posts from transport business organizations could provide information about where additional emergency forces might be needed. Similar to the location, the provided information can enhance the user’s understanding of the situation, and thus the situation awareness. In combination with a time series variation analysis of the disaster damage, the relevant tweets will not only be useful during real-time operations but also to discuss a disaster mitigation plan.

3.2 Usefulness of Tweets for Situation Awareness Support

The second round of expert interviews was conducted to answer our second research question and allowed us to identify the types of tweets that emergency operators perceived as useful to achieve situation awareness. Our study confirms previous research results stating that emergency management organizations in Germany are still not incorporating the full potential of social media analytics (Eismann et al. 2016; Stieglitz et al. 2018c; Tapia et al. 2013). We found that traffic information from transport business organizations is perceived as particularly useful and trustworthy. One of the situation awareness experts explained: “We have this road company and they have this official map where you could get this information from. But they are not too quick with putting in the information. It’s not very implemented in their procedures I think. Maybe in some huge events, social media could be faster” (SA01). In addition, the interviewees described eyewitness reports as useful but questioned the trustworthiness of private persons. Most tweets were rated as not useful, e.g. posts from Twitter profiles of media channels as well as emotional, action, and opinion tweets. However, in our view this represents an advantage for the integration of social media data into the work environment of emergency operators. If there is only a small number of tweets that is actually perceived as useful, filter algorithms can be used to exclude the large amount of irrelevant information. This way, the integration in of social media data into control rooms can provide an added value without causing information overload, which would negatively affect situation awareness. Consequently, filter strategies that specifically display traffic information and eyewitness reports are needed to improve the usefulness and increase the adoption of social media analytics solutions for emergency management. In previous research, it has already been proposed that “a customization of filtering algorithms for the needs of emergency management agencies might be needed for them to respond to their specific crisis situations” (Stieglitz et al. 2018c). Related to this, the eighth design principle for situation awareness demands the careful use of computer-driven filtering and recommends the use of customizable filter options (Endsley et al. 2003). Our research also revealed that individual operators have different perceptions of the usefulness and trustworthiness of different types of tweets. For example, some interview partners perceived weather information from media organizations as relevant while others considered them to be redundant. We therefore suggest to provide customizable filter options allowing emergency operators to adapt the visualization of social media data according to their needs.

The first step towards the development of filter strategies is improving the data collection process in existing algorithms by selecting only useful information for further processing and visualization. While there are already multiple approaches that focus on the extraction of eyewitness reports (e.g. Zahra et al. 2020), the development of algorithms to extract information from relevant business organizations such as railway traffic companies has been neglected so far. In this matter, our findings reveal that new crowdsourcing approaches or machine learning algorithms are needed. In a second step, the filtered set of relevant social media data needs to be visualized to support the situation awareness of emergency operators. As we identified the informativeness as the main attribute of useful posts, the visualization of social media data should be combined with a relevancy indicator that highlights the most important tweets. To assess the relevancy, the number of retweets can be considered, as previous research concluded that users mostly retweet posts that are either from official sources or from eyewitnesses (Starbird and Palen 2010). The interviewees also mentioned other aspects that could contribute to the relevance indicator: These include the @-function that marks messages directly addressed towards emergency management organizations, searching images in Google Pictures to ensure that they are up-to-date or have been used before, and the style of writing that can identify concrete and detailed information. The latter was used by Verma et al. (2011), who found that actionable tweets were objective and formal and could identify disaster relevant posts with this method. As not only relevancy but also trustworthiness is important to emergency operators, the relevant social media data should also be visualized in combination with a trustworthiness indicator, e.g. a traffic light system. Factors that could be integrated into such an indicator could be the verified badge that Twitter offers for accounts of public interest. Other criteria like the author’s description and name, number of followers, formal language, and confirming the person’s location could be applied for accounts without this label. Trustworthy networks could be organized through subscriptions to local officials and Twitter’s list function, which can be used to incrementally organize private persons’ accounts, whose trustworthiness might already have been proven in past events, into content groups (e.g. “local population”) and monitor their tweets without subscribing to them. To the best of our knowledge, the combination of the previously named indicators for relevance and trustworthiness have not been implemented in previous research, although emergency managers often associate their willingness for the adoption of social media analytics with these two aspects (Zade et al. 2018). Most algorithms use informativeness as main criterion for data extraction and incorporate relevancy in their selection process. For instance, there are algorithms using the increase of attention and retweets to identify crisis-related tweets (Hiltz and Plotnick 2013; Starbird et al. 2012) and sorting their tweet list according to the number of retweets (Ludwig et al. 2015b). Other researchers rather focus on the trustworthiness of social media information (Lazreg et al. 2018; Tapia et al. 2013). Related to this, Hiltz et al. (2020) identified a rating of trustworthiness as one of 16 features that experts would find useful in software to support social media use in emergency management. Our proposed indicators can inform the implementation of this important feature.

3.3 Conceptual Design Augmented Reality Design

To answer our third research question, we synthesized the results and proposed a conceptual augmented reality design that integrates social media data into control rooms. The design includes a customizable filter function based on the previously presented filter strategies. Thus, the proposed design integrates only useful and trustworthy Twitter data so that situation awareness can be increased without creating information overload. At the same time, the design fulfills the eight design principles for situation awareness (Endsley et al. 2003) and addresses the situation awareness requirements of emergency operators we identified through the goal-directed task analysis. In previous research, there are already situation awareness support systems that integrate social media data and provide support for emergency responders; however, the needs of emergency operators have not been addressed so far. As a result, our proposed design can support situation awareness in a new field of emergency management. Furthermore, we emphasized a user-centered approach by conducting interviews with experts from the application domain to secure the usefulness of our design. There have also been other researchers who derived requirements for social media analytics from interviews with emergency managers (Hiltz et al. 2020; Stieglitz et al. 2018c) and the tool CrowdMonitor, for instance, was also developed based on expert opinions (Ludwig et al. 2015b). Nevertheless, the development of a design has not been combined with the goal directed task analysis, which ensures that our design fits the specific needs of emergency operators. Similarly to other approaches, e.g. CrowdMonitor, Senseplace2, Scatterblogs, Whistland or CrisMap, our design includes a map where emergency tweets can be located in a well-structured arrangement. However, we have deliberately avoided to include tweet lists or timelines in our design as it can be found in Whistland or Senseplace2, so that the emergency operators will not experience information overload. In addition, our conceptual design includes visualization using augmented reality technologies, which enable quick interpretation (Luchetti et al. 2017) and decision making through the unique presentation (Chandler et al. 2015; ElSayed et al. 2016). Regarding other implementations for augmented reality, mobile solutions like Smart Augmented Field of Emergency or THEMIS-AR have been designed to guide, inform and connect first responders, but they do not incorporate social media data, even though it was found to increase situation awareness. One of the augmented reality experts further highlighted how our proposed design extends previous research: “Many augmented reality applications are just 2D screens with many layers. But what you mostly don’t have is a bird eye view where you have a look at the scenery from top and can walk around and can enter different perspectives” (AR01). With our design, we think outside the patterns of 2D screens and propose to implement a 3D view of the map on demand which could help the operators to interpret the perspectives of social media pictures and videos.

4 Conclusion

In the present study, we examined the potential of social media and augmented reality to enhance the situation awareness of emergency operators working in control rooms. In a first step, we conducted a goal-directed task analysis to identify situation awareness requirements of emergency operators. We were able to determine that information about the location constitutes the most important requirement for emergency operators to achieve situation awareness. In a second step, we collected Twitter data during two storms in Germany and conducted expert interviews with emergency operators to evaluate the perceived usefulness of Twitter data for achieving situation awareness. The interviews revealed that traffic information from transport business organizations and eyewitness reports from private persons were rated as most useful. In contrast, other types of informational tweets as well as emotion, opinion, and action tweets were either evaluated as not useful or trustworthy enough to be integrated into control rooms. In a third step, we synthesized our results by proposing filter strategies for social media data and a conceptual augmented reality design that illustrates how useful and trustworthy social media data could be integrated into control rooms. We evaluated and improved our design based on the results of a workshop with situation awareness and augmented reality experts.

With our research, we contribute a hierarchical overview about the goals, decisions, and situation awareness requirements of emergency operators that has not been presented in such a detailed form, yet. The identified requirements serve as an important foundation for the design of situation awareness support systems for control rooms in the future. We further shed light upon the potential of emerging technologies such as social media and augmented reality for emergency operators while previous studies mostly focused on other actors such as emergency response teams in the field. In addition, we propose criteria that can be used to develop filter algorithms for extracting eyewitness reports and traffic information from transport business organizations. So far, most studies either focused on extracting eyewitness reports or informational tweets from trustworthy sources. However, we found that many informational tweets from trustworthy sources were not perceived as useful by emergency operators, and thus we highlight the need for more targeted social media filter algorithms.

Our research has some limitations. First, the results might not be representative of the whole emergency management landscape due to the small sample size and restriction to emergency operators working in control rooms of German fire departments. To validate the results, future studies should examine whether different types of emergency agencies worldwide perceive the situation awareness requirements and usefulness of social media data in the same way. Second, the usefulness of Twitter data was discussed with the emergency operators based on original tweets collected during a storm event in Germany. However, retweets and replies might also include relevant information and could be examined in future research. Furthermore, it might be conceivable that the perceived usefulness of Twitter data differs depending on the type and severity of the crisis situation as well as the usage patterns of Twitter in a given country. Related to this, future studies could also investigate whether the perceived usefulness of tweets depends on the different stages of emergency management (e.g. preparation, response, and recovery). Third, we proposed a conceptual design based on the insights gained through the goal-directed task analysis and expert interviews. The subject matter experts were not experts on interactive systems, therefore we invited situation awareness and augmented reality experts to provide further feedback on the design. Furthermore, the implementation and evaluation of the design regarding its usefulness for enhancing situation awareness and reducing information overload of emergency operators represents another avenue for further research. Nevertheless, the statements of our interview partners and workshops participants allow us to conclude that the control room work environment constitutes a promising application area for social media and augmented reality technologies.