Keywords

1 Introduction

Agile Software development (ASD) gains in popularity in today’s business world due to enabling immediately changes in the direction of product development. These short-term changes in direction require a flexible approach to Requirements Engineering (RE) as well. In addition, agile methodologies (such as Scrum [1], Kanban [2] or Extreme Programming [3]) are often combined with Human-Centered Design (HCD) [4] activities in order to emphasize a value-driven approach to product development [5, 6]. To this end, the field of agile RE has emerged during the last decade.

Focusing on user needs and value delivery becomes an important aspect in product development due to the increasing competition in all areas. With regard to ASD, plan-driven organizations moved away to value-driven organizations. On the one hand, people in plan-driven organizations often negotiate about project plans, pricing models and the amount of features they can develop with the available resources. They are emphasizing the generated outputs such as number of created features during a time period. On the other hand, people in value-driven organizations discuss visions, experiences and human values as well as the way to address them through the product. They focus on the outcomes that the delivered outputs entail.

Compared to sequential approaches to RE, which comprise a requirement analysis phase before the development can even begin, agile RE is carried out along with the development itself. Therefore, continuous management of requirements is a crucial attribute. Requirements are regularly described from a user perspective in the form of epics and user stories [7] instead of creating a requirements document [8]. Recent research is showing that there are several ways of running RE in an agile environment while involving users and stakeholders [5, 9,10,11,12].

Performing agile RE can lead to challenges organizations have to deal with. In literature, there can be found some studies investigating challenges in agile RE (see [11,12,13,14,15]). However, the related work still lacks in giving a general overview of the challenges in current industry.

This study pursues the main objective of identifying the most important challenges in agile RE industry has to address today. We aim to build a shared understanding concerning these challenges among voices that matter by means of experts in the field of agile RE. Thus, the research questions we pose are listed below:

  • RQ1: What are the key challenges in Agile Requirements Engineering?

  • RQ2: How can we deal with the identified key challenges?

The paper is structured as follows: Sect. 2 briefly summarizes the related work and points out the research gap. Section 3 presents the applied research method and describes the iterative expert judgement process. Then, Sect. 4 identifies the findings and discusses both on their meaning and on the limitations of this study. Finally, Sect. 5 provides the conclusions as well as an outlook on future research.

2 Related Work

There are related studies in the literature that investigate challenges in agile RE by means of different research methods. Table 1 shows an overview of the reported challenges and used research methods.

Table 1. Challenges in agile RE reported by related work
Full size table

Analyzing the related work, we can state that the authors use two different kinds of research approaches in general. On the one hand, Ramesh et al. [13] and Bjarnason et al. [14] utilize case studies to investigate the challenges in the field. On the other hand, Inayat et al. [11], Heikkila et al. [15] and Soares et al. [12] report challenges in agile RE by analyzing primary studies with the aim to identify available evidence in existing research.

Ramesh et al. [13] results were published in 2010. However, as ASD is a rapidly changing research area and the body of knowledge has evolved over the last years, we need to clarify whether the reported challenges are still relevant today. For instance, NFRs may not be longer a challenge for industry since the concept of the Definition of Done and the usage of acceptance criteria are widely spread. Bjarnason et al. [14] carry out a case study in only one company, therefore the results may not be applicable to other companies and may not be representative in general. In comparison, Inayat et al. [11], Heikkila et al. [15] and Soares et al. [12] review primary studies by analyzing existing literature, which is a good approach to get an impression of relevant aspects from a theoretical viewpoint. Nevertheless, one could argue that this is not an appropriate approach to investigate the existing challenges in practice.

To this end, the aim of this study is to identify the most important challenges in agile RE industry has to face up today by getting insights from 26 experts in the field. To the best of our knowledge there is no existing study investigating these challenges by means of a qualitative study with practicing experts in ASD working for many different companies.

3 Research Method

We used an iterative expert judgement process rooted in a Delphi study [16,17,18] in order to respond to our RQs. We applied a modified Delphi study where measuring consensus and stability at group level among several iterations was not the most crucial part. On the contrary, we shifted the focus to applying the valuable features of Delphi for conducting our iterative expert judgement process [19]:

  • Anonymity among experts to avoid influence of dominant individuals

  • Iterative approach

  • Controlled feedback with statistical group response

The main benefit of our modified approach was utilizing the learnings from a previous iteration for carrying out the following ones.

3.1 General Study Design

The study was performed in three complementary rounds. Figure 1 gives a general overview of the process. At the beginning of each round, we started designing the questionnaire, optimized by a pretest. Once finished, the invitation was sent to the experts via email. In the second and third round, we attached the results of the previous rounds to the invitation in order to share the outcomes among the panel. The experts had two weeks to fill in the questionnaire. During the following two weeks we evaluated the results, created the report, specified the criteria for drop** items for the following round and designed the questionnaire for the next round.

Fig. 1.
figure 1

General process of study

Full size image

We conducted the study in German since most of the experts are native speaker. Since we are aware that the term agile RE is not very accepted in the agile community and some experts understand this as a contradiction in itself, we decided not to ask for challenges in agile RE directly. On the contrary, we phrased our questions differently and described the context of our study within the introduction part of each questionnaire.

We used google forms for the first and second round, whereas limesurvey was used for the third round due to the complexity of the questionnaire. In general, we decided to use 7-point Likert items since this has been proven to be the best choice in terms of avoiding interpolations within related research fields [20]. Besides, we adapted the quality criteria proposed by Diamond et al. [17] so as to ensure the quality of our study.

3.2 Panel of Experts

We selected our panelists specifically for their knowledge or position regarding the issue under study. As shown in previous work, the research field of agile RE is very close to existing work practices in industry [5]. To this end, we defined the reproducible criteria for selecting participants as follows:

  • Many years of experience as professional in the field of ASD

  • Working experience in one or more of the following roles: Product Owner, Scrum Master, Agile Coach, Consultant for Agile Transition, Kanban Expert or Lean Startup Expert

The panel consisted of 26 experts who are working in 19 different companies located in Germany and Switzerland. In general, they had 2–10 years of experience working in ASD (average = 6.14 years). In comparison, experts have about 0–16 years of experience with RE (average = 6.65 years). Even though one expert stated that he had no experience with RE at all, we decided to include his answers into the study, since he has long experience in ASD and in general there do not exist a specific role of a requirements engineer.

Figure 2 shows the kind of process models experts have been working with. It is worth mentioning that most of the experts have experience both with sequential approaches and with agile approaches.

Fig. 2.
figure 2

Process models used by experts

Full size image

In addition, Table 2 displays the know-how level in terms of ASD rated by experts themselves.

Table 2. Know-how of panel in terms of ASD
Full size table

3.3 Round 1

The questionnaire of the first round comprised two open questions, repeated 15 times. On the one hand, the experts were asked what the most important challenges with requirements in terms of ASD were. On the other hand, they should give a statement for each challenge to clarify why they considered this challenge as important. The minimum number of required answers was 3, whereas the maximum was 15. In sum, we received 107 answers (items) from 26 experts. Table 3 shows an example of an item consisting of a challenge and a statement concerning importance. The full results can be found in [21].

Table 3. Exemplary item in round 1
Full size table

With respect to data analysis, each challenge was categorized by the authors during a workshop. Those items, which could not be categorized easily, were discussed within the group of authors. We used the following categories: stakeholder and user involvement, collaboration within the team, vision and big picture, iteration planning and estimation, granularity of requirements, dependencies of requirements, understanding agile and agile values, continuous delivery of value, roles and responsibilities, need for security, requirement validation, RE methods, format of requirements, clarity of requirements, prioritization, refinement, discovery and transparency.

Additionally, the reported challenges were categorized according to their agile RE activity (see Table 4).

Table 4. Agile RE activities
Full size table

3.4 Round 2

We checked each item of round one critically, whether or not it was appropriate for answering our RQs and being queried in the next round. Thus, items of round 1 were consolidated or excluded. In the end, we identified 34 items as relevant for assessing them in round 2. Based on those items, we created the questionnaire for the second round. The resulting questionnaire assessed 34 items related to the following topics: stakeholder and user involvement (6 items), understanding agile and agile values (6 items), RE methods (10 items), iteration planning and estimation (6 items) and format of requirements (6 items).

The experts rated each item using 7-point Likert items (see Fig. 3). Moreover, they could choose giving no statement. To sum up, we received responses from 23 experts. For each item we calculated mean, variance and standard deviation. Additionally, we created a diagram showing the distribution of experts’ opinion (see Fig. 3) and discussed on the meaning of findings. The results of round two can be found in [22].

Fig. 3.
figure 3

Exemplary item of round 2

Full size image

3.5 Round 3

We reduced the number of items when designing the questionnaire for the third round. Considering items from round 2, we assessed each item according to (a) its relevance in terms of our RQs, (b) the importance in terms of the attributes of agile RE, (c) the opinion of the experts and the comprehensibility of the items.

The final questionnaire comprised two parts. The first part queried in sum 20 potential key challenges of agile RE (see Appendix). The experts were asked to rate each item, whether or not it is a challenge in agile RE. Moreover, they had the option to choose giving no statement. Then, the second part evaluated those items that experts identified as challenge in terms of importance, following 7-point Likert items (totally important, important, rather important, neutral, rather unimportant, unimportant, totally unimportant, no statement). In addition, experts optionally had the chance to provide a solution for solving the challenge.

In sum, 22 experts filled in the questionnaire. We classified each of the 20 items as challenge in Agile RE since we derived all items from the results of the previous rounds. Besides, we calculated the number of experts who rated each item as a challenge. Then, we defined challenges as key in those cases where 2/3 of the experts’ answers were: “Yes, it is a challenge”. Finally, we calculated the importance for those items. The results of round 3 can be found in [23].

4 Results and Discussion

Summarizing the results of the three complementary rounds, we derived 20 challenges that companies have to cope with in terms of agile RE (see Appendix). We categorized such challenges into stakeholder and user (3 items), requirements management (7 items), methods and artifacts (5 items) and format of requirements (5 items).

4.1 (RQ1) What Are the Key Challenges in Agile Requirements Engineering?

We identified six key challenges industry has to face today in terms of agile RE (see Table 5). In general experts weighted the identified challenges as important [23] and none of them rated one of the six key challenges as unimportant.

Table 5. Key challenges in agile RE
Full size table

All challenges related to the category stakeholder and user are classified as key challenges (C2, C5, C6). Therefore, we can conclude that organizations still struggle to the agile transition. Evolving an agile mindset within a whole organization even in parts that are not close to development is still a challenge companies have to address.

Typically, agile transformation starts in development-oriented parts of an organization. Transforming an organization to become more agile implies a change within the whole organization. The results show that there is a gap between knowledge and understanding agile values [24] within organizations. Development-oriented techniques evolve rapidly. In comparison, there are still challenges involving stakeholders and users into the agile processes (C2, C5, C6).

Two challenges (C1, C4), related to category requirements management, are key in agile RE. On the one hand, companies have an issue with the continuous management of requirements. On the other hand, they have a problem with technical or functional dependencies due to raising effort in coordination. Besides, one challenge of methods and artifacts (C3) is a key challenge.

ASD is commonly used in environments where people have to solve complex adaptive problems [25]. Concerning C1, C3, and C4 we can state that there are still challenges to be solved, due to the complexity of problems, which are not addressed by agile techniques properly. To this end, existing techniques and methods must be adapted or new techniques need to be found.

Figure 4 offers an overview of the categorized key challenges.

Fig. 4.
figure 4

Categorized key challenges in agile RE

Full size image

4.2 (RQ2) How Can We Deal with the Identified Key Challenges?

Experts recommend techniques, methods and tools in order to deal with the challenges in agile RE. Below, we will list the techniques and methods proposed by the panel for each key challenge.

C1: In agile software development functional or technical dependencies with other teams are a challenge because a considerable coordination effort is required.

More than three experts recommended using scaled frameworks such as LeSS, SAFe or Scrum of Scrum. Moreover, they proposed the use of the following techniques: creating a common understanding among all, enhancing continuous communication and collaboration, training the ability to solve dependencies, holding weekly coordination meetings, organizing teams in matrix management, building communities of practices for transcending topics, release planning (SAFe), team-transcending availability of product und sprint backlogs, involving temporary representatives in other teams, enforcing continuous integration, improving API-driven development and microservices.

C2: In agile software development it is a challenge that stakeholders understand that the development team can make independent (detailed) decisions.

The following techniques were suggested: continuous coordination and presenting possible solutions to stakeholder, providing transparency about rationales of the decisions, strengthening product owner with competency in decision making and hel** stakeholders become aware of the consequences of interfering into detailed decisions.

More than three experts recommended providing alternative solutions for one requirement. In addition, it is useful to demonstrate that the recommended solution of a stakeholder is an alternative out of many. In previous rounds, more than one expert stated that product owner and stakeholder altogether decide what to be developed. In contrast, the development team decides how the requirement should be developed.

C3: In agile software development it is a challenge not to lose sight of the big picture during the implementation of complex requirements.

The following techniques were recommended: creating a shared understanding regarding the meaning of the big picture by means of a product vision, defining epics or subgoals in the beginning, managing the big picture as a responsibility of the product owner, providing transparency concerning changes among all, understanding connections among user stories by means of story map**, visualizing customer journey in the beginning, involving users continuously in order to focus on the problem to be solved and identifying central contact person for related topics to enable rapid coordination. Moreover, the experts advised to use visualization by means of roadmaps, sketches of the system and processes, and value streams.

C4: In agile software development continuous management of requirements is a challenge since not all of them are fixed at the beginning and they may change over the course of the project.

The experts proposed the following techniques, methods and tools: collaborating closely with the requesting stakeholder, communicating regularly within the team, refining and prioritizing continuously the product backlog, grooming on demand (Kanban), describing in detail the requirements in the sprint backlog, reviewing the results regularly, discussing the maturity level of a requirement with the team, grou** user stories to epics, using Kano analysis, screening and scoring the theme, weighting relatively, utilizing spike stories to evaluate uncertainty in requirements and using ticketing tools (e.g. JIRA).

C5: In agile software development it is a challenge to work out user requirements and quality of use in cooperation with direct users (end users) of the product.

The experts recommended utilizing the following techniques: prototypes, interviews, observing users by the think aloud method, A/B testing, UX labs, analyzing usage behavior, friendly user tests, alpha/beta/silent launches, improving continuously a released version, utilizing a UX-board for play back user insights and testing hypotheses with real users. In addition, one expert suggested adapting user research to ASD by reducing the methods to the minimal, evaluate within the team without report creation, reduce financial restrictions for user involvement as well as problems of accessing real user by means of panels or a prior recruitment.

C6: In agile software development it is a challenge to involve stakeholders throughout the whole development process in regular iterations so that product development will succeed.

The following techniques were proposed: defining stakeholders and their involvement in regular iterations, proposing goals instead of prescribing solutions, involving all possible stakeholders in the beginning and reducing the amount of people over time.

More than eight experts suggested involving stakeholders by regular planning and review meetings to gather feedback and useful information. In light of this, they recommended clarifying the purpose of the meetings and the importance of the outcomes to be discussed beforehand.

4.3 Meaning of Findings

Comparing our findings to the identified challenges of the related work (see Table 1), we can conclude that 16 out of our challenges are not reported by the related studies.

Our key challenge C5 (user involvement) is reported by all related studies. In addition, three studies [11,12,13] report issues with non-functional requirements, which is comparable to our challenge C13. There is also a relation between the key challenge C4 (continuous requirements management) and the challenge “requirements change and its evaluation” reported by [11]. Moreover, the key challenge C1 (technical or functional dependencies to other teams) is reported by [12] in a slightly different manner since they phrase it like “dependence between requirements”.

Moreover, the results show that the identified challenges are often not limited to ASD, but they rather refer to software development in general. Therefore, we can conclude that organizations still struggle with agile transition and understanding agile values, in particular, in terms of stakeholder and user involvement.

4.4 Limitations

We are aware that the design of a questionnaire is important for the process of data gathering. To this end, we made several pretests of each questionnaire we used with participants matching our criteria of expert selection. Nevertheless, we observed two experts struggling with the user experience of the questionnaire tool (Google Forms) used in round 1. Therefore, we decided to use another tool (LimeSurvey) for the questionnaire in round 3, which was more complex than the previous two.

To carry out the study, the group of authors created summaries of the results and made decisions concerning the kind of items they had to query in the following rounds. That may lead to bias in the opinion building process of the panel. We tried to prevent this point by being very accurate in terms of data analysis and by creating the reports. In addition, we selected items for the following rounds through the selection criteria defined earlier.

5 Conclusions and Future Work

This paper has addressed the identification of the most important challenges in agile RE industry has to face up today. Moreover, we examined how to deal with those challenges. For that purpose, we carried out an iterative expert judgement process comprising three complementary rounds. The learnings from previous iterations were used for carrying out the following ones. Our panel consisted of 26 experts in the field of ASD working for 19 different companies.

We have contributed to the body of knowledge of software development by identifying 20 challenges industry has to address at present in terms of agile RE. Six of these challenges have been defined as key challenges. In addition, we have analyzed options to deal with those key challenges by means of agile techniques recommended by the experts.

Future research may specifically identify challenges in agile RE by means of an international panel of experts, for instance with experts from Scandinavian countries. Our aim is to conduct a comparative analysis among the statements of German-speaking experts with the viewpoint of international experts. In addition, we are creating a tool that supports practitioners solving the identified challenges using agile techniques. Therefore, we are working on agile RE patterns. Some experts stated that the queried challenges are not limited to ASD. To this end, future studies may analyze whether the challenges appear in terms of RE in general.