Keywords

1 Introduction

Nowadays, both usability and user experience (UX) are considered key factors for the success of a software product [1]. The current competitive market has forced companies to concern not only in the functionality, but also in the user experience. Graphical user interfaces of a software product must be understandable and easy-to-use enough, to allow end users to achieve their goals with effectiveness, efficiency and satisfaction [2]. Moreover, the concept of usability has evolved, and the main interest nowadays is related to ensure a positive perception and responses that result from the use or anticipated use of a product, system or service [3]. Usability as well as positive user experience can be achieved if a user-centered design (UCD) process is followed during the software development [4]. However, there is not still a consensus about the methods that must be used in each phase of the UCD framework.

The user-centered design process, described in the ISO 13407 standard, establishes four activities that cover, from the conception of considering the user’s opinion for the development of the software products, until the evaluation of obtained results. Likewise, the iterative nature of these activities is instituted. User-centered design involves iterating until the goals are achieved, and the ISO 13407 establishes the basic principles and guidelines without defining the methods. In this work, we present a systematic literature review to determine the methods and tools that can be used in each activity of UCD. This proposal is based on the review of success stories reported on redesigns of software products. This study is intended for developers and designers to be used and serve as a guide for decision making in the development and improvement of software systems.

2 Background

2.1 Usability

According to the ISO 9241-220 standard, usability is the “extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [5].

Jakob Nielsen defines usability as “a quality attribute that assesses how easy user interfaces are to use” [6]. This author also mentions that usability “refers to methods for improving ease-of-use during the design process” [6] and it is defined by five quality components:

  • Learnability: How easy it is for users to accomplish basic tasks the first time they encounter the design.

  • Efficiency: Once users have learned the design, how quickly they can perform tasks.

  • Memorability: When users return to the design after a period of not using it, how easily they can reestablish proficiency.

  • Errors: Number of errors made by users, the severity of these errors and how easily users can recover from the errors.

  • Satisfaction: How pleasant it is to use the design.

2.2 User-Centered Design (UCD)

User-Centered Design (UCD) is defined as “a software design methodology for developers and designers. Essentially, it helps them make applications that meet the needs of their users” [7].

Another definition of UCD is given by the ISO 9241-210 standard, which defines it as an “approach to systems design and development that aims to make interactive systems more usable by focusing on the use of the system and applying human factors/ergonomics and usability knowledge and techniques” [8].

Likewise, according to the ISO 13407 standard [9], UCD is composed of four activities, which are:

  1. (1)

    To understand and specify the context of use

  2. (2)

    To specify the user and organizational requirements

  3. (3)

    To produce design solutions

  4. (4)

    To evaluate designs against requirements

2.3 Usability Evaluation Methods

According to Fernandez et al., [10], the usability evaluation methods are defined as procedures which are “composed of a set of well-defined activities for collecting usage data related to end-user interaction with a software product and/or how the specific properties of this software product contribute to achieving a certain degree of usability”.

Because of the importance of usability “in the context of the software development process” [11], several methods have been proposed to evaluate it. These methods can be classified into three categories: Inspection, Testing and Inquiry [12]. The objectives of inspection methods are to identify usability issues and improve “the usability of an interface design by checking it against established standards” [13]. On the other hand, the testing category is composed of a set of methods that involve the participation of the end users of the software product to be evaluated. The purpose of this is to obtain information about how the users use the system and the problems they have when performing some tasks with the graphical interfaces [13]. Finally, the aim of inquiry methods is to evaluate the usability of a software product by obtaining information about the experiences of the end users with the system. This is done through interviews, observations while users are using the software product or through surveys [14].

3 Conducting the Systematic Review

A systematic literature review is “a means of identifying, evaluating and interpreting all available research relevant to a particular research question, or topic area, or phenomenon of interest” [15].

The present work was performed following the methodology defined by Kitchenham and Charters [15] in order to carry out an objective search. Four research questions were formulated to guide this review and a search string based on the PICOC criteria (Population, Intervention, Comparison, Output, Context) was developed. The databases used for the primary search were Scopus, IEEE Xplore and ACM Digital Library.

3.1 Research Questions

The purpose of this systematic review was to identify the techniques and tools that have been employed in order to make redesigns of graphical user interfaces of software products following the User-Centered Design (UCD) approach and that have been successful in usability measurements. Additionally, it shows the most reported usability evaluation methods in these cases, and the reasons why this process took place. In this way, the following research questions were formulated:

RQ1::

What techniques have been used for the redesign of graphical user interfaces of software products following the User-Centered Design framework and that have shown successful results in usability evaluations?

RQ2::

What have been the most reported software tools in the literature that have been used for the redesign of graphical user interfaces of software products following the User-Centered Design framework?

RQ3::

What methods are the most reported in the literature to evaluate the usability of software products within the UCD framework?

RQ4::

What are the reasons for redesigning the graphical user interfaces of a software product following the User-Centered Design methodology?

In order to structure the research questions and the information search to perform this systematic review the general concepts based in PICOC were defined. These concepts are Population, Intervention, Comparison, Output and Context. The “Comparison” criterion was not taken into consideration because this research is not intended to compare interventions (Table 1).

Table 1. PICOC criteria defined for systematic review
Full size table

3.2 Source Selection

Three recognized databases were selected to perform the search process because they are the most relevant in the area of computer engineering. For this work, grey literature was not considered since it is not peer-reviewed. The selected databases were the following:

  • Scopus

  • IEEE Xplore

  • ACM Digital Library

3.3 Search String

Definition of Search Terms.

For the elaboration of the search string, five general concepts were proposed taking into consideration the Population and Intervention criteria previously defined. Different terms were established for each general concept (Table 2).

Table 2. Defined terms for the search string
Full size table

Definition of the Search String.

After establishing the search terms, the following basic search strings were defined:

  • CG1: “redesign”

  • CG2: “user interface*” OR “GUI” OR “graphical user interface” OR “interface” OR “UI”

  • CG3: “software” OR “system*” OR “application*” OR “app*”

  • CG4: “user centered design” OR “user-centered design” OR “UCD” OR “user experience” OR “UX” OR “usability”

  • CG5: “methodolog*” OR “method*” OR “technique*”

Then, the final search string was the following:

(“redesign”) AND (“user interface*” OR “GUI” OR “graphical user interface” OR “interface” OR “UI” OR “software” OR “system*” OR “application*” OR “app*”) AND ((“user centered design” OR “user-centered design” OR “UCD”) OR ((“user experience” OR “UX” OR “usability”) AND (“methodolog*” OR “method*” OR “technique*”))).

Finally, the search strings adapted to the syntax used by the search engine of each database were established:

SCOPUS:

TITLE-ABS((“redesign”) AND (“user interface*” OR “GUI” OR “graphical user interface” OR “interface” OR “UI” OR “software” OR “system*” OR “application*” OR “app*”) AND ((“user centered design” OR “user-centered design” OR “UCD”) OR ((“user experience” OR “UX” OR “usability”) AND (“methodolog*” OR “method*” OR “technique*”)))).

IEEE Xplore:

(“redesign”) AND (“user interface*” OR “GUI” OR “graphical user interface” OR “interface” OR “UI” OR “software” OR “system” OR “systems” OR “application” OR “applications” OR “app” OR “apps”) AND ((“user centered design” OR “user-centered design” OR “UCD”) OR ((“user experience” OR “UX” OR “usability”) AND (“methodolog*” OR “method*” OR “technique*”))).

ACM Digital Library:

(“redesign”) AND (“user interface*” OR “GUI” OR “graphical user interface” OR “interface” OR “UI” OR “software” OR “system*” OR “application*” OR “app*”) AND ((“user centered design” OR “user-centered design” OR “UCD”) OR ((“user experience” OR “UX” OR “usability”) AND (“methodolog*” OR “method*” OR “technique*”))).

In order to obtain updated results and analyze the current state of the art, only relevant studies whose publication date was since 2015 were considered for this review.

3.4 Inclusion and Exclusion Criteria

Each article obtained as a result of the search string was principally examined through its title and abstract in order to determine its inclusion in the present systematic review. Likewise, inclusion and exclusion criteria were defined to carry out this process.

The inclusion criteria were the following:

  1. 1.

    The article is written in English.

  2. 2.

    The article presents a case study of redesign of graphical user interfaces of a software product following the User-Centered Design framework and has been successful in usability evaluations.

  3. 3.

    The article presents the use of methods, tools or techniques for the design of graphical interfaces or for the usability evaluation of software products whose interfaces have been redesigned.

On the other hand, to determinate which studies will not be considered, the following exclusion criteria were established:

  1. 1.

    Articles not related to usability studies of software products.

  2. 2.

    Studies about redesign or usability evaluation of software products for people with disabilities.

  3. 3.

    Articles related to usability studies of virtual reality or 3D virtual environments software products.

3.5 Data Collection

The automated search for this systematic review was performed on September 17th, 2019 in the Scopus database and on September 19th, 2019 in IEEE Xplore and ACM Digital Library. A total of 146 results were obtained from the three consulted databases. After the inclusion and exclusion criteria were applied, 19 articles were selected as relevant for this review process.

Table 3 shows the number of articles that were found during the search process, and Table 4 shows more details about the selected articles.

Table 3. Summary of search results
Full size table
Table 4. Details of selected articles
Full size table

4 Report and Analysis of Results

4.1 Techniques Used for the Redesign of Graphical User Interfaces of Software Products Following the User-Centered Design Framework

Results Report

In order to determine the techniques used to redesign graphical user interfaces, the number of relevant articles in which each technique was reported was identified. The results are summarized in Table 5.

Table 5. UCD techniques reported in the systematic review
Full size table

Analysis of Results

According to the results obtained from the systematic review, a large number of techniques used to work following the UCD methodology were identified and the most reported were prototy**, usability evaluation of the original graphical user interfaces, identification of stakeholders/end users, interviews and focus group.

Prototy** is a technique that aims to design the initial or “draft” version of a system, which allows the designers to explore their ideas and show them to end users before investing resources into development [16]. This has been the most reported technique in the selected articles (a total of twelve papers), since according to many authors “the prototypes are primarily used for the communication, exploration, refinement, and evaluation of design ideas” [17].

The second most reported technique was the usability evaluation of the original graphical user interfaces. In nine of the selected articles, a usability evaluation method was applied to these interfaces in order to identify the problems they had and correct them in the redesign.

The third most used technique was the identification of stakeholders/end users, which was reported in six articles. The fourth most employed technique was the interview, which was used in A17 in order to identify the needs of the users [18] and in A13 to determine the requirements from the perspective of the interviewees [19]. This technique is relevant since according to the UCD methodology it is important to know the needs and perspectives of the users to develop a good design.

Focus group was another of the main techniques employed. It consists of bringing together a group of users to discuss a specific topic through their personal experience. The application of this technique in A03 aimed to know the needs and suggestions that users had regarding perceived usability when they used existing rental applications [20].

4.2 Software Tools Used for the Redesign of Graphical User Interfaces of Software Products Following the User-Centered Design Framework

Report and Analysis of Results

Many of the articles obtained from the systematic review did not report the software tools that were used for the redesign of the graphical interfaces, because of that, it was only possible to identify three tools for the prototy** and four that were used during the usability evaluations.

With respect to prototy** tools, one of them is Drupal, a content management system (CMS). In A06 Drupal was used for the elaboration of the prototype skeleton and its importance is that it allowed the design team to “accelerate the development and modifications of the prototype according to the users’ feedback” obtained during the focus group [21]. Another tool reported was POP 2.0, this was used in A07 to design high fidelity prototypes [17]. The last tool reported was Balsamiq Mockups, which allows the creation of wireframes or low-fidelity prototypes [22]. This tool was employed in A10.

As for the software tools used during usability evaluation, one of them is Prometheus, which was used in A09. Prometheus is a web tool developed to detect usability issues in different types of web pages. It provides “a percentage score to determine the level of usability achieved on the website and a list of failed criteria sorted by priority” [23]. During the systematic review process, support tools for usability evaluations were also identified, such as: OBS Studio and AZ screen Recorder (which were used in A07); and Morae (used in A10, A11 y A12). The last one has been the most reported (in a total of three articles) and it is used to capture “video of the computer screen actions” [24].

4.3 Methods to Evaluate the Usability of Software Products Within the UCD Framework

In this section, the most reported usability evaluation methods are detailed. The results are summarized in Table 6.

Table 6. Usability evaluation methods reported in the systematic review
Full size table

Results Report

Analysis of Results

Through the systematic review, twelve methods used for usability evaluation could be found. The five most reported were:

  1. 1.

    User testing: It consists in selecting a representative amount of end users to perform a set of pre-defined tasks in a software product. The aim of this method is to identify usability issues in the system evaluated. Normally, this evaluation is applied in a usability laboratory [11].

  2. 2.

    Surveys/Questionnaires: “It is a list of questionnaire items that representative users have to answer according to a Likert scale” [11]. The objective of each element is to measure an usability aspect of the system or a dimension of the user satisfaction [11].

  3. 3.

    Interviews: In this method, both end users and usability specialists “participate in a discussion session about the usability of a software application” [11].

  4. 4.

    Usability metrics: The objective of this evaluation method is “to establish quantitative measurements” [11]. In order to quantify the usability of a software product regarding effectiveness, efficiency and satisfaction; usability metrics are used. In this method “the participation of a representative number of users is required to generalize the obtained results” [11].

  5. 5.

    Heuristic evaluation: The heuristic evaluation consists in evaluating the graphical user interfaces of a software product according to certain rules. The purpose of this evaluation is to detect usability problems in a system [25].

According to the results obtained, one of the most reported usability evaluation method was user testing, this is because it provides direct information about the problems that users could have with the interfaces they are testing. That is why, according to Nielsen, it is considered as “the most fundamental usability method and is in some sense irreplaceable” [25].

4.4 Reasons for Redesigning the Graphical User Interfaces of a Software Product Following the User-Centered Design Methodology

Report and Analysis of Results

According to the articles resulting from the systematic review, there are different reasons for redesigning the graphical user interfaces of a software product following the User-Centered Design methodology. These reasons may vary depending on the use of the software. These include adapting the software product to the needs and requirements of the users (articles A06 and A13) [19, 21], improving user participation in the effective use of software (A06) [21] and improving the quality of information dissemination (A13) [19].

In addition, two reasons were identified as the most reported. One of them was to improve the usability of the software product (reported in nine articles) and the other was to improve the quality of the user experience (five articles). With this, it can be concluded that UCD is a framework that helps to design software products with good usability and an adequate user experience, since the articles selected for the systematic review have been reported as successful case studies in usability evaluations. Likewise, it is important to remember that UCD is a methodology based on the interests and needs of end users and that is why its use results in a software product that is usable, understandable and has a good user experience [5].

5 Conclusions and Future Works

From the systematic review of the literature we can conclude that the User-Centered Design methodology is one of the most suitable for designing graphical user interfaces (GUI) of software products that show good results in usability evaluations.

Following a predefined protocol, 146 studies were identified, from which 18 were selected. This work allowed to determine that: (1) prototy**, (2) usability evaluation of the original graphical user interfaces, (3) identification of stakeholders/end users, (4) interviews and (5) focus group are the most reported techniques for redesigning graphical interfaces according to the literature. Moreover, in this study we have identified that one of the most used usability evaluation method is User Testing, since it provides very useful information to detect the problems that users have with the GUI of a system. In addition, according to the systematic review, we have determined that the main reasons for redesigning the graphical user interfaces of a software product following the UCD framework are to improve the usability and user experience with respect to the original interfaces.

As future work, each reported technique could be analyzed more deeply and a list of criteria that take into account the characteristics of the software projects and the stakeholders could be established; this in order to identify which technique would be the most appropriate to apply in each phase of the UCD methodology according to the defined criteria.