Keywords

1 Introduction

More than 50 years ago, John Shepherd-Barron invented the first ATM (Automatic Teller Machine) in London at the request of Barclays Bank [1]. Since then, ATMs have evolved. At the beginning, it was only aimed at withdrawing cash, over time they became more complex machines and were added another financial transactions of a very varied nature, such as mobile phone, balance inquiries, data updates, among others [2].

Considering that, the most used transaction in ATM is cash dispensing, we cannot ignore that when adopting new functions, ATM should be as friendly as possible [3]. An important aspect to improve the usability, and, in general, the user experience is to consider the emotional state, feelings, and emotions that the final user experiences before, during and after interacting with the ATM [4].

In addition, when we are talking about ATMs, security and safety aspects should be considered since ATM are targets of different criminal acts, so that financial institutions add surveillance cameras, electronic devices and others, so that users can perform their operations in a safe way in ATM [5]. Additionally, several studies mentioned that security is an important aspect in the UX [6, 7], especially for users in the ATM domain [8, 9].

In this context, the industry needs to use techniques to obtain quantitative results over the ATM interfaces, to objectively measure aspects of usability and security of ATM applications.

In this paper, the authors present a systematic literature review about metrics that are reported in the literature to assess usability and security in banking software. The objective is to identify whether there are currently specific metrics for ATM interfaces and others that evaluate bank software that can be input to build the mentioned metrics. The works that were taken into account are those published from 2014 to 2019. The final intention of this work is to carry out as a future work a proposal of consolidated metrics for the ATM domain.

2 Background

2.1 Automatic Teller Machine

Automatic Teller Machine (ATM) is a computerized telecommunications device that provides, in real time, access to the clients of a financial institution to their bank accounts in a public space without intervention of the administration of the financial institution [10].

The customer is identified by inserting the card and entering a personal identification number (PIN). This process allows customers to access their bank accounts and perform the operations available according to the bank.

2.2 Metric

It is a measurement scale and method used for the measurement of attributes that influence one or more sub-quality characteristics [11].

2.3 Usability

There are many usability concepts proposed by different authors, but Jakob Nielsen provides a more complete definition, which covers most of the characteristics mentioned by other specialists [12].

Nielsen states that usability has multiple components and is associated with the following attributes: Learning, Efficiency, Memory, Mistakes and Satisfaction [13].

2.4 Usability Metrics

They quantitatively demonstrate whether the evaluated software can be understandable, learned, operated, attractive and compatible with the standards and usability guidelines [11].

An example of usability metrics can be the time that a user takes to perform a certain task, in order to find the ease that users have to perform a task. This time being closer to 0 will show that the user has managed to perform the task quickly and efficiently [14].

2.5 Security

Security will be interpreted as the perception of security that customers have when making a transaction from the beginning to the end of it. The lack of security perception causes the client not to use a certain channel to carry out their transactions. For this reason, perceived security is the extent to which a customer believes that a channel is safe to perform their bank transaction [15].

2.6 Security Metrics

Security metrics are designed to facilitate the decision-making process and improve results. They anticipate user needs to ensure compliance the security objectives [16].

3 Systematic Literature Review

This systematic literature review was conducted as a starting point to identify the current state of research related to usability and security metrics of ATM interfaces or other banking systems. This review was carried out following the methodology established by B. A. Kitchenham and S. Charters [17].

The definition of research questions was carried out based on the PICOC method, where the following criteria are considered: Population, Intervention, Comparison, Outcomes and Context. In this work, a comparison between variables will not be made, for this reason in Table 1 this criterion does not apply.

Table 1. Definition of concepts using PICOC
Full size table

The research questions that were established for this Review Literature Systematic are the following:

  • What metrics have been reported in the literature in the last five years for the evaluation of usability and safety that can be applied to ATM?

  • What metrics have been reported in the literature in the last five years for the evaluation of usability and security of banking software?

According to the research questions, the terms used to compose the search strings were defined. To perform the search, synonyms and acronyms were taken into account to structure the search strings to avoid omitting any relevant results. The established search strings are the following:

  • C1: (ATM OR automatic teller machine OR automated teller machine OR banking OR bank OR financial)

  • C2: (metrics OR measurement)

  • C3: (Security OR secure interface)

  • C4: (Usability OR UX).

The string used for the search was formed as follows:

C1 AND C2 AND C3 AND C4

The search was carried out in the following relevant databases in the research area of this work:

  • Scopus

  • IEEEXplore

  • ACM Digital Library

  • SpringerLink.

The search strings of primary studies that will be used for each of the specified databases are detailed below:

  • Scopus: (TITLE-ABS-KEY (metrics OR measurement) AND TITLE-ABS-KEY (ux OR usability OR “secure interface”) AND TITLE-ABS-KEY (banking OR bank OR financial OR atm)) AND PUBYEAR > 2013 AND (LIMIT-TO (SUBJAREA, “COMP”) OR LIMIT-TO (SUBJAREA, “ENGI”) OR LIMIT-TO (SUBJAREA, “BUSI”) OR LIMIT-TO (SUBJAREA, “DECI”)) AND (LIMIT-TO (LANGUAGE, “English”))

  • IEEEXplore: (“Publication Title”:metrics OR measurement) AND (“Abstract”:usability OR “secure interface” OR UX) AND (“Abstract”:bank OR financial OR banking OR ATM)

  • ACM Digital Library: acmdlTitle:(metrics measurement) AND content.ftsec:(+interface bank financial ATM) AND recordAbstract:(usability security UX)

  • SpringerLink: (metrics or measurement) AND banking AND (usability OR “secure interface”) AND NOT(game AND health AND traffic)

  • Subdiscipline: User Interfaces and Human Computer Interaction

  • Date published: 2014–2019.

To consider a primary study, it must have the following inclusion criteria:

  • Information about usability or security metrics for ATM.

  • Relevant information about usability or security metrics used in other channels and that could be used for ATM.

  • Case studies on usability or security evaluations of financial channels.

  • Aspects to be considered in the elaboration of usability and security metrics.

The following criteria were taken to exclude a primary study:

  • Information and aspects that do not correspond to the banking or financial field.

  • Information not found in the English or Spanish language.

  • Articles published before 2014.

4 Search Results

After searching in the mentioned databases, 354 articles were found. In order to make the selection of the relevant articles, the title and the summary of all articles found in the search were reviewed. Table 2 shows the search results in each of the databases, duplicate articles and relevant articles.

Table 2. Number of papers founded
Full size table

The relevant papers of the area of interest are the following:

  • “Evaluating mobile banking application: Usability dimensions and measurements” [18].

  • “Measurement on Usage of the Internet Banking in Colombia” [19].

  • “Online Banking Security and Usability - Towards an Effective Evaluation Framework” [20].

  • “A model for evaluating the security and usability of e-banking platforms” [21].

  • “Development of Questionnaire to Measure User Acceptance Towards User Interface Design” [22].

Table 3 shows a summary of the content of each of the relevant papers:

Table 3. Summary of the relevant papers
Full size table

Of these five articles, the most significant contribution is that of 160 metrics divided into 13 categories of metrics to evaluate security and usability of Internet Banking. The authors indicate the points that would be evaluated in each of these categories, which are divided into 6 categories to evaluate safety (Table 4) and 7 categories to evaluate usability (Table 5).

Table 4. Security metrics
Full size table
Table 5. Usability metrics
Full size table

5 Discussion

After the Systematic Literature Review carried out, we were able to answer the two research questions mentioned at the beginning.

  • What metrics have been reported in the literature in the last five years for the evaluation of usability and safety that can be applied to ATM?

No documents were found with could be answered the first question, this evidences the absence of specific metrics to evaluate the usability of ATM interfaces or to evaluate the security of the interfaces from this same channel.

  • What metrics have been reported in the literature in the last five years for the evaluation of usability and security of banking software?

It is observed that by extending the scope of search to other financial systems, contributions are found since the issue has been addressed and deepened in the case of Internet Banking and mobile banking. The most important contribution was the list of metrics found for Internet Banking to assess usability and security.

6 Conclusions and Future Work

In the papers founded, it is expressed the importance of the evaluation of two aspects: Usability and Security, and the close relationship between them. It should be noted that several of the points evaluated for security are not related to the interface but to the communication issue of the banking channel and the banking entity, an aspect that is not relevant for the purposes of this investigation.

According to the above, we can conclude that there is an absence of specific metrics to evaluate the usability and security of ATM interfaces.

As future work, we will adapt the usability and interface security metrics obtained for other banking systems in the systematic literature review to identify a proposal that is valid and that applies to the ATM domain. To validate this proposal of metrics, the opinion of industry experts will be considered, who should be interviewed and conducted surveys in order to define the points that can be replicated in ATM and those that cannot.