Keywords

1 Introduction

Crowdsourcing over the world-wide web has been connecting large groups of people together to enable organizations to tap into their knowledge, expertise, time or resources for productive purposes [5]. Senior citizens in Japan and Singapore are beginning to turn to crowdsourcing platforms for productive aging activities including both paid work and volunteering [13, 25]. As the likes of such technical platforms as Amazon’s Mechanical Turk (mTurk), 99designs and Uber grow in popularity, a large amount of worker behaviour data have been accumulated to enable artificial intelligence (AI) approaches to collaborate with human workers to streamline various aspects of crowdsourcing. Increasingly, software algorithms have been designed to evaluate [10, 16], optimize [4, 18] and delegate [27, 29,30,31] work among diverse populations of crowdsourcing workers. These software algorithms that assume managerial roles in order to allow crowdsourcing companies to oversee many workers in an optimized manner are referred to as Algorithmic Management (AM) [14].

While AM helps crowdsourcing systems leverage workers’ intelligence and diverse skills effectively and efficiently, the human aspects of crowdsourcing workers are often overlooked [2]. This prevailing notion has resulted in the workers’ experience during the crowdsourcing processes not being taken into consideration by existing AM approaches. The benefits of rest on worker productivity and wellbeing have been well-established in management science [3, 11]. Yet, most existing AM approaches do not account for allocating time to rest for workers in their computational processes. So far, only one published AM research proposed an Affective Crowdsourcing (AC) framework to dynamically leverage changes in workers’ mood in view of other situational factors to schedule rest-breaks in an optimized manner [32].

The AC framework is based on the assumption that changes in workers’ mood result in linear changes in their productivity. However, existing empirical research from social sciences only support that mood is positively correlated to productivity [20]. Due to individual heterogeneity, it is possible that the map** function between a worker’s mood and his productivity follows non-linear forms. This assumption limits the applicability of AC in practical situations.

To address this limitation, we extend the AC framework to propose the Opportunistic Work-Rest Scheduling (OWRS) approach. It takes into account local information on a worker’s mood, current workload and desire to rest to produce dynamic work-rest schedules. By scheduling work during favourable situations and scheduling rest during unfavourable situations, OWRS jointly minimizes crowdsourcing workers’ collective effort output while maximizing their collective productivity. Compared to AC, OWRS is able to operate under more diverse mood–productivity map** functions including step, logarithmic and exponential forms. As it is a distributed approach with time complexity of O(1), it can be implemented as a personal assistant agent [21, 24, 26] responsible for scheduling activities.

Through extensive numerical experiments based on a large-scale real-world dataset released by Taobao.com, we demonstrate that OWRS outperforms three baseline scheduling approaches. It significantly conserves the collective effort output while making the smallest sacrifice in terms of task completion rates, thereby achieving superlinear collective productivity [23]. The proposed approach establishes a framework which accounts for workers’ heterogeneity to enhance their user experience and productivity.

2 Related Work

In the algorithmic management for crowdsourcing literature, existing approaches concerning the human aspects mostly focus on providing incentives to motivate workers to work harder or inducing good mood among workers to enhance their productivity.

In [15], the authors studied how paid crowdsourcing workers perform compared to volunteers. They observed that per-task payment schemes induce workers to work faster but with reduced quality of work, while wage-based payment schemes cause workers to work more slowly but produce better quality results. This work laid the empirical foundation for computational incentive mechanisms for crowdsourcing which dynamically trade off precision, recall, speed, and total attention on tasks. However, it does not provide insight into how to advise workers to rest efficiently. Following [15], the incentives for crowdsourcing workers are further linked to their past performance so that their competence in the system can be used as a sanctioning mechanism to determine their future expected payoff [16]. This approach induces workers to produce higher quality results, but does not help workers determine the most opportune time to rest either.

In [17], the authors studied the relationship between crowdsourcing workers’ mood and their creative output capacities. They then proposed two approaches for enhancing worker performance in creative tasks: affective priming and affective pre-screening. Their results confirmed that workers who are in a good mood exhibit enhanced creativity, and the approaches induce good mood among workers to create favourable conditions for them to work.

Designing work-rest schedules for workers is an important topic in management science as well [1]. Most of the work in this field focuses on maximizing the workers’ wellbeing as a function of how long they have been working without considering how to simultaneously achieving how collective productivity. Outcomes from this branch of research are static schedules at a coarse granularity (e.g. rest breaks over a day).

[32] is the most closely related work to this research. It proposed an Affective Crowdsourcing framework to model the dynamics of crowdsourcing work while accounting for the effect of workers’ mood on their productivity to produce optimal work-rest schedules. This paper extends [32] to propose new solutions to this optimization problem under more diverse general forms of mood–productivity map** functions to deal with individual differences more effectively.

3 The OWRS Approach

In this section, we discuss the model and formulation of the optimization objective function to minimize effort output while maximizing collective productivity. Then, we provide efficient distributed solutions to this optimization problem under three mood–productivity map** functions.

3.1 Preliminaries

In the context of defined task crowdsourcing systems such as mTurk, the system model consists of a set of workers \(\mathbf {W}=\{w_{1},w_{2},...,w_{N}\}\), a set of competence \(\mathbf {C}=\{c_{1},c_{2},...,c_{K}\}\) workers may possess, and a set of tasks \(\mathbf {T}=\{\tau _{1},\tau _{2},...,\tau _{L}\}\) at any given time slot t.

  • Competence refers to know how in performing certain types of tasks. It is quantified as a scalar value in the range of [0, 1] with 0 indicating no competence in a given type of tasks and 1 indicating completely competent in performing a given type of tasks. It can be treated as the probability for a worker to complete a task requiring \(c_{m}\) with quality acceptable to the crowdsourcer.

  • Workers are associated with personal profiles. Each profile contains information on a worker i’s competence in all available types of tasks, his pending workload queue \(q_{i}(t)\), and his maximum productivity \(\mu _{i}^{\max }\) which indicates the maximum workload he can complete in a given time slot t (the granularity of t depends on requirements by specific crowdsourcing platforms). The ground truth of the competence and the maximum productivity values may not be directly observable. With analytics tools such as Turkalytics [9], they can be tracked and approximated over time.

  • Tasks are associated with task profiles. Each profile contains information on the competence or combinations of competence required perform a task \(\tau _{l}\), the reward for performing the task \(r_{\tau _{l}}\) and the deadline, \(d_{\tau _{l}}\) (expressed in terms of number of time slots) before which this task must be completed.

The number of workers, tasks and competence available in a crowdsourcing system at different time slots may differ.

Following well-established models of the dynamics of the pending task queue for each worker [27, 28], we can express a worker i’s pending workload at time slot \(t+1\) as:

$$\begin{aligned} q_{i}(t+1)=\max [0,q_{i}(t)+\lambda _{i}(t)-\mu _{i}(t)] \end{aligned}$$
(1)

where \(\lambda _{i}(t)\) is the new workload entering i’s backlog queue during time slot t; and \(\mu _{i}(t)\) is the workload completed by i during time slot t. Based on the ‘happy-productive worker’ thesis [8, 20], \(\mu _{i}(t)\) can be expressed as a function of i’s mood and effort output as:

$$\begin{aligned} \mu _{i}(t)=\mu (\xi _{i}(t),m_{i}(t)) \end{aligned}$$
(2)

where \(\xi _{i}(t)\in [0,1]\) is the normalized effort output by worker i during time slot t; and \(m_{i}(t)\in [0,1]\) is i’s mood during time slot t, where 1 denotes the most positive mood. Based on this function, the amount of work done by a worker depends on his current mood and how much effort he spends on the tasks. Following the same Lyapunov optimization [19] based derivation process explained in [32], the optimization objective function is:

Minimize:

$$\begin{aligned} \frac{1}{T}\sum _{t=0}^{T-1}\sum _{i=1}^{N}[\phi _{i}\xi _{i}(t)-q_{i}(t)\mu (\xi _{i}(t),m_{i}(t))] \end{aligned}$$
(3)

Subject to:

$$\begin{aligned} 0\leqslant \xi _{i}(t)\leqslant 1, \forall i, \forall t \end{aligned}$$
(4)
$$\begin{aligned} 0\leqslant \mu (\xi _{i}(t),m_{i}(t))\leqslant \mu _{i}^{\max }, \forall i, \forall t \end{aligned}$$
(5)

where \(\mu _{i}^{\max }\) is worker i’s maximum productivity based on historical observations tracked by the crowdsourcing system. By minimizing Eq. (3), we simultaneously minimize the time-averaged total worker effort output while maximizing the time-averaged collective task throughput (i.e. productivity).

Fig. 1.
figure 1

General forms of the mood–productivity map** function investigated in this paper: (a) Step map**; (b) Logarithmic map**; and (c) Exponential map**.

Full size image

Opportunistic Work-Rest Scheduling

In order to derive solutions for the above objective function, we require a computational model on how mood influences productivity. Existing literature generally agrees that good mood leads to higher productivity. However, due to individual heterogeneity, the mood–productivity map** function may take various forms. In the proposed OWRS approach, we make provisions for three possible general forms of the mood–productivity map** function which follow the “happy-productive worker” thesis. They are:

  1. 1.

    The Step map** function: the general shape and the range of possible values of this relationship is shown in Fig. 1(a). Let \(f(m_{i}(t))\) be a map** function between mood and productivity with 5 steps:

    $$\begin{aligned} f(m_{i}(t))=\left\{ \begin{array}{ll}0&{}, \text {if } m_{i}(t)\in m_{vl}\\ 0.25&{}, \text {if } m_{i}(t)\in m_{l}\\ 0.5&{}, \text {if } m_{i}(t)\in m_{m}\\ 0.75&{}, \text {if } m_{i}(t)\in m_{h}\\ 1&{}, \text {if } m_{i}(t)\in m_{vh}. \end{array}\right. \end{aligned}$$
    (6)

    There can be multiple potential ways to implement the \(\mu (\xi _{i}(t),m_{i}(t))\) for this situation. In this paper, we adopt an approach which divides the mood value between 0 to 1 into 5 equal partitions labelled as “very low (vl)”, “low (l)”, “medium (m)”, “high (h)”, and “very high (vh)” denoted as \(\{m_{vl},m_{l},m_{m},m_{h},m_{vh}\}\), respectively. In this case, changes in a worker’s mood result in five different levels of productivity. It represents a possible situation in which small changes in a person’s mood do not result in significant change in productivity immediately. The general trend of productivity increasing with mood follows the “happy-productive worker” thesis. As the number of steps approaches infinity, the trend will approach the linear mood–productivity map** function. If \(\xi _{i}(t)=0\), \(\mu (0,m_{i}(t))=0\) regardless of \(\mu (0,m_{i}(t)\). Let \(\Psi _{i}(t)=\phi _{i}-q_{i}(t)\mu (1,m_{i}(t))\), Eq. (3) can be optimized by determining the values of \(\xi _{i}(t)\) for all workers at a given time slot as:

    $$\begin{aligned} \xi _{i}(t)=\left\{ \begin{array}{ll}0, \text {if }\Psi _{i}(t)\geqslant 0 \text { or } m_{i}(t)\in m_{vl}\\ \min \left[ 1,\frac{q_{i}(t)}{f(m_{i}(t))\mu _{i}^{\max }}\right] , \text {otherwise.} \end{array}\right. \end{aligned}$$
    (7)

    Thus, \(\mu (\xi _{i}(t),m_{i}(t))\) can be computed as:

    $$\begin{aligned} \mu (\xi _{i}(t),m_{i}(t))=\left\{ \begin{array}{ll}0, \text {if }\Psi _{i}(t)\geqslant 0 \text { or } m_{i}(t)\in m_{vl}\\ \lfloor f(m_{i}(t))\mu _{i}^{\max }\rfloor , \text {otherwise.} \end{array}\right. \end{aligned}$$
    (8)
  2. 2.

    The Logarithmic map** function: the general form of this relationship can be expressed as:

    $$\begin{aligned} f(m_{i}(t))=\log _{2}(1+m_{i}(t)). \end{aligned}$$
    (9)

    The general shape and the range of possible values of this map** function is shown in Fig. 1(b). It represents a possible situation in which the effect of changes in a person’s mood on his productivity is more pronounced when the mood is low than when the mood is high. Thus, the actual number of tasks that can be completed by worker i under such a level of productivity can be expressed as:

    $$\begin{aligned} \mu (\xi _{i}(t),m_{i}(t))=\lfloor \mu _{i}^{\max }\log _{2}(1+m_{i}(t)\xi _{i}(t))\rfloor . \end{aligned}$$
    (10)

    In this case, to minimize Eq. (3), we compute its first order derivative with respect to \(\xi _{i}(t)\) for an individual worker i as:

    $$\begin{aligned}&\frac{d}{d \xi _{i}(t)}[\phi _{i}\xi _{i}(t)-q_{i}(t)\mu _{i}^{\max }\log _{2}(1+m_{i}(t)\xi _{i}(t))] \nonumber \\&\,\,\,\,=\phi _{i}-\frac{q_{i}(t)m_{i}(t)\mu _{i}^{\max }}{[1+m_{i}(t)\xi _{i}(t)]\ln 2}. \end{aligned}$$
    (11)

    By equating Eq. (11) to 0, we obtain the effort output \(\xi _{i}(t)\) by worker i at time slot t which can minimize the objective function Eq. (3) as follows:

    $$\begin{aligned} \xi _{i}(t)=\min \left[ \max \left[ 0, \frac{q_{i}(t)\mu _{i}^{\max }}{\phi _{i}\ln 2}-\frac{1}{m_{i}(t)}\right] ,1\right] . \end{aligned}$$
    (12)

    By substituting Eq. (12) into Eq. (10), we obtain the solutions \(\mu _{i}(t)\).

  3. 3.

    The Exponential map** function: this general form of this relationship between mood and productivity can be expressed as:

    $$\begin{aligned} f(m_{i}(t))=\frac{1}{e-1}[e^{m_{i}(t)}-1]. \end{aligned}$$
    (13)

    The general shape and the range of possible values of this map** function is shown in Fig. 1(c). It represents a possible situation in which the effect of changes in a person’s mood on his productivity is more pronounced when the mood is high than when the mood is low. Thus, the actual number of tasks that can be completed by worker i under such a level of productivity can be expressed as:

    $$\begin{aligned} \mu (\xi _{i}(t),m_{i}(t))=\left\lfloor \frac{\mu _{i}^{\max }}{e-1}[e^{m_{i}(t)\xi _{i}(t)}-1]\right\rfloor . \end{aligned}$$
    (14)

    In this case, to minimize Eq. (3), we compute its first order derivative with respect to \(\xi _{i}(t)\) for each worker i as:

    $$\begin{aligned}&\frac{d}{d \xi _{i}(t)}\left\{ \phi _{i}\xi _{i}(t)-\frac{q_{i}(t)\mu _{i}^{\max }}{e-1}[e^{m_{i}(t)\xi _{i}(t)}-1]\right\} \nonumber \\&\,\,\,\,=\phi _{i}-\frac{q_{i}(t)\mu _{i}^{\max }m_{i}(t)e^{m_{i}(t)\xi _{i}(t)}}{e-1}. \end{aligned}$$
    (15)

    By equating Eq. (15) to 0, we obtain the effort output \(\xi _{i}(t)\) by each worker i at time slot t which can minimize the objective function Eq. (3) as follows:

    $$\begin{aligned} \xi _{i}(t)=\min \left[ \max \left[ 0, \frac{1}{m_{i}(t)}\ln \left( \frac{\phi _{i}(e-1)}{q_{i}(t)\mu _{i}^{\max }m_{i}(t)}\right) \right] ,1\right] . \end{aligned}$$
    (16)

    By substituting Eq. (16) into Eq. (14), we obtain the solutions \(\mu _{i}(t)\).

figure a

The proposed OWRS approach is summarized by Algorithm 1. It implements computationally the intuition that the more pending tasks in a worker’s backlog queue, the less emphasis placed on conserving effort output and the higher the worker’s current mood, the worker should spend more effort on completing pending tasks during the current time slot (subject to the physical limitations of the worker’s maximum productivity). The recommendations from OWRS are given to workers in the form of the number of tasks he should complete in a given time slot so as to make it actionable for the worker to follow. As OWRS only requires a worker’s local information as inputs, it can be implemented as a distributed algorithm with time complexity of O(1), making it highly scalable.

4 Experimental Evaluation

To evaluate the performance of OWRS under realistic settings, we compare it against three baseline approaches via extensive simulations. To do so, we synthesize a population of crowdsourcing worker agents whose performance characteristics are based on the Tianchi datasetFootnote 1 released by Taobao.com. This real-world dataset allows us to construct realistic scenarios. The simulations enable us to study the performance of OWRS under different situations.

4.1 Experiment Settings

Released by Taobao.com, this real-world dataset contains information regarding the competence and maximum productivity (as measured by the maximum number of tasks that can be completed per time slot) of 5,547 workers. The distributions of productivity and competence values in the dataset are shown in Fig. 2. It can be observed that the majority of the workers are able to process 10 tasks per time slot while a relatively small percentage of them are able to complete up to 100 tasks per time slot. The distribution of workers’ competence roughly follows a Normal distribution with mean value centred around 0.5. This dataset allows us to construct realistic simulated scenarios to facilitate the evaluation of OWRS under different conditions.

Fig. 2.
figure 2

The distribution of ground truth maximum productivity and competence values for 5,547 worker agents in the simulations.

Full size image

The four comparison approaches included in the simulations are:

  1. 1.

    The MaxEffort (ME) approach: under this approach, a worker i always works as long as there are tasks in his backlog queue regardless of their mood. Subject to there being enough tasks in the backlog queue, worker i exerts up to \(\xi _{i}(t)=1\) effort at time slot t.

  2. 2.

    The Mood Threshold (MT) approach: under this approach, a worker i works whenever \(m_{i}(t)\geqslant \theta _{1}\) and there are tasks in his backlog queue. \(\theta _{1}\in [0,1]\) is a predetermined mood threshold used by MT. Subject to there being enough tasks in his backlog queue, worker i exerts up to \(\xi _{i}(t)=1\) effort at time slot t; otherwise, worker i rests.

  3. 3.

    The Mood and Workload threshold (MW) approach: this approach jointly considers a worker i’s current mood and workload to determine how much effort to exert. Whenever \(q_{i}(t)\mu (1,m_{i}(t))\geqslant \mu _{i}^{\max }\mu (1,\theta _{2})\), worker i exerts up to \(\xi _{i}(t)=1\) effort subject to there being enough tasks in his backlog queue; otherwise, worker i rests. \(\theta _{2}\in [0,1]\) is a predetermined mood threshold used by MW.

  4. 4.

    The OWRS approach proposed in this paper.

The task allocation approach used for all comparison approaches is DRAFT [27] which dynamically distributes tasks among workers in a situation-aware manner in order to avoid over concentration of workload. At each time slot, DRAFT takes each worker i’s current competence and workload as inputs to determine how many new tasks to assign to i (i.e. DRAFT determines \(\lambda _{i}(t)\) for all i and t). The principle implemented by DRAFT is that the higher a worker i’s competence and the lower his current workload, the more tasks should be assigned to him. In practice, the DRAFT approach can be replaced by other similar approaches which determine \(\lambda _{i}(t)\).

In the simulations, the value of \(\phi \) is varied between 5 and 100 in increments of 5. The values of \(\theta _{1}\) and \(\theta _{2}\) are varied between 0.1 and 1 in increments of 0.05. As the workload is measured in relation to the collective task throughput of the generated worker agent population, we compute the maximum collective task throughput as \(\Omega =\sum _{i=1}^{N}r_{i}\mu _{i}^{\max }\). In this equation, \(r_{i}\) is a worker agent i’s competence, and \(N=5,547\) in our experiments. We adopt the concept of load factor (LF) from [30] to denote the overall workload placed on the crowdsourcing system. It is computed as the ratio between the number of tasks allocated among the worker agents during time slot t, \(W_{req}(t)\), and the maximum collective productivity \(\Omega \) of the system (i.e. \(LF=\frac{W_{req}(t)}{\Omega }\)). We vary LF between 5% to 100% in 5% increments in the simulations.

In the simulations, the mood for each worker i during time slot t, \(m_{i}(t)\), is randomly generated in the range of [0, 1] following a uniform distribution. This eliminates the possibility for any of the four approaches to predict a worker agent’s future mood, thereby focusing the experimental comparisons on the effectiveness of the scheduling strategies. Under each LF setting, the simulation is run for \(T=1,000\) time slots. Task deadlines are randomized following a uniform distribution. On average, a task must be completed within 3 time slots after it is assigned to a worker agent. The simulations are executed under three mood–productivity map** functions \(M\in \){Step, Logarithmic, Exponential}.

The performances of the four approaches are compared using the following metrics:

  1. 1.

    Time-averaged worker effort output:

    $$\begin{aligned} \bar{\xi }=\frac{1}{TN}\sum _{t=0}^{T-1}\sum _{i=1}^{N}\xi _{i}(t). \end{aligned}$$
    (17)

    The smaller the value of \(\bar{\xi }\), the better the performance.

  2. 2.

    Time-averaged task completion rate:

    $$\begin{aligned} \bar{\mu }=\frac{1}{T}\sum _{t=0}^{T-1}\frac{\sum _{i=1}^{N}\mu _{i}(t)}{L}. \end{aligned}$$
    (18)

    The higher the value of \(\bar{\mu }\), the better the performance.

  3. 3.

    Efficiency:

    $$\begin{aligned} \bar{\epsilon }=\frac{\bar{\mu }}{\bar{\xi }}. \end{aligned}$$
    (19)

    The higher the value of \(\bar{\epsilon }\), the better the performance.

4.2 Results and Analysis

As worker agents adopting the ME approach consistently achieve the highest task completion rate at the expense of not resting whenever there are pending tasks, we use ME as the reference for comparing the performance of the other three approaches under different LF, \(\phi \), \(\theta _{1}\) and \(\theta _{2}\) settings. In sub-figures (a)–(c), (f)–(h) and (k)–(m) of Fig. 4, each data point for the OWRS approach is averaged over all \(\phi \) values; whereas each data point for the MT and MW approaches is averaged over all \(\theta _{1}\) and \(\theta _{2}\) values, respectively. In sub-figures (d), (i) and (n) of Fig. 4, each data point is averaged over all LF values. In sub-figures (e), (j) and (o) of Fig. 4, each data point is averaged over all experimental settings. As the experiments are simulations, exact values are less important than relative performance.

Fig. 3.
figure 3

The average effort output by worker agents experiencing different mood and workload conditions following each of the four comparison approaches under LF = 50%, \(\theta _{1}=0.5\), \(\theta _{2}=0.5\) and \(\phi =50\). (a)–(d) Step mood–productivity map**; (e)–(h) Logarithmic mood–productivity map**; and (i)–(l) Exponential mood–productivity map**.

Full size image

Figure 3 contains snapshots of the average effort output by worker agents experiencing different mood and workload conditions following each of the four comparison approaches under different mood–productivity map** functions. The snapshots are taken under the settings \(\theta _{1}=0.5\), \(\theta _{2}=0.5\) and \(\phi =50\) with LF = 50%. The personal workload values in the x-axis are computed as \(\frac{q_{i}(t)}{\mu _{i}^{\max }}\) for each worker agent i during each time slot t in the simulations. Under LF = 50%, DRAFT ensures that the workload in any worker agent’s pending task queue never exceeds \(200\%\times \mu _{i}^{\max }\).

Under ME (Figs. 3(a), (e) and (i)), worker agents always exert enough effort to complete as many tasks as allowed by their respective \(\mu _{i}^{\max }\) limits. Since higher mood values results in higher productivity in general, worker agents under ME can spend less effort when their mood is high and workload is below 100%. Worker agents under MT follows the same effort output patterns as those under ME except for when their mood drops below the \(\theta _{1}\) which is 0.5 in this case study (Figs. 3(b), (f) and (j)). When \(m_{i}(t)<\theta _{1}\), MT worker agents rest. Under MW (Figs. 3(c), (g) and (k)), worker agents jointly consider if their mood and workload pass a combined threshold before determining how much effort to exert. When the worker agents experience either high mood, high workload or both, they exert as much effort as required by the situation. For the remaining conditions, MW worker agents rest. Under OWRS (Figs. 3(d), (h) and (l)), worker agents also jointly consider their mood and workload when making effort output decisions. However, as OWRS is not a threshold-based approaches, worker agents do not rest completely. Instead, even when worker agents are experiencing low mood or low workload, OWRS worker agents still exert effort, albeit at lower levels. This allows OWRS worker agents to achieve comparatively higher task completion rates.

Fig. 4.
figure 4

Experiment results: (a)–(e) Step map**; (f)–(j) Logarithmic map**; and (k)–(o) Exponential map**.

Full size image

Figures 4(a)–(e) show the simulation results under the step mood–productivity map** function. The time-averaged effort output \(\bar{\xi }\) achieved by all four approaches are shown in Fig. 4(a). Compared to ME, MT, MW and OWRS achieve significant saving in effort as LF increases. This is partially due to the DRAFT task allocation approach used in the simulations. When LF is low, tasks are mostly concentrated on worker agents with high competence. In this case, the task backlog on individual workers who have been allocated tasks tend to be relatively high, which makes scheduling approaches allocate less time for these workers to rest in order to meet task deadlines. As LF increases and the workload is spread more evenly among a larger segment of the worker agent population, there are more opportunities for scheduling approaches to slot in opportunistic rest sessions. The \(\bar{\xi }\) values achieved by MT, MW and OWRS stabilize around 40% with MW recommending the lowest worker effort output and OWRS recommending the highest. The time-averaged task completion rate \(\bar{\mu }\) achieved by all four approaches are shown in Fig. 4(b). It can be observed that OWRS achieves the second highest \(\bar{\mu }\) values which stabilize around 70% and are 27% and 40% higher than MT and MW, respectively. Figure 4(c) shows the performance of all four approaches in terms of efficiency. The efficiency of ME is constant as is does not attempt to conserve worker agents’ effort. OWRS achieves the highest efficiency which stabilizes at around 50% higher than that achieved by ME as it makes the smallest sacrifice in time-averaged task completion rate to achieve significant saving in worker effort. The efficiency values achieved by MT and MW under the step mood–productivity map** function are roughly the same. OWRS achieves over 20% higher efficiency compared to MT and MW.

Figure 4(d) shows the performance landscape of MT, MW and OWRS as a percentage of ME under different control parameter settings. By varying the \(\phi \) value in OWRS, we can achieve different trade-offs between worker effort conservation and task completion rate. The performance of OWRS varies between spending 78% of the ME effort output and achieving 99% of the ME task completion rate, to spending 20% of the ME effort output and achieving 54% of the ME task completion rate. Under MT and MW, the performance varies between spending 90% of the ME effort output and achieving 99% of the ME task completion rate, and spending 0% of the ME effort output and achieving 0% of the ME task completion rate. OWRS consistently and significantly outperforms both MT and MW, conserving significant worker effort while achieving high task completion rates. Overall (Fig. 4(e)), OWRS achieves superlinear collective productivity, spending 44% of effort to complete 73% of the tasks on average. Whereas, ME, MT and MW are only able achieve linear collective productivity.

Figures 4(f)–(j) and (k)–(o) show the simulation results under the logarithmic and exponential mood–productivity map** functions, respectively. The overall trend of relative performance of the four approaches are similar to those under the step mood–productivity map** function. The overall efficiency achieved by OWRS under these two map** functions are slightly lower than that under the step map** function. Nevertheless, OWRS is able to handle the changes in the map** function to achieves superlinear collective productivity and outperform other approaches significantly.

In the cases in which \(\theta _{1}\) and \(\theta _{2}\) are set to 1, indicating that workers are unwilling to work under any mood condition, the worker effort output and the task completion rates achieved by both MT and MW are 0 under all experimental settings. This is expected as in MT and MW, mood is directly used to control effort output. However, under OWRS, mood becomes one of the situational factors considered by the approach. By dynamically minimizing the joint objective function (Eq. 3), even when \(\phi \) is set to 100 indicating that workers place very high emphasis on effort conservation in general, OWRS is able to maintain a long-term average effort output of between 20% to 30% taking advantage of favourable working conditions whenever possible to achieve average task completion rates of more than 50% under all three mood–productivity map** functions investigated. By making workers sacrifice the completion of some tasks under situations which do not support high productivity, OWRS achieves significant savings in worker effort and superlinear collective productivity in view of the reduced total effort output, putting the Parrondo’s paradox [7, 22] into practice on a large scale.

5 Conclusions and Future Work

In this paper, we extended the AC framework to propose OWRS. It takes into account information on a worker’s mood, current workload and desire to rest to produce dynamic work-rest schedules. By leveraging on favourable situations to work and rest when the situations do not support high productivity, OWRS minimizes crowdsourcing workers’ effort output while maximizing collective productivity. Compared to AC, OWRS is able to handle more diverse mood–productivity map** functions including step, logarithmic and exponential forms. As it is a distributed approach, the time complexity of OWRS is O(1). It can be implemented as a personal scheduling agent to advise each worker on when (and how much) to work and when to rest in order to balance personal wellbeing with collective productivity goals in an optimized manner. Simulations based on a large-scale real-world dataset demonstrate that OWRS outperforms three baseline scheduling approaches. It significantly conserves the collective effort output while making the smallest sacrifice in terms of task completion rates, thereby achieving superlinear collective productivity. OWRS establishes a framework which accommodates workers’ heterogeneity to enhance their experience and productivity, enabling AI-power algorithmic management to collaborate with workers in a more human-centric way.

In subsequent research, we will combine OWRS with explainable AI techniques [6, 12] to provide stronger persuasion in order to enhance worker compliance to the recommended schedules.