Abstract
Online crowdsourcing platforms have rapidly become a popular source of data collection. Despite the various advantages these platforms offer, there are substantial concerns regarding not only data validity issues, but also the ethical, societal, and global ramifications arising from the prevalent use of online crowdsourcing platforms. This paper seeks to expand the dialogue by examining both the “internal” aspects of crowdsourcing research practices, such as data quality issues, reporting transparency, and fair compensation, and the “external” aspects, in terms of how the widespread use of crowdsourcing data collection shapes the nature of scientific communities and our society in general. Online participants in research studies are informal workers who provide labor in exchange for remuneration. The paper thus highlights the need for researchers to consider the markedly different political, economic, and socio-cultural characteristics of the Global North and the Global South when undertaking crowdsourcing research involving an international sample; such consideration is crucial for both increasing research validity and mitigating societal inequities. We encourage researchers to scrutinize the value systems underlying this popular data collection research method and its associated ethical, societal, and global ramifications, as well as provide a set of recommendations regarding the use of crowdsourcing platforms.
Similar content being viewed by others
Change history
17 February 2024
The original version of this article was revised: Modifications have been made to the author names and affiliations of Mayowa T. Babalola, Edina Dóci and Alexander Newman. Full information regarding the corrections made can be found in the correction for this article.
28 February 2024
A Correction to this paper has been published: https://doi.org/10.1007/s10551-024-05636-9
Notes
In the late twentieth century, the terminology ‘Global North’ and ‘Global South’ replaced previous descriptors of the global order such as East-West, developed-develo** nations, First-Third World, core-periphery, and modern-traditional societies. Global North and Global South do not refer to a geographic region in any conventional sense, but rather to the relative power, privilege, wealth, and development of countries in distinct parts of the world. The Global North encompasses the rich, powerful, and developed regions of the world such as North America, Europe, and Australia. It includes countries such as the United States, Canada, the United Kingdom, nations of the European Union, Australia, New Zealand, Singapore, Japan, and South Korea. The Global North countries have mature economies and stable polities and are technologically advanced with low population growth rates and high quality of life metrics. The Global North has roughly 25% of the world’s population, but earns 80% of the world’s wealth and tends to dominate the Global South both politically and economically. The Global South encompasses the poor, less powerful, and less developed countries of the world in areas such as South America, Africa, and Asia, including formerly colonized countries. Many of these countries are still marked by the social, cultural, and economic repercussions of colonialism, even after achieving national independence. The Global South remains home to the majority of the world’s population, but that population is relatively young and resource-poor, living in economically dependent nations which have unstable democracies and are marked by large inequalities in living standards and access to resources as well as low per capita income and excessive unemployment (Braff and Nelson, 2021; Dados and Connell, 2012; World Population Review, 2023a and b). Both the Global North and Global South encompass internal variations (Braff and Nelson, 2021).
Online platform workers, also known as ‘crowdworkers’, operate via online platforms that digitally connect workers, clients, organizations, and businesses across borders spread over large geographic distances (e.g., Amazon Mechanical Turk, Upwork, Innocentive). Offline platform workers engage in ‘work on-demand via apps’, undertaking place-based and geographically-limited work facilitated by platforms through their applications but requiring direct interface between workers, customers, clients, organizations, and businesses (e.g., Uber, Swiggy, Urban Company) (de Stefano, 2015).
For example, Berg et al. (2018) found that across various platforms, including MTurk, Prolific, and Clickworker, the average hourly wage for online workers in North America (US$4.70 per hour) and Europe and Central Asia (US$3.00 per hour; Central Asia includes Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, and Uzbekistan) is higher than that for online workers in other regions of the world, where the average hourly wage varied between US$1.33 (Africa) and US$2.22 (Asia and the Pacific; Asia includes Bangladesh, Brunei, China, India, Indonesia, Pakistan, Philippines, and Vietnam).
References
Abrahamson, S., Ryder, P., & Unterberg, B. (2013). Crowdstorm: The future of innovation, ideas, and problem solving. Wiley.
Aguinis, H., Cummings, C., Ramani, R. S., & Cummings, T. G. (2020a). “An A Is An A”: The new bottom line for valuing academic research. Academy of Management Perspectives, 34(1), 135–154. https://doi.org/10.5465/amp.2017.0193
Aguinis, H., Villamor, I., & Ramani, R. S. (2020b). MTurk research: Review and recommendations. Journal of Management. https://doi.org/10.1177/0149206320969787
Anwar, M. A., & Graham, M. (2021). Between a rock and a hard place: Freedom, flexibility, precarity and vulnerability in the gig economy in Africa. Competition & Change, 25(2), 237–258.
Barnes, S. A., Green, A., & de Hoyos, M. (2015). Crowdsourcing and work: Individual factors and circumstances influencing employability. New Technology, Work and Employment, 30(1), 16–31. https://doi.org/10.1111/ntwe.12043
Beerepoot, N., & Lambregts, B. (2015). Competition in online job marketplaces: Towards a global labour market for outsourcing services? Global Networks, 15(2), 236–255.
Berg, J., & de Stefano, V. (2018). Employment and regulation for clickworkers. In Work in the digital age (pp. 175–184). Rowman & Littlefield International.
Berg, J., Furrer, M., Harmon, E., Rani, U., & Silberman, M. S. (2018). Digital labour platforms and the future of work: Towards decent work in the online world. International Labour Office.
Braff, L., & Nelson, K. (2022). The Global North: Introducing the region. In N. T. Fernandez and K. Nelson (Eds.), Gendered Lives: Global Issues. Accessed 2 February 2023 from https://milnepublishing.geneseo.edu/genderedlives/chapter/chapter-15-the-global-north-introducing-the-region/
Bridgland, V. M. E., Barnard, J. F., & Takarangi, M. K. T. (2022). Unprepared: Thinking of a trigger warning does not prompt preparation for trauma- related content. Journal of Behavior Therapy and Experimental Psychiatry, 75, 101708.
Brink, W. D., Eaton, T. V., Grenier, J. H., & Reffett, A. (2019). Deterring unethical behavior in online labor markets. Journal of Business Ethics, 156, 71–88.
Buhrmester, M. D., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5. https://doi.org/10.1177/1745691610393980
Buhrmester, M. D., Talaifar, S., & Gosling, S. D. (2018). An evaluation of Amazon’s Mechanical Turk, its rapid rise, and its effective use. Perspectives on Psychological Science, 13(2), 149–154. https://doi.org/10.1177/1745691617706516
Casilli, A. A. (2017). Global digital culture | Digital labor studies go global: Toward a digital decolonial turn. International Journal of Communication, 11, 21.
Chandler, J., Mueller, P., & Paolacci, G. (2014). Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavioral Research Methods, 46(1), 112–130. https://doi.org/10.3758/s13428-013-0365-7
Cheung, J. H., Burns, D. K., Sinclair, R. R., & Sliter, M. (2017). Amazon Mechanical Turk in organizational psychology: An evaluation and practical recommendations. Journal of Business and Psychology, 32(4), 347–361. https://doi.org/10.1007/s10869-016-9458-5
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS ONE. https://doi.org/10.1371/journal.pone.0057410
Dados, N., & Connell, R. (2012). The Global South. Contexts, 11(1), 12–13.
D’Cruz, P. (2017). Partially empowering but not decent? The contradictions of online labour markets. In E. Noronha & P. D’Cruz (Eds.), Critical perspectives on work and employment in globalizing India (pp. 173–195). Springer.
D’Cruz, P., & Noronha, E. (2016). Positives outweighing negatives: The experiences of Indian crowdsourced workers. Work Organisation, Labour and Globalisation, 10(1), 44–63.
D’Cruz, P., & Noronha, E. (2018a). Abuse on online labour markets: Targets’ co**, power and control. Qualitative Research in Organizations and Management: An International Journal, 13(1), 53–78.
D’Cruz, P., & Noronha, E. (2018b). Target experiences of workplace bullying on online labour markets: Uncovering the nuances of resilience. Employee Relations, 40(1), 139–154.
D’Cruz, P., & Noronha, E. (2023). India’s platform economy experience: A site for the commodification–decommodification dynamic. In I. Ness (Ed.), Platform Labour and Global Logistics: A Research Companion. Routledge.
D’Cruz, P., Noronha, E., Banday, M. U. L., & Chakraborty, S. (2022a). Place matters:(Dis) embeddedness and child labourers’ experiences of depersonalized bullying in Indian Bt cottonseed global production networks. Journal of Business Ethics, 176(2), 241–263.
D’Cruz, P., Noronha, E., & Katiyar, S. (2022b). Meaningfulness and impact of academic research: Bringing the global south to the forefront. Business & Society, 61(4), 839–844.
de Stefano, V. M. (2015). The rise of the “just-in-time” workforce. Paper presented at the 4th ILO Conference on Regulating for Decent Work, Geneva, July 8–10, 2015.
Ettlinger, N. (2017). Paradoxes, problems and potentialities of online work platforms. Work Organisation, Labour and Globalisation, 11(2), 21–38.
Fairwork. (2022). Fairwork Annual Report 2022. Oxford, Berlin.
Financial Time. (2016). The humans behind Mechanical Turk’s artificial intelligence. Retrieved June 3, 2023 from https://www.ft.com/content/17518034-6f77-11e6-9ac1-1055824ca907
Follmer, D. J., Sperling, R. A., & Suen, H. K. (2017). The role of MTurk in education research: Advantages, issues, and future directions. SAGE: Educational Researcher.
Gastrow, M., & Oppelt, T. (2018). Big science and human development-what is the connection? South African Journal of Science, 114(11–12), 1–7.
Gleibs, I. H. (2017). Are all “research fields” equal? Rethinking practice for the use of data from crowdsourcing market places. Behavior Research Methods, 49(4), 1333–1342.
Goodman, J. K., & Paolacci, G. (2017). Crowdsourcing consumer research. Journal of Consumer Research, 44(1), 196–210.
Haidar, J., & Keune, M. (2021). Introduction: Work and labour relations in global platform capitalism. In Work and Labour Relations in Global Platform Capitalism (pp. 1–27). ILO & Edward Elgar.
Hammett, D., & Sporton, D. (2012). Paying for interviews? Negotiating ethics, power and expectation. Area, 44(4), 496–502.
Hauser, D. J., & Schwarz, N. (2016). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1), 400–407. https://doi.org/10.3758/s13428-015-0578-z
Hauser, D., Paolacci, G., & Chandler, J. J. (2020). Common concerns with MTurk as a participant pool: Evidence and solutions. In F. R. Kardes, P. M. Herr, & N. Schwarz (Eds.), Handbook of research methods in consumer psychology. Routledge.
Holtgrewe, U. (2014). New new technologies: The future and the present of work in information and communication technology. New Technology, Work and Employment, 29(1), 9–24.
Horton, J. J., Rand, D. G., Zeckhauser, R. J., Chandler, D., Dreber, A., Haque, R., & Zeckhauser, R. (2011). The online laboratory: Conducting experiments in a real labor market. Springer, 14(3), 399–425. https://doi.org/10.1007/s10683-011-9273-9
Hunt, N. C., & Scheetz, A. M. (2019). Using MTurk to distribute a survey or experiment: Methodological considerations. Journal of Information Systems, 33(1), 43–65. https://doi.org/10.2308/isys-52021
Huws, U., Spencer, N., Syrdal, D. S., & Holts, K. (2018). Working in the gig economy: insights from Europe. In Work in the digital age (pp. 153–162). Rowman & Littlefield International.
Islam, G., & Greenwood, M. (2023). Ethical research in business ethics. Journal of Business Ethics, 182, 1–5.
Islam, G., & Greenwood, M. (2021). Reconnecting to the social in business ethics. Journal of Business Ethics, 170, 1–4.
Jammulamadaka, N. P. (2020). Author (ing) from post-colonial context: Challenges and jugaad fixes. Qualitative Research in Organizations and Management: An International Journal., 15(3), 388–401. https://doi.org/10.1108/QROM-07-2019-1791
Johnston, H., & Land-Kazlauskas, C. (2018). Organizing on-demand: Representation, voice, and collective bargaining in the gig economy.
Kantor, P., Rani, U., & Unni, J. (2006). Decent work deficits in informal economy: Case of Surat. Economic and Political Weekly, 2089–2097.
Kees, J., Berry, C., Burton, S., & Sheehan, K. (2017). An Analysis of Data Quality: Professional Panels, Student Subject Pools, and Amazon’s Mechanical Turk. Journal of Advertising, 46(1), 141–155. https://doi.org/10.1080/00913367.2016.1269304
Keith, M. G., Harms, P., & Tay, L. (2019). Mechanical Turk and the gig economy: Exploring differences between gig workers. Journal of Managerial Psychology, 34(4), 286–306.
Keith, M. G., Tay, L., & Harms, P. D. (2017). Systems perspective of amazon mechanical turk for organizational research: Review and recommendations. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2017.01359
Landers, R. N., & Behrend, T. S. (2015). An inconvenient truth: Arbitrary distinctions between organizational, Mechanical Turk, and other convenience samples. Industrial and Organizational Psychology, 8(2), 142–164.
Leung, W.-F., D’Cruz, P. and Noronha, E. (2021) ‘Freelancing globally: upworkers in China and India, Neo-liberalisation and the New International Putting-out System of Labour (NIPL)’. In Work and Labour Relations in Global Platform Capitalism, edited by J. Haidar and M. Keune. ILO and Edward Elgar.
Levay, K. E., Freese, J., & Druckman, J. N. (2016). The demographic and political composition of Mechanical Turk samples. SAGE Open, 6(1), 1–17. https://doi.org/10.1177/2158244016636433
Linder, C., & Farahbakhsh, S. (2020). Unfolding the black box of questionable research practices: Where is the line between acceptable and unacceptable practices? Business Ethics Quarterly, 30(3), 335–360.
Lombe, M., Newransky, C., Crea, T., & Stout, A. (2013). From rhetoric to reality: Planning and conducting collaborations for international research in the global south. Social Work, 58(1), 31–40.
MacInnis, C. C., Boss, H. C. D., & Bourdage, J. S. (2020). More evidence of participant misrepresentation on MTurk and investigating who misrepresents. Personality and Individual Differences, 152, 109603. https://doi.org/10.1016/j.paid.2019.109603
Mawdsley, H., Amaugo, A., & Thirlwall, A. (2018). Lost in translation? Testing the NAQ-R in an African context. In 11th International conference on workplace bullying and harassment, 6–8 June, Bordeaux, France.
Mawdsley, H., Thirlwall, A., Amarachi, A., D’Cruz, P., Noronha, E., Essiaw, M. N., & Chimere, E. (2021). Cultural influences on the interpretation of Workplace Bullying: A four country study. In 12th International conference on workplace bullying and harassment, 12–14 April, Online via Dubai.
McCredie, M. N., & Morey, L. C. (2019). Who are the Turkers? A characterization of MTurk workers using the personality assessment inventory. Assessment, 26(5), 759–766.
Miron-Spektor, E., Ingram, A., Keller, J., Smith, W. K., & Lewis, M. W. (2018). Microfoundations of organizational paradox: The problem is how we think about the problem. Academy of Management Journal, 61(1), 26–45.
Newman, A., Bavik, Y. L., Mount, M., & Shao, B. (2021). Data collection via online platforms: Challenges and recommendations for future research. Applied Psychology, 70(3), 1380–1402.
Ophir, Y., Sisso, I., Asterhan, C. S. C., Tikochinski, R., & Reichart, R. (2020). The Turker Blues: Hidden factors behind increased depression rates among Amazon’s Mechanical Turkers. Clinical Psychological Science, 8(1), 65–83. https://doi.org/10.1177/2167702619865973
Palan, S., & Schitter, C. (2018). Prolific.ac—A subject pool for online experiments. Journal of Behavioral and Experimental Finance, 17, 22–27. https://doi.org/10.1016/j.jbef.2017.12.004
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017). Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70, 153–163. https://doi.org/10.1016/j.jesp.2017.01.006
Peer, E., Vosgerau, J., & Acquisti, A. (2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods, 46(4), 1023–1031. https://doi.org/10.3758/s13428-013-0434-y
Pittman, M., & Sheehan, K. (2016). Amazon’s Mechanical Turk a digital sweatshop? Transparency and accountability in crowdsourced online research. Journal of Media Ethics, 31(4), 260–262.
Porter, C. O. L. H., Outlaw, R., Gale, J. P., & Cho, T. S. (2019). The use of online panel data in management research: A review and recommendations. Journal of Management, 45(1), 319–344. https://doi.org/10.1177/0149206318811569
Robinson, J., Rosenzweig, C., Moss, A. J., & Litman, L. (2019). Tapped out or barely tapped? Recommendations for how to harness the vast and largely unused potential of the Mechanical Turk participant pool. PLoS ONE, 14(12), e0226394.
Rosioru, F., & Kiss, G. (2013). The changing concept of subordination. Recent Developments in a Labour Law, 1, 150–185.
Sears, D. O. (1986). College sophomores in the laboratory: Influences of a narrow data base on social psychology’s view of human nature. Journal of Personality and Social Psychology, 51, 515–530.
Shank, D. B. (2016). Using crowdsourcing websites for sociological research: The case of Amazon Mechanical Turk. The American Sociologist, 47, 47–55.
Srnicek, N. (2017). Platform capitalism. John Wiley & Sons.
State of Working India. (2021). State of Working India Report. Retrieved February 5, 2023 from https://cse.azimpremjiuniversity.edu.in/wp-content/uploads/2021/08/SWI2021_August_WEB.pdf
Stone, A. A., Walentynowicz, M., Schneider, S., Junghaenel, D. U., & Wen, C. K. (2019). MTurk participants have substantially lower evaluative subjective wellbeing than other survey participants. Computers in Human Behavior, 94, 1–8.
StrideOne. (2021). Start-up economy report 2022. Retrieved February 6, 2023 from https://origin.strideone.in/wp-content/uploads/2022/12/Startup-Economy-Report-2022-by-StrideOne.pdf
Visser, M., & Arnold, T. C. (2022). Recognition and work in the platform economy: A normative reconstruction. Philosophy of Management, 21(1), 31–45.
Walter, S. L., Seibert, S. E., Goering, D., & O’Boyle, E. H. (2019). A tale of two sample sources: Do results from online panel data and conventional data converge? Journal of Business and Psychology, 34(4), 425–452. https://doi.org/10.1007/s10869-018-9552-y
Wessling, K. S., Huber, J., & Netzer, O. (2017). MTurk character misrepresentation: Assessment and solutions. Journal of Consumer Research, 44(1), 211–230.
Williams, B. (1985). Ethics and the limits of philosophy. Harvard University Press.
Williams, C. C. (2017). Reclassifying economies by the degree and intensity of informalization: The implications for India. Critical perspectives on work and employment in globalizing India (pp. 113–129). Springer.
Williamson, V. (2016). On the ethics of crowdsourced research. PS: Political Science & Politics, 49(1), 77–81.
Wood, A. J., Graham, M., Lehdonvirta, V., & Hjorth, I. (2019). Networked but commodified: The (dis) embeddedness of digital labour in the gig economy. Sociology, 53(5), 931–950.
World Population Review. (2023a). The Global North. Retrieved February 4, 2023 from https://worldpopulationreview.com/country-rankings/global-north-countries
World Population Review. (2023b). The Global South. Retrieved February 4, 2023 fromhttps://worldpopulationreview.com/country-rankings/global-south-countries
Young, I. M. (2003). Political responsibility and structural injustice. University of Kansas.
Zhou, H., & Fishbach, A. (2016). The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology, 111(4), 493.
Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontier of power. Profile Books.
Zyphur, M. J., & Pierides, D. C. (2017). Is quantitative research ethical? Tools for ethically practicing, evaluating, and using quantitative research. Journal of Business Ethics, 143, 1.
Acknowledgement
We thank Tracy Wilcox for her co-ordination role during the initial phase of this article.
Funding
No funding was received for this paper.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interest.
Ethical Approval
This paper does not involve human participants.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Editors at the Journal of Business Ethics are blinded from decisions on manuscripts on which they are listed as authors. Such manuscripts are handled by an independent editor at the journal.
The original version of this article was revised: Modifications have been made to the author names and affiliations of Mayowa T. Babalola, Edina Dóci and Alexander Newman. Full information regarding the corrections made can be found in the correction for this article.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Du, S., Babalola, M.T., D’Cruz, P. et al. The Ethical, Societal, and Global Implications of Crowdsourcing Research. J Bus Ethics (2024). https://doi.org/10.1007/s10551-023-05604-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10551-023-05604-9