SpringerLink
Log in

Advances in automatically rating the trustworthiness of text processing services

  • Original Research
  • Published:
AI and Ethics Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

AI services are known to have unstable behavior when subjected to changes in data, models or users. Such behaviors, whether triggered by omission or commission, lead to trust issues when AI works with humans. The current approach of assessing AI services in a black-box setting, where the consumer does not have access to the AI’s source code or training data, is limited. The consumer has to rely on the AI developer’s documentation and trust that the system has been built as stated. Further, if the AI consumer reuses the service to build other services which they sell to their customers, the consumer is at the risk of the service providers (both data and model providers). Our approach, in this context, is inspired by the success of nutritional labeling in food industry to promote health and seeks to assess and rate AI services for trust from the perspective of an independent stakeholder. The ratings become a means to communicate the behavior of AI systems, so that the consumer is informed about the risks and can make an informed decision. In this paper, we will first describe recent progress in develo** rating methods for text-based machine translator AI services that have been found promising with user studies. Then, we will outline challenges and vision for a principled, multimodal, causality-based rating methodologies and its implication for decision-support in real-world scenarios like health and food recommendation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

Data and further details of experiments are available at our project page - https://ai4society.github.io/projects/rating_page/index.html.

Notes

  1. At http://opennmt.net/

  2. At: https://github.com/joelescudefont/genbiasmt

References

  1. Agarwal, V., Chafle, G., Mittal, S., Srivastava, B.: Understanding Approaches for Web Service Composition and Execution. In Proceedings of the 1st Bangalore Annual Compute Conference, COMPUTE ’08. New York, NY, USA: Association for Computing Machinery. ISBN 9781595939500 (2008)

  2. Antun, V., Renna, F., Poon, C., Adcock, B., Hansen, A.C.: On instabilities of deep learning in image reconstruction and the potential costs of AI. Proceedings of the National Academy of Sciences 117(48), 30088–30095 (2020)

    Article  Google Scholar 

  3. Bagdasaryan, E., Shmatikov, V.: Spinning Language Models for Propaganda-As-A-Service. CoRR, abs/2112.05224 (2021)

  4. Bernagozzi, M., Srivastava, B., Rossi, F., Usmani, S.: Gender Bias in Online Language Translators: Visualization, Human Perception, and Bias/Accuracy Trade-offs. In In IEEE Internet Computing, Special Issue on Sociotechnical Perspectives, Nov/Dec (2021)

  5. Bernagozzi, M., Srivastava, B., Rossi, F., Usmani, S.: VEGA: a Virtual Environment for Exploring Gender Bias vs. Accuracy Trade-offs in AI Translation Services. Proceedings of the AAAI Conference on Artificial Intelligence, 35(18): 15994–15996 (2021b)

  6. Blodgett, S. L., Barocas, S., au2, H. D. I., Wallach, H.: Language (Technology) is Power: A Critical Survey of "Bias" in NLP. In On Arxiv at: 2 (2020) https://arxiv.org/abs/2005.14050

  7. Brandon, E.: Why older citizens are more likely to vote. In (2020) https://money.usnews.com/money/retirement/aging/articles/why-older-citizens-are-more-likely-to-vote

  8. Chatelan, A., Bochud, M., Frohlich, K.L.: Precision nutrition: hype or hope for public health interventions to reduce obesity? Int. J. Epidemiol. 48(2), 332–342 (2019)

    Article  Google Scholar 

  9. Dashtipour, K., Poria, S., Hussain, A., Cambria, E., Hawalah, A.Y.A., Gelbukh, A., Zhou, Q.: Multilingual sentiment analysis: state of the art and independent comparison of techniques. Cognit. Comput. 8, 757–771 (2016). https://doi.org/10.1007/s12559-016-9415-7

    Article  Google Scholar 

  10. Davidson, T., Warmsley, D., Macy, M., Weber, I.: Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of the 11th International AAAI Conference on Web and Social Media, ICWSM ’17 (2017)

  11. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, 248–255 (2009)

  12. Dua, D., Graff, C.: UCI Machine Learning Repository (2017)

  13. FDA. Food Labeling & Nutrition. In https://www.fda.gov/food/food-labeling-nutrition (2022)

  14. Feder, A., Keith, K. A., Manzoor, E., Pryzant, R., Sridhar, D., Wood-Doughty, Z., Eisenstein, J., Grimmer, J., Reichart, R., Roberts, M. E., Stewart, B. M., Veitch, V., Yang, D.: Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond. CoRR, abs/2109.00725 (2021)

  15. Font, J. E., Costa-jussà, M. R.: Equalizing Gender Biases in Neural Machine Translation with Word Embeddings Techniques. CoRR, abs/1901.03116 (2019)

  16. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., Crawford, K.: Datasheets for Datasets. In On Arxiv at: http://arxiv.org/abs/1803.09010. Cite arxiv:1803.09010Comment: Working Paper, comments are encouraged (2018)

  17. Goodfellow, I., Shlens, J., Szegedy, C.: Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations (2015)

  18. Gu, T., Liu, K., Dolan-Gavitt, B., Garg, S.: BadNets: evaluating backdooring attacks on deep neural networks. IEEE Access 7, 47230–47244 (2019)

    Article  Google Scholar 

  19. Guess, A., Nagler, J., Tucker, J.: Less than you think: Prevalence and predictors of fake news dissemination on Facebook. Sci. Adv. 5(1): eaau4586 (2019)

  20. Henderson, P., Sinha, K., Angelard-Gontier, N., Ke, N. R., Fried, G., Lowe, R., Pineau, J.: Ethical Challenges in Data-Driven Dialogue Systems. CoRR, abs/1711.09050 (2017)

  21. Hoffman, R. R., Mueller, S. T., Klein, G., Litman, J.: Metrics for Explainable AI: Challenges and Prospects. In On Arxiv at https://arxiv.org/abs/1812.04608 (2018)

  22. Hutto, C. J., Folds, D. J., Appling, S.: Computationally Detecting and Quantifying the Degree of Bias in Sentence-Level Text of News Stories. In The First International Conference on Human and Social Analytics (HUSO) (2015)

  23. Jiang, S., Min, W.: Food Computing for Multimedia. In Proceedings of the 28th ACM International Conference on Multimedia, 4782–4784 (2020)

  24. Joshi, H. C., Yadav, U., Srivastava, B., Singh, R. M.: Learning About People’s Attitude Towards Food Available in India and Its Implications for Fair AI-based Systems. In ICDM Workshop on AI for Nudging and Personalization (WAIN) (2022)

  25. Kiritchenko, S., Mohammad, S.: Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, 43–53. New Orleans, Louisiana: Association for Computational Linguistics (2018)

  26. Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J.R., Jurafsky, D., Goel, S.: Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences 117(14), 7684–7689 (2020)

    Article  Google Scholar 

  27. Lakkaraju, K., Srivastava, B., Valtorta, M.: Rating Sentiment Analysis Systems for Bias through a Causal Lens. In Under review (2023)

  28. Liao, Q. V., Srivastava, B., Kapanipathi, P.: A Measure for Dialog Complexity and its Application in Streamlining Service Operations. CoRR, abs/1708.04134 (2017)

  29. Mann, C.B.: Can conversing with a computer increase turnout? Mobilization using chatbot communication. J. Exper. Polit. Sci. 8(1), 51–62 (2021)

    Article  Google Scholar 

  30. Mao, C., Cha, A., Gupta, A., Wang, H., Yang, J., Vondrick, C.: Generative interventions for causal learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3947–3956 (2021)

  31. Min, W., Jiang, S., Liu, L., Rui, Y., Jain, R.: A survey on food computing. ACM Computing Surveys (CSUR) 52(5), 1–36 (2019)

    Article  Google Scholar 

  32. Muppasani, B., Pallagani, V., Lakkaraju, K., Lei, S., Srivastava, B., Robertson, B., Hickerson, A., Narayanan, V.: On Safe and Usable Chatbots for Promoting Voter Participation (2022)

  33. Narayanan, V., Robertson, B. W., Hickerson, A., Srivastava, B., Smith, B. W.: Securing social media for seniors from information attacks: Modeling, detecting, intervening, and communicating risks. In The Third IEEE International Conference on Cognitive Machine Intelligence (IEEE CogMI) (2021)

  34. Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M.-E., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., Kompatsiaris, I., Kinder-Kurlanda, K., Wagner, C., Karimi, F., Fernandez, M., Alani, H., Berendt, B., Kruegel, T., Heinze, C., Broelemann, K., Kasneci, G., Tiropanis, T., Staab, S.: Bias in Data-driven AI Systems – An Introductory Survey. In On Arxiv at: https://arxiv.org/abs/2001.09762 (2020)

  35. Oasis. Universal Description, Discovery and Integration v3.0.2 (UDDI). In https://www.oasis-open.org/specs/index.php#uddiv3.0.2; Last accessed 19 Jan 2023 (2023)

  36. Pallagani, V., Ramamurthy, P., Khandelwal, V., Venkataramanan, R., Lakkaraju, K., Aakur, S. N., Srivastava, B.: A Rich Recipe Representation as Plan to Support Expressive Multi Modal Queries on Recipe Content and Preparation Process (2022)

  37. Patki, N., Wedge, R., Veeramachaneni, K.: The Synthetic Data Vault. In 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), 399–410 (2016)

  38. Pearl, J.: Causality. Cambridge University Press, Cambridge (2009)

    Book  Google Scholar 

  39. PewResearchCenter. An examination of the 2016 electorate, based on validated voters. In https://www.pewresearch.org/politics/2018/08/09/an-examination-of-the-2016-electorate-based-on-validated-voters/ (2018)

  40. Prates, M. O. R., Avelar, P. H. C., Lamb, L. C.: Assessing Gender Bias in Machine Translation - A Case Study with Google Translate. CoRR, abs/1809.02208 (2018)

  41. PW.: Programmable Web. In https://www.programmableweb.com/apis; Last accessed 19 Jan 2023 (2023)

  42. Ribeiro, M. T., Singh, S., Guestrin, C.: Semantically Equivalent Adversarial Rules for Debugging NLP models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 856–865. Association for Computational Linguistics (2018)

  43. Seedat, F., Taylor-Phillip, S.: UK guidance on evaluating AI for use in breast screening. In https://nationalscreening.blog.gov.uk/2022/08/01/guidance-on-evaluating-ai-for-use-in-breast-screening/ (2022)

  44. Srinivasan, R., Chander, A.: Biases in AI Systems. In Communications of the ACM, August 2021, Vol. 64 No. 8, Pages 44-49, https://doi.org/10.1145/3464903 (2021)

  45. Srivastava, B.: Did chatbots miss their “Apollo Moment’’? Potential, gaps, and lessons from using collaboration assistants during COVID-19. In Patterns 2(8), 100308 (2021)

    Article  Google Scholar 

  46. Srivastava, B., Rossi, F.: Towards Composable Bias Rating of AI Systems. In 2018 AI Ethics and Society Conference (AIES 2018), New Orleans, Louisiana, USA, Feb 2-3 (2018)

  47. Srivastava, B., Rossi, F.: Rating AI Systems for Bias to Promote Trustable Applications. In IBM J. Res. Develop. (2020)

  48. Srivastava, B., Rossi, F., Usmani, S., Bernagozzi, M.: Personalized Chatbot Trustworthiness Ratings. In IEEE Transactions on Technology and Society (2020)

  49. UC-Davis. Food Labeling. In https://ucfoodsafety.ucdavis.edu/processing-distribution/regulations-processing-food/food-labeling (2022)

  50. UCS. Transparency in Food Labeling. In https://www.ucsusa.org/resources/transparency-food-labeling (2016)

  51. UnitedHealthRankings.: Senior Report: Vote Participation - Age 65+ (Midterm). In https://www.americashealthrankings.org/explore/senior/measure/voter_turnout_Senior/state/ALL (2022)

  52. US-BLS.: List of occupations. In https://www.bls.gov/bls/occupation.html; Last accessed 19 Jan 2023 (2023)

  53. Varshney, K. R.: Trustworthy machine learning. ISBNL 979-8411903959 (2022)

  54. Verma, S., Rubin, J.: Fairness Definitions Explained. In Proceedings of the International Workshop on Software Fairness, FairWare ’18, 1–7. New York, NY, USA: Association for Computing Machinery. ISBN 9781450357463 (2018)

  55. Wallach, W., Allen, C.: Moral Machines: Teaching Robots Right from Wrong. USA: Oxford University Press, Inc. ISBN 0195374045 (2008)

  56. Wang, D.D., Hu, F.B.: Precision nutrition for prevention and management of type 2 diabetes. Lancet Diabetes Endocrinol. 6(5), 416–426 (2018)

    Article  Google Scholar 

  57. Wang, W., Chen, L., Thirunarayan, K., Sheth, A. P.: Cursing in English on Twitter. In CSCW (2014)

  58. Wang, W., Feng, F., He, X., Wang, X., Chua, T.-S.: Deconfounded Recommendation for Alleviating Bias Amplification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery &; Data Mining, KDD ’21, 1717–1725. New York, NY, USA: Association for Computing Machinery. ISBN 9781450383325 (2021)

  59. **ang, A., Raji, I. D.: On the Legal Compatibility of Fairness Definitions. In On Arxiv at: https://arxiv.org/abs/1912.00761 (2019)

  60. Xu, S., Tan, J., Heinecke, S., Li, J., Zhang, Y.: Deconfounded Causal Collaborative Filtering (2021)

Download references

Acknowledgements

This research was partially funded by a fellowship by the Government of India’s VAJRA program and a gift from Cisco Research.

Author information

Authors and Affiliations

  1. AI Institute, University of South Carolina, Columbia, South Carolina, USA

    Biplav Srivastava, Kausik Lakkaraju & Marco Valtorta

  2. IBM Research, Yorktown Heights, New York, USA

    Mariana Bernagozzi

Authors
  1. Biplav Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kausik Lakkaraju
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mariana Bernagozzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marco Valtorta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Biplav Srivastava.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Srivastava, B., Lakkaraju, K., Bernagozzi, M. et al. Advances in automatically rating the trustworthiness of text processing services. AI Ethics 4, 5–13 (2024). https://doi.org/10.1007/s43681-023-00391-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s43681-023-00391-5

Keywords

Search

Navigation