Abstract
This work presents an approach to creating interpreted vector representations of words in which each component of the vector corresponds to a certain interpreted semantic category. To obtain such categories, a lexical and semantic resource in the form of the semantic network RuWordNet is used, as well as a representative corpus of Russian-language texts for generating vector representations. The resulting interpreted vector representations were tested for the ability to display different models in the same vector space.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
E. Agirre, E. Alfonseca, K. Hall, J. Kravalova, M. Paşca, and A. Soroa, “A study on similarity and relatedness using distributional and WordNet-based approaches,” in Proc. Human Language Technologies: The 2009 Annu. Conf. of the North Am. Chapter of the Assoc. for Computational Linguistics NAACL’09, Boulder, Colo., 2009 (Association for Computational Linguistics, Stroudsburg, Pa., 2009), pp. 19–27. https://doi.org/10.3115/1620754.1620758
C. Aloui, C. Ramisch, A. Nasr, and L. Barque, “SLICE: Supersense-based lightweight interpretable contextual embeddings,” in Proc. 28th Int. Conf. on Computational Linguistics, Barcelona, 2020 (International Committee on Computational Linguistics, 2020), pp. 3357–3370. https://doi.org/10.18653/v1/2020.coling-main.298
Yu. D. Apresyan, Lexical Semantics: Synonymical Means of Language (Moscow, 1974).
M. Artetxe, G. Labaka, and E. Agirre, “Learning principled bilingual map**s of word embeddings while preserving monolingual invariance,” in Proc. 2016 Conf. on Empirical Methods in Natural Language Processing, Austin, Texas, 2016 (Association for Computational Linguistics, 2016), pp. 2289–2294. https://doi.org/10.18653/v1/d16-1250
S. Brin and L. Page, “The anatomy of a large-scale hypertextual Web search engine,” Comput. Networks ISDN Syst. 30, 107–117 (1998). https://doi.org/10.1016/s0169-7552(98)00110-x
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minn., 2019 (Association for Computational Linguistics, 2019), Vol. 1, pp. 4171–4186. https://doi.org/10.18653/v1/N19-1423
C. Fellbaum, WordNet: An Electronic Lexical Database (MIT Press, Cambridge, Mass., 1998).
L. Flekova and I. Gurevych, “Supersense embeddings: A unified model for supersense interpretation, prediction, and utilization,” in Proc. 54th Annu. Meeting of the Assoc. for Computational Linguistics, Berlin, 2016 (Association for Computational Linguistics, 2016), Vol. 1, pp. 2029–2041. https://doi.org/10.18653/v1/p16-1191
I. M. Kobozeva, Linguistic Semantics (URSS, Moscow, 2004).
A. Kutuzov and M. Kunilovskaya, “Size vs. structure in training corpora for word embedding models: Araneum Russicum Maximum and Russian National Corpus,” in Analysis of Images, Social Networks and Texts. AIST 2017, Lecture Notes in Computer Science, Vol. 10716 (Springer, Cham, 2017), pp. 47–58. https://doi.org/10.1007/978-3-319-73013-4_5
H. Le, L. Vial, J. Frej, V. Segonne, M. Coavoux, B. Lecouteux, A. Allauzen, B. Crabbé, L. Besacier, and D. Schwab, “FlauBERT: Unsupervised language model pre-training for French,” in Proc. 12th Language Resources and Evaluation Conf., Marseille, 2020 (2020), pp. 2479–2490. https://aclanthology.org/2020.lrec-1.302.
I. Leviant and R. Reichart, “Separated by an un-common language: Towards judgment language informed vector space modeling,” (2015). https://doi.org/10.3115/v1/w15-15
N. V. Loukachevitch, A. A. Gerasimova, B. B. Dobrov, G. Lashevich, and V. V. Ivanov, “Creating Russian WordNet by conversion,” in Computational Linguistics and Intellectual Technologies: Proc. Int. Conf. Dialogue 2016, Moscow, 2016 (2016), pp. 22–30.
T. Mikolov, Q. V. Le, and I. Sutskever, “Exploiting similarities among languages for machine translation,” (2013). https://doi.org/10.48550/ar**v.1309.4168
T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in Proc. 26th Int. Conf. on Neural Information Processing Systems, Lake Tahoe, Nevada, 2013, Ed. by C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Curran Associates, Red Hook, N.Y., 2013), pp. 3111–3119.
R. Navigli and S. P. Ponzetto, “BabelNet: The automatic construction, evaluation, and application of a wide-coverage multilingual semantic network,” Artif. Intell. 193, 217–250 (2012). https://doi.org/10.1016/j.artint.2012.07.001
A. Panchenko, D. Ustalov, N. Arefyev, D. Paperno, N. Konstantinova, N. Loukachevitch, and C. Biemann, “Human and machine judgements for Russian semantic relatedness,” in Analysis of Images, Social Networks and Texts, Ed. by D. I. Ignatov, M. Yu. Khachay, V. G. Labunets, N. Loukachevitch, S. I. Nikolenko, A. Panchenko, A. V. Savchenko, and K. Vorontsov, Communications in Computer and Information Science, Vol. 661 (Springer, Cham, 2016), pp. 221–235. https://doi.org/10.1007/978-3-319-52920-2_21
M. Radovanović, A. Nanopoulos, and M. Ivanović, “Hubs in space: Popular nearest neighbors in high-dimensional data,” J. Mach. Learn. Res. 11, 2487–2531 (2010).
S. Ruder, I. Vulić, and A. Søgaard, “A survey of cross-lingual word embedding models,” J. Artif. Intell. Res. 65, 569–631 (2019). https://doi.org/10.1613/jair.1.11640
F. Scozzafava, A. Raganato, A. Moro, and R. Navigli, “Automatic identification and disambiguation of concepts and named entities in the multilingual Wikipedia,” in Advances in Artificial Intelligence. AI*IA 2015, Ed. by M. Gavanelli, E. Lamma, and F. Riguzzi, Lecture Notes in Computer Science, Vol. 9336 (Springer, Cham, 2015), pp. 357–366. https://doi.org/10.1007/978-3-319-24309-2_27
J. Yamane, T. Takatani, H. Yamada, M. Miwa, and Yu. Sasaki, “Distributional hypernym generation by jointly learning clusters and projections,” in Proc. COLING 2016, the 26th Int. Conf. on Computational Linguistics: Technical Papers, Osaka, Japan, 2016 (The COLING 2016 Organizing Committee, 2016), pp. 1871–1879. https://aclanthology.org/C16-1176.
Funding
This work was supported by a grant from the Russian Science Foundation (project no. 21-71-30003).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
The authors declare that they have no conflicts of interest.
Additional information
Mikhail Tikhomirov (born in 1993) graduated from the Faculty of Computational Mathematics and Cybernetics of the Lomonosov Moscow State University, defended his dissertation for the degree of Candidate of Physics and Mathematics on the topic of “Methods of Automated Replenishment of Knowledge Graphs Based on Vector Representations” in 2022. Currently, he is an employee of the Research Computing Center of the Lomonosov Moscow State University and is engaged in research in the field of automatic text processing. Research interests: natural language processing, neural network models, large language models, knowledge graphs.
Natalia V. Loukachevitch (born in 1964) graduated from the Faculty of Computational Mathematics and Cybernetics of the Lomonosov Moscow State University, defended her dissertation for the degree of Doctor of Engineering Sciences in 2016. Currently, she is a leading researcher at the Research Computing Center of the Lomonosov Moscow State University. Lectures on automatic text processing and information search at the Faculty of Computational Mathematics and Cybernetics and the Faculty of Philology of the Lomonosov Moscow State University, as well as at the Bauman Moscow State Technical University. She has more than 300 publications in the field of automatic text processing, information search, and presentation of knowledge. Scientific interests: automatic text processing, information search, ontology.
APPENDIX
APPENDIX
The appendix provides a list of supersenses for the model (the first 30 supersenses) which were obtained in the described experiment:
Supersense 0: component part: strip, piece, pulp, barrel, component
Supersense 1: product of work: work, production, goods, product, ware
Supersense 2: occupation, activity: occupation, industry, hobby, profession, interest
Supersense 3: to be in a state: languishing, vitality, adoration, thawing, starvation
Supersense 4: image (result): photo, drawing, image, engraving, snapshot
Supersense 5: group united by a common trait: three, thirty, ten, hundred, twenty
Supersense 6: subject of activity: debtor, opponent, buyer, benefactor, organizer
Supersense 7: substance: perfume, preservative, powder, ingredient, substance
Supersense 8: vary, change: improvement, increase, decrease, slow down, impregnation
Supersense 9: place in space: bend, bulge, speck, surface, ravine
Supersense 10: natural phenomenon: weather, sediment, cloud, wind, cyclone
Supersense 11: spend, consume: expenditure, economy, consumption, overconsumption, cost
Supersense 12: computer program: utility, application, subprogram, program, update
Supersense 13: biological essence: cell, tissue, mitochondria, organism, chromosome
Supersense 14: movement, displacement: riding, movement, inclination, travel, jump
Supersense 15: state, internal circumstances: orderliness, dryness, moisture, sputum, situation
Supersense 16: population: urban, Latino, European, Uralian, Sverdlovsk
Supersense 17: physiological process: climax, regeneration, secretion, heartbeat, menopause
Supersense 18: physical property: permeability, density, conductivity, fragility, property
Supersense 19: unit of measurement of information: Mb, gigabyte, Mbit, kbit, megabyte
Supersense 20: unit of volume: gallon, decaliter, half liter, barrel, liter
Supersense 21: change, make different: improve, lighten, increase, decrease, change
Supersense 22: unit of mass: kilogram, gram, kilo, ton, ounce
Supersense 23: ability: tact, musicality, skill, flexibility, disposition
Supersense 24: material for manufacture: foam, latex, plastic, winding, rubber
Supersense 25: unit of length: micron, meter, centimeter, nanometer, millimeter
Supersense 26: construction, structure: construction, building, structure, hut, formation
Supersense 27: object, thing: ball, bar, piece, patch, decoration
Supersense 28: age: adolescence, childhood, age, property, old age
Supersense 29: shape, appearance: roundness, bulge, trim, cut, style
Supersense 30: God: God, Lord, Christ, god-man, Jesus
Rights and permissions
About this article
Cite this article
Tikhomirov, M.M., Loukachevitch, N.V. Generation of Interpreted Vector Representations of Words Based on Supersenses. Pattern Recognit. Image Anal. 33, 517–524 (2023). https://doi.org/10.1134/S1054661823030446
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S1054661823030446