SpringerLink
Log in

An Evaluation of the Multi-engine MT Architecture

  • Conference paper
  • First Online:
Machine Translation and the Information Soup (AMTA 1998)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 1529))

Included in the following conference series:

Abstract

The Multi-Engine MT (MEMT) architecture combines the outputs of multiple MT engines using a statistical language model of the target language. It has been used successfully in a number of MT research systems, for both text and speech translation. Despite its perceived benefits, there has never been a rigorous, published, double-blind evaluation of the claim that the combined output of a MEMT system is in fact better than that of any one of the component MT engines. We report here the results of such an evaluation. The combined MEMT output is shown to indeed be better overall than the output of the component engines in a Croatian ↔ English MT system. This result is consistent in both translation directions, and between different raters.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 35.99
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 44.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. J. J. Bartko and W. T. Carpenter. 1976. On the Methods and Theory of Reliability. The Journal of Nervous and Mental Disease, 163(5):307–317.

    Article  Google Scholar 

  2. I. N. Bronstein and K. A. Semendyayev. 1985. Handbook of Mathematics. Verlag Harri Deutsch, Thun and Frankfurt/Main, Frankfurt/Main, third edition.

    Google Scholar 

  3. Ralf Brown and Robert Frederking. 1995. Applying Statistical English Language Modeling to Symbolic Machine Translation. In Proceedings of the Sixth International Conference on Theoretical and Methodological Issues in Machine Translation (TMI-95), pages 221–239.

    Google Scholar 

  4. Ralf Brown. 1996. Example-based Machine Translation in the Pangloss System. In Proceedings of the 16th International Conference on Computational Linguistics (COLING-96).

    Google Scholar 

  5. David Carter et al. 1997. Translation Methodology in the Spoken Language Translator: An Evaluation. In Steven Krauwer et al., editors, Spoken Language Translation: Proceedings of a Workshop, pages 73–81, Madrid, Spain, July. Association of Computational Linguistics and European Network in Language and Speech.

    Google Scholar 

  6. G. Dunn. 1989. Design and Analysis of Reliability Studies: The Statistical Evaluation of Measurement Errors. Oxford University Press, New York.

    MATH  Google Scholar 

  7. David Farwell and Yorick Wilks. 1991. Ultra: A Multi-Lingual Machine Translator. In Proceedings of Machine Translation Summit III, Washington D. C., July.

    Google Scholar 

  8. Robert Frederking and Ralf Brown. 1996. The Pangloss-Lite Machine Translation System. In Proceedings of the Second Conference of the Association for Machine Translation in the Americas (AMTA), pages 268–272, Montreal, Quebec, Canada, October.

    Google Scholar 

  9. Robert Frederking and Sergei Nirenburg. 1994. Three Heads are Better than One. In Proceedings of the Fourth Conference on Applied Natural Language Processing (ANLP-94), Stuttgart, Germany.

    Google Scholar 

  10. Robert Frederking et al. 1993. An MAT Tool and its Effectiveness. In Proceedings of the DARPA Human Language Technology Workshop, Princeton, New Jersey.

    Google Scholar 

  11. Robert Frederking et al. 1994. Integrating Translations from Multiple Sources with the Pangloss Mark III Machine Translation System. In Proceedings of the First Conference for Machine Translation in the Americas (AMTA), Columbia, Maryland, October.

    Google Scholar 

  12. Robert Frederking, Alexander Rudnicky, and Christopher Hogan. 1997. Interactive Speech Translation in the DIPLOMAT Project. In Steven Krauwer et al., editors, Spoken Language Translation: Proceedings of a Workshop, pages 61–66, Madrid, Spain, July. Association of Computational Linguistics and European Network in Language and Speech.

    Google Scholar 

  13. Robert Frederking. 1994. Statistical Language Models for Symbolic MT. In Language Engineering on the Information Highway Workshop, Santorini, Greece, September.

    Google Scholar 

  14. Donna Gates et al. 1996. End-to-End Evaluation in Janus: A Speech-to-Speech Translation System. In Elisabeth Maier et al., editors, Dialogue Processing in Spoken Language Systems, volume 1236 of Lecture Notes in Artificial Intelligence, pages 195–206. Springer-Verlag, Berlin.

    Google Scholar 

  15. Juliane House. 1981. A Model for Translation Quality Assessment. Gunter Narr Verlag.

    Google Scholar 

  16. Martin Kay. 1967. Experiments with a Powerful Parser. In Proceedings of the 2nd International COLING, August.

    Google Scholar 

  17. K. Krippendorf. 1970. Bivariate Agreement Coeffcients for Reliability of Data. In E. F. Borgatta and G. W. Bohrnstedt, editors, Sociological Methodology. Jossey-Bass, San Francisco.

    Google Scholar 

  18. R. J. Light. 1971. Measures of Response Agreement for Qualitative Data: Some Generalizations and Alternatives. Psychological Bulletin, 76:365–377.

    Article  Google Scholar 

  19. R. R. MacDonald. 1963. General Report 1952-1963. Number 30 in Georgetown University Occasional Papers in Machine Translation. Washington, D. C.

    Google Scholar 

  20. A. K. Melby. 1983. Computer-Assisted Translation Systems: The Standard Design and a Multi-Level Design. In Conference on Applied Natural Language Processing, Santa Monica, California, February.

    Google Scholar 

  21. Teruko Mitamura, Eric Nyberg, and Jaime Carbonell. 1991. Interlingua Translation System for Multi-Lingual Document Processing. In Proceedings of Machine Translation Summit III, Washington, D. C., July.

    Google Scholar 

  22. M. Nagao. 1984. A Framework of a Mechanical Translation between Japanese and English by Analogy Principle. In A. Elithorn and R. Banerji, editors, Artificial and Human Intelligence. NATO Publications.

    Google Scholar 

  23. Sergei Nirenburg et al. 1995. The Pangloss Mark III Machine Translation System. Technical Report Issued as CMU Technical Report CMU-CMT-95-145, Joint Technical Report, Computing Research Laboratory (New Mexico State University), Center for Machine Translation (Carnegie Mellon University), Information Sciences Institute (University of Southern California).

    Google Scholar 

  24. Sergei Nirenburg et al. 1996. Two Principles and Six Techniques for Rapid MT Development. In Proceedings of the Second Conference of the Association for Machine Translation in the Americas, pages 96–105, Montreal, Quebec, Canada, October.

    Google Scholar 

  25. William H. Press, Brian P. Flannery, Saul A. Teukolsky, and William T. Vetterling. 1986. Numerical Recipes: The Art of Scientific Computing. Cambridge University Press, Cambridge, England.

    Google Scholar 

  26. Manny Rayner and David Carter. 1997. Hybrid Processing in the Spoken Language Translator. In Proceedings of ICASSP-97, Munich, Germany.

    Google Scholar 

  27. S. Sato and M. Nagao. 1990. Towards Memory Based Translation. In Proceedings of COLING-90, Helsinki, Finland.

    Google Scholar 

  28. J. S. White and T. A. O’Connell. 1994. The ARPA MT Evaluation Methodologies: Evolution, Lessons, and Future Approaches. In Proceedings of the First Conference of the Association for Machine Translation in the Americas (AMTA).

    Google Scholar 

  29. Terry Winograd. 1983. SyntaxLanguage as a Cognitive Process. Volume 1: Syntax. Addison-Wesley.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Language Technologies Institute, Carnegie Mellon University, 4910 Forbes Avenue, Pittsburgh, PA, 15213, USA

    Christopher Hogan & Robert E. Frederking

Authors
  1. Christopher Hogan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert E. Frederking
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computing Research Lab, New Mexico State University, Box 30001 / 3CRL, Las Cruces, NM, 88003, USA

    David Farwell

  2. SYSTRAN Inc., 7855 Fay Avenue, Suite 300, P.O. Box 907, La Jolla, CA, 92037, USA

    Laurie Gerber

  3. Information Sciences Institute, University of Southern California, 4676 Admiralty Way, Marina del Rey, CA, 90292-6695, USA

    Eduard Hovy

Rights and permissions

Reprints and permissions

Copyright information

© 1998 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Hogan, C., Frederking, R.E. (1998). An Evaluation of the Multi-engine MT Architecture. In: Farwell, D., Gerber, L., Hovy, E. (eds) Machine Translation and the Information Soup. AMTA 1998. Lecture Notes in Computer Science(), vol 1529. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49478-2_11

Download citation

  • DOI: https://doi.org/10.1007/3-540-49478-2_11

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-65259-5

  • Online ISBN: 978-3-540-49478-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation