本書(shū)是"計(jì)算語(yǔ)言學(xué)與語(yǔ)言科技原文叢書(shū)"中的一冊(cè)。文本和語(yǔ)音系統(tǒng)是計(jì)算語(yǔ)言學(xué)和語(yǔ)言技術(shù)的主要應(yīng)用方向，涉及的范圍很好廣泛，幾乎包括所有的語(yǔ)言技術(shù)應(yīng)用系統(tǒng)。本書(shū)涵蓋的內(nèi)容很好豐富，既詳細(xì)介紹了各種文本和語(yǔ)音系統(tǒng)及其評(píng)測(cè)的基礎(chǔ)理論，又結(jié)合大量實(shí)例針對(duì)系統(tǒng)評(píng)測(cè)進(jìn)行了明確的說(shuō)明和解釋。內(nèi)容深入淺出，適合有一定背景知識(shí)的初學(xué)者或相關(guān)領(lǐng)域的科研工作者閱讀，也可作為計(jì)算語(yǔ)言學(xué)專業(yè)教材。
戴布克耶Dybkjaer，丹麥南丹麥大學(xué)教授。
導(dǎo)讀1
Preface 15
Contributing Authors 17
Introduction 21
Laila Dybkjar， Holmer Hemsen and Wolfgang Minker
1 SPEECH AND SPEAKER RECOGNITION EVALUATION 1
Sadaoki Furui
1.Introduction 1
2.Principles of Speech Recognition 2
3.Categories of Speech Recognition Tasks 3
4.Evaluation of Speech Recognition Systems 6
5.Principles of Speaker Recognition 12
6.Categories of Speaker Recognition Tasks 13
7.Normalization and Adaptation Techniques 15
8.Evaluation of Speaker Recognition Systems 17
9.Factors Affecting the Performance and Evaluation Paradigm
Design for Speech and Speaker Recognition Systems 22
10.Svstem-Level Evaluation of Speech and Speaker Rognition23
11.Conclusion 24
References 24
2 EVALUATION OF SPEECH SYNTHESIS 29
Nick Campbell
1.Introduction 29
2.Components of Computer Speech 30
3.Evaluation Methodologies 36
4.Organised Evaluations and Assessment 41
5.Speaking to (and on Behalf of) People 45
6.Conclusion 45
References 48
3 MODELLING AND EVALUATING VERBAL AND NON-VERBAL
COMMUNICATION IN TALKING ANIMATED INTERFACE AGENTS
Bjarn Granstram and David House
1.Introduction 65
2.KTH Parametric Multimodal Speech Synthesis 67
3.Data Collection and Data-Driven Visual Synthesis 69
4.Evaluating Intelligibility and Information Presentation 71
5.Evaluating Visual Cues for Prominence 77
6.Evaluating Prosody and Interaction 80
7.Evaluating Visual Cues to Sentence Mode 84
8.Evaluation of Agent Expressiveness and Attitude 85
9.Agent and System Evaluation Studies 87
10.Future Challenges in Modelling and Evaluation 91
References 92
4 EVALUATING PART-OF-SPEECH TAGGING AND PARSING 99
Patrick Paroubek
1.POS Tagging 99
2.Parsing 102
3.Evaluation and Natural Language Processing 105
4.POS Tagging Evaluation Methodology 110
5.Methodology and Evaluation Measures for Parsing 114
6.Conclusion 117
References 118
5 GENERAL PRINCIPLES OF USER-ORIENTED EVALUATION 125
Margaret King
1.A Historical Note 126
2.What is User-Oriented Evaluation？ 128
3.A First Principle： Quality is Decided by Users 129
4.A Second Principle： Users do not Have the Same Needs 130
5.A Third Principle： Quality can be Characterized 135
6.A Fourth Principle： Quality can be Measured 148
7.Combining the Particular and the General： The Ideal 153
8.Conclusion 154
References 156
6 AN OVERVIEW OF EVALUATION METHODS IN TREC AD HOC INFO-
RMATION RETRIEVAL AND TREC QUESTION ANSWERING163
Simone Teufel
1.Introduction 163
2.Evaluation Criteria 169
3.Evaluation Metrics 172
4.Real-World Performance 182
5.Conclusion 183
References 185
7 SPOKEN DIALOGUE SYSTEMS EVALUATION 187
Niels Ole Bernsen， Laila Dybkjar and Wolfgang Minker
1.Introduction 187
2.Evaluation Methods and Criteria 188
3.Evaluation of the NICE Hans Christian Andersen Prototype 193
4.Evaluation of the SENECA Prototype 207
5.Conclusion 216
References 218
8 LINGUISTIC RESOURCES， DEVELOPMENT，AND EVALUATION OF
TEXT AND SPEECH SYSTEMS221
Christopher Cieri
1.Introduction 222
2.The Linguistic Resource Landscape 222
3.Background on Linguistic Data and Annotation 234
4.Data Planning for Technology Development and Evaluation 237
5.Finding Resources 240
6.Building Resources 242
7.Conclusion 259
References 260
9 TOWARDS INTERNATIONAL STANDARDS FOR LANGUAGE
RESOURCES263
Nancy Ide and Laurent Romary
1.Introduction 263
2.Background 265
3.The Linguistic Annotation Framework 268
4.Putting it All Together 278
5.Conclusion 282
References 283
Index 285
Akers, G.and Lennig, M.(1985).Intonation in Text-to-Speech Synthesis: Evaluation of Algorithms.Journal of the Acoustical Society of America, 77(6):2157-2165.
Benoit, C.and Pols, L.C.W.(1992).On the Assessment of Synthetic Speech.In Bailly, G., Benoit, C., and Sawallis, T., editors, Talking Machines: Theories, Models and Designs, pages 435-441, Elsevier, North Holland, Amsterdam, The Netherlands.
Benoit, C., van Erp, A., Grice, M., Hazan, V., and Jekosch, U.(1989).Multilin- gual Synthesiser Assessment Using Semantically Unpredictable Sentences.In Proceedings of the European Conference on Speech Communication and Technology (Eurospeech), volume 2, pages 633-636, Paris, France.
Bernstein, J.(1982).Evaiuating Synthetic Speech.In Proceedings of the NBS Workshop on Standardization for Speech I/O Technology, pages 87-91, Gaithersburg, Maryland, USA.
Bernstein, J.and Pisoni, D.B.(1980).Unlimited Text-to-Speech System: Deion and Evaluation of a Microprocessor Based Device.In Proceedings of the Internarional Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 576-579, Denver, Colorado, USA,
Bladon, A.(1990).Evaluating the Prosody of Text-to-Speech Synthesizers.In Proceedings of Speech Tech, pages 215-220, New York, USA.
Boeffard, O., Cherbonnel, B., Emerard, F., and White, S.(1993).Automatic Segmentation and Quality Evaluation of Speech Unit Inventories for Concatenation-Based, Multilingual PSOLA Text-to-Speech Systems.In Proceedings of the European Conference on Speech Communication an.d Technology (Eurospeech), volume 2, pages 1449-1452, Berlin, Germany.
Boogaar, T.and Silverman, K, (1992).Evaluating the Overall Comprehensibility of Speech Synthesizers.In Proceedings of the International Conference on Spoken Language Processing (ICSLP), pages 1207-1210, Banff, Canada.
