JAVELIN I System (2002-2005)
During Phase I of the AQUAINT program, we built our first question answering system (JAVELIN I). This system was developed for the AQUAINT newswire corpus, and was evaluated in the TREC QA track (2002, 2003, 2005).
The system incorporated the following main components:
Question Analyzer: A module that analyzes the user's questions to produce a representation of the information need (question type, expected answer type, key terms, etc.).
Retrieval Strategist: A module that formulates a series of queries to search for documents relevant to the information need using a search engine and document index.
Request Filler: A module that extracts candidate answers from the documents that were retrieved. During Phase I, we developed a variety of information extraction approaches for extracting answer candidates, most of them based on statistical question/answer modeling.
Answer Generator: A module that merges and ranks the candidate answers to produce a final ranked answer list.
Planner & Domain Model: A dynamic decision-making module that determines the incremental strategy for the entire system; for example, if processing at one phase fails to produce an output with an acceptable score, the Planner determines how to recover (e.g., by clarifying the question with the user, or selecting a different extraction module to look for answers). The Domain Model is the set of question-answering operations that the system can apply at each stage in processing, depending on the current state.
Execution Manager: A framework class which hides the low-level details of the JAVELIN pipeline from the Planner and User Interface.
Data Repository: A relational database which stores all of the questions, intermediate data structures, and final results for each user session.
Selected Publications
"JAVELIN I and II Systems at TREC 2005", Eric Nyberg, Robert Frederking, Teruko Mitamura, Matthew Bilotti, Kerry Hannan, Laurie Hiyakumoto, Jeongwoo Ko, Frank Lin, Lucian Lita, Vasco Pedro, and Andrew Schlaikjer, Proceedings of TREC 2005 PDF
"Multi-Strategy Information Extraction for Question Answering", Hiyakumoto L., Lita L.V., Nyberg, E., Proceedings of the Florida Artificial Intelligence Research Society Conference (FLAIRS 2005). PDF
"Unsupervised Question Answering Data Acquisition From Local Corpora", Lita L.V., Carbonell J., Proceedings of the 13th International Conference on Information and Knowledge Management (CIKM 2004) PDF
"Resource Analysis for Question Answering", Lita L.V., Hunt W., Nyberg E, Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL 2004). PDF
"Instance-Based Question Answering: A Data Driven Approach", Lita L.V., Carbonell J., Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2004) PDF
"An Information Repository Model For Advanced Question Answering Systems", Pedro, V., Ko J., Nyberg E., Mitamura T., International Conference on Language Resources and Evaluation (LREC 2004) PDF "Planning in the JAVELIN QA System", Laurie S. Hiyakumoto, Carnegie Mellon Computer Science Technical Report CMU-CS-04-132 (2004) PDF
"Gazetteers, WordNet, Encyclopedias, and The Web: Analyzing Question Answering Resources", Hunt W.,Lita L.V., Nyberg E., Language Technologies Institute, Carnegie Mellon Technical Report CMU-LTI-04-188 (2004) PDF
"The JAVELIN Question-Answering System at TREC 2003: A Multi-Strategy Approach with Dynamic Planning", E. Nyberg, T. Mitamura, J. Callan, J. Carbonell, R. Frederking, K. Collins-Thompson, L. Hiyakumoto, Y. Huang, C. Huttenhower, S. Judy, J. Ko, A. Kupse, L. Lita, V. Pedro, D. Svoboda and B. Van Durme, Proceedings of the 12th Text REtrieval Conference, November 2003 (TREC 2003) PDF
"The JAVELIN Question-Answering System at TREC 2002", Nyberg, E., T. Mitamura, J. Carbonell, J. Callan, K. Collins-Thompson, K. Czuba, M. Duggan, L. Hiyakumoto, N. Hu, Y. Huang, J. Ko, L. Lita, S. Murtagh, V. Pedro and D. Svoboda, Proceedings of the 11th Text REtrieval Conference, November 2002 (TREC 2002) PDF