TREC KBA 2014: Accelerate & Create
Knowledge Base Acceleration (KBA) is an open evaluation in NIST's Text Retrieval Conference (TREC). KBA addresses this fundamental question:
filter a stream of documents to
accelerate users filling in knowledge gaps.
New Open Task: in KBA 2014, we are introducing this new third task called Accelerate & Create, in which we invite participants to try cool ideas of their own creation using the track data.
Requirements
- Must address KBA's fundamental question (above)
- Must use the track data, e.g. the streamcorpus, entity profiles, ground truth, other system runs.
- Must also submit a CCR run.
- Should produce some kind of artefact, e.g. data file, plot, picture, factoid, code, demo, etc.
- Send it before the track's official deadline.
We seek submissions that teach, inspire, bemuse, engage, or otherwise intrigue you.
Anything meeting the above requirements is acceptable. Everything
from analyzing the corpus to find plagiarism to visualization
techniques for the data to extracting content for a particular
knowledge domain to cross-language ideas to efficient compression and
indexing of streamcorpus data --- and more.
To submit a run to the Accelerate & Create task, send something by email to the track organizers before the submission deadline, and describe it in your notebook paper submitted before the conference. The track will provide a venue for discussing KBA Create submissions at the TREC conference.
For example, active learning is a hot area in both natural language processing and information retrieval. Since the entire ground truth data will be available, you can simulate different modes of training, such as those illustrated in the time series diagrams to the right.