TREC Knowledge Base Acceleration


TREC KBA 2014: Overview

Knowledge Base Acceleration (KBA) is an open evaluation in NIST's Text Retrieval Conference (TREC). KBA addresses this fundamental question:

Given a rich dossier on a subject,
filter a stream of documents to
accelerate users filling in knowledge gaps.

KBA 2014 is completed! See papers and data!

Specfically, KBA posses this question in a stream filtering context and uses entities (people, organizations, buildings) as the subjects of interest. The dossiers are profiles about the entities from a knowledge base (KB). As the stream of documents progresses, the entities change and evolve, so KBA systems must detect when vital, new information appears that would motivate an update to the KB.

KBA provides training data in the form of ground truth labels from the beginning of the stream. This simulates a human user training a filter with feedback (positive and negative examples) and then leaving the system to run open loop as the stream continues.

KBA provides a predefined list of entity profiles as query targets and a corpus organized in hourly batches of 105 texts. Considering each hourly batch in chronological order, KBA systems perform three tasks:

  1. Vital Filtering: Identify "vital" documents containing timely, new information that should update the profile.
  2. Streaming Slot Filling: Identify short phrases that fill-in basic attributes of the entity such as alternate names and birth date.
  3. Accelerate & Create (new open task!): Help the user rapidly assimilate new information into the KB.

Timeline leading up to TREC 2014 conference

now: NIST accepting KBA Data Access Agreements.
End of May: NIST accepting TREC registration closes! You must register to get KBA ground truth data and queries.
May: Updated 2014 corpus available for download.
End of June: Queries and all ground truth data released.
End of August: Run submissions due at NIST.
October: Submission deadline for TREC Notebook papers.
November: TREC Conference in Gaithersburg, MD.


KBA 2014 has important improvements over KBA 2012 and KBA 2013: