Review process
Kili Technology allows annotation workflow management, and when required, a fine-grained data validation process to analyze and validate the quality of production.
This structured process facilitates collaboration within the team and helps to produce better quality annotated data.
The combination of quality scores, which makes it possible to filter quickly on the labels that raise questions, and the process of review, are key to producing better annotations.
The review Interface
The review interface is accessed from the Audit Labelers tab, by clicking on Start Reviewing.
The interface, similar to the annotation interface, allows you to review annotated assets, correct labels if necessary, and validate the labels of an asset.
Search labels & assets
To search and filter labels before reviewing, you can play with the panel size to get a better view on the list.
To combine filters, just add a space between two fields. For instance status:labeled consensus<80
- External ID =
externalId:XXX - Creation Date =
creationDate>25/11/2019//creationDate<25/11/2019 - Type =
type:prediction/default/review - Priority = filter not yet available in search
- Labeled by =
labeledBy:XXX - Duration = filter not yet available in search
- Status =
status:todo/ongoing/labeled/reviewed - Honeypot =
honeypot<10//honeypot>10 - Consensus =
consensus<90//consensus>90 - Skipped =
skipped:true/false - Label Honeypot =
labelHoneypot>40//labelHoneypot<10 - JSON response (ie. the content of the label) =
response:OBJECT_A
