Kili Docs

Kili Docs

›Recipes

Introduction to Kili Technology

  • Introduction to Kili Technology
  • Kili Technology allows
  • Compatible browser

Hosting

  • SaaS
  • On-Premise Data
  • On-Premise Entreprise

Definitions

  • Definitions

Users and roles

  • Roles by project
  • Users
  • Users and roles management

Projects

  • Audit labelers
  • Customize interface
  • Dataset
  • New project
  • Project overview
  • Projects
  • Projects list
  • Settings
  • Shortcuts

Image interfaces

  • Bounding Box
  • Classification
  • Point
  • Polygon
  • Polyline
  • Segmentation
  • Simple and intuitive interfaces

Text & PDF interfaces

  • Classification
  • Image transcription / OCR
  • Named entities recognition
  • Relations extraction

Video interfaces

  • Classification
  • Multi-frames classification
  • Multi-frames object detection
  • Transcription

Audio interfaces

  • Voice transcription / Speech to text

Data ingestion

  • Data ingestion made easy
  • Load data from a workstation
  • Load data from a public cloud
  • Data on premise or on private cloud
  • How to generate non-expiring signed URLs on AWS

Quality management

  • Consensus
  • Honeypot or Gold Standard
  • Instructions
  • Quality KPIs
  • Quality management
  • Questions and Issues
  • Review Process
  • Workload distribution

Automation

  • Human in the loop
  • Model based preannotation
  • Online learning
  • Queue prioritisation

Data export

  • Data export
  • Data format
  • Example

Python - GraphQL API

  • Authentication
  • Create a Honeypot
  • Create a user
  • Creating Consensus
  • Delete the data
  • Export data
  • Export labels
  • GraphQL API
  • Import data
  • Import labels
  • Prioritize assets
  • Python API
  • See the Consensus of an annotation
  • See the Honeypot of an annotation
  • Throttling

Recipes

  • AutoML for faster labeling with Kili Technology
  • Create a project
  • Exporting a training set
  • Importing medical data into a frame project
  • Importing assets
  • # Kili tutorial: Import rich-text assets
  • Importing predictions
  • Reading and uploading dicom image data
  • How to query using the API
  • Labelled Image Data & Transfer Learning
  • Webhooks

Change log

  • Change log

# Kili tutorial: Import rich-text assets

Kili tutorial: Import rich-text assets

When dealing with textual data, style can convey a lot of meaning. If you annotate a long list or a legal text, displaying structured text instead of plain boring text allows your annotator to rapidly grasp patterns within the document.

import os

# !pip install kili # uncomment if you don't have kili installed already
from kili.authentication import KiliAuth
from kili.playground import Playground

api_key = os.getenv('KILI_USER_API_KEY')
api_endpoint = os.getenv('KILI_API_ENDPOINT') # If you use Kili SaaS, use the url 'https://cloud.kili-technology.com/api/label/v2/graphql'

kauth = KiliAuth(api_key=api_key, api_endpoint=api_endpoint)
playground = Playground(kauth)

Let's create a named-entity recognition project. We are going to label American law texts:

project = playground.create_empty_project(user_id=playground.auth.user_id)
project_id = project['id']
json_interface = {
    "jobRendererWidth": 0.25,
    "jobs": {
        "JOB_0": {
            "mlTask": "NAMED_ENTITIES_RECOGNITION",
            "instruction": "Categories",
            "required": 1,
            "isChild": False,
            "isVisible": True,
            "content": {
                "categories": {
                    "INSTRUCTIONS": {
                        "name": "Instructions",
                        "children": [],
                        "color": "#cc4125"
                    },
                    "PREAMBULE": {
                        "name": "Preambule",
                        "children": [],
                        "color": "#ffd966"
                    },
                    "RIGHTS": {
                        "name": "Rights",
                        "children": [],
                        "color": "#76a5af"
                    },
                    "REFERENCE_TO_GOD": {
                        "name": "Reference to God",
                        "children": [],
                        "color": "#c27ba0"
                    }
                },
                "input": "radio"
            }
        }
    }
}
playground.update_properties_in_project(project_id=project_id, consensus_tot_coverage=100,
                                        min_consensus_size=2, json_interface=json_interface,
                                        input_type='TEXT', title='massive')

When you insert normal text in Kili, you provide a content array. In case of rich text, you provide a json_content array. Given a JSON content, let's write a function that inserts a new asset:

from random import random

def create_asset_from_json_content(json_content):
    json_content_array = [json_content]
    external_id_array = [f'#{int(random()*1000)}']
    playground.append_many_to_dataset(project_id=project_id,
                                      content_array=[''],
                                      json_content_array=json_content_array,
                                      external_id_array=external_id_array)

json_content can contain nodes or leaves.

  • A node can have children (that is a list of nodes or leaves).

    • By default, nodes are <div />.
    • Possible types for a node are:
      • quote
      • h1
      • h2
      • p
      • li
      • ol
    • Possible styles for a node are:
      • border (reference)
      • display (reference)
      • marginBottom (reference)
      • marginLeft (reference)
      • marginRight (reference)
      • marginTop (reference)
      • textAlign (reference)
  • A leaf can contain text.

    • By default, leaves are <span />.
    • Leaves are identified by an id. The ID must be unique accross all the document. This will allow for overlapping entities between two or more leaves.
    • Possible styles for a leaf are:
      • bold: true
      • code: true
      • italic: true
      • underline: true
      • display (reference)
      • paddingBottom (reference)
      • paddingLeft (reference)
      • paddingRight (reference)
      • paddingTop (reference)

Let's see some real examples with the corresponding result in Kili's interface!

WARNING: Here, IDs are randomly generated, but you probably do not want this!

# One simple text leaf
json_content = [{
    'id': f'{random()}',
    'text': 'The unanimous Declaration of the thirteen United States of America.'
}]
create_asset_from_json_content(json_content)

# Some basic text style
json_content = [{
    'id': f'{random()}',
    'bold': True,
    'underline': True,
    'text': 'The unanimous Declaration of the thirteen United States of America.'
}]
create_asset_from_json_content(json_content)

# You can mix styled leaves with plain-text leaves
json_content = [
    {
        'children': [
            {
                'type': 'p',
                'children': [
                    {
                        'id': f'{random()}',
                        'bold': True,
                        'underline': True,
                        'text': 'The unanimous Declaration',
                    },
                    {
                        'id': f'{random()}',
                        'bold': True,
                        'text': ' of the thirteen United States of America.',
                    },
                    {
                        'id': f'{random()}',
                        'text':
                        "When in the Course of human events, it becomes necessary for one people to dissolve the political bands which have connected them with another, and to assume among the powers of the earth, the separate and equal station to which the Laws of Nature and of Nature's God entitle them, a decent respect to the opinions of mankind requires that they should declare the causes which impel them to the separation.",
                    }
                ]
            }
        ]
    }
]
create_asset_from_json_content(json_content)

# Finalize with a title, a sub title and proper margins
json_content = [
    {
        'children': [
            {
                'type': 'h1',
                'children': [
                    {
                        'border': '1px solid black',
                        'textAlign': 'center',
                        'children': [{'id': f'{random()}', 'text': 'Declaration of Independence'}],
                    },
                ],
            },
            {
                'type': 'h2',
                'children': [{'id': f'{random()}', 'text': 'In Congress, July 4, 1776'}],
            },
            {
                'type': 'p',
                'children': [
                    {
                        'id': f'{random()}',
                        'bold': True,
                        'underline': True,
                        'text': 'The unanimous Declaration',
                    },
                    {
                        'id': f'{random()}',
                        'bold': True,
                        'text': ' of the thirteen United States of America.',
                    },
                    {
                        'id': f'{random()}',
                        'text':
                        "When in the Course of human events, it becomes necessary for one people to dissolve the political bands which have connected them with another, and to assume among the powers of the earth, the separate and equal station to which the Laws of Nature and of Nature's God entitle them, a decent respect to the opinions of mankind requires that they should declare the causes which impel them to the separation.",
                    },
                ],
            },
            {
                'type': 'p',
                'marginLeft': '30px',
                'marginRight': '30px',
                'border': 'red',
                'children': [
                    {
                        'id': f'{random()}',
                        'text':
                        'We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness.',
                    },
                    {
                        'type': 'ul',
                        'children': [
                            {
                                'type': 'li',
                                'children': [
                                    {
                                        'id': f'{random()}',
                                        'text':
                                        'That to secure these rights, Governments are instituted among Men, deriving their just powers from the consent of the governed,',
                                    },
                                ],
                            },
                            {
                                'type': 'li',
                                'children': [
                                    {
                                        'id': f'{random()}',
                                        'text':
                                        'That whenever any Form of Government becomes destructive of these ends, it is the Right of the People to alter or to abolish it, and to institute new Government, laying its foundation on such principles and organizing its powers in such form, as to them shall seem most likely to effect their Safety and Happiness.',
                                    },
                                ],
                            },
                        ],
                    },
                    {
                        'id': f'{random()}',
                        'text':
                        'Prudence, indeed, will dictate that Governments long established should not be changed for light and transient causes; and accordingly all experience hath shewn, that mankind are more disposed to suffer, while evils are sufferable, than to right themselves by abolishing the forms to which they are accustomed. But when a long train of abuses and usurpations, pursuing invariably the same Object evinces a design to reduce them under absolute Despotism, it is their right, it is their duty, to throw off such Government, and to provide new Guards for their future security.--Such has been the patient sufferance of these Colonies; and such is now the necessity which constrains them to alter their former Systems of Government. The history of the present King of Great Britain is a history of repeated injuries and usurpations, all having in direct object the establishment of an absolute Tyranny over these States. To prove this, let Facts be submitted to a candid world.',
                    },
                ],
            },
        ],
    },
]
create_asset_from_json_content(json_content)


← Importing assetsImporting predictions →