span_marker.label_normalizer module¶

span_marker.label_normalizer.Entity¶

Tuple of:

  • Entity label

  • Word start index

  • Word end index

alias of Tuple[int, int, int]

class span_marker.label_normalizer.LabelNormalizer(config)[source]¶

Bases: ABC

Class to convert NER training data into a common format used in the SpanMarkerTokenizer.

The common format involves 3-tuples with entity labels, word start indices and word end indices.

Parameters:

config (SpanMarkerConfig) –

abstract ner_tags_to_entities(ner_tags)[source]¶
Parameters:

ner_tags (List[int]) –

Return type:

Iterator[Tuple[int, int, int]]

class span_marker.label_normalizer.LabelNormalizerScheme(config)[source]¶

Bases: LabelNormalizer

Parameters:

config (SpanMarkerConfig) –

ner_tags_to_entities(ner_tags)[source]¶

Assumes a correct IOB or IOB2 annotation scheme

Parameters:

ner_tags (List[int]) –

Return type:

Iterator[Tuple[int, int, int]]

class span_marker.label_normalizer.LabelNormalizerIOB(config)[source]¶

Bases: LabelNormalizerScheme

Parameters:

config (SpanMarkerConfig) –

class span_marker.label_normalizer.LabelNormalizerBIOES(config)[source]¶

Bases: LabelNormalizerScheme

Parameters:

config (SpanMarkerConfig) –

class span_marker.label_normalizer.LabelNormalizerBILOU(config)[source]¶

Bases: LabelNormalizerScheme

Parameters:

config (SpanMarkerConfig) –

class span_marker.label_normalizer.LabelNormalizerNoScheme(config)[source]¶

Bases: LabelNormalizer

Parameters:

config (SpanMarkerConfig) –

ner_tags_to_entities(ner_tags)[source]¶
Parameters:

ner_tags (List[int]) –

Return type:

Iterator[Tuple[int, int, int]]

class span_marker.label_normalizer.AutoLabelNormalizer[source]¶

Bases: object

Factory class to return the correct LabelNormalizer subclass.

static from_config(config)[source]¶
Parameters:

config (SpanMarkerConfig) –

Return type:

LabelNormalizer