span_marker.pipeline_component module

class span_marker.pipeline_component.SpanMarkerPipeline(model, tokenizer=None, feature_extractor=None, image_processor=None, modelcard=None, framework=None, task='', args_parser=None, device=None, torch_dtype=None, binary_output=False, **kwargs)[source]

Bases: Pipeline

A Pipeline component for SpanMarker.

The pipeline function is pipeline(), which you can also import with from transformers import pipeline, but you must also import span_marker to register the "span-marker" pipeline task.

Example:

>>> from span_marker import pipeline
>>> pipe = pipeline(task="span-marker", model="tomaarsen/span-marker-mbert-base-multinerd", device_map="auto")
>>> pipe("Amelia Earhart flew her single engine Lockheed Vega 5B across the Atlantic to Paris.")
[{'span': 'Amelia Earhart', 'label': 'PER', 'score': 0.9999709129333496, 'char_start_index': 0, 'char_end_index': 14},
 {'span': 'Lockheed Vega 5B', 'label': 'VEHI', 'score': 0.9050095677375793, 'char_start_index': 38, 'char_end_index': 54},
 {'span': 'Atlantic', 'label': 'LOC', 'score': 0.9991973042488098, 'char_start_index': 66, 'char_end_index': 74},
 {'span': 'Paris', 'label': 'LOC', 'score': 0.9999232292175293, 'char_start_index': 78, 'char_end_index': 83}]
Parameters:

  • model (PreTrainedModel | TFPreTrainedModel) –

  • tokenizer (PreTrainedTokenizer | None) –

  • feature_extractor (SequenceFeatureExtractor | None) –

  • image_processor (BaseImageProcessor | None) –

  • modelcard (ModelCard | None) –

  • framework (str | None) –

  • task (str) –

  • args_parser (ArgumentHandler) –

  • device (int | torch.device) –

  • torch_dtype (str | torch.dtype | None) –

  • binary_output (bool) –