span_marker.evaluation module¶

span_marker.evaluation.compute_f1_via_seqeval(tokenizer, eval_prediction, is_in_train)[source]¶

Compute micro-F1, recall, precision and accuracy scores using seqeval for the evaluation predictions.

Note

We assume that samples are not shuffled for the evaluation/prediction. With other words, don’t use this on the (shuffled) train dataset!

Parameters:

  • tokenizer (SpanMarkerTokenizer) – The model its tokenizer.

  • eval_prediction (EvalPrediction) – The predictions resulting from the evaluations.

  • is_in_train (bool) –

Returns:

Dictionary with "overall_precision", "overall_recall", "overall_f1"

and "overall_accuracy" keys.

Return type:

Dict[str, float]