Evaluate

Module containing ROUGE implementations.

RougeScore

Python implementation of ROUGE score.

Taken and adopted from:

• https://github.com/miso-belica/sumy/blob/master/sumy/evaluation/rouge.py
• https://github.com/google/seq2seq/blob/master/seq2seq/metrics/rouge.py

class clstk.evaluation.rougeScore.RougeScore(tokenizer=None, stemmer=None)

Bases: object

Implementation of ROUGE score.

__init__(tokenizer=None, stemmer=None)

x.__init__(…) initializes x; see help(type(x)) for signature

rouge_n(summary, model_summaries, n=2)

Computes ROUGE-N of two text collections of sentences.

rouge_l_sentence_level(evaluated_sentences, reference_sentences)

Computes ROUGE-L (sentence level) of two text collections of sentences.

rouge_l_summary_level(evaluated_sentences, reference_sentences)

Computes ROUGE-L (summary level) of two text collections of sentences.

rouge(hyp_refs_pairs, print_all=False)

Calculates and prints average rouge scores for a list of hypotheses and references

Parameters:

• hyp_refs_pairs – List containing pairs of path to summary and list of paths to reference summaries
• print_all – Print every evaluation along with averages

__weakref__

list of weak references to the object (if defined)

ExternalRougeScore

Integration with external ROUGE tool-kit.

We recommend the use of https://github.com/nisargjhaveri/ROUGE-1.5.5-unicode

ROUGE_HOME variable needs to be set to run this.

class clstk.evaluation.externalRougeScore.ExternalRougeScore

Bases: object

Integration with external ROUGE tool-kit.

rouge(summaryRefsList)

Runs external ROUGE-1.5.5 and prints results

Parameters:summaryRefsList – List containing pairs of path to summary and list of paths to reference summaries

__weakref__

list of weak references to the object (if defined)