Package featuregenerator :: Module lengthfeaturegenerator :: Class LengthFeatureGenerator

Class LengthFeatureGenerator

                       object --+    
                                |    
featuregenerator.FeatureGenerator --+
                                    |
                                   LengthFeatureGenerator

Class that provides a feature generator able to count the number of the tokens in the given simplesentences

Instance Methods

[hide private]

dict

get_features_simplesentence(self, simplesentence, parallelsentence=None)
Uses NLTK toolkit in order to tokenize given simplesentence and provide a feature with the number of tokens

source code

Inherited from featuregenerator.FeatureGenerator: add_features_batch, add_features_dataset, add_features_parallelsentence, add_features_simplesentence, add_features_src, add_features_tgt, get_annotation_name, get_features_parallelsentence, get_features_src, get_features_string, get_features_tgt, process_dataset

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __init__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Properties

[hide private]

Inherited from object: __class__

Method Details

[hide private]

get_features_simplesentence(self, simplesentence, parallelsentence=None)

source code

Uses NLTK toolkit in order to tokenize given simplesentence and provide a feature with the number of tokens

Parameters:

simplesentence (sentence.sentence.SimpleSentence) - The SimpleSentence whose words are to be counted

Returns: dict

dictionary containing lenght attribute

Overrides: featuregenerator.FeatureGenerator.get_features_simplesentence