gsitk is a library on top of scikit-learn that eases the development process on NLP machine learning driven projects. It uses numpy, pandas and related libraries to easy the development.
gsitk manages datasets, features, classifiers and evaluation techniques, so that writing an evaluation pipeline results fast and simple.
Installation and use
Installation
gsitk can be installed via pip, which is the recommended way:
pip install gsitk
Alternatively, gsitk can be installed by cloning this repository.
Using gsitk
gsitk saves into disk the datasets and some other necessary resources.
By default, all these data are stored in /data.
The environment variable $DATA_PATH can be set in order to specify an alternative directory.
Feature extractors
SIMON feature extractor
gsitk includes the implementation of the SIMON feature extractor. To use it, two things are needed: - A sentiment lexicon - A word embeddings model that is gensim compatible.
For example, using only the lexicon from Bing Liu and a embeddings model that is in the current directory:
from gsitk.features import simon
from nltk.corpus import opinion_lexicon
from gensim.models.keyedvectors import KeyedVectors
lexicon = [list(opinion_lexicon.positive()), list(opinion_lexicon.negative())]
embedding_model = KeyedVectors.load_word2vec_format('GoogleNews-vectors-negative300.bin', binary=True)
simon_transformer = simon.Simon(lexicon=lexicon, n_lexicon_words=200, embedding=embedding_model)
# simon_transformer has the fit() and transform() methods, so it can be used in a Pipeline
To enhance performance, it is recommendable to use a more complete scikit-learn pipe that implements normalization and feature selection in conjuction with the SIMON feature extraction.
from gsitk.features import simon
simon_model = simon.Simon(lexicon=lexicon, n_lexicon_words=200, embedding=embedding_model)
model = simon.simon_pipeline(simon_transformer=simon_model, percentile=25)
# model also implemtens fit() and transform()
