Skip to contents

The goal of textdata is to provide access to text-related data sets for easy access without bundling them inside a package. Some text datasets are too large to store within an R package or are licensed in such a way that prevents them from being included in an OSS-licensed package. Instead, this package provides a framework to download, parse, and store the datasets on the disk and load them when needed.

Installation

You can install the not yet released version of textdata from CRAN with:

install.packages("textdata")

And the development version from GitHub with:

# install.packages("remotes")
remotes::install_github("EmilHvitfeldt/textdata")

Example

The first time you use one of the functions for accessing an included text dataset, such as lexicon_afinn() or dataset_sentence_polarity(), the function will prompt you to agree that you understand the dataset’s license or terms of use and then download the dataset to your computer.

After the first use, each time you use a function like lexicon_afinn(), the function will load the dataset from disk.

Included text datasets

As of today, the datasets included in textdata are:

Dataset Function
v1.0 sentence polarity dataset dataset_sentence_polarity()
AFINN-111 sentiment lexicon lexicon_afinn()
Hu and Liu’s opinion lexicon lexicon_bing()
NRC word-emotion association lexicon lexicon_nrc()
NRC Emotion Intensity Lexicon lexicon_nrc_eil()
The NRC Valence, Arousal, and Dominance Lexicon lexicon_nrc_vad()
Loughran and McDonald’s opinion lexicon for financial documents lexicon_loughran()
AG’s News dataset_ag_news()
DBpedia ontology dataset_dbpedia()
Trec-6 and Trec-50 dataset_trec()
IMDb Large Movie Review Dataset dataset_imdb()
Stanford NLP GloVe pre-trained word vectors embedding_glove6b()
embedding_glove27b()
embedding_glove42b()
embedding_glove840b()

Check out each function’s documentation for detailed information (including citations) for the relevant dataset.

Community Guidelines

Note that this project is released with a Contributor Code of Conduct. By contributing to this project, you agree to abide by its terms. Feedback, bug reports (and fixes!), and feature requests are welcome; file issues or seek support here. For details on how to add a new dataset to this package, check out the vignette!