LeNER-Br: a Dataset for Named Entity Recognition in Brazilian Legal Text

This page holds the dataset and source code described in the paper below, which was generated as a collaboration between two institutions of the University of Brasília: NEXT (Núcleo de P&D para Excelência e Transformação do Setor Público) and CiC (Departamento de Ciência da Computação).

• Pedro H. Luz de Araujo, Teófilo E. de Campos, Renato R. R. de Oliveira, Matheus Stauffer, Samuel Couto and Paulo Bermejo
  LeNER-Br: a Dataset for Named Entity Recognition in Brazilian Legal Text
  International Conference on the Computational Processing of Portuguese (PROPOR), September 24-26, Canela, Brazil, 2018.
  Download PDFs of the paper and slides.

      @InProceedings{luz_etal_propor2018,
          author = {Pedro H. {Luz de Araujo} and Te\'{o}filo E. {de Campos} and
          Renato R. R. {de Oliveira} and Matheus Stauffer and
          Samuel Couto and Paulo Bermejo},
          title = {{LeNER-Br}: a Dataset for Named Entity Recognition in {Brazilian} Legal Text},
          booktitle = {International Conference on the Computational Processing of Portuguese
          ({PROPOR})},
	  publisher = {Springer},
	  series = {Lecture Notes on Computer Science ({LNCS})},
	  pages = {313--323},
          year = {2018},
          month = {September 24-26},
          address = {Canela, RS, Brazil},	  
	  doi = {10.1007/978-3-319-99722-3_32},
	  url = {https://teodecampos.github.io/LeNER-Br/},
	  }	  
      

We kindly request that users cite our paper in any publication that is generated as a result of the use of our source code, our dataset or our pre-trained models.

Provide the LSTM-CRF model described in the paper, which achieved an average f1-score of 92.53% on the test set. This paper and up to date results are listed in PapersWithCode.

The sections below describe the requirements and the dataset and model files.

Download

Requirements

1. Python 3
2. pip

LeNER-Br Dataset

The directory structure is as follows:

• the train, test and dev folders hold space separated text files where the first column are the words and the second column are the correspondent named entity tags. Sentences are separeted by empty lines. In addition, each folder has a file that is the concatenation of all the other conll files of the same folder (train.conll, dev.conll and test.conll).
• metadata holds json files with additional information from each annotated document.
• raw_text holds the source txt files that originated the conll files.
• scripts hold an abbreviation list used for sentence segmentation and the script that generated the conll files. To run it, use this script:

python textToConll.py path/to/txtfile
      

Model

The model code is adapted from this repo and implements a NER model using Tensorflow (LSTM + CRF + chars embeddings). All code files modified are marked as such at the beginning. The section below summarizes the use of the model. For more in depth explanations of how to use the model and change its configurations refer to the README of the original implementation.

Evaluation

• To install the required python packages, run from the model folder:

pip install -r requirements.txt
      

• To obtain the f1 scores for each class on each part of the dataset:

python classScores.py train
python classScores.py dev
python classScores.py test
      

• To tag a raw text file:

python evaluateText path/to/txtfile

• To tag sentences in a interactive way:

python evaluate.py

or

python evaluateSentence.py

• To retrain the model from scratch:

python train.py