Transformer Based Approaches to Named Entity Recognition (NER) and Relationship Extraction (RE)


Named Entity Recognition (NER) and Relationship Extraction (RE) are foundational for many downstream NLP tasks such as Information Retrieval and Knowledge Base construction. While pre-trained models exist for both NER and RE tasks, they are usually specialized for some narrow application domain. If your application domain is different, your best bet is to train your own models. However, the costs associated with training, specifically generating training data, can be a significant deterrent for doing so. Fortunately, Language Models learned by pre-trained Transformers learn a lot about the language of the domain it is trained and fine-tuned on, and therefore NER and RE models based on these Language Models require fewer training examples to deliver the same level of performance. In this workshop, participants will learn about, train, and evaluate Transformer based neural models for NER and RE.

Session Outline
"* Background (20 mins)
* Problem Definition
* Overview of previous (non-neural) approaches to NER and RE
* Named Entity Recognition (1 hour)
* Neural and Transformer based architectures for Named Entity Recognition
* Hands-on case study – train a Transformer based NER using the Groningen Meaning Bank (GMB) dataset to predict entities such as person, organization, geographical entity, etc.
* Relationship Extraction (1 hour)
* Neural and Transformer based architectures for Relationship Extraction
* Hands-on case study – train a Transformer based RE model using the New York Times Relation Extraction dataset to predict relations such as nationality, place of birth, company founder, etc.
* Conclusion (25 mins)
* Applications of NER and RE
* Q/A session"

Background Knowledge
* Languages -- Python (intermediate)
* Tools -- PyTorch (intermediate), HuggingFace Transformers (intermediate)
* Methodologies -- Natural Language Processing (intermediate)


Sujit Pal builds intelligent systems around research content that help researchers and medical professionals achieve better outcomes. His areas of interest are Information Retrieval, Natural Language Processing and Machine Learning (including Deep Learning). As an individual contributor in the Elsevier Labs team, he works with diverse product teams to help them solve tough problems in these areas, as well as build proofs of concept at the cutting edge of applied research.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google