Model, Task and Data Engineering for NLP


With the advent of deep learning and neural methods, NLP research over the last decade has shifted from feature engineering to model engineering, primarily focusing on inventing new architectures for NLP problems. Two other related factors that are getting more attention only recently are: (i) which objectives (or tasks) to optimize, and (ii) how to better use the available data; referred to as task engineering and data engineering, respectively. In this talk, I will present our recent work along these three dimensions. In particular, I will first present novel neural architectures for parsing texts into hierarchical structures and efficient parallel encoding of such structures for better language understanding and generation. I will then present a new objective for natural language generation (NLG) tasks that aims to mitigate the degeneration issues prevalent in neural generation models. Finally, I will present effective data augmentation methods for supervised and unsupervised machine translation and other cross-lingual tasks. With empirical results, I will argue that while model engineering is crucial to the advancement of the field, the other two factors are more important to build robust NLP systems.

Background Knowledge


Shafiq Joty is an Asst. Prof. in the School of Computer Science and Engineering (SCSE) at NTU, where he leads the NTU-NLP group. He is also a senior manager of NLP research and a founding member at Salesforce AI Research Asia. His work has primarily focused on developing language analysis tools (e.g., syntactic parsers, NER, discourse parser, coherence models) and downstream NLP applications including machine translation, question answering, text summarization, image/video captioning and visual question answering. A significant part of his current research focuses on multilingual processing and robustness of NLP models. His work has mostly relied on deep learning for better representation of the input text and on probabilistic graphical models and reinforcement learning for capturing dependencies in the output. He served (or will serve) as a (senior) area chair for ACL'19-21, EMNLP'19,21 and NAACL’21, EACL’21, and a senior program committee member for AAAI’21 and IJCAI'19. He gave tutorials at ACL-2019 and ICDM-2018. He was an associate editor for ACM Transactions on Asian and Low Resource Language Processing. He has published more than 95 papers in top-tier NLP/AI conferences and journals including ACL, EMNLP, NAACL, NeurIPS, ICML, ICLR, CVPR, ECCV, ICCV, CL and JAIR.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google