Seminar – Phong Le, Amazon – 3rd March 2021

Can Language Models be Weak Annotators

We are happy to have Phong Le, from Amazon, talk on Teams on Wed 3 March at 12 noon on Teams.


Deep language models e.g. BERT and GPT3 are the breakthrough in Natural Language Processing in the last 3 years. Being trained on massive raw text data, they capture useful priors for several tasks such as syntactic parsing, information extraction, and question answering. Moreover, they are capable of answering factual and commonsense cloze questions such as “Dante was born in _____”. In this talk, I will firstly give an overview about what language models “know”. I will then present our work on exploiting their knowledge as weak supervision for a specific task called relation classification.

Relation classification, the identification of a particular relation type between two entities in text, requires annotated data. Data annotation is either a manual process for supervised learning, or automated, using knowledge bases for distant learning. However, both methodologies are costly and time-consuming since they depend on intensive human labour for annotation or for knowledge base creation. Using language models as annotators, on the contrary, is very cheap but the annotation quality is low. We hence propose NoelA, an auto-encoder using a noisy channel, to improve the accuracy by learning from the low quality annotated data. NoelA outperforms BERT and a bootstrapping baseline on TACRED and reWIKI datasets.

Bio: I’m an applied scientist at Amazon Alexa. Before that, I was a tenure-track research fellow at the University of Manchester. I did a postdoc with Ivan Titov at the University of Edinburgh, and got a PhD from the University of Amsterdam under the supervision of (Jelle) Willem Zuidema. I’m interested in neural networks and deep learning. My current work is to employ them to solve natural language processing tasks such as entity linking, coreference resolution, and dependency parsing. I’m also interested in formal semantics, especially learning semantic parsing.

For more details, please visit my homepage


Please note the session will not be recorded, to preserve the like-for-like nature of physical seminars and also avoid any privacy/rights issues.

Event details

  • When: 3rd March 2021 12:00 - 3rd February 2021 13:00
  • Format: Seminar

Seminar: ‘Formalizing Garbage: Mathematical Models of Memory Management’ by Jeremy Singer


Garbage collection is no longer an esoteric research interest. Mainstream programming languages like Java and C# rely on high-performance memory managed run time systems. In this talk, I will motivate the need for rigorous models of memory management to enable more powerful analysis and optimization techniques. I will draw on a diverse range of topics including thermodynamics, economics, machine learning and control theory.


Jeremy Singer is a lecturer at the School of Computing Science, University of Glasgow, Scotland. He has research interests in programming languages,compilation, run time code optimization and memory management. Singer received his PhD from Cambridge in 2006. Website:


Event details

  • When: 6th October 2015 14:00 - 15:00
  • Where: Cole 1.33a
  • Series: School Seminar Series
  • Format: Seminar, Talk

School Seminar: Complex Networks and Complex Processes

Simon Dobson, School of Computer Science, University of St Andrews


Complex networks provide a way of modelling systems with lots of
dependent elements, such as traffic networks or social networks. By
running processes over these networks we can explore how the topology of
the network affects the way the process evolves, and potentially
identify factors that accelerate or impede it. This opens-up
possibilities both for study (science) and control (engineering).

This talk will briefly introduce the mechanics of complex networks and
the processes that run on them, review some recent results we have
obtained, and look to future research programme where we will combine
simulation with sensing to give us new ways of looking at the world.

Event details

  • When: 4th November 2014 14:00 - 15:00
  • Where: Cole 1.33
  • Series: School Seminar Series
  • Format: Talk

The Design and Implementation of Feldspar

By: Josef Svenningsson, Chalmers University of Technology, Sweden

Feldspar is a domain specific language with the goal of raising the
level of abstraction for performance sensitive, low-level code.
Feldspar is a functional language embedded in Haskell, which offers a
high-level style of programming. The key to generating generating
efficient code from such descriptions is to use a high-level
optimisation technique called vector fusion. Feldspar achieves
vector fusion for free by employing a particular way of embedding the
language in Haskell by combining deep and shallow embeddings.

Bio: Josef Svenningsson is an Assistant Professor in the Functional
Programming group at Chalmers University of Technology. He has a broad
range of interest and has published papers on wide variety of topics,
including: program analysis, constraint solving, security, programming
language design, testing and high-performance computing.

Event details

  • When: 21st October 2014 14:00 - 20th October 2014 15:00
  • Where: Cole 1.33
  • Series: School Seminar Series
  • Format: Seminar