Can Language Models be Weak Annotators
We are happy to have Phong Le, from Amazon, talk on Teams on Wed 3 March at 12 noon on Teams.
Deep language models e.g. BERT and GPT3 are the breakthrough in Natural Language Processing in the last 3 years. Being trained on massive raw text data, they capture useful priors for several tasks such as syntactic parsing, information extraction, and question answering. Moreover, they are capable of answering factual and commonsense cloze questions such as “Dante was born in _____”. In this talk, I will firstly give an overview about what language models “know”. I will then present our work on exploiting their knowledge as weak supervision for a specific task called relation classification.
Relation classification, the identification of a particular relation type between two entities in text, requires annotated data. Data annotation is either a manual process for supervised learning, or automated, using knowledge bases for distant learning. However, both methodologies are costly and time-consuming since they depend on intensive human labour for annotation or for knowledge base creation. Using language models as annotators, on the contrary, is very cheap but the annotation quality is low. We hence propose NoelA, an auto-encoder using a noisy channel, to improve the accuracy by learning from the low quality annotated data. NoelA outperforms BERT and a bootstrapping baseline on TACRED and reWIKI datasets.
Bio: I’m an applied scientist at Amazon Alexa. Before that, I was a tenure-track research fellow at the University of Manchester. I did a postdoc with Ivan Titov at the University of Edinburgh, and got a PhD from the University of Amsterdam under the supervision of (Jelle) Willem Zuidema. I’m interested in neural networks and deep learning. My current work is to employ them to solve natural language processing tasks such as entity linking, coreference resolution, and dependency parsing. I’m also interested in formal semantics, especially learning semantic parsing.
For more details, please visit my homepage https://sites.google.com/site/lephongxyz/
Please note the session will not be recorded, to preserve the like-for-like nature of physical seminars and also avoid any privacy/rights issues.
- When: 3rd March 2021 12:00 - 3rd February 2021 13:00
- Format: Seminar
The School of Computer Science is offering the following scholarships for 3.5 years of study in our PhD programme. All UK/EU and International students are eligible:
• 6 fully funded scholarships consisting of tuition + stipend
• 6 additional tuition-only scholarships
This award is part-funded through the University’s new ‘handsels’ scheme.
Value of Award
• Tuition scholarships cover PhD fees irrespective of country of origin.
• Stipends are valued £15,285 per annum.
We are looking for highly motivated research students willing to be part of a diverse and supportive research community. Applicants must hold a BSc or MSc in Computer Science or related area appropriate for their proposed topic of study.
International applications are welcome. We especially encourage female applicants and underrepresented minorities to apply.
22 January 2021 for scholarship eligibility. Late applications will be considered if funding allows.
How to Apply
Every PhD application indicating interest, if accepted, will automatically be considered for these scholarships; there is no need for a separate application.
The best way to win one of our scholarships is to make a strong PhD application. You are also encouraged to approach supervisors before formal submission to discuss your project ideas with them.
The School’s main groups are Artificial Intelligence and Symbolic Computation, Computer Systems and Networks, Human-Computer Interaction, and Programming Languages. It is highly recommended that applicants identify potential supervisors in their applications. A list of existing faculty and areas of research can be found at https://www.st-andrews.ac.uk/computer-science/prospective/pgr/supervisors/).
Full application instructions can be found at https://www.st-andrews.ac.uk/study/apply/postgraduate/research/.
Inquiries and questions may be directed to firstname.lastname@example.org.
The second school seminar on 5th November at 2pm, on Teams. If you do not have the Teams link available please contact the organiser, Ian Gent.
The work on deep learning based understanding of ancient coins by Jessica Cooper, who is a Research Assistant and a part-time PhD student supervised by Oggie Arandjelovic and David Harrison has been chosen as a featured, “title story” article by the Journal Sci where it was published in a Special Issue Machine Learning and Vision for Cultural Heritage.
Congratulations to Zoë Nengite who has been awarded The Principal’s Medal in recognition of outstanding academic achievement and exceptional activities within the University and the wider St Andrews community. The Medal is awarded to students who have both excellent academic accomplishments and those who have inspired and supported their peers and who have often undertaken extensive advocacy work, which has improved life for many of their fellow students.
Zoë sent us a reflection on time spent studying in the School and a photo celebrating with Mum.
“I’m really sad that my time at St Andrews has come to an end. I will especially miss the School of Computer Science. We are such a close community of students and staff alike. I will even miss the Jack Cole labs, despite spending many hours with my head in my hands stuck on a problem gripping my mug of coffee. I always knew that help wasn’t too hard to find.
“Some of my best memories are from my time at St Andrews. Most of them spent with my closest friends who also studied Computer Science. Coming from London, I was apprehensive about St Andrews, but it quickly became a place I called home. I think even years from now, it will always be somewhere I call home.”
The award was announced during the virtual conferral of degrees in July. Zoë hopes to attend a rescheduled Class of 2020 Ceremony in the future where we look forward to celebrating with her in person.
MANAGING OPEN SOURCE PROJECTS ON GITHUB — SUCCESS FACTORS AND PERFORMANCE INDICATORS
As a part of my, Julia Seeger’s, MSc Dissertation in the School of Computer Science at the University of St Andrews I am looking for volunteers for an interview. This interview is a part of a research project focussed on success factors and performance indicators of managing open source projects hosted on GitHub.
I am looking for core contributors to open source projects hosted on GitHub. Ideally, the project should have configured and make use of Travis CI, and should have a history of pull requests before and after the configuration of Travis CI.
I would firstly be interested in your opinion about success factors and performance indicators that I have identified by analysing the public GitHub repository of your project with the help of the GitHub API. I will ask if, as a core contributor of the project, you would agree or disagree with my findings. Secondly, I am interested in your personal experience in managing a repository of an open source project on GitHub, and the factors and managing techniques you identified to be important for a successful project.
The interview will take place in a form of a video or an audio call via Skype for Business or Microsoft Teams. The interview will take place during July 2020, consists of 6 questions and will last around 25 minutes. If you agree to participate, questions will be given to you at least three days in advance.
If you are willing to participate, please get in touch using the contact details below. You will then be given a Participant Information Sheet that further details my research, and will have the opportunity to ask questions, before being asked whether you consent to participate.
Researcher: Julia Seeger
Supervisor: Dr. Alexander Konovalov