Summer School on Experimental Methodology in Computational Science Research
St Andrews, Scotland, UK
Monday 4 August to Friday 8 August 2014
The purpose of this summer school is to bring together interested computer scientists and other researchers who work in the broadly-defined area of “computational science”, and to explore the state-of-the-art in methods and tools for enabling reproducible and “recomputable” research. Reproducibility is crucial to the scientific process; without it researchers cannot build on findings, or even verify these findings. The development and emergence of new tools, hardware and processing platforms means that reproducibility should be easier than ever before. But to do so, we also need to effect “a culture change that will integrate computational reproducibility into the research process”.
The school will be hands on, comprising lectures, tutorials and practical sessions in topics including statistical methods, using cloud computing services for conducting and sharing reproducible experiments, methods for publishing code and data, legal issues surrounding the publication and sharing of code and data, and generally the design of experiments with replication in mind. Speakers include academics from mathematics, computer science and law schools, and other researchers and industrial speakers from Microsoft Azure, the Software Sustainability Institute and more. Practicals will include the replication of existing experiments and a “hackathon” to improve tools for replication. The aim of the school will be to create a report that will be published in arXiv by the end of the week, and in a suitable journal later on.
- Steve Crouch (Software Sustainability Institute): Developing Sustainable Research Software – why is it important and what resources are out there?
- Ian Gent (St Andrews): Reproducibility in Computer Science
- Tristan Henderson (St Andrews): Data Sharing: the good, the bad and the ugly
- Neil Chue Hong (Software Sustainability Institute), Better Software, Better Research: Why reproducibility is important for your research
- Darren Kidney (St Andrews): Data Analysis using R
- Alexander Konovalov (St Andrews): Using Microsoft Azure (Tutorial)
- Lars Kotthoff (University College Cork): Recomputation
- Miguel Nacenta (St Andrews): Experimental Methodology in HCI Research
- Burkhard Schafer (Edinburgh): Legal Issues in Computer Science
- Kenji Takeda (Microsoft Research Cambridge), Reproducible Research and the Cloud
The intended audience of the summer school is researchers doing computational experiments, or who might do so in the future. One audience is therefore almost all computer scientists, but we also welcome scientists from other disciplines in which computational experiments play a key role – which is almost all of science! The school is open to all researchers interested in computational scientific experiments. PhD students, Postdoctoral researchers, academic faculty, and non-academic researchers are all welcome to attend. Participants are expected to have familiarity with programming (in at least one programming language but not a specific one), version control systems such as git or mercurial, and UNIX command line interfaces.
Bring an Experiment!
Participants are expected to bring to the school the suggestion of a computational experiment to be reproduced as part of the summer school. Requirements for experiments to bring are that:
- The experiment should be either from a published paper or, if unpublished, you should have the agreement of all authors concerned for this activity including the publication of a replication.
- The paper should appear to be reproducible without major domain-specific work at the summer school. This condition is met if, for example, all code concerning the paper is available on an open source basis. But this summer school is not the place for re-implementing specialist algorithms.
At the end of the summer school, replications will be published and (where possible) released on recomputation.org.
At the end of the week, participants will work in groups to build tools or resources to encourage reproducible experiments and to make them easier to perform.
During the week of the Summer School, the staff and participants of the Summer School will collaborate on a paper to be published on arXiv before the end of the School on Friday. All participants will (if they wish) be authors of this paper, and it will be a report on the week, the success or otherwise of replications, and tool descriptions from the hackday. If appropriate, this paper will be submitted for publication following any post-conference polishing.
The summer school will take place in a well equipped Computer Science department. One desktop machine will be available per participant, and participants are also welcome to bring their own laptops, for which wifi is available. Thanks to the generosity of Microsoft Research, a 6 month Microsoft Azure pass is available free to all participants. Residential accommodation is a single room, with shared bathroom facilities. It should be possible to arrange options such as staying before or after the summer school. However there is no discount for arriving late or leaving early.
The full residential cost will be £500 including the summer school itself, accommodation, breakfast, lunch and dinner for the whole duration of the school. This includes accommodation on the Sunday night before the school starts, and the Friday after it ends, so you can work hard all week!
There will be also a non-residential option for those willing to commute each day or arrange their accommodation personally. It will cost £250 and will also cover lunches.
Sponsorship from SICSA means that the school will have a limited number of free places available for Scottish PhD students. These will be allocated using a selection procedure details of which will be published at the time of opening the registration.
We gratefully acknowledge support from
- SICSA for providing support for Scottish PhD students.
- The Software Sustainability Institute, a UK national facility for cultivating world-class research through software, for support with preparation of the school.
- Microsoft Research providing a free 6-month Microsoft Azure pass for each participant.
- The School of Computer Science at the University of St Andrews for providing facilities and staff time.
A limited number of free places are available to PhD students in Computer Science at Scottish Universities. Students wishing to apply for one of these places should not register online but should email to emcsr2014 [at] cs.st-andrews.ac.uk requesting a free place, and also ask their PhD supervisor to email a recommendation that they should take part. Applicants in this category should apply by 20 June. UPDATE!!! We have informed first successful applicants about their places at the school. There are still two more places left, please get in touch with organisers as soon as possible if you are interested.
Otherwise, for the online registration for the event and paying full or non-residential costs, please use the EMCSR 2014 entry in the University of St Andrews Online Shop. The deadline for the full registration is July 18th. After that date we may only be able to provide some flexible arrangements for the non-residential registration.
The summer school will be held in the School of Computer Science at the University of St Andrews. St Andrews is a beautiful town on the east coast of Scotland. Known around the world as the birthplace of golf, St Andrews also boasts clean and attractive beaches, historical buildings and a dry climate (by Scottish standards!). More information can be found here.
Email: emcsr2014 [at] cs.st-andrews.ac.uk