In January 2016, I’ve asked on the Mathematics Q&A site the question called “Most wanted reproducible results in computational algebra“. I hope that making a list of suggested experiments to reproduce will be useful to those interested in checking them twice ;-). For example, one could submit their findings to a journal like ReScience which “targets computational research and encourages the explicit replication of already published research, promoting new and open-source implementations in order to ensure that the original research is reproducible”. If you have any suggestions, please consider posting them as answers to that question. Today I’ve also added my own answer, which I am reproducing below.
I believe that enumeration of finite groups of a given order is definitely among most wanted reproducible experiments. Here “enumeration” means providing complete and non-redundant list of groups, “complete” means that no groups are missing in this list, and “non-redundant” means that groups from this list are non-isomorphic pairwise. Guaranteeing these properties is crucial for results that rely on checking all groups of a given order, or that refer to a particular group by its “catalogue number”. (more…)
How do you denote the dihedral group of order n?
This is not a question of “vim or emacs” or “tabs or spaces” sort, but still there are two notations in use. Some sources use Dn to denote a dihedral group of order n, and some others – a dihedral group of order 2n. Both notations are also reflected in mathematical software.
For example, in GAP
DihedralGroup(n) returns a group of order n:
and in SageMath – a group of order 2n:
Let us look at other systems: (more…)
This post is based on my earlier comment on the pull request number one submitted to the GAP repository on GutHub a little bit more than a year ago.
Of course, GAP is not new to the version control. The first revision in the CVS repository for GAP 4 is dated July 3rd 1996. Then 16 years later, in summer 2012 the repository had been converted to Mercurial (thanks to Max Horn!) after the release of GAP 4.5. In February 2015 the Mercurial repository had been converted to Git (thanks to Chris Jefferson!), and we started to host it on GitHub at https://github.com/gap-system/gap.
Below there are some highlights of what happened during the first year after that:
- On 26th February 2016 there were In total 640 pull requests and issues in this repository, namely:
- 120 open + 133 closed issues
- 32 open + 356 closed pull requests
- That’s more than one new pull request a day!
- While the ratio for open/closed issues is nearly 50:50, “if things are done, they are done”: pull requests are normally reviewed and getting merged, so the ratio open/closed for pull requests is about 1:10.
- We had several very productive GAP Days in Aachen, Trondheim and St Andrews.
- We have a gap-system virtual organisation on GitHub which also hosts repositories for other development tools, GAP Website, etc.
- We have an expanding virtual organisation for gap-packages.
- There is even more activity with GAP packages, since some of them are openly developed elsewhere (we are trying to keep an up-to-date list at this page).
- There is an active open GAP development mailing list.
- There is also a growing Wiki with documentation for developers.
- In 2015, we made two minor releases based on the stable-4.7 branch of the GitHub repository (4.7.8 and 4.7.9).
- We put a lot of efforts in making GAP more robust and test it better, in particular, we now run nightly check of the code coverage and publish code coverage reports.
- And we have just announced the first major release of GAP 4.8.2, which is the first major release of GAP that has been made from this repository! The overview of changes between GAP 4.7 and GAP 4.8 highlights most significant ones, including support for partially variadic functions, better tools for testing, debugging and profiling, function calls with timeouts, 12 new packages redistributed with GAP, and other changes.
Congratulations and thanks to everyone who worked on making all of this possible!!!
In addition, this visualisation, produced with gource, shows files and directories that were modified in the master branch of the core GAP system during the 1st year on GitHub. The large cluster in the top right corner corresponds to the GAP library. The top left corner is occupied by the GAP kernel. The pink cluster in the bottom represents GAP regression tests, and the main GAP manuals are marked by green. There are already changes waiting for GAP 4.9!
GAP 4.8.2 includes 130 GAP packages, and I am amazed how many of them are in the working order in the container – including all those requiring compilation, with non-trivial dependencies on external libraries and other third-party software. I have previously used the container for GAP 4.7.9 on an Ubuntu machine as a reference GAP installation, and this morning I was thrilled to upgrade it to GAP 4.8.2 using just two commands and waiting for no more than a couple of minutes!
I still haven’t seen GAP Docker container running on Windows – if anyone manages to do that, I would be very glad to know. This would be very beneficial for users of some packages working only in the UNIX environment.
The GAP Bibliography currently contains 2217 records. Among others, it contains 1961 journal publication (the rest are books, PhD theses, preprints, etc.). These journal publications represent 331 journal, but almost a half of them (1042 papers) are published in the following 20 journals:
- J. Algebra : 258
- Comm. Algebra: 137
- J. Symbolic Comput.: 88
- (*) Discrete Math.: 66
- (*) Experiment. Math.: 42
- J. Group Theory: 41
- Internat. J. Algebra Comput.: 41
- (*) European J. Combin.: 38
- (*?) LMS J. Comput. Math.: 37
- (*) J. Pure Appl. Algebra: 37
- J. Algebra Appl.: 36
- (*) Des. Codes Cryptogr.: 35
- Arch. Math. (Basel): 33
- Math. Comp.: 26
- (*) J. Combin. Theory Ser. A: 25
- Electron. J. Combin.: 25
- Israel J. Math.: 22
- J. Combin. Des.: 20
- J. Algebraic Combin.: 18
- Adv. Math.: 17
It’s interesting, however, which of these journals suggest publishing code together with the paper? I have marked with (*) journals where the guidelines for authors (click on the journal title to see them) at least mention an opportunity to submit supplementary material, though the wording used may be quite different. For example, Elsevier’s Discrete Math., European Journal of Combinatorics, J. Pure Appl. Algebra, J. Combin. Theory (Ser.A) and Advances in Mathematics all say “Supplementary files offer the author additional possibilities to publish supporting applications, high-resolution images, background datasets, sound clips and more”, so the code could go under “more”. Experimental Mathematics is more explicit in suggesting that “detailed code can be submitted as a computer supplement”.
There is no information available at the moment for the LMS Journal of Computation and Mathematics, which is currently closed for new submissions, awaiting for the LMS Special General Meeting tomorrow, but there were some in the past. You may see an example how they worked in the paper by Meinolf Geck linked from “In which journals should I publish my software” post by Neil Chue Hong of the Software Sustainability Institute.
Finally, Designs, Codes and Cryptography accepts “electronic multimedia files (animations, movies, audio, etc.) and other supplementary files to be published online along with an article or a book chapter”. In particular, it says that “specialized format such as .pdb (chemical), .wrl (VRML), .nb (Mathematica notebook), and .tex can also be supplied”.
I haven’t discovered suggestions to submit the code in other journals from the list above. I am happy to be corrected in case I’ve overlooked them, but in this case other authors could probably overlook them too…
This week I was mainly wearing my Research Software Engineer (RSE) hat. RSE support for the GAP system has three related strands: development of the core GAP system; support of package authors and other GAP users; training future users and contributors to the system. They are all important for the thriving community of users and developers, and this week I’ve done something to advance each of them.
I’ve recently developed the Software Carpentry lesson “Programming with GAP“. This lesson is intended for GAP beginners and has been beta-tested at the Software Carpentry Workshop that we run in Manchester as a part of the First CoDiMa training school in Computational Discrete Mathematics. The lesson itself is available here, and its repository is located here.
What next? I can teach my lesson myself, but is it written clearly enough to be taught by others? Is it possible for the reader to follow it for self-study? Is there any introductory material missing, or is there an interest in having more advanced lesson(s) on some (which?) aspects of the system? If you would like to contribute to its further development, issues and pull requests on GitHub are most welcome!
What will happen if we will declare
Random as an attribute? Let’s try!
First we will create new attribute
MyRandom for objects in the category
MyRandom := NewAttribute("MyRandom", IsCollection);
Now we install a generic method (more…)
I analysed 185 papers papers citing GAP, published in 2014-2015 and contained in MathSciNet as on August 18th, 2015. The diagram below (click on the picture to enlarge) shows the number of papers citing GAP releases by the year of the release. It also shows releases that were published (though not necessary cited in the analysed papers) in corresponding years.
Some facts discovered:
- 44% (81 paper) refer to GAP 4.5.4 (the 1st public release of GAP 4.5 in June 2012) or some newer version.
- 24% (44 papers) cite GAP 4.4.12 (December 2008), which was the stable GAP release for several years until the appearance of GAP 4.5.
- 14 papers cite some GAP package but not GAP, so this only gives some hints to the version of GAP that might be used.
- 9 papers cite GAP without GAP version and/or release year.
- 19% (35 papers) cite GAP 4 releases published during 2000-2007.
- 1 paper genuinely cites GAP 3.4.4 because it uses the CHEVIE package.
It is not surprising that there is a long tail of citations of older versions. Papers that appeared in 2014 may be submitted prior to GAP 4.5 release, or, even if they were submitted after, describe experiments that were performed using GAP 4.4.12. But it is interesting what’s behind those 35 citations of the older GAP releases. Are these real usages of outdated GAP installations, re-used bibliographies from earlier papers, or something else?
Specifying the exact version of the system (as well as specific GAP packages, if their use was essential) is very important for the reproducibility of computational results. Furthermore, each GAP release contains new features and bugfixes, and comes with updated and expended set of GAP packages (see the history of GAP releases). Therefore, we suggest to always try to use the latest release, and to cite it correctly (GAP function Cite may help with that).
- Use command line for object exploration and prototyping, then write the proper code.
- Use LogTo to save inputs and outputs to a logfile which then you can edit in a text editor.
- Save code in text files and use Read to load them. Find some advanced text editor to edit these files.
- Make your code modular and reusable by organising it into functions.
- Write comments in the code – this will help you when you will return to it after a while.
- Use command line history instead of retyping commands! It even could be made available in the next GAP session.
- Understand break loops: you may explore variables on the current break level and use Where to show the last commands before the error occurred.
- Use preferences mechanism to customise GAP.
- Understand the theory behind calculations: theoretical improvements could improve the performance much more than optimised code for brute-force calculations.
- Cite GAP if you use it! This helps for the community to grow, and this will help you.