I am a visiting scientist at UTD working in Dr. Stephen Levene’s group. I started working with the group after obtaining a PhD in Biomedical Engineering (also at UTD). Our research focuses on modeling the physical behavior of DNA strands and their topology especially in their interaction with enzymes that affect DNA topology. The simulations that we run rely largely on MC serial runs on a medium to large scale.
Experience with HPC :
I have used HPC systems before, but I am by no means an expert. I have mainly used the systems available at UTD.
My Workflow :
Most of the code development that I do happens on my personal laptop. Our code is a mix of FORTRAN90 for the core libraries, and Python2/3 for data interpretation, analysis, and visualization. When things become more (CPU) intensive I switch to an HPC system. I am definitely guilty of doing all minor changes on the IDE from my personal laptop and rsyncing to the HPC system often. However, I do use Unix editors (vi) for minor changes.
Challenge :
We are sharing our code within the group and it gets messy, quickly. I have been trying to suggest a collaboration platform (GitHub) for storing and maintaining the code, but it hasn’t been embraced so far. I would like some help with establishing an efficient code development that works well with small groups, such as a university lab.
Greetings! We definitely feel your pain regarding collaboration. Assuming most of your collaboration is internal (at least for now), we can get you setup on our local Gitlab instance. In the future, we plan to create a public Gitlab instance that will provide some things we aren’t currently able to provide.
If you would like to setup your account, simply visit https://gitlab.circ.utdallas.edu and click on UT Dallas shibboleth login. If you need me to setup any groups, feel free to send a ticket to circ-assist@utdallas.edu.
In our group we use Box extensively to share notes, data, and files including code. It is intended for general storage, not code maintenance, but it does have crude version control and individual user permissions. Because Box is integrated into the UTD network, your group members may be more willing to adopt it.
Do you think some training would help or is it more of culture? As @solj said, we have an internal gitlab server that people can use. Sometimes people are just wary of sharing their work on the public internet (new programmers are often “embarrassed”).
On our gitlab server, you can have everything private, or public, or any combination. Longer term, we will be launching a TRECIS-branded gitlab and associated services to help with software development and documentation.
One non-traditional solution might be to try http://www.sparkleshare.org/
Sparkleshare allows you to set up a git repo on a remote server and then configure sparkleshare to work like “Box” or “Dropbox” but the backend is a git repo.
You, as a git user, can work with the repo directly but your other users (until they get trained up) can simply edit the source code directly in their local “synced” sparkleshare folder. Then, in the background, sparkleshare will do the git commits to the remote repo.
Finally, I’m teaching a class in the spring where we will cover all of this and more.
PHYS 5v48, Practical Research Computing
The class will be 100% virtual and anyone can attend the lectures and do the work (regardless if they are enrolled or not).