Providing Python

Posted on August 4, 2014 by Catherine E. Pitt

The Python programming language has become very popular in the scientific computing world in the last few years. In Chemistry we have always provided a wide range of compilers for the widely used scientific languages Fortran and C, but our Python offerings have been more patchy. Different machines had different versions of Python available, and the same Python modules were not always available everywhere.
Recently we’ve made an effort to fix this by making a standard installation available on as many machines as we can. We’re using the Anaconda Python distribution for this. Anaconda provides an up to date version of Python along with hundreds of popular modules for scientific computing. The current release comes in both a Python 2 and a Python 3 version so both the old and new languages are supported. It’s also redistributable and comes with an excellent installer so we should be able to continue to make it available on new machines as we get them.
Currently Anaconda is available on every compute cluster other than a couple of the very old ones, and also on the Ubuntu 14.04 Linux workstations. To use Anaconda load the appropriate environment module
module add anaconda/python2
or
module add anaconda/python3
The next version of Anaconda is supposed to install even on the older machines so when that comes along we’ll install it on those too.

Posted in Software | Comments Off on Providing Python

Mathematica 10.0 now available

Posted on July 31, 2014 by Catherine E. Pitt

Mathematica 10.0 is now available to those groups who contributed to the cost of the Mathematica licence.
For those using managed Windows, Mac, or Linux workstations the software is available through the usual means. For more details or to get the installers for non-managed machines please see
http://downloads.ch.cam.ac.uk/mathematica/

Posted in Software | Comments Off on Mathematica 10.0 now available

Thunderbird Upgrades

Posted on January 23, 2014 by Adam Thorn

We’ve been having some troubles with users who have upgraded from Thunderbird 17 to Thunderbird 24 (despite the jump in numbering, that’s only one version different – Thunderbird recently changed their version numbering scheme to match Firefox). The symptom was that after upgrading, you no longer see any of your email or folders despite the account settings being all present and correct.
We manage our software centrally using wpkg, and as part of that we deploy a prefs.js file which sets a number of default preferences. One preference we set is
pref("toolkit.telemetry.prompted", true)
to disable a prompt which asks if one wants to send usage data back to Mozilla.
It turns out that Thunderbird changed this configuration value from a boolean to an integer, for compatibility with Firefox. This means that if you have an old preferences file with the boolean value, then rather than failing gracefully, Thunderbird 24 becomes totally unusable as described above. You instead need to set
pref("toolkit.telemetry.prompted", 2)
to achieve the old behaviour.
If you’re writing software and need to make a change to the format of your config file: please try to fail gracefully if you receive a config value you weren’t quite expecting!

Posted in Software | Comments Off on Thunderbird Upgrades

Homeward Bound

Posted on September 2, 2013 by Russell Currie

As part of the ongoing refurbishment of out Unilever server room, we cleared out the room. All of the sticky carpeting has been removed, the floor cleaned and treated to prevent damp seeping in. The old air conditioning has been replaced with four shiny new units. As you can see from the rack, our servers are starting their migration back home.
New Air Conditioning in UB02

Posted in Server rooms | Comments Off on Homeward Bound

Overtaken by events

Posted on July 12, 2013 by Catherine E. Pitt

We’re doing a complete refurbishment of the Unilever server room this summer. The original plan was to move all the servers out by the end of July in order for work to start in August. The room holds three compute clusters and half the servers that provide the Chemistry IT infrastructure, so we’d planned to do this gradually over a period of weeks.
Unfortunately the ageing air conditioning in the room didn’t quite make it to August. It failed last Sunday. Luckily it’s been possible to bring all the refurb work forward by three weeks. So in the last few days we’ve gone from this
UB02 full
to this
UB02 half full
to this.
UB02 empty
That last server sitting on the trolley has to be moved out of working hours, and will be gone by Monday. The rack to the left is the only thing that will remain in the room, and that’s because it’s supplying all the phones and networking to the Unilever centre.
The servers that came out of the Unilever room have been spread around the department. We’ve moved some less important machines out of the other server rooms to make space for them. Some of the displaced machines have been switched off for the duration (if that affects you then you will have had an email from us) and quite a few others have found a temporary home in a lab that’s awaiting a refurb.
Temporary server room
Now all we have to do is move them all back in a month’s time.

Posted in Hardware, Server rooms | Comments Off on Overtaken by events

Server room refurb: the sticky carpet

Posted on May 16, 2013 by Catherine E. Pitt

This summer we’re hoping to do a major refurbishment on one of our server rooms. The room has a number of things that need fixing, but perhaps the most obvious is the floor. The glue under the carpet tiles has come to the surface! The carpet is going to be removed and the floor cleaned and sealed.
Server room carpet
Other goals are replacing the ageing air conditioning and rearranging the racks to get better airflow. Hopefully we’ll remember to take pictures as things progress and post them here.

Posted in Server rooms | Comments Off on Server room refurb: the sticky carpet

VPN for iPhone and iPad

Posted on February 27, 2013 by Catherine E. Pitt

The department of Chemistry has a popular VPN service which uses software called OpenVPN. VPN is a service that allows you to log into the department network from anywhere on the Internet and then appear to the network as if you are in the department, behind our firewall.
Until recently there was a gap in our VPN provision because there was no OpenVPN client available for iOS. iPhones and iPads are popular around here so this wasn’t very satisfactory. We’d even started working on providing a second, completely different, VPN package to provide a service for them. This wasn’t the best solution: we already have one legacy VPN service and running three different services is very confusing for everyone. We were therefore very pleased when an OpenVPN iOS client was released in the App Store in February. The COs have prepared a configuration file and instructions for it which are available at http://www-co.ch.cam.ac.uk/facilities/vpn/ios-openvpn.html. So if you’re an iPhone or iPad user, please give it a go.

Posted in Remote access | Comments Off on VPN for iPhone and iPad

Buffering…

Posted on February 18, 2013 by Frank Lee

The word “buffering” often appears when watching an online video – the display pauses for a bit while the next bunch of data downloads and then the cute kittens start moving again. In this example, “buffering” is the process of using a bit of memory (the “buffer”) to smooth out the supply of data from the network before sending it to the video player – the display pauses until sufficient data has come from the network for it to be worth playing the video again.

The arrivals hall at an airport is another example of buffering – people arrive in batches, one aircraft-load at a time, and join queues snaking around barriers. From the point of view of the immigration officer, there’s a steady flow of people showing their passports. The queue has served to smooth out the very lumpy arrival of several hundred people into a steady stream.

Such buffers have a cost – it takes a certain amount of time for the first frame of the video to make it through the buffer, or the first person off the flight to get to the immigration officer – but they can speed things up immensely.

We’ve been moving big datasets around recently – several TB in size – and one thing which has sped this up has been buffering. There are three major processes: reading the data on one machine, moving the data across the network and then writing the data on another machine. All of these are subject to quite lumpy behaviour – perhaps we’re requesting some files which happen to be adjacent to each other on the disk then some files stored in quite separate places or the network isn’t used by anything else for a bit then someone starts another big transfer. By putting some buffers between each stage, we can smooth out the delays and get a speedier overall transfer.

We used mbuffer to aid in a ZFS send/recv operation, running
zfs send -R Foo/Bar/Baz@snap1 | mbuffer -s 128k -m 4G -O OtherHost:9090
on the source and
nc -l 9090 | mbuffer -s 128k -m 4G | zfs recv Foo/Bar/Baz
on the destination. This combination takes about 25% of the time of the more traditional
zfs send -R Foo/Bar/Baz@snap1 | ssh OtherHost zfs recv Foo/Bar/Baz.

Posted in technical | Comments Off on Buffering…

A tale of two lifts

Posted on January 30, 2013 by Adam Thorn

We’re currently planning a major reorganisation of one of the two main server rooms in the department, and one of the first things we needed to do is put an additional rack into the room. We had a suitable spare rack already, so it was just a case of moving it from one end of the department to the other – and also from the 3rd floor down to the basement.

So obviously, we had to put it in one of the lifts. This one, in fact:

On the left is a picture of the lift door, taken on the third floor. On the right is the door for the same lift, but this time taken in the basement. They look pretty similar, and indeed you might even expect them to be identical – we certainly assumed that! However, there’s an important difference: the doorway on the right is a few millimetres shorter than the one on the left. Getting the rack through the lefthand doorway was tight but manageable. When we came to take the rack out in the basement, though, there wasn’t quite enough space and the rack got stuck.

Thankfully we had the toolbag with us, and were able to quickly remove enough bits of metal from the rack to let us get it out of the lift. The rack is now sitting happily in its new home, waiting for us to fill it with servers…

….and apologies to anyone who was wondering why the lift took longer than usual to arrive a few days ago!

Posted in Server rooms | Comments Off on A tale of two lifts

Hobbiting

Posted on January 29, 2013 by Adam Thorn

We use a system called Hobbit to monitor all of the computers in the department. (Catherine wrote about this system not too long ago). In fact, that’s what drives this webpage which tells you the current status of all the IT services in Chemistry. This lets us see all sorts of interesting things about our infrastructure, and it produces lots and lots of graphs!

For example, here’s a graph showing the disk I/O activity on one of the servers which runs multiple Xen virtual machines:

The y-axis on the graph is “transactions per seconds” – roughly, the number of requests to write or read to the disk. The one that’s been interesting me recently is that green line, averaging around 1000 transactions/sec pretty constantly. The fact that the server concerned is constantly accessing the disk is slightly odd.

(Oh, and in case you were wondering, the regular large spikes at midnight in the yellow line arise from a cron job which syncs software packages to some of the clusters in the deptartment.)

It turns out that not only did our Hobbit monitoring let us identify that this is going on, but also it was the cause of all the disk accesses! One part of how Hobbit works is that there’s a client installed on each computer being monitored, and one of the things the client does is scan through the log files so it can report any errors back to the Hobbit server. However, the log files on this server had grown rather big, and so each time the Hobbit client scans them it leads to lots and lots of disk reads – as you can see in the graph above!

As you might be able to tell from the graph, some time this morning I fixed the settings which will now ensure that the log file doesn’t grow to too large a size – the green line dropped pretty much instantly to zero after making the changes, which confirms that the change had the desired effect. So hobbit turned out to simultaneously be the cause of the problem whilst also guiding us towards a solution.

Posted in Linux | Comments Off on Hobbiting