Typed with a slightly dodgy keyboard into Arcturus at JISC Managing Research Data Meeting

[I’m at a 2-day meeting run by Simon Hodson of JISC on Managing Research Data – it’s a good meeting with about 12 JISC projects represented and epitomises how JISC funds a wide range of activities which are designed to test out new infrastructures, develop new tools and protocols, collect key info etc. I have been impressed by the presentations. #jiscmrd. But that’s not what I’m blogging about.]

In the bar and dinner I ranted on about the increasing power of commercial organizations in grabbing our digital outputs from academia and more generally the cultural commons. I believe that funded academic research is worth about 500billion USD – I can’t get an accurate figure – and that much of this is being grabbed in a digital gold rush. Is this scaremongering? I don’t think so, if we look at the signs. The dissemination of this information is not under the control of the institutions – much is under the control of income-driven publishers for “full-text” .

I am using DRM as an example in this post but it could apply to many other restrictions. I am appalled to see the widespread use of Digital Rights Management technology on academic output – this could easily become the norm, just in the way that video cameras in our streets and everywhere else are the norm. It may be possible to make a case for DRM (I have an open mind), but this case has not been made. We are hearing about data management and exposure. If we don’t get this right we shall lose another slice of the digital commons.

The common response when I ask about the restriction of access to knowledge is “I can’t do anything. It’s the fault of the academics/library/vice-chancellors/publishers [strike out the one you belong to].” But in the Internet age everyone can do something. It may be small but it’s critical. So here are steps to change the world in the digital age:

1. Care passionately. I find that few people appear to care about the encroaching digital grab. If you don’t care, then you can’t do anything. Practice caring, each day. Try reading a one-page outline of DRM in academia : http://www.openrightsgroup.org/ourwork/speeches/google-books-hearing
2. Tell the world. If we don’t know what the problems are, then we can’t do anything about them. Alert us to the next restriction to knowledge. You can do this anonymously. If anyone mails me or my blog I won’t reveal names.
3. Find other like-minded people. In the Internet age this is easy. Just type “digital rights management in academia” into Google and you’ll find the openrights group.

That’s a simple start. We’re not asking for heroes. No-one (yet) has been sacked for raising issues like DRM.

If you care more passionately you can do something. Here are some simple things:

• (perhaps jointly) Write a responsible, well-argued position paper. What are the effects of DRM? Clearly argued cases are worth more than rants
• Bring it up within your organization. This takes courage. If you don’t have the right balance of passion/courage, don’t do it.
• Bring it up externally. This takes courage. If you don’t have the right balance of passion/courage, don’t do it.
• Find the facts. Public bodies in UK are governed by FOI. It’s their legal duty to reply. Again it takes courage. If you don’t have enough of this, I’ll do it on your behalf (which I am currently doing for 2 “deepthroats”).
• Write to your MP or MEP. This is your right, and many believe, your duty. The MySociety tools are superb.
• Create alternative information tools. A classic example of this is openstreetmap.org . One person’s passion has been transformed into a gut-wrenching superb resource by hundreds of thousands of volunteers. Rather than complaining that map data is owned by X, Y, Z and we can’t do anything, go out and create 0.0001% or Open streemap. And it’s healthy. Of course it’s not easy to spot the opportunities, but they are there. OKF has created Open Shakespeare. We could, by volunteer activity, pull zillions of books out of copyright into digital form. If you want to make it happen it can.

Women in the UK have the vote. That’s partly because people like my grandmother campaigned for the vote. Uncomfortable, and some went to prison. But that is how freedom is won. In the Internet age the battle is different and so are the weapons.

Just care enough!

Typed/scraped with a slightly dodgy keyboard into Arcturus

[I have been catalysed to write an apologia. An Apologia is the defence of a position, not a plea for forgiveness. See http://en.wikipedia.org/wiki/Apologetics ]
I have just discovered that the Techrights blog has been following this blog, and particularly the discussion on the BL. I haven’t read back-issue posts and so may not address all the points that Techrights have covered. I take Techrights very seriously and clearly have a great deal in common. I expect to come back to this discussion so the current reply is a first pass. This post sets out the basis of my position on various aspects of Openness. As with any single human I have imperfect knowledge of the issues and I have to make personal judgments and trade-offs – I do not have the clarity of purpose of a Gandhi.
Techrights specifically concern my Microsoft connection:

http://techrights.org/2010/05/15/british-library-and-trust/

Posted in DRM, Europe, Intellectual Monopoly, Microsoft at 6:28 am by Dr. Roy Schestowitz

Summary: An open access hero asks whether or not he should work with Microsoft; Techrights responds

“Should I work with Microsoft?”

That is the question asked by Peter Murray-Rust, who sought a reply from readers. We’ll come to this in a moment, right after providing some background.

“Microsoft loves artificial scarcity, on which its entire business model is built.”Peter Murray-Rust is an admirable person and a high achiever. He is at the forefront of the open access (OA) movement and he advocates strongly the sharing of knowledge. Here at Techrights we have great respect for this man, but circumstances at work seem to have led him to the biggest enemy of open source (and by extension one of the principal foes of OA).

PMR: An important technical clarification. In my own area of chemistry I identify four separate (orthogonal) areas. In general they are all subsumable into Open Knowledge. In the Blue Obelisk (http://www.blueobelisk.org ) of which I am a founder member we concentrate on three in chemistry: Open Data, Open Source, Open Standards (but not Open Access, primarily for pragmatic and historical reasons)

• Open Access. This is the philosophy and practice that the fruits of scholarship (primarily academic research, and emphatically publicly funded research) presented in “full-text” manuscripts and articles should be available to every person on the globe on a FaiF (free as in freedom) basis. In general there is a near-zero cost of delivery in electronic form, so these are also FaiB (free as in beer) at point of access. I support Open Access and work specifically with Open Access pubklishers such as BiomedCentral, Public library of Science, and the International Union of Crystallography (which publishes several thousand Open Access articles each year alongside closed ones). Open Access has different models (e.g. Stevan Harnad’s “green” self-archiving FaiB against the above publishers’ CC-BY FaiF). On a pragmatic basis I favour FaiF OA publishing and do not believe thqt FaiB is very useful in science [note – this will probably trigger a reply from Stevan who argues that the whole academic world should go green FaiB and only then should be address rights and re-use]. So my activities in OA are primarily confined to urging publishers to move to FaiF and CC-BY (and avoid CC-NC and CC-SA at all costs). In so far as Open Data overlaps with OA I am often invited to OA meetings.
• Open Source. Most chemical computations are done with closed source. Some of the companies are at least as monopolist as MS. For example it is widely accepted that one leading company will sue any user of their software if they report bugs in public or publish benchmarks. I will not name them for obvious reasons. I believe that the closed source practice has seriously retarded chemistry, often leads to bad science (as the algorithms are not public or reproducible) and leads to a culture of supine mediocrity. As a member of the editorial board of BMC’s J. Cheminformatics I am trying to change the approach to closed systems – I have yet to carry a majority of the editorial board with me. The Blue Obelisk is working to replace the closed source inchemistry with Open Source as we believe this leads to better science – there is encouraging evidence that our software has overcome a 15-year backlog of closed source stagnation and is now competitive with closed source offerings. The main enemy is the public apathy of the pharmaceutical industry which continues to use closed source software, primarily through inertia.
• Open Standards. This relates to the requirement that the specifications for our information is openly available. In chemistry it’s very messy as the closed source has left a serious mess of “de facto” specifications which are semi-proprietary. My own contribution (with Henry Rzepa) has been to create an Open Standard for chemical information (Chemical Markup Language, CML) – this is relevant later.
• Open Data. This is my top priority and one that gaining a lot of attention. 4 years ago the term was virtually unused – now it is ubiquitous. In my case it is the requirement to make available the data that supports a scientific paper (e.g. climate measurements, genome sequences, etc.). It is very unusual in chemistry to make data specifically Open FaiF and that is why we have developed the Panton Principles instrument for specifying this. Note that Open Data is bitterly opposed by major players in chemistry such as the American Chemical Society. When the quasi-FaiF pubchem service was started by the NIH, the SCS lobbied congress to have it shut down as unfair competition. So I have to deal with monopolists at all levels.

Peter Murray-Rust is currently pressuring the British Library to drop DRM. What he may or may not know is Microsoft’s role in opposition to his goals. What Microsoft is doing to the British Library is a subject that we covered some days ago (citing Peter Murray-Rust six times in the process). We quoted Microsoft’s CEO as saying that “DRM is the future.” One article worth reading comes from Forbes Magazine (2007) and bears the title “Microsoft: We Like DRM”.

I will address this in a later post after I have read the arguments.

By this point, we pretty much get the picture. Microsoft loves artificial scarcity, on which its entire business model is built.

Peter Murray-Rust continues his fair and balanced struggle to rid the British Library of DRM. He speaks to an ex-Adobe person:

[… posted earlier on this blog …]

This is excellent stuff.

As we pointed out on some past occasions (11 separate posts in Techrights cite his blog), Microsoft occasionally works with Peter Murray-Rust’s employer or affiliates

[PMR: for the record. I used to be employed by the University of Cambridge which has accepted considerable donations from Microsoft and also allowed Microsoft Research to be located on the West Cambridge University site. I now am employed by Churchill College in the University of Cambridge, but continue to accept the moral responsibility of working within a University which has accepted MS funding.]

, which makes it hard for him to have Microsoft estranged. Very recently he published this post which asks, “Should I work with Microsoft?”

[… PMR snipped]

Let it be clarified that Microsoft is probably worse than before. It just employs a lot of PR agencies to embellish its image and bury criticisms. We have given a very large number of examples and it would require a long time to assemble a long list that convinces the reader/observer (a lot of unethical PR efforts, propaganda, bribed bloggers, and AstroTurfing as exemplified in the previous post are partly documented).

Microsoft’s latest strand of offences is racketeering [1, 2, 3, 4, 5, 6, 7]. People must not be misled by memes like “new Microsoft” because this new Microsoft is more aggressive than ever and more dishonest about its intentions (it’s part of the PR spiel, along the lines of a green BP logo or Blackwater changing names).

No proponent of OA should befriend Microsoft. To repeat what we wrote yesterday, “dissociation from Microsoft is a survival skill, not intolerance.” █

PMR: I shall read these and then comment. It may not be immediate. I have no first-hand knowledge of Microsoft’s involvement in the BL’s DRM (if any). I do know they are working with the BL on research information systems. However I regard DRM as a potentially serious problem.
In my own area the issues are intertwined. Microsoft is funding Chem4Word in our grouop which from the start was planned to promote Open Data and Open Standards (in chemistry). We acknowledged it was a hybrid in that it relies on a closed source platform (.NET). It can be argued (and I would be convinced by sales figures) that it helps to sell Microsoft systems in chemistry. During the project Microsoft agreed that the whole of the Chem4Word project (though not the underlying .NET, of course) would be Open Source and this has now available at Codeplex. The primary document standards that the project uses are Chemical Markup Language and OOXML (whose genesis has been heavily criticized, but is an ISO standard. The route to many accepted current political positions in the world has often not been pretty).
In principle, therefore, it is possible to take the current Chem4Word software with WITHOUT Microsoft’s permission (or ours, as it is FaiF) and port it to another platform. I do not have the time or resources but would be delighted to help those who might.
Finally I have made it clear to my immediate collaborators in Microsoft that their funding does not reduce my ability to speak my mind on any issue.

Typed with a slightly dodgy keyboard into Arcturus

Late-breaking insights into DRM@BL

Henry Rzepa [Professor of Chemistry, Imperial College, and co-author of Chemical Markup Language] says:

May 14, 2010 at 12:54 pm  (Edit)

I would like the BL to clarify these points

1. DRM in the Broadcast sense that we are all familiar with (BBC iPlayer) imparts finite lifetimes to the “creative materials” (7-14 days). Science has and should not have any lifetime. Why then is the digital lifetime of the BL’s SED document 14 days?

2. The “E” of SED (secure electronic delivery) implies a digital document. Digital information is important for one reason. It can be repurposed/re-used in a context not necessarily the same as the original one. Much of our understanding of science derives from this concept. So why is the BL delivering documents where this very essence has to be immediately destroyed by by the recipient?

Under the DOI [PMR: assumed to be FOI], I would like to see if the BL has any document which both recognizes the importance of preserving digital information, and the need to do so with STM information specifically (as opposed to content generated by the “creative industries).

[PMR –

1. there appears to be no time limit in the Act and Statutory Instrument mentioned by Andrew Walker in http://wwmm.ch.cam.ac.uk/blogs/murrayrust/?p=2385

2. This is the essence of my FOI request and I hope I make it clear

2a. I shall be asking for all public documents relating to this.

And from a newcomer (deepthroat2) via deepthroat0. Deepthroat2 is in an organization doing Web 2.0 stuff in science and potentially on BL ILLs.

[deepthroat2] But that’s absolutely fascinating!! I’m almost speechless how something like this can seriously be introduced… “

Typed with a slightly dodgy keyboard into Arcturus

In my rather unsystematic research into the use of Digital Rights Management DRM by the British Library BL I have discovered a flurry of activity in 2005/6 and almost nothing until 2009. (I shall be quoting these in my FOI request) The later versions have messages such as:

The British Library Improves Electronic Access with New DRM Platform (http://www.bl.uk/news/2009/pressrelease20091126.html )

which suggests that the BL has given in to DRM and now wishes to promote it as a public good.

It was not ever thus: in 2006 they challenged DRM (APIG = All Party Parliamentary Internet Group ):

The British Library submits evidence to APIG on DRMs (http://www.bl.uk/news/2006/pressrelease20060206.html )

With excerpts such as:

It is essential that we ensure that DRMs cannot interfere with the responsibilities of the legal deposit libraries to acquire, store, preserve and give access to digital items in perpetuity. The Legal Deposit Libraries Act 2003 already provides for the delivery by the publisher of “a copy of any computer program and any information necessary in order to access the work”. It would be preferable if publications came without any DRM wrapping. The Act does not explicitly provide for the circumvention of a DRM in the event of harvest from the Internet by a library .

And:

DRMs are still in their relative infancy. Any particular DRM is likely to be short-lived. Thus DRMs are not easily susceptible to legislative regulation except at a very generic level. However, there does need to be an accepted and constantly revised code of practice for the design and operation of DRMs to ensure that they cannot constrain statutory rights in the form of exceptions and limitations of copyright.

And perhaps most tellingly of all:

Finally, the British Library is a member of Share the Vision and attaches particular importance to addressing the needs of visually impaired users in the DRM world.

It looks as if the BL has lost sight of its 2006 vision and no-one in the UK (or probably global) library community cares.

Dictated because of a slightly dodgy keyboard into Arcturus

I shall be composing my freedom of information letter to the British Library today. A week ago I asked informally on the blogosphere and twittersphere for feedback from academic librarians on the implementation, use, and impact of digital rights management (DRM) systems on the supply of academic material from the British Library (inter-library loans, ILL). This post represents what I shall write to the BL under FOI unless I get more feedback today. (I am stunned by the passive acceptance of the UK academic library system to the BL’s DRM (and presumably DRM in general), but will retain that phrase unless they can show that they have taken action to challenge the system, even if unsuccessful. It also appears that they have no public views challenging any sort of DRM and they accept what is given them).

Over the last week I have discovered that:

• The DRM systems (Adobe Digital Editions, FileOpen) had an extremely detrimental effect on the material and service received by the reader. This has been confirmed by the ex director of the Digital Curation Centre, Chris Rusbridge and the ex director of Adobe Digital Editions, Bill McCoy. The system implemented by the BL has been beset by technical difficulties which mean that many readers cannot access documents on a current setup.
• There is a markedly reduced accessibility as admitted by bill McCoy. I shall be asking the British library whether the accessibility levels are compatible with legal requirements.
• The UK academic library system apparently does not care. It has accepted the DRM system without question and without any apparent consultation either within their institutions or more widely. I have been unable to get a single comment from within the system and I have found no evidence on the Internet for any concern about the use of DRM. I therefore write this letter with imperfect knowledge of the UK academic library system, and I shall be getting further information through freedom of information requests to the individual institutions.

In writing to the British library I shall provide the following background and clarifications. My query:

• only relates to journal articles (“serials”) and not to books (“monographs”).
• only relates to material provided by the BL to members of academic institutions operated by their local library.
• only relates to the use of DRM and not to the change from paper to electronic form. I wish the comparison to be between electronic delivery (ED) and electronic delivery plus DRM (ED+DRM).
• relates to “born-digital” material (e.g. as supplied by a current scientific scholarly publisher) and material copied from paper by the BL.
• relates both to material in copyright and also presumed to be out of copyright . I shall query the basis of how the BL decides which is which.

When the BL replied to my last FOI request there was an indication that where copyright or other issues were unclear the BL imposed charges as that had the effect of maximising revenue and cost recovery.

I shall also ask about the consultation process. When the BL started to provide ED, did it:

• consider the adverse effects (technical, accessibility and public opinion) in introducing DRM
• take legal advice as to how the 1989 copyright/library act affected ED.
• consult its advisory boards and committees
• consult among UK academic libraries or receive representation from these libraries
• consult among the wider public (such as academics and other readers)
• take legal advice as to whether the DRM systems were legal (primarily accessibility).

And I shall ask for any material produced by this consultation.

I shall also ask whether suppliers of material (“publishers”) have required the BL to impose DRM delivery or whether they have set contractual conditions which can only be met by adding DRM.

I shall explain that my motivation in this FOI is:

• to redress the unacceptable situation on the ground.
• To try to redress the impaired public perception of the BL
• To lobby my new MP (who is a research scientist) and the new government to change legislation and statutory instruments to remove DRM.
• To get information on which UK academic libraries, if any, challenged the introduction of DRM and to prepare FOI requests to UK academic libraries asking for their consultation processes on DRM.
• To have clear and accurate information in my dealings with the BL in my formal and informal positions.

Dictated because of a slightly dodgy keyboard into Arcturus

I have praised the skill and dedication that Jim downing brought to our group and that is now permanently embedded in its culture. This has been to take a group of committed scientists with self taught computer skills show them the cutting edge of group software development.

In many areas of science it is common for individuals to write programs with out interacting with their neighbours. There are indeed some very good single person programmes but there are also very many more are very bad single person programmes. I believe that group working should be mandatory for scientific programmers for several reasons:

• Programming is hard and complex even the gurus know that they frequently get it wrong.
• There are many operations which are common to most programs and with out group working and sharing there will be unnecessary duplication which in any case usually leads to errors and confusion.
• There are a number of established patterns which are highly valuable in organizing software. These are not normally taught and not normally acquired when scientists start writing programs.
• There is a discipline to writing code which is initially somewhat painful. In the same way as musicians have to learn scales, programmers have to learn the value of unit tests. Again this does not come naturally.

There are a number of tools that Jim has introduced us to, probably in order:

• A source control system. (SCS) over the years we have moved from CVS to SVN and now to Mercurial.
• An integrated development environment (IDE). We use eclipse but there are several others.
• Unit testing and test driven development. This is essential for almost all our code. As you will see the environment enforces and encourages it.
• A build environment. Again this is essential. The build pulls together all the resources which are required, checks on their availability and produces and tests the code.
• A continuous integration environment (Hudson) .

The change to our productivity has been enormous and the change to our culture likewise. It is now expected that anybody can have access to anybody else’s code and work. Of course, since our projects are open source, there are simpler ways of doing this such as source forge and bit bucket. But we also expose our code publicly to the group on Friday mornings. No one, not even Jim, is beyond criticism but all criticism is constructive.

To show what we have achieved the screen shot shows some of our current projects (https://hudson.ch.cam.ac.uk/ ). There are about 30 in the system and they can range from a few hundred lines to tens of thousands. Almost all projects have dependencies which means that if you break something early in the build many other people may suffer. Hudson runs every 20 minutes and rebuilt everything that has changed or which depends on something that has changed. You can visit the site yourself which is probably the best way to understand it. In the current case I have one project which depends linearly on five other projects and each of these has to build. As you can see from the numbers some systems have been built over 100 times. Unless you have tried this yourself you will not appreciate the enormous amount of confidence that it gives you that your software is fit to share with the rest of the group. It does not mean that the software necessarily gives the right answers but it should mean that it does not fall over in a heap before starting.

Most academic software does not get redistributed beyond its point of creation. Sometimes for acceptable reasons where it is developed to tackle a particular short term problem. However short term software has a habit of turning into a long-term lava flow (a well known anti Pattern). But much software is simply not fit for distribution. It takes time and effort which cannot be credited against the traditional narrow minded approach of evaluation through H indices.

We are proud of our software, much of it supported by grants from JISC and we are in the process of making it is available as possible to the community. By combining the components in the system we can address a number of problems in semantic chemistry, particularly textmining, and computational chemistry. With Crystaleye we have the most advanced semantic system for chemical crystallography and this has only been possible because of the framework on which we built.

Dictated because of a slightly dodgy keyboard into Arcturus

I have had two responses to my post about Bio med central and their 10 year celebration at which they are honouring open data. The responses highlight and comment on the fact that Microsoft is a sponsor of the occasion.

 Twitter Trackbacks for Biomed Central and Microsoft honour Open Data – oh look, Microsoft trying to snuggle up to openness again… #opendata [cam.ac.uk] on Topsy.com says:

May 8, 2010 at 8:58 am  (Edit)

; glynmoodyHighly Influential: “Biomed Central and Microsoft honour Open Data – http://bit.ly/9eZHIW oh look, Microsoft trying to snuggle up to openness again… #opendata ” 3 minutes ago view tweet retweet Filter tweets […]

 Rusconi says:

May 13, 2010 at 10:23 am  (Edit)

I still have to be convinced that a convicted monopolist will bona fide support anything truly “open source” or “free software” or “open data” or anything “open xxx”. I sincerely hope you are not getting netted by a purely PR operation for M$.

Sincerely,
Filippo Rusconi
Scientist at CNRS, Paris, France
Free software developer for scientific applications

These are serious concerns and I will address them. I have previously blogged about the relationship of me and my group to Microsoft (http://wwmm.ch.cam.ac.uk/blogs/murrayrust/?p=2249 ) where I have shown my reasons for working with them. The current concerns are different in that they relate to sponsorship but basically have the same concern that Microsoft are guilty of actions which put them beyond the bounds of acceptability.

I do not know Filippo, but I know Glyn Moody well. He and I share positions on the Open Knowledge Foundation advisory board and most of our views coincide. However he and I differ on Microsoft. I believe his view is that Microsoft is inherently “evil” in a way beyond the natural commercial orientations and activities of any large company. I do not take this view at present although as I have acknowledged some of Microsoft’s past actions were clearly unacceptable.

Filippo describes Microsoft as a convicted monopolist, and I believe this to be essentially accurate although I do not know the precise details. From Wikipedia (http://en.wikipedia.org/wiki/Microsoft )

Throughout its history the company has been the target of criticism, including monopolistic business practices and anti-competitive strategies including refusal to deal and tying. The U.S. Department of Justice and the European Commission, among others, have ruled against Microsoft for antitrust violations.^[14][15] (See also United States v. Microsoft, European Union Microsoft competition case.)

I also believe that they have been brought to justice and have been made to pay. Whether the punishment fitted the crime is a matter of for the courts and the society that convicted them. These views mirror our own approach to criminality in society: that a criminal is basically always a criminal; or that a criminal once brought to justice can change their ways.

There is nothing inherently evil in manufacturing software and information systems. I personally believe that some industries must be challenged, such as tobacco and armaments. I am not an expert on company reorientation after conviction, but I don’t believe it to be impossible. The policy of a company is made of a complex mixture of market perception (which can include ethical issues), shareholder values and the attitude of its staff. I have primarily interacted with Microsoft Research and found the staff there to be highly aware of the need to change and looking for the directions.

Dictated because of a slightly dodgy keyboard into Arcturus

[I’m giving up on trying to get Librarians motivated about DRM. They clearly accept it in a supine manner as being “the law” (it isn’t). They aren’t interested enough in the harm it causes their reader community. Here’s something more exciting – and if I mention DRM to this community you bet I would get a reaction.]

One of the strengths of JISC and a very positive reason for working with them is a the strength of their programme management. (I always confuse programme and project management and if I get it wrong here please correct me). If you are interested in bidding for a JISC grant then liaising with the programme manager is extremely useful to both sides (the simple secret is that JISC wants your project to succeed! They are friends not enemies). The programme managers call Town meetings to discuss bids before the event and workshops after it. I have two JISC workshop meetings next week for example. It seems very OTT in management, but it’s a good investment.
Dave Flanders is one of the programme managers at JISC and we were chatting in the bar at Dev8D (I haven’t even blogged about that wonderful event). The topic turned to the productivity of programmers (i.e. those who write programs (not programmes)) – often called developers (Dilbert calls them engineers). We recognise that there is a huge difference between the mediocre and the top ones. I proposed a scale and Dave has written it up. I’ll comment at the end.

Good, Great and Guru Developers – An Algorithmic Skill Scale

It was Peter Murray-Rust (PMR) who proposed this scale to me at dev8D which helped refine a straw man diagram I’d been working on for a while since a chat with Jim Downing at OR09 on scales for learning curves on picking up programmatic concepts.   The vertical access is the percentage of work that can be completed by a single developer on a system/project; the horizontal access is a one to ten rating scale for developer ability (by “ability” I mean a full spectrum of coding skills as well as user, culture and business knowledge by which the software is contextuliased within); I ‘d welcome comments/refinements.

The significance between having goodDev, greatDev and guruDev is the observable fact that greatDev and guruDev can achieve significantly more than goodDev.  I’d argue that a single guruDev is significantly more productive than having a dozen goodDev.

None the less good software teams (like PMR’s as lead by Jim Downing) manage to balance all three types of developers. Aside:  I’m hoping to attend one of Jim’s Friday code reviews soon (my colleague Andy was recently in attendance and was very impressed, especially with their team motivational use of Hudson).  None the less, there is a significant risk in taking on goodDev who potentially have a negative effect on the production of the team (not that they won’t advance their skills to become productive).

The point of this post being, that developers are not just developers.  One is not the same as the other.  Projects looking to use developers on their team need to understand how significant the selection of a developer is.  Unfortunatly, I don’t think we have a good system for even interviewing developers let alone a way to acknowledge their skills.

More on this later.  Lots of thoughts abound.

And some comments

 I’m not a believer in the Guru Dev as a silver bullet, that is I don’t agree with your assertion that “a single guruDev is significantly more productive than having a dozen goodDevs”, at least not when we think about sustainable software.

It is true that a guru who knows her stuff can produce more code in less time, but that doesn’t mean anyone else can understand that code or how it was created.

For me a real guru developer is actually someone who can recognise the strengths of the good and the great and provide an environment in which they can deliver. The combined production of those developers will far outstrip what a single guru can produce. Furthermore, the skills of the majority will have increased (thanks to the guru’s leadership_ and the knowledge of the software will have been spread across more bases.

Ross Gardler said this on March 25, 2010 at 3:09 pm | Reply

• [DaveF] Agreed a dozen developers for one developers is a bit extreme though personally I’d rather have one Ben O’Steen or Jim Downing than a dozen fresh out of CS undergrads on my project <- though agreed that the more minds on a problem the better (there is a separate discussion around all software projects having sustainability as their output, as opposed to visionary or instruction, but we can save that for the pub).
  

  Also my point is more towards the fact that we don’t have systems by which to recognise the good, great and guru (and I’m including the ability to write sustainable code as one of the skills that a great and guru developer should have). Also the blatant problem that Universities do not have any recognised tiered position for developers to progress up the salary scale. We need to make a case towards having junior, senior and CTOs within Universities and that will require metrics of some kind (mythical man month or not).
  
  dfflanders said this on March 26, 2010 at 10:33 am | Reply
  
  

PMR: I am prepared to defend these order of magnitude differences (though the exact scale is questionable). There are of course many tasks which bring every one down to the same level. For example if a developer has to create a periodic table of the elements and type the data in (to avoid copyright problems) then a guru will not do it quicker and may in fact do it worse. What I am talking about is the ability to create a great piece of working, a reusable, distributable piece of software. In many cases getting a project 90% correct is equivalent to getting it zero per cent correct. So the ability to create the correct design is as important as the ability to understand arcane pieces of code. Indeed is the code is arcane then it is almost certainly poorly written or depends on a poorly created library.
Dave praises Jim Downing and rightly so. Jim is one of the great developers that I have known. In the time that he has been with us (somewhere between three and five years depending on how you count) he has given us some truly revolutionary designs for our software. His system Lensfield will be a milestone in Scientific Software. It’s truly revolutionary it so it will take some time to understand, but I have used its principles over the last three weeks to do things that I could not possibly do otherwise. This is why I think that a factor of one hundred is not impossible.
Jim is not a top guru. Jim listens to the top gurus, like Jeff Attwood and Joel Spolsky. Kent Beck created one of the truly great tools for developers, JUnit tests. The code itself is easy to write. However the impact of this small amount of code has been sensational. So I would put Kent Beck somewhere in the stratosphere, adding at least one more orders of magnitude given the influence that his work has had on the community. So where does Jim come on this diagram? Probably about 8.5. Where do I come? Perhaps around 6.5. This includes the fact that I have many years of experience of what not to do, and that I have had the opportunity and a determination to set up a group where Jim has been able to develop and impart his skills.
Originally I proposed a logarithmic graph but this does not account for the fact that some developers have a negative influence. We’ve had experience of this. Obviously no names, but one developer (nickname suggested by a red dwarf episode) when confronted with a failing unit test would change the test so that it “passed” – the version control system that we were forced to use did not record that there had been changes.
Want to Jim has shown us is that it is possible for people to move up the curve. This is by introducing and using modern methods of software development, which are relatively uncommon in scientific programming. This is an area or we have debated with JISC programme managers; how can we jointly work towards a better standard of programming in the scientific arena so that individual projects do not simply die. In future blog post I shall show some of our tools and some of the results, which are a tribute to Jim’s vision and methodical commitment to delivery.
We shall miss him very much indeed.

Typed on a slightly dodgy keyboard into Arcturus

I have had an excellent, long and thoughtful reply from Bill McCoy. I quoted from his blog at Adobe (http://wwmm.ch.cam.ac.uk/blogs/murrayrust/?p=2396 ), where he highlighted the shortcomings of some Adobe products. I thought this was brave and applaud it – it is rare to see employees being honest – in public – when sales are at stake.

He’s no longer with Adobe and he has written a useful account of the way that scholarly publications should be managed – in the browser, not at the server. I like his analysis. I am not against access control per se, and it works well in most cases – certainly far better than DRM.

The last paragraph is very useful and shows the difficult path that institutions such as the BL have to tread. I truly don’t know what flexibility of action they have but I still intend to ask.

I am, however, concerned about the apparent lack of interest in this area from UK research librarians. OK, I’m not a Librarian but I have been asked to server on a BL advisory board, I have been asked to comment in detail to the BL on copyright, and I have today received an invitation to speak at a meeting on Library matters in November – more later. So there has been useful contribution to this discussion from several sections – everyone agrees that the current DRM is awful – but nothing from librarians. This has been going on since 2005/6 – the CEO of the BL, dame Lynne Brindley highlighted the problems – but I can find no trace on the net of concern within university libraries.

If you don’t alert us to problems, we can’t engage in whatever is the appropriate way.

Bill McCoy says:

May 12, 2010 at 3:15 pm  (Edit)

As you may have noticed from the corporate blog, I’ve left Adobe since making that post. I will note that, 7 months later, no such “next major release” of Adobe Digital Editions has as yet taken place.
So, arguably, the commitment I made at that time has not been violated.

But to me this points out another objective issue with reliance on a de facto standard DRM scheme proprietary to and controlled by a single corporation: lack of choice. With no alternative PC/Mac solution for reading Adobe-DRM-protected eBooks, we are all hostage to Adobe’s schedule releases of its software, and more to the point, Adobe’s overall corporate priorities and resulting resource allocation decisions.

Personally, I’ve become convinced that the browser is the right platform for digital document distribution, with open standards (HTML5) now enabling capabilities such as offline access and higher-quality typography. In a browser-centric model “heavyweight” DRM be replaced with user authentication mechanisms that we are already accustomed to using.

The browser market is increasingly competitive with 5 significant players having market share (IE, Firefox, Safari, Chrome, and Opera), with 3 of these built on open source rendering engines (Webkit & Gecko). And Webkit is fast becoming the standard browser engine on mobile devices. To me this represents a very healthy foundation upon which to build a digital publishing ecosystem, a foundation that will ensure choice for publishers, distributors, and consumers: choice at the browser level, and choice of distribution solutions that build on the browser and open Web Standards.

I do empathize with content distributors, such as British Library, who have a tough row to hoe (as we say over here). By and large, distributors do not want DRM any more than end users do, and recognize that more times than not, DRM is counter-productive. I have found them in general to be quite concerned about accessibility. But distributors must honor commercial agreement with the underlying copyright holders, which puts them in a difficult position. I trust that the emergence of browser-based distribution solutions for digital publications, built on open standards, will give content distributors more options in the near future.

Typed on a slightly dodgy keyboard into Arcturus

I have received a useful reply from the Electoral Commission. You may remember that I wished to vote in the recent election without showing my face (http://wwmm.ch.cam.ac.uk/blogs/murrayrust/?p=2331 ), was requested to do so and was unclear whether this is legal. The Commission have made it clear that there is no law defining this and so it appears that some Presiding officers may be given one set of guidance and others other.

I’d like to congratulate all concerned – the Presiding Officer who was firm but courteous, the Commission for their speedy reply, and WhatDoTheyKnow (links at bottom) for this wonderful service. My question was a serious one and I hope the answer in WDTK can help others as being a definitive one. (Does WDTK have a method for highlighting general questions that may be of fairly widespread interest?)

And since we are not having an election for 5 years, but we are having a referendum, the bear knows when it has to come out.

James Pack
Electoral Commission

12 May 2010

Dear Mr. Murray-Rust,

Thank you for your query regarding your experience on polling day.

The Electoral Commission give no guidance on the subject of voters wearing clothes that cover their face. Our guidance simply states that the polling station staff should ‘Greet electors, ask them to confirm their name and address, and make sure that they are eligible to vote’.

This guidance is derived from the current legislation on the process for the issue of a ballot paper. The Electoral Commission is not responsible for the writing of that legislation or the debate and agreement to the legislation as that is a matter for the Ministry of Justice and the UK Parliament respectively. The Electoral Commission gives guidance on the interpretation of that legislation.

The Commission does review the current legislation and gives recommendations for legislative change. Our view on this subject is that the Commission has not ruled out the possibility of introducing measures requiring voters in Great Britain to provide identification at the polling station. However, we believe that the introduction of a system of individual electoral registration with personal identifiers in Great Britain should be prioritised in order to provide a more secure foundation for the electoral system.

Regards,

James Pack, AEA (Cert)
Senior Adviser (Electoral Guidance and Events)
The Electoral Commission
Trevelyan House
Great Peter Street
London SW1P 2HW
Tel: 020 7271 0564
Fax: 020 7271 0505
Textphone: 18001 020 7271 0571
www.electoralcommission.org.uk

www.aboutmyvote.co.uk

show quoted sections

Link to this | Reply to this message