Lensfield: In the Beginning was the FileSystem

This post introduces the command line and the file system which are the bedrock of the Lensfield system we have built to support scientific computing .

Neal Stephenson has a marvellous essay/book http://en.wikipedia.org/wiki/In_the_Beginning…_Was_the_Command_Line . It’s primarily a discussion on proprietary operating systems, but it highlights the role of the command line in supporting one of the most natural and most durable ways of humans communicating with machines. It’s so “natural” that we may feel it’s a fundamental part of human intelligence, mapped onto the machine.

But there were machines before command lines. When I started (what an awful phrase – but please don’t switch off)… there was no keyboard input into machines. Input was via paper tape:

[thanks to wikimedia]

Ours was even worse – it had only 5 holes and would tear at the slightest chance. And ask yourself how you would edit it? – yes, it was possible and we did it… You encoded your data on an ASR33. To get your results out the computer had a tape punch and you fed your tape into the ASR33 which punched it out at 10 chars/second – rather slower than a competent typist.

And where was the filestore? Yes, you’ve guessed it – rolls of paper tape. Howe were they indexed? By writing on them with pens. If the information was too big it was split over two or more tapes. If you read them in the wrong order, the result was garbage. And if the rolls were large the middle could fall out. A beautiful effect, but topologically diabolical. Because every layer in the spiral contributed to a twist that had to be undone by hand. It could take an hour to rewind a tape.

The next advance was magnetic tape. This was almost infinite in storage.

[from Wikimedia]

From Wikipedia (http://en.wikipedia.org/wiki/IBM_729 ):

Initial tape speed was 75 inches per second (2.95 m/s) and recording density was 200 characters per inch, giving a transfer speed of 120 kbps^[1]. Later 729 models supported 556 and 800 characters/inch (transfer speed 480 kbps). At 200 characters per inch, a single 2400 foot tape could store the equivalent of some 50,000 punched cards (about 4,000,000 six-bit bytes, or 3 MByte).

A tape could therefore hold many chunks of information. I can’t remember whether we called them “files” – it depended on the machine. But it was really the first filing system. Let’s assume we had 1000 files, each about 32Kb. The files were sequential. To find one at the end of the tape you had to read through all the other 999. This could take ages. Writing a file could only be done if you have a tape with space at the end (even if you wanted to delete a file in the middle and overwrite it, problems with tape stretch might make this dangerous). So generally you read from one tape and wrote to another.

And how did you input instructions to the machine? Not with a commandline. Not with buttons and guis. But with switches. Here’s a PDP-8 (which controlled my X-ray diffractometer in the mid-1970’s ).

[thx Wikimedia]

To enter a program you have to key in the operating system – set the toggle switches (orange/white at the bottom) to a 12-bit number – enter it. Set it to another number. Enter it. Do this for about 10 minutes without mistake (a single mistake meant back to the start). Then you read the rest of the operating system on paper tape. Then you could read in your program!

The point is that back then…

• No command line
• No file system

That’s why when Dennis Ritchie and others introduced the hierarchical file system and the command line in UNIX it was a major breakthrough.

The magnificent thing is that 40 years on these two fundamentals are still the most natural way for many humans to interact with machines. 40 years in which the tools and approaches have been tested and honed and proved to work.

And that’s why Lensfield builds on the filesystem and the command line to provide an infrastructure for modern scientific computing and information management. And why we’ve used Lensfield for the Green Chain Reaction and will be providing it to the Quixote project (and anyone else for anything else).