CSM: .We have already extracted 10s of thousands of chemical names and will be linking them up to ChemSpider structures to enable Open Access papers to be structure/substructure searchable. However, we’ve hit a bit of a hurdle…more details on this will follow shortly but we have been asked to remove thousands of articles indexed according to what we believe is a standard search engine policy from the ChemRefer index. During our conversation today with the publisher the conversion of chemical names to chemical structures to provide a structure searchable index of the articles was deemed to be “re-purposing” of the Open Access articles and is NOT allowable. Peter Corbett and Peter Murray Rust are engaged in similar activities so will likely run into the same challenges. If they manage to get around this issue with this and other publishers then they will be working in a “permissive” role where they will need to get permission from publishers to perform semantic markup. Their semantic markup is also “re-purposing”. The “permissive challenge” is far away from Peter’s stance in terms of Open Data for all.
By “open access” to this literature, we mean its free availability on the public internet, permitting any users to read, download, copy, distribute, print, search, or link to the full texts of these articles, crawl them for indexing, pass them as data to software, or use them for any other lawful purpose, without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself. The only constraint on reproduction and distribution, and the only role for copyright in this domain, should be to give authors control over the integrity of their work and the right to be properly acknowledged and cited.
PMR: Publishers do not make the rules. They think they do, but they do not. If they call themselves Open Access then they are expected to abide by the rules. Their licence must be OA compatible or they will be taken off the list of approaved journals. There is and will be increasing pressure from the community to make sure publishers conform. Calling yourself Open Access when you are not is fraudulent (I assume this publisher is taking money from people).
Any proper Open Access publisher who – mistakenly or implicitly – fails to remove permission barriers will remove them once the error is pointed out. It is LEGAL to index articles from ANY publisher. You do not have to ask the publishers’ permission to create an index. You do not have to ask their permission to re-use facts. Facts are not copyright. Elsevier made this clear on this blog. Molecules are not copyright. Molecule names are not copyright. The time of sunset is not copyright. It is LEGAL to extract these from any article. It is totally unacceptable that any publisher, let alone an “Open Access” one should disallow indexing. And to insist that you remove the index is preposterous.
What now worries me greatly is the you (Chemspiderman) are giving in to this extortion. It gives the worst possible message – allowing a publisher to make up non-existent rules to which you have to kow-tow. Don’t do it. Don’t negotiate. There is nothing to negotiate about.
There is only one aspect where you might be in error – the actual spider. If this site wishes to forbid spidering it should have a “robots.txt” file. That indicates which files and directories can be spidered, but this is ONLY to avoid inconvenience to the server. If, for example, I wish to click manually through the whole site while watching the rugby, robots.txt cannot stop me.
I am now worried that you will muddy the waters by suggesting to publishers that they have rights that they do not have. And that when I come to index the material – and re-purpose the facts as we have done with crystalEye – the publisher might object.
I do not know who the publisher is – I have a fair idea. Unlike Chemspiderman I have no hesititation in publishing correspondence and I see no reason why the publisher should have any cloak of anonymity. But I have a strategy for dealing with them
That I SHALL keep secret.
pm286 
October 13th, 2007 at 8:31 pm ePeter, I will not announce the publisher at present because I made a commitment to not do so until we had a mutually agreeable blog posting for our users and accurately representing the conversation and agreements between us. I have an urge to co-exist in the world with publishers since they put a lot of value into the world. With the changes going on in Open Access figuring out how to co-exist is very necessary. I hope we can get the information out shortly. It is possible we have mis-stepped but more likely that there is a policy issue with spidering policy that needs addressing by the publisher.