OSCAR reviews a journal

In the last post I described OSCAR, which can review and extract chemical data from published articles. Here is how I used it to review the Beilstein Journal of Organic Chemistry
The BJOC unlike most other chemistry journals encourages reader’s comments, so I thought OSCAR would like to add some. Since I did this on a Saturday none of the comments have been moderated (or at least none have appeared). I first added comments to the journal announcement about what I intended to do, and gave links to the OSCAR home page. I then started at the first paper and found the “Additional File 1” which contains a pointer to the chemical data. (The process seems overly convoluted, and I have commented on this). I first downloaded OSCAR (the adventurous among you can try this and the following), started it (click the jar file), opened the BJOC (Word) file with the data, selected all of it and pasted it into OSCAR.
This is a very well presented file (and worthy of the authors’ orgnaisations – GSK and Leeds) – not all chemical manuscripts are as well prepared. OSCAR reveals only two errors, which are missing commas. (These are more important than they sound as we rely on them for the parsing). Typical results can be seen in the previous post. I therefore added this to the comments section for the paper. I assume the comments will appear in a day or two. I don’t know whether the authors will be automatically informed – I expect so – and whether the deposited data can then be corrected either by authors or editorial staff. If so, this is a real mechanism for cleaning up the current literature. Of course if the authors use OSCAR in future they will get a clean sheet!
I then applied OSCAR to all the papers in the Journal that contained chemical synthetic data – about 27. There is no standard place for the data – sometimes they occur in free text and sometimes in “Additional File n” (this name is not very helpful and I have suggested it should be changed to something with chemical semantics). I commented on the variability in navigation which made it difficult for me (and very difficult for OSCAR if it wished to review the journal systematically). OSCAR discovered several important errors – for example a chemical formula was wrong (this matters) and many suggestions about style improvements. (I did not comment on these as OSCAR’s rules don’t yet include BJOC policy). I also noted that some papers didn’t include data. I did not comment on the chemistry at all – its merit or its correctness – as I am not a specialist except on data. But perhaps this will stimulate expert readers to do so in future.
OSCAR raised concerns in almost all papers – ranging from punctuation to incorrect formula. I stress that this is common in ALL chemistry papers – and should not be used to measure BJOC against others. They all need cleaning up.
I made addiitonal comments on the accessibility of crystallographic data – these were not added as supplemental data and I argue strongly that they should be. I’ll write later about this.
I am hoping this will be seen as positive critiquing – it would be in compsci or crystallography. Certainly the adoption of data standards will make an enormous impact in the standard and re-usability of chemistry.
(Note: Our two summer students this year- Richard Moore and Justin Davies – again financed by RSC, have been refactoring OSCAR – we call this OSCAR-Data. OSCAR-Data uses OPSIN (OSCAR3) and allows for several inputs – SciXML, HTML, converts them into CML and then applies a set of custom rules (which could be publisher-specific). )