This is my third and final post in a (rather impromptu) mini-series on academic journals and their place in the 21st century landscape. You may want to read my previous two posts here and here before reading this one?
To find a new enlightening and inspiring idea (as distinct from finding a recipe for getting safely through the peer-built barricade), browsing through thousands of journal pages is all too often called for. With my tongue in one cheek only, I’d suggest that were our Palaeolithic ancestors to discover the peer-review dredger, we would still be sitting in caves… (Zygmunt Bauman)
In my previous posts on academic journals I’ve compared them unfavourably – either explicitly or implicitly – with the kind of informal ‘peer review’ that happens through blogs and social media. Some commenters have assumed that this means that, like Bauman (see above) I’m completely against peer review. I’m not.
Peer review is valuable. In fact, it’s so important we need a (re)new(ed) academic ecosystem to protect it.
I’m all for new systems such as hypothes.is which provides an open, distributed peer review layer for the web. Although I don’t want to go into it in too much depth here, academia is one of the few unreformed areas with outdated power structures and glass ceilings.
As Stephen Thomas pointed out in the comments to my previous post, academic journals have, and still do, play an important role in both establishing precedent and providing a quality filter. This is important (most of the time).
But, as Dan Meyer pointed out in the quotation making up the bulk of my first post in this series, it’s the edifice that’s built upon the academic journal system that’s problematic:
The incentive seems strange to me… I don’t understand this brass ring I’m chasing. (Dan Meyer)
This academic edifice is built upon other perceived ‘advantages’ of academic journals, including:
Dissemination of work
Contact with others inside and outside field
Academics, unfortunately, have ended up inventing a stick with which they can be beaten. In the UK, the Research Excellence Framework (REF) is a crude instrument looking a research outputs. Career progression (and therefore status) depends upon disseminating work in journals that are, all too often, closed and paywalled.
Part of the answer, I agree, comes through academic journals becoming open access. That’s a step in the right direction (even if it does smack a little of Henry Ford’s ‘faster horses‘). Going further, something more like Alan Cann’s experiments around open peer review could work. But, realistically, we need something a bit more radical.
How can we save peer review whilst democratising and reforming higher education?
I leave you with the words of Frances Bell, who commented on my previous post:
What I suspect is that more research needs to be done on how, for example. scholarly societies can support research, scholarship and practice in a digital age. (Frances Bell)
Until the late seventeenth century, communication between scholars depended heavily on personal contact and attending meetings arranged by the early learned societies (e.g. the Royal Society). As the membership of these societies increased, more people could not attend the meetings and so the Proceedings, usually circulated as a record of the last meeting became a place to publish papers that had not been presented at the meetings at all and moved towards what we now recognise as scholarly journals. (Wells, 1999)
So journals are a replacement for personal contact.
Are they good for anything else? Brown (1997) cites the following:
distributed (many copies are stored in many places)
scholars trust and understand the system
journals have prestige built up over many years
portable and easy to read
Which of the above benefits either (a) cannot, or (b) are not currently able to be replicated by another system?
Some would argue that an important difference between (for example) a blog post and a journal article is that the latter has been formally peer reviewed.
However, as even the editor of The Lancet points out:
The mistake, of course, is to have thought that peer review was any more than a crude means of discovering the acceptability — not the validity — of a new finding. Editors and scientists alike insist on the pivotal importance of peer review. We portray peer review to the public as a quasi-sacred process that helps to make science our most objective truth teller. But we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong. (Horton, 2000)
Just how big do the cracks in the ivory tower have to get before the whole edifice tumbles?
Odlyzko (September 1997) points out that there was an “extensive resistance to print by scholars” in Gutenberg’s time which included calls to ban the new technology because only trash was getting into print and books were not as durable as parchment. The reaction to the Web of today’s scholars has largely echoed the reaction of scholars to the printing press in the 15th century. (Well, 1999)
Is the only reason we persist with journals and their articles is because they provide a convenient means to weigh the pig?
Although I’m still yet to have an academic article published, I’ve made a public commitment to do so only in open-access journals. I’ve already dedicated this blog to the public domain (see CC0 license in footer) and shared my thesis online. Whilst for me it’s a logical continuation of my position as an open educator/academic/researcher/individual, I’ve been waiting for a compelling reason for others to ditch closed journals.
In this, my third blog post quoting Zygmunt Bauman (from a recent interview with Simon Dawes, editor of Theory, Culture & Society), I want to consider briefly the ways in which the whole edifice of the peer-review system is flawed. It’s not just about the binary distinction between whether a journal is ‘open’ or ‘closed’.
Simon Dawes: One final question, TCS is committed to the process of peer-review, and many of our (both rejected and accepted) contributors are grateful for the feedback given by our editors and anonymous reviewers, and for the subsequent strengthening of their articles, but you are critical of peer-review and no longer act as a referee for us. Could you tell us why?
Zygmunt Bauman: There are, by the most conservative counting, two grave and deeply regrettable collateral victims of the peer-review gruesome strategem: one is the daring of thought (wished-washed to the lowest common denominator), and the other is the individuality, as well as the responsibility of editors (those seeking shelter behind the anonymity of ‘peers’, but in fact dissolved in it, in many cases without a trace).
Last but not least, I would single out yet another collateral damage: the multitude of the trails blazed and heterogeneity of inspirations. I suspect that the peer-review system carriers a good part of blame for the fact that something like 60 percent or more of journal articles are never quoted (which means leaving no trace on our joint scholarly pursuits), and (in my reception at any rate) the ‘learned journals’… ooze monumental boredom. To find a new enlightening and inspiring idea (as distinct from finding a recipe for getting safely through the peer-built barricade), browsing through thousands of journal pages is all too often called for. With my tongue in one cheek only, I’d suggest that were our Palaeolithic ancestors to discover the peer-review dredger, we would still be sitting in caves…
That’s a fairly damning verdict from an Emeritus Professor of Sociology, don’t you think? I haven’t met anyone who thinks that the REF (Research Excellence Framework) is a good idea and does the job it’s intended to do. The ‘peer-built barricade’ that Bauman mentions evolved in a world before real-time communication between academics; although it feels obvious to say so, the internet changes everything. As proved with <a href="my thesis (see Appendix 3) the amount and quality of feedback does not depend either upon journals or anonymity. We can, and should, build a better (more democratic, fairer, transparent) system.
Before Google Teacher Academy last week I was revisiting Google tools I don’t use every day. One of these is Google Knol. Like Google Wave, it had changed a lot since I last used it, so I experimented further on the train down to London. Below is a link to the New Literacies ‘knol’ I produced along with a video overview of some of Knol’s features. I really do think the current peer-reviewed academic journal system is broken and needs a replacement. Something like Knol could do the job!
I’ve been very impressed with my Sony Reader PRS-600 since I got it last week. It’s a great device for reading, highlighting and taking notes on academic articles. Since before I couldn’t find much useful video on how the highlighting and note-taking functionality works, I’ve quickly put together the above two minutes by way of demonstration.
Hope it helps. 🙂
Note: those reading via RSS/email may need to click through to see the video – or view it on YouTube!
I learned today that the best gadget purchases are those that solve a problem. Whilst it’s wonderful to have the latest and greatest (I’ll be getting a free iPad via my attendance at the Handheld Learning Conference later this year) it’s very satisfying when something plugs a gap.
I’ve got lots (probably hundreds) of journal articles to read for my Ed.D. thesis.
I use a computer screen for my work much more than I used to, meaning on-screen PDFs is problematic.
I get the train (c.30 minutes each way) and then walk to work. I don’t want to have to carry around anything heavy.
Today I bought (or should I say my parents, who are extremely supportive of my studies, bought me) a Sony Reader PRS-600. It’s the one with the touch screen for highlighting and annotation. It’s got an e-ink screen meaning it appears like a physical book instead of a flickering screen.
What I’ve tried previously:
Printing out articles (cumbersome, expensive and not environmentally-friendly)
Dropbox iPhone app (doesn’t ‘reflow’ PDFs meaning horizontal scrolling which isn’t very user-friendly)
GoodReader iPhone app (iPhone screen too small for annotation)
I considered an Amazon Kindle, but after seeing and handling the Sony Reader at the JISC Conference earlier this week, I was sold on it. JISC had funded a project where the Sony Readers were used by previously technophobic academic staff to mark student essays. They loved them and if they’re good enough for that purpose, it’s good enough for me!
It’s still (very) early days. I’ll let you know how I get on! 🙂
I learned several things in my Skype chat with my Ed.D. thesis supervisor (Steve Higgins) this week, notably:
I’m a pearl-grower
People can be metaphorical in different ways
How ambiguity can be productive
Steve was very interested in what I’d mentioned in our brief email conversation earlier this month about different types of ambiguity. After reading William Empson’s Seven Types of Ambiguity (Wikipedia)recently I’ve realised that the terms that I’ve been analysing in the literature – ‘digital literacy’, ‘electracy’, information competence’, and the like – are actually ambiguous in different ways.
In fact, the level of ambiguity and lack of clarity can depend upon the ‘level’ at which people are working. So, for the teenager at school talking about ‘digital literacy’ isn’t productive or helpful. For the university professor, on the other hand, it could create a productive space in which to discuss one or more concepts.
Steve mentioned that I need to be careful about different positions taken by thinkers about literacy. Radicalists, for example, would argue that the way that you become literate affects the way you are literate later. This all links into the seven types of ambiguity mentioned above. I think that, given that Empson was considering mainly poetry and works of literature, there may be scope in my thesis to collapse or discard two or more types of ambiguity.
The problem about New Literacies is that they potentially lead to ‘infinite literacies’. For example, I’ve seen studies that seem to consider the use of Flickr as requiring a separate ‘literacy’ than, for example, Facebook. If there is such a thing as ‘digital literacy’ then it would need to be a level up from this – a complex organizing construct that could explain such affinity spaces. 🙂
Moving on to the structure of my thesis, I explained to Steve that I’m thinking about moving the location of my discussion on the Pragmatic method to the beginning of my thesis. If it went after the Literature Review, as originally planned, then it would seem a bit out-of-kilter. If so, that would give me the following structure:
Pragmatic method (plus discussion of other lenses that can be used)
Literature Review (including evolution of New Literacies)
Discussion and analysis of seven types of ambiguity
Analyse digital literacy (and other concepts) as constructs (do they have pragmatic utility?)
Review of terminology used in government, NGO and media reports (e.g. Singapore, BBC, NGOs, UK, USA – what do people mean by digital literacy? is it a productive ambiguity?)
Digital Flow as a more coherent and less ambiguous construct
Steve noted that ‘digital literacy’ is a term used mainly by digital enthusiasts. It creates a positive space or construct for people to explore shared meanings. I thought this was an interesting idea and brought up the idea of a continuum of positions people could take. Steve expanded on this to say that it’s as if the whole of the Venn diagram can be discussed rather than just the intersect:- includes the whole of the Venn diagram rather than just the intersect:
My ‘Norwegian analysis’ (as Steve calls it) contains a metaphorical aspect which is at heart of way the term has evolved. It creates a ‘productive space’ which may or may not be useful. By moving computer skills into the ‘literacy’ domain the debate is altered as a ‘value space’ has been created. After all, no-one is going to argue about the utility of a literacy. This could be a useful segué into the section about the what the media and governments say about digital literacy and related concepts.
Regarding my final section on ‘flow’ I mentioned how in the literature it is used most in relation to computer games. Steve mentioned that he has an unpublished presentation/paper on how interactive whiteboards (IWBs) develop teacher didactic ‘flow’ (in that they control IWB with one hand and class with another). The IWB encourages whole-class presentation, with more answers from students, but less depth. The teacher ends up in some kind of bubble which becomes less and less permeable. They are in control of resources; lessons ‘progress’ but without the cognitive dissonance required.
I was concerned that I wasn’t discovering academic journal articles in the most efficient mannter. I explained to Steve my method:
Find recent relevant, high-quality journal article using Google Scholar
Scour reference section for relevant articles.
Go through relevant articles and repeat.
Steve calls this pearl-growing and apparently it’s a popular method. He did, however, mention that ‘forward-tracking’ via services such as Google Scholar is also useful. Steve also recommended keyword searching at the Web of Science and directly within the archives of relevant journals. The problem with everyone ‘pearl-growing’ I pointed out is that, like the iPhone App Store, once something becomes slightly popular, it will become exponentially more so. Steve said that his colleague, Peter Timms, had found that there is evidence to this effect as there is no strong correlation between the quality of references in academic papers and the quality of the article itself.
Moving on we discussed the problem of ‘solidification’. As I’ve referenced before, Allan Martin has written on the idea of ‘liquid modernity’. This is the idea that by the time research has been done and social norms have begun to emerge, the technology has moved on. What Steve wondered, however, was whether this is true from this point forward (i.e. we will be in this situation forever more) or whether it’s for a finite period of time. I pointed out that my historical research has shown that people pretty much always think they’re living through times of rapid change! Steve quite rightly pointed out the difference between technological and conceptual things changing. Something to dwell on… :-p
The problem with such a nebulous concept as ‘digital literacy’ is that it can collapse to meaning no more than ‘having access to digital texts’. Steve and I discussed the importance of ‘knowing’ (in terms of internalization and understanding). All an individual can do on the first attempt is to assimilate information and value judgements into what W.V. Quine would call an individual’s web of belief. It would be interesting to explore the extent to which new concepts such as ‘digital literacy’ restructure belief systems (as oppose to merely being assimilated)
This led onto my wondering about ‘digital epistemologies’ – especially as Richard Rorty claims that pragmatists don’t have an epistemology. Steve though that, although he would have to check, Rorty means by this that pragmatists don’t have a single epistemology. They are more agile than thinkers holding different positions.
I thought that this potentially linked together our discussion of ‘liquid modernity’ with the idea of a ‘web of beliefs’. Pragmatists have a more fluid and changeable web of beliefs, meaning they are better situated when faced with fluid change. It’s something I’m going to think about putting into the first section of my thesis. 😀
Finally, Steve and I discussed the difference between C.S. Peirce (a community of inquirers would find truth in the long run) and Richard Rorty (there is no ‘truth’ to find). Peirce, coming before Thomas Kuhn, didn’t take into account paradigm shifts where the graph of what constitutes ‘normal science’ is forever having to restart. Although Peirce can be interpreted in a Postmodern way, Steve is more comfortable interpreting him towards the Realist end of the spectrum. That is to say: Peirce believed that the external world exists and there are truths to be known, it’s just that we as humans aren’t in a position to do that. The difference between Rorty and earlier pragmatists, suggested Steve, is that Rorty came from a strongly libertarian standpoint, whereas Dewey, for example, was more of a communitarian.
The hour went very quickly. Lots to think about and lots of work to do! 🙂