Open Science meets Open Education

Introduction

On 26 October the Dutch SIG Open Education, together with SURF, the collaborative organisation for ICT in Dutch education and research, organised the seminar Open Science meets Open Education. The goal was to bring the communities of both fields together and explore ways to better cooperate, using each others good practices.

The program consisted of an interview with two institutional policy makers about their ideas and experiences, crash courses on both Open Science and Open Education to inform both communities about the basics of the two fields, speeddates between community members to explore options for cooperation, presentation of a practice and three in-depth sessions on respectively Policy, Quality and Infrastructure.

During the speeddates, findings were reported on paper. These findings and impressions will be used as input for both SURF and the SIG to define follow-up activities for 2019.

My take aways from this seminar

In the past, I have blogged already (here and here) about obvious similarities and differences between these two branches of open. Here my main findings for this seminar.

Role libraries

Many of the participants came from institutional libraries. Historically, they have a key role in supporting publishing of research results (both open and closed). More and more, they are also involved in supporting teachers with sharing and reusing OER (e.g. with their expertise in copyright issues and open licenses). Therefore, for me it is clear that libraries are key partners in realizing a closer cooperation in implementing both Open Science and Open Education within an institution. There are however differences between research universities (where research is considered the main activity, although in theory both research and education should be equally important) and universities of applied sciences (UoAS) (where education is historically the main activity, and practice based research becoming more and more important). The distance between teachers (in research universities) respectively researchers (in UoAS) and the library should be smaller than currently is experienced by several librarians.

Cultural issues

John Doove from SURF made the observation that in research a culture of publishing results, based on peer review, is common practice for a long time already. This has resulted in more formal, top down  organized, support processes. For education, activities to come to more openness, like sharing and reusing OER, are relatively new. These activities are in many cases initiated bottom-up. This creates more flexibility, but also makes it harder to implement in a more formal structure. I support this observation.

Awareness about and willingness for publishing Open Access or publishing research data openly is not common yet. In this sense, both Open Science and Open Education face similar challenges. Experiences and attitudes for sharing in the areas of both science and education could be useful in advancing both Open Science and Open Education. Local institutional activities could be defined to collect these experiences (e.g. by organizing similar seminars as the one from last Friday locally within an institution).

Quality

As mentioned in my previous blog on this topic, quality assurance of scientific results is a well-organized process, based on rigid peer review, when it comes to publications. Participants of the seminar made some comments on this.

  • Publishing research data openly, following the FAIR principles (findable, accessible, interoperable and reusable), is relatively new for researchers. Support mechanisms are currently implemented in institutions, but awareness and attitude of researchers for this phenomenon has a comparable level as making education more open by sharing and reusing OER or engage in Open Educational Practices among teachers.
  • Adoption of alternative, more open ways of peer review, are in many cases dependant on disciplines. E.g. peer review of pre-prints like on ArXiv.org, is more and more common in fields like Physics, Mathematics and Computer Science. In these forms of peer review, authors and/or reviewers are not anonymous and the review is openly shared. Realizing similar processes for OER could maybe solve the challenge where teachers hesitate to reuse because of uncertainty on the quality of the OER.
  • Especially in practice based research, the main results can be different from a paper or a data set. An example is creating a software prototype for innovating some production process in IT. In such cases, these results should also be made openly available when financed with public money, e.g. by sharing the code as open source, with sufficient documentation.
  • Attention for FAIR principles on open data can be extended to OER. Adding sufficient metadata is one of the means to realize.

Conclusion

This seminar added to my ideas. For me, it is obvious to join these two communities to advance adoption of both open science and open education. Lessons learned and similar challenges in both fields can accelerate the process of adoption. As mentioned before, SURF and the SIG Open Education will work on activities to support institutions in this endeavour.

A light-hearted illustration: Big Bang Theory

For many years, I am a big fan of the sitcom The Big Bang Theory. One of the nice things I find in this show is informing the audience on recent findings in science.

In one of the episodes (The Troll Manifestation), the principle of peer review of a pre-print is illustrated, when two of the main characters, physicists Sheldon Cooper and Leonard Hofstadter, publish a paper with a revolutionary idea on a pre-print server. It then appears that one of the reviews is rather nasty and insulting, with the reviewer using a nickname.

Finally, the anonymous reviewer appears to be rather famous (for those unfamiliar with this sitcom: he had appeared in the series several times previously to this episode).

Besides a cOAlition S also a cOERalition S?

CC0 Wikilmages @ https://pixabay.com/en/solar-system-big-bang-11188/This blogpost is a co-production by Ben Janssen (OpenEd Consult) and me. It is translated from a Dutch version, slightly adapted.

Two weeks ago cOAlition S was launched by 11 national research funders (including NWO, the Netherlands Organisation for Scientific Research), with the support of the European Commission and the European Research Council (ERC). In their words (source):

…an initiative to make full and immediate Open Access to research publications a reality. It is built around Plan S, which consists of one target and 10 principles.
cOAlition S signals the commitment to implement, by 1 January 2020, the necessary measures to fulfil its main principle: “By 2020 scientific publications that result from research funded by public grants provided by participating national and European research councils and funding bodies, must be published in compliant Open Access Journals or on compliant Open Access Platforms.”

This initiative has rightly received a great deal of publicity and is generally regarded as a major step towards achieving the goal of Open Science, where research publications become immediately available free of charge to all interested parties, without conditions. After all, growth in science is rooted in a tradition that builds on previously achieved results. In order to be accepted for publication, articles must demonstrate what previous research has yielded and what new contribution the publication has for the subject in question. In order for this to work optimally, it is essential that research results are as accessible as possible. Open Access publications are an excellent means of achieving this.

In one of these reactions, NWO indicated that it wants to go further than just the 10 steps mentioned in the initiative, and also to strive for a different appreciation of scientific achievements than counting publications in peer reviewed journals. This reduces the pressure of publishing, which is still preventing many researchers from publishing their results in an Open Access journal. This pressure also leads to another perversity in the current system, the predatory journals. These journals tempt researchers to publish their results quickly, without a thorough peer review process, often after payment of a considerable amount.

>> The Economist: “European countries demand that publicly funded research be free”
>> Stan Gielen (Chairman NWO) “NWO wants to move away from the impact factor” (in Dutch)
>> Article about predatory journals

Previously I wondered why a similar initiative is not launched for the open availability of educational resources too. Educational resources made with public funds should be available to everyone. After all, it has already been paid for (the moral argument). As with the publication of research results under Open Access, open sharing of educational resources contributes to free access to knowledge, making an important contribution to the realisation of UNESCO’s SDG 4: “to ensure inclusive and equitable quality education and promote lifelong learning opportunities for all” (the access argument).

We may witness more and more governments and the EU requiring educational resources created in a government-funded or EU-funded project to be made publicly available. But an initiative as comprehensive as cOAlition S is not available for OER.

Why is this? What factors are at stake that could give us more insight into why a broad coalition for the free provision of publicly funded educational resources has not taken off? So far, we have identified the following arguments. We invite readers to respond, to propose additions and changes.

Costs
One of the factors seems to be costs. The costs of purchasing journals, in order to be able and allowed to consult publications, are direct expenses that are tangible for educational institutions. They are also visible in the annual accounts of libraries and institutions.

To give you an idea of expenditure: in the Netherlands it was 43 million euros (source, in Dutch) in 2015; worldwide in 2015 it was a profit margin (!) of 7.6 billion euros (source), which is roughly 30% of the turnover. Because institutions are directly affected by the still rising costs, this is an important factor in our eyes in explaining why the international research and education world has formed a coalition such as the cOAlition S.

Costs for making (open) educational resources are usually indirect and invisible in the annual accounts of educational institutions. They are mainly hidden in the “costs for employees”. The costs for students and self-learners do not even appear in the financial statements of institutions. In addition, there are the hidden costs of teachers who create and adapt materials in their own time.

We have not (yet) been able to find an estimate of the costs that are spent annually on the creation of learning materials. But based on the fact that in 2016 around 46,000 FTEs in the Netherlands were working as teachers or researchers (source, in Dutch) in higher education and research, with an average hourly wage bill of €60, and the assumption that 40 hours per FTE per year are spent on developing and adapting learning materials, we arrive at a conservative estimated annual cost of 40*46000*€60=€110 million per year. The situation in other countries may not differ. In our opinion, the amount of money involved justifies an initiative for OER, similar to cOAlition S.

Ecosystem
For scientific publications, there is an internationally recognized, highly developed and well-functioning ecosystem. This ecosystem ensures scientific progress. The fact that the revenues of this ecosystem are distributed and appropriated very unevenly is an accelerator in the public debate. This ecosystem is crucial for the functioning of other ecosystems, such as international scientific research and higher education (scientific and vocational). It is somewhat less connected to other forms of education, such as primary and secondary education and the TVET sector (Technical and Vocational Education and Training).

In the case of OER, such an ecosystem must in fact still be built up, and the ecosystem to be developed must also be linked to the existing education and training ecosystems.

Quality assurance system
The ecosystem for scientific publications comprises a well-known and recognized quality assurance system based on peer review. Therefore, anyone interested in a publication has the certainty that the publication meets a certain minimum quality standard. The undermining of this certainty by the aforementioned predatory journals is therefore disastrous; the system is in danger of providing false certainty.

Such a system does not (yet?) exist for OER. Institutions each have their own quality assurance system for OER, but it is often unknown on what such a system is based and on what aspects OER are considered. As a result, users of OER remain uncertain about their quality, and will have to make greater efforts to determine whether the learning materials they find actually have the quality they are looking for.

Value for the professional
Scientific, peer-reviewed publications make a significant contribution to the researcher’s reputation and thus to assessment and career development. Increasing the visibility of these results by publishing Open Access helps to increase that reputation, although, as mentioned above, there are also comments to be made by making the assessment depend too much on the number of publications.

Open sharing of educational resources has little or no impact on the reputation of teachers. As far as we know, open sharing of educational resources is hardly a factor in the assessment of instructors. Nobody mentioned this in our research in 2017 (source, in Dutch).

Conclusion
We have mentioned a number of factors that may play a role in the explanation of less interest in arriving at an initiative for OER, cOERalition S, comparable to cOAlition S for Open Access. However, we still have an unsatisfactory feeling that we have not yet been able to identify this difference sufficiently.

However, it is clear to us that as an OER community, we must work towards the creation of institutional OER ecosystems and, at the same time, a national OER ecosystem (including a known and recognized quality assurance system). This goes beyond ‘mainstreaming of OER’. At national level, an important argument is that educational resources that are made with public funds in education must be available under open licenses.

Naast een cOAlition S ook een cOERalition S?

alt="Photo by geralt on Pixabay"

Deze blogpost is een coproductie van Ben Janssen (OpenEd Consult) en mij.

Vorige week werd de cOAlition S gelanceerd door 11 nationale onderzoeksfinanciers (waaronder NWO), met support van de Europese Commissie en de European Research Council (ERC). In hun woorden (bron):

…an initiative to make full and immediate Open Access to research publications a reality. It is built around Plan S, which consists of one target and 10 principles.
cOAlition S signals the commitment to implement, by 1 January 2020, the necessary measures to fulfil its main principle: “By 2020 scientific publications that result from research funded by public grants provided by participating national and European research councils and funding bodies, must be published in compliant Open Access Journals or on compliant Open Access Platforms.”

Dit initiatief kreeg terecht veel publiciteit en wordt algemeen beschouwd als een forse stap naar realisatie van het streven van Open Science, waar onderzoekspublicaties direct en zonder voorwaarden kosteloos beschikbaar komen voor iedere belangstellende. Groei in de wetenschap is tenslotte geworteld in een traditie waarbij voortgebouwd wordt op eerder bereikte resultaten. Artikelen moeten, om geaccepteerd te worden voor publicatie, aantonen wat eerder onderzoek heeft opgeleverd en welke nieuwe bijdrage de publicatie heeft voor het betreffende onderwerp. Om dit optimaal te laten werken is een zo groot mogelijke toegankelijkheid van onderzoeksresultaten essentieel. Open Access publicaties zijn hiervoor een prima middel.

In één van die reacties gaf het NWO aan verder te willen gaan dan alleen de 10 stappen die in het initiatief worden genoemd, en ook te streven naar een andere waardering van wetenschappelijke prestaties dan het tellen van publicaties in peer reviewed journals. Daarmee wordt de druk op publiceren verminderd; een druk die nu nog veel onderzoekers ervan weerhoudt hun resultaten in een Open Access journal te publiceren. Die druk leidt overigens ook tot een andere perversiteit in het huidige systeem, de neptijdschriften (predatory journals). Deze journals verleiden onderzoekers hun resultaten snel gepubliceerd te hebben, zonder een gedegen peer review proces, uiteraard na betaling van vaak een aanzienlijk bedrag.

>> Interview met Stan Gielen “NWO wil weg van de impact factor”
>> Artikel over neptijdschriften

Ik heb me al eerder afgevraagd waarom een soortgelijk initiatief niet ook voor open beschikbaarheid van leermaterialen van de grond komt. Leermaterialen die met publieke gelden worden gemaakt zouden voor iedereen beschikbaar moeten zijn. Er is tenslotte al voor betaald (het morele argument). Net als bij onder Open Access publiceren van onderzoeksresultaten draagt open delen van leermaterialen bij aan kosteloze toegang tot kennis, waarmee een belangrijke bijdrage wordt geleverd aan het realiseren van SDG 4 van UNESCO: “to ensure inclusive and equitable quality education and promote lifelong learning opportunities for all” (het toegangsargument).

We kunnen constateren dat meer en meer overheden en de EU vereisen dat leermaterialen die tijdens een door de overheid of door de EU gefinancierd project worden gecreëerd, publiek beschikbaar moeten komen. Maar een zo alomvattend initiatief als cOAlition S is er niet voor open leermaterialen.

We hebben ons afgevraagd waar dat aan kan liggen. Welke factoren zijn in het geding die ons meer inzicht kunnen geven in waarom een brede coalitie voor het vrijelijk beschikbaar maken van publiek gefinancierde leermaterialen niet van de grond komt? Wij zijn tot de volgende argumenten gekomen. Wij nodigen lezers uit om daarop te reageren, om aanvullingen en wijzigingen voor te stellen.

Kosten
Een van de factoren lijkt ons de kosten te zijn. De kosten voor aanschaf van tijdschriften, om publicaties te kunnen en mogen raadplegen, zijn directe uitgaven die voelbaar zijn voor instellingen. Ze zijn ook zichtbaar in de jaarrekeningen van bibliotheken en instellingen.

Om een idee te geven van uitgaven: in Nederland ging het in 2015 om 43 miljoen euro (bron); wereldwijd in 2015 om een winstmarge (!) van €7,6 miljard (bron), hetgeen neerkomt op grofweg 30% van de omzet. Omdat instellingen direct met de nog steeds stijgende kosten te maken hebben, is dit onze ogen een belangrijke factor in de verklaring waarom internationaal de onderzoeks- en onderwijswereld tot een coalitie als de cOAlition S kan worden bewogen.

Kosten voor het maken van (open) leermaterialen zijn doorgaans indirect en onzichtbaar in de jaarrekeningen van onderwijsinstellingen. Ze zijn vooral verscholen in de “kostenpost werknemers”. De kosten voor studenten en self-learners komen überhaupt niet voor in de financiële overzichten van instellingen. Daarnaast zijn er nog de verborgen kosten van docenten die in hun eigen tijd materialen maken en aanpassen.

Een schatting van de kosten die jaarlijks aan het creëren van leermateriaal wordt besteed hebben we (nog?) niet kunnen vinden. Maar met het gegeven dat in Nederland in 2016 rond de 46.000 fte in hbo en wo werkzaam zijn als docent of onderzoeker (bron), met gemiddelde loonlasten per uur van €60, en de aanname dat per fte per jaar 40 uur besteed wordt aan ontwikkelen en aanpassen van leermaterialen, komen wij uit op geschatte jaarlijkse kosten van 40*46000*€60=€110 miljoen per jaar. Dit bedrag rechtvaardigt ons inziens een soortgelijk initiatief als cOAlition S voor open leermaterialen.

Ecosysteem
Voor wetenschappelijke publicaties bestaat er een internationaal erkend, hoog-ontwikkeld en goed functionerend ecosysteem. Dit ecosysteem zorgt voor wetenschappelijke vooruitgang. Dat de revenuen van dat ecosysteem zeer ongelijk verdeeld en toegeëigend worden is een versneller in de publieke discussie. Dit ecosysteem is van cruciaal belang voor het functioneren van andere ecosystemen, zoals het internationaal wetenschappelijk onderzoek, en het hoger (wetenschappelijk en beroeps) onderwijs. Het is wat minder verbonden met andere vormen van onderwijs, zoals PO, VO en de TVET-sector (Technical and Vocational Education and Training).

Bij Open Educational Resources moet een dergelijk ecosysteem feitelijk nog worden opgebouwd, en moet dat te ontwikkelen ecosysteem ook nog een keer verbonden worden met de bestaande ecosystemen van onderwijs en training.

Systeem van kwaliteitsborging
Het ecosysteem voor wetenschappelijke publicaties omvat een bekend en erkend systeem voor kwaliteitsborging, gebaseerd op peer review. Een belangstellende voor een publicatie heeft daarmee de zekerheid dat de publicatie voldoet aan een zekere minimum kwaliteit. Het ondermijnen van die zekerheid door de eerder genoemde predatory journals is daarom funest; het systeem dreigt een schijnzekerheid te geven.

Voor open leermaterialen bestaat een dergelijk systeem (nog?) niet. Instellingen hebben ieder hun eigen systeem van kwaliteitsborging van leermaterialen, maar het is veelal onbekend waarop een dergelijk systeem is gebaseerd en op welke aspecten leermaterialen bekeken wordt. Een gebruiker van open leermaterialen blijft daardoor onzeker over de kwaliteit en zal meer inspanningen moeten verrichten om te bepalen of het gevonden leermateriaal inderdaad de kwaliteit heeft die hij/zij zoekt.

Waarde voor de professional
Wetenschappelijke, peer reviewed publicaties dragen in hoge mate bij aan de reputatie van de onderzoeker en daarmee aan beoordeling en carrière. Grotere zichtbaarheid van die resultaten door Open Access publiceren helpt die reputatie te verhogen, hoewel er zoals eerder vermeld ook kanttekeningen hierbij te plaatsen zijn door teveel de beoordeling te laten afhangen van aantallen publicaties.

Open delen van leermaterialen heeft deze impact op reputatie niet of nauwelijks voor docenten. Open delen van leermaterialen speelt voor zover wij weten ook nauwelijks mee bij beoordelingen van docenten. In ons onderzoek uit 2017 is dit door niemand genoemd (bron)

 

Conclusie
We hebben een aantal factoren genoemd die een rol kunnen spelen in de verklaring van mindere belangstelling om te komen tot een initiatief voor open leermaterialen, cOERalition S, te vergelijken met cOAlition S voor Open Access. We blijven echter zitten met een onbevredigend gevoel dat we nog onvoldoende de vinger hebben kunnen leggen op de kern van dit verschil.

Voor ons is evenwel duidelijk dat we als OER community moeten werken aan het tot stand brengen van institutionele OER-ecosystemen en tegelijkertijd ook een nationaal OER-ecosysteem (waaronder een gekend en erkend systeem voor kwaliteitsborging). Dit gaat verder dan “mainstreaming of OER”. Nationaal is een belangrijk argument dat leermaterialen die in het onderwijs met publieke gelden worden gemaakt, onder open licenties beschikbaar moeten zijn.

Op 26 oktober organiseert SURF in samenwerking met de SIG Open Education een seminar “Open Science meets Open Education”. In dit seminar geef ik een sessie over het kwaliteitsvraagstuk waarover ik in deze blog spreek. Meer details.

Open Education & Open Science: an integrated approach?


Last Friday I was present at the Open Science seminar, organized by the TU Delft as an event in the Open Education Week. The intended audience were the people at the TU Delft, but also some attendants from other institutions (among myself) were present. Through this seminar we were informed about developments in both Open Education and Open Science, in general and more specific at the TU Delft.
After an introduction by Anka Mulder about openness in general, in two presentations the fields of Open Education and Open Science were introduced. A more detailed report about these presentations and the discussions afterwards is made by Pim Bellinga.
The lively discussions between the attendants were based on several dilemmas on both open education and open science. To get an impression of these dilemmas: “If I share my education resources openly, I give away my complete education” and “If anyone can use my measurement data, then I did all the work and had the costs, and someone else might get the success” are two examples.
Afterwards, I was reflecting on the seminar. I had hoped to hear more about cross-overs, examples where approaching open education and open science simultaneously would make a difference that would not be possible when they are considered two separate fields. Obvious examples of cross-overs are Open Access papers becoming OER when used in an educational setting or using MOOCs as a means to collect research data or as a means to disseminate results of research to a wider audience. Examples of the latter two were presented at the seminar. A less obvious example comes from Javiera Atenas and Leo Havemann, using open data as OER.
But (to quote Peggy Lee) “Is that all there is?” Does that justify an integrated approach, e.g. in policy making, at an institution? Reality is still that in many institutions education and research are two separate branches with different processes and different stakeholders. And although some descriptions of Open Science includes OER (example), other do not mention them (as in this interactive example from the website of the European Union, presented at the seminar).
Discussing this with Martijn Ouwehand from the TU Delft, I realized that the dilemmas both fields are facing are similar:

  • Fear of commercial use of openly available resources
  • Fear of free riding
  • Uncertainty about copyright issues
  • Fear of harming your career when involved in openness, because of possible quality flaws. The latter can also be looked at from a broader perspective, where being involved in education is considered harmful for a scientific career. Last week in his inaugural speech, professor Jan-Willem van Groenigen (Wageningen University & Research) pronounced ambitious researchers to minimize educational activities, because these activities do not count when applying for important research grants in the Netherlands (see interview (in Dutch) in the Volkskrant)

Also, the current focus in open education is to go beyond just publishing and reuse resources and have more attention for Open Educational Practices and Open Pedagogy. One way to realize this is to have students performing research activities in projects with stakeholders outside of the institution as a means to acquire certain knowledge and skills: a perfect example of intertwining education and research in an open way. One of the attendants pointed at the phenomenon of Citizen Science, where connections with educational projects under the umbrella of Open Pedagogy seem rather straightforward.
So the cross-over is not only determined by examples like I mentioned (and I am hoping to find out about more types of examples), but also by the dilemmas and comparable challenges both fields are facing and the opportunities that intertwining education and research in an open way can bring. We can learn from each other about how to overcome those challenges and cooperate in realizing the intertwining.
For me, this is sufficient justification for an approach where policy making, creating awareness and activities to adopt openness will be integrated for both research and education, with an open eye for the differences in both fields that also exist. And, by the way, let’s not forget to learn from the field of open source software, with a much longer history than at least OER.