Open Praxis,
Apr 09, 2020
Abstract
Open Education gained more visibility as a result of the emergence of Open Educational Resources (OER) and Massive Open Online Courses (MOOCs). This article discusses whether MOOCs should be considered as OER. Open Education and OER can be treated as two strands with different historical roots even though, in theory, OER are an aspect of Open Education. Different OER definitions and typologies are analyzed in relation to their dimensions and categorizations. Furthermore, the four conditions and two original categories of MOOCs are discussed, leading to a debate on their quality. It turns out that there are two perspectives on MOOCs: from an OER perspective, MOOCs as a product can be called OER. From an Open Education perspective, MOOCs are going beyond OER as enablers of Open Education and are understood as an innovative way of changing education. These perspectives are reflected by the OpenEd Quality Framework. The short answer to our leading question is: sometimes, and it depends on your perspective.
Keywords: Open Education, Open Learning, Massive Open Online Courses, Open Educational Resources, literature review, OpenEd Quality Framework
Introduction
The topic of Open Education has become increasingly complex in recent years. The first of a series intended to contribute to a better understanding of that complexity, this article considers whether Massive Open Online Courses (MOOCs) should be thought of as Open Educational Resources (OER). This question is important because it addresses the distinction between a conception of Open Education as based on open content and Open Education as based on open and innovative pedagogy.
Open Education is a broad concept with a lively history (Nyberg, 1975; Stracke, 2018). Unfortunately, there is no stable and commonly shared definition of Open Education. This has led to differences of opinions and to a certain confusion regarding the term (Cronin, 2017). The meaning of Open Education has changed over time. In the last century, characterised by the rise of institutions such as the Open Universities, Open Education was associated with open admissions and distance education (Weller, Jordan, DeVries, & Rolfe, 2018). More recently, Open Education has been thought of in association with the introduction of OER and MOOCs (Gaskell & Mills, 2014; Stracke, 2015). Thus, the current focus of Open Education is different from the past (Mulder, 2013; Nascimbeni, Burgos, Campbell & Tabacco, 2018).
Within the broad field of Open Education, MOOCs and OER are both quite new concepts, though they correspond to elements of the original definition: the course itself, and the course resources (or course package). As a starting point, the authors follow the UNESCO definition of OER as "teaching, learning and research materials in any medium, digital or otherwise, that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation and redistribution by others with no or limited restrictions" (UNESCO, 2012, p. 1). Meanwhile, the first MOOC was offered in 2008. Since then, the number of MOOCs, MOOC providers, and the number of MOOC-taking students have continuously increased (Hilton, Fischer, Wiley, & Williams, 2017). In this paper, the authors consider both the historical and contemporary roots of OER and MOOCs.
This paper has three major parts. In the first part, different definitions and typologies of OER are analyzed and compared in relation to their dimensions and categorizations. Following this, definitions and usages of the term MOOCs are presented and related to the standard definition of OER. Furthermore, the quality of MOOCs is discussed –introducing the OpenEd Quality Framework as a theoretical basis. Finally, the leading question "Are MOOCs Open Educational Resources?" is debated and answered from the two perspectives of OER and Open Education.
History, Definitions and Typologies of OER
The concept of OER is based on a long history with multiple roots. On the one hand, OER is associated with the trajectory of Open Education and Open Learning that can be traced back for several thousands of years (Nyberg, 1975; Stracke, 2019). Thus, the nature of OER has its roots in the principles of instructional design for open and distance learning and education. On the other hand, OER has more recently been associated with the idea of open content (e.g., as defined by Wiley, 2007), which in turn was based on the idea of free and open source software. Hence, there is a more recent emphasis on licensing in OER that was not reflected in the original Open Education movement. Thus, Open Education and OER can be seen and treated as two strands with different developments (and their own citation circles) even though, in theory, OER is an aspect of Open Education. In the following pages, we describe the rise and history of the OER movement starting at the beginning of our century and compare proposed definitions and typologies of OER (Downes, 2007; D'Antoni, 2009; McAndrew, 2010).
The origin of OER is based on the common and widespread practice of creating and sharing learning resources. While MIT's OpenCourseWare project is often described as the first instance of OER (e.g., by Knox, 2013), it is a relative newcomer, having been launched only in 2001, and was preceded, among others, by shared lesson plans, libraries of resources available through Gopher, early websites (such as Downes, 1996), open software documentation, and more.
The commonly accepted origin of the term OER is the 2002 UNESCO Forum on the Impact of Open Courseware for Higher Education in Developing Countries. Its Final Report defines Open Educational Resources as "the open provision of educational resources, enabled by information and communication technologies, for consultation, use and adaptation by a community of users for non-commercial purposes" (UNESCO, 2002, p. 24).
In 2007, a revised definition was proposed in a report to the William and Flora Hewlett Foundation, which had funded many early OER initiatives (Atkins, Brown, & Hammond, 2007). This OER definition includes non-digital resources and focuses on different types of OER: "OER are teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use or re-purposing by others. Open educational resources include full courses, course materials, modules, textbooks, streaming videos, tests, software, and any other tools, materials, or techniques used to support access to knowledge" (Atkins, Brown, & Hammond, 2007, p. 4) In the years that followed, several declarations and guidelines were developed to support the spread of the OER movement, such as the Cape Town Open Education Declaration (2007), the Dakar Declaration on OER (2009) and the Guidelines on Open Educational Resources in Higher Education (2011) published by Commonwealth of Learning and UNESCO.
A milestone was the first World OER Congress organized by UNESCO. It approved the 2012 Paris OER Declaration (UNESCO, 2012) with its broader OER definition: "teaching, learning and research materials in any medium, digital or otherwise, that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation and redistribution by others with no or limited restrictions. Open licensing is built within the existing framework of intellectual property rights as defined by relevant international conventions and respects the authorship of the work" (UNESCO, 2012, p. 1).
It is worth underlining that two restrictions of the earlier OER definitions from UNESCO (2002) and from the William and Flora Hewlett Foundation (Atkins, Brown, & Hammond, 2007) are not considered in current versions: the non-commercial purposes and the enabling by information and communication technologies. Thus, any purposes and any resources (digital as well as non-digital) are covered by the term OER today, according to UNESCO (2012).
The second World OER Congress organized by UNESCO took place in 2017 and led to the Ljubljana OER action plan (UNESCO, 2017). This ambitious policy calls for the development of OER as enablers of Open Education and changes towards innovative education and pedagogical strategies and has resulted in the production of a UNESCO Recommendation focusing on OER to all its member states. The Recommendation focuses on five objectives: 1) building capacity of stakeholders to create access, use, adapt and redistribute OER; 2) developing supportive OER policy; 3) encouraging the development of inclusive and equitable quality OER; 4) nurturing the creation of sustainability models for OER; and 5) facilitating international cooperation on OER. In this recommendation, OER are defined as "teaching, learning and research materials in any medium that may be composed of copyrightable materials released under an open license, materials not protected by copyright, materials for which copyright protection has expired, or a combination of the foregoing" (UNESCO, 2019).
With regards to the typologies of OER, there are many proposals (see Conole & Brown, 2018). One early popular proposal was Wiley's 4R framework, based on the four usage types of OER: reuse, revise, remix, and redistribute (Wiley, 2007). He later amended it to the 5R framework adding a fifth usage type: retain (Wiley, 2014). Another categorization was proposed by Tuomi (2013) that defined four hierarchical types of OER: type OER I guarantees access, OER II adds usage rights, OER III adds adaptation rights and OER IV finally adds re-distribution rights (Tuomi, 2013). These categorisations of OER focus mainly on the legal and operational dimensions and do not address other dimensions such as open recognition, methodologies and innovations (Stracke, 2018).
For the application and re-usage of OER, several frameworks were developed for the learning design and quality development of Open Education:
- Tuomi (2013) analyses learning with OER as being based on the four pillars for holistic and learner-centered education and learning defined by the UNESCO Report (1996).
- Puentedura (2013) employs the SAMR model focusing four levels of technology integration for the learning design: substitution, augmentation, modification and redefinition.
- The ICAP Framework by Chi and Wylie (2014) underlines the importance of four modes for the learners' engagement behaviours: Interactive, Constructive, Active, and Passive.
- Conole (2015) introduced the 7Cs of Learning Design Framework: Conceptualise (for vision building), Create, Communicate, Collaborate, Consider (as five key activities), Combine (for synthesis building) and Consolidate (for implementation).
Concerning the overall benefits of OER, Butcher and Moore (2015) distinguish three main aspects of OER:
- "Increased availability of high quality, relevant learning materials can contribute to more productive students and educators [...]
- The principle of allowing adaptation of materials provides one mechanism amongst many for constructing roles for students as active participants in educational processes [...]
- OER has potential to build capacity by providing institutions and educators access, at low or no cost, to the means of production to develop their competence in producing educational materials and carrying out the necessary instructional design [...]" (Butcher, & Moore, 2015, p. 13).
The OpenEdOz project identified six key benefits of Open Education (OpenEdOz, 2016): 1) economies of scale by the collaborative co-production of learning resources, 2) quality of learning can be raised at decreased time and financial cost, 3) OER are richer and more appropriate to the learning contexts and styles of an increasingly diverse student community, 4) learning opportunities for disadvantaged communities globally and for remote and regional areas, 5) greater collaboration between learning providers through peer review and collegial development of learning materials, and 6) facilitation of greater levels of transparency into the educational processes. Furthermore, the United Nations' Sustainable Development Goal 4 (SDG) calls for actions to "Ensure inclusive and quality education for all and promote lifelong learning" and promotes OER for the realization and implementation (United Nations, 2015).
We have seen therefore that since the introduction of OER in 2002 several categorizations have been proposed, based on usage rights, applicability to learning design, and benefits. These typologies speak not only to the applicability of OER, but also to the changing conception, over time, of what OER are.
History, Definitions and Typologies of MOOCs
The history of open online courses did not begin with Massive Open Online Courses (MOOCs). Arguably, open online learning began with e-mail-based courses in the 1990s (Smith, Whiteley, & Smith, 1999; Abdolrasulnia et al., 2004; Hodges, 2008). Additionally, open online learning in the form of self-paced web-based courses began almost as soon as the web was popularized in the late 1990s and early 2000s (Wiley & Gurrell, 2009). Thus, MOOCs were preceded by both open online courses as well as by OER movement. The first open online course to be called a 'MOOC' was "Connectivism and Connective Knowledge" (CCK08) organized by Stephen Downes and George Siemens in the year 2008 (Bozkurt, Kilgore, & Crosslin, 2018). CCK08 was not content-focused; instead it emphasized network formation among participants and the sharing of resources and contributions across those networks. This type of MOOC, based on a 'connectivist' pedagogy, was later called a 'cMOOC'.
A second type of MOOC emerged in 2011. Called the 'xMOOC', its design emphasized traditional educator-led instruction with the focus on providing content to a massive public audience (Downes, 2007; Stracke, 2017a). The first xMOOC is widely thought to have been Norvig and Thrun's 'Artificial Intelligence', which attracted more than 150,000 participants, though some educators have subsequently made their own claim to being the first (Davidson, 2013).
Since then, the number of MOOCs has continually grown (Gaskell, & Mills, 2014). A highlight of MOOC development was the calling of 2012 as the "Year of the MOOCs" by the New York Times. At the same time, the concept of MOOC was criticized as the "educational buzzword of 2012" (Daniel, 2012, p. 1). In the years that followed, educators evaluated and debated the quality of MOOCs and their educational value (Liyanagunawardena, Adams, & Williams, 2013; Veletsianos, & Shepherdson, 2016; Stracke, 2018; Zawacki-Richter, Bozkurt, Alturki, & Aldraiweesh, 2018). Despite some misgivings, the number of registered MOOCs (9,400 as of 2018), participating MOOC learners (81 Million) and MOOC providers (800+) have been continuously increasing, according to the MOOC aggregator website Class Central (Shah, 2018).
Any definition of the concept 'MOOC' will start with the four components that make up the abbreviation: massive, open, online and course. But questions have been raised about each of the four terms and their definitions and interpretations:
1. MOOCs as "Massive". The term 'massive' may be thought in terms of impact, that is, a course is a MOOC if (and only if?) it enrolls massive numbers of students, or in terms of design, where a course is a MOOC if it could enroll massive numbers of students, even if it actually fails to do so. cMOOCs and xMOOCs create mass differently, the former through the use of decentralized networks, and the latter through scalable cloud services and automation. As a quantity, the term 'massive' is open to multiple interpretations, though as a starting point a threshold of 150 learners may be considered, based on Dunbar's (1998) number, signifying the point at which a MOOC graduates from being a 'group' where everyone knows each other, to a 'network' characterized by interactions. As the number of MOOCs is growing and as a result of international competition, the number of registered MOOC learners per course is decreasing, but most MOOCs are still register far more than several hundred participants.
2. MOOCS as "Open". Openness can be considered the biggest challenge for MOOCs and their quality. On the one hand, openness means open access (no requirement to sign up, no admission requirements, no fee, etc.) but some courses called 'MOOCs' are not freely available and so it was argued they should not be labelled open. Critics argued that the courses offered by Coursera and Udacity should not be called open because the contents are not openly licensed. Disagreement in the MOOC community about the meaning of "open" deepened with the development of MOOC-based business models that, for example, often charge certification fees for having completed a MOOC. Further, "Open" does not mean necessarily "Universal". A MOOC can be open for a whole learning community (e.g. a university), counting thousands of users, but restricted for outer login. In addition, others argue that openness should be related to open methodologies, i.e., to innovative approaches for learning and education (Gaskell & Mills, 2014; Stracke, 2017a).
3. MOOCs as "Online". This condition is almost always met and easy to achieve: MOOCs have to be offered and provided online as otherwise they cannot reach the masses of interested MOOC learners and participants. That means that there should be no requirement for offline activities for full participation in the MOOC, even though from the moment the first MOOCs appeared there were initiatives like MeetUps organized by local learners' groups. But there are also a few MOOCs distributed for offline usage by learners that are lacking online internet connectivity. In addition, some institutions employed the concept of 'wrapped MOOCs' which limited participation to those registered for an associated in-person course 'wrapped' around the MOOC content (Zawacki-Richter et al., 2018, Jaffer, Govender, & Brown, 2017). 4. MOOCs as "Courses". The term 'course' can be defined specifically to mean a series of events with a fixed start date, a fixed end-date, and a common theme in the middle. The original cMOOC was based on the old model of 'a course of lectures', which would be organized by students and offered by a professor, but without the trappings of what we now call a 'traditional' course with assignments and grades, etc. xMOOCs, meanwhile, resembled the traditional model of educator-led instruction. Today, most MOOCs are offering a blend of different models and are offered over a short period of time, normally between five and eight weeks.
Following the popularity of MOOCs, many different models of MOOC-like courses were proposed, often with the intention of addressing perceived shortcomings in the original MOOC model. Some examples include the SPOC (Small Private Online Course), developed to meet the need for more personal contact in courses next to many other proposed combinations and acronym inventions leading to a rather diverse landscape of current MOOC practices and raising the overall quality of MOOCs (Daniel, 2012; Gaskell, & Mills, 2014; Reich, 2015; Stracke, 2019).
The Quality of MOOCs and OER
If we are asking whether MOOCs are OER, then it matters what MOOCs and OER are intended to be. One way to consider what MOOCs are intended to be is to ask what would constitute quality in a MOOC. That is the approach we take in this section.
Since their introduction, the quality of MOOCs has been challenged and questioned by numerous researchers (Stracke, 2017a, 2017b). For example, Weller et al. (2018) argue that many MOOC designers and providers have largely ignored previous literature on quality in distance and e-learning. Additionally, some early studies (e.g., the University of Pennsylvania study by Christensen et al., 2013) focused on metrics like student demographics and course completion.
More recently, an organization called the MOOQ Alliance developed a Quality Reference Framework (QRF) for evaluating and improving the quality of MOOCs (Stracke et al., 2018a). It addresses the adoption, the design, the delivery and the evaluation of MOOCs in order to better enable MOOC designers, facilitators and providers to support the benefit of the learners. The QRF is based on a mixed methods research methodology and included a Global MOOC Quality Survey (GMQS), literature review, interviews, and MOOQ presentations and workshops, at regional, European and international conferences involving more than 10,000 MOOC learners, designers, facilitators and providers. Initial findings suggest that a gap exists between MOOC designers' perspectives and learners' preferences on interactions (Stracke et al., 2018b; Stracke & Tan, 2018).
With respect to OER, the OpenEd Quality Framework (Stracke, 2019) can serve as an additional instrument (figure 1). Research supporting the OpenEd Quality Framework (Stracke, 2018) is based on the transfer of the three generic dimensions of quality ('potential', 'process', 'result') to educational applications. These dimensions are derived from Total Quality Management (TQM) with a continuous improvement cycle introduced mainly by Deming (1982; 1986) and Juran (1951; 1992). Their implementation here is similar to the way Donabedian (1980) implemented them in health care. Here they are adapted to Open Education such that they can be combined with the three educational levels (macro, meso, micro) and represented as 'objectives', 'realizations' and 'achievements' (Stracke, 2019).
Figure 1: The OpenEd Quality Framework.
Does the research and literature on the quality of MOOCs support the idea that the OpenEd Quality Framework could apply to them as well as to OER? This is what we examine in the next section.
From a Quality Perspective: are MOOCs a Special Type of OER?
We suggest that whether a MOOC is considered to be a type of OER will depend on the perspective that we take.
If we look at MOOCs from a resources point of view (having in mind Wiley (2014)'s 5 Rs, for example), that is, if we consider them as content-based courses, the value of which is based mainly in the quality of their content, then in many cases MOOCs are not OER, since they are much more difficult to re-use and redistribute by virtue of their size and complexity or even cannot be re-used and redistributed due to restrictions in their licensing. From this perspective, MOOCs are normally not thought of as OER, except in rare cases. These rare cases are those in which the MOOCs are licensed to allow re-use and adaptation. Such MOOCs could be categorized either as a single OER, which would create a specific sub-type of OER due to their typical large size, or they could be considered as collections of multiple OER, raising the question of how easily MOOCs could be opened up to provide access to these resources (Nascimbeni, 2018).
If we look at MOOCs from a learning innovation point of view, they are potentially much more than merely OER as they include not only resources but also pedagogical methods and pathways (even though as many MOOCs are less innovative than one might hope for). If Open Education is primarily understood as way to include innovative concepts and methodologies for the creation of collaborative and supportive learning experiences, then MOOCs can go beyond OER as a strong instrument to transform and improve the educational quality with a focus on peer learning and online communities.
Normally, MOOCs are not understood as static, as in the first perspective, but as involving lively processes and contents over several weeks that encourages communication and collaboration with other learners and are supported by moderation and tutoring, as in perspective 2. This is especially the case for the cMOOC, though we observe that students develop their own collaborative and supportive elements also in xMOOCs. Additionally, MOOCs offering and benefitting from re-used and adapted OER can be labelled as OER if considered and addressed as a whole and a product. Furthermore, some MOOCs are licensing all their materials as OER and curating them outside the MOOC platform for easy re-usage and adaptation. Also, the intent of the possible MOOCs intermediaries (i.e., teachers, facilitators and tutors) is key. It mainly depends how an educator (or a learning community) is using a MOOC, whether using parts of MOOCs as content nuggets, embedding a full MOOC in a course or laboratories, recommending MOOCs as additional course content or using MOOCs as triggers for international collaboration or virtual mobility experiences. These are all ways of using MOOCs that can support their function as OER.
Conclusion and Outlook
In this paper we examined the history and nature of both OER and MOOCs. We found that in both cases their nature can be understood by taking a quality-centred perspective. The quality of these resources, in turn, depends and can be represented in both cases according to the objectives, realizations and outcomes of the resource. If one takes an open resources perspective, the quality of the content per se is insufficient to establish quality of process and outcome, since quality content by itself supports neither licensing for reuse and adaptation, not does it support innovation in learning experiences. Thus, many MOOCs (and especially, many xMOOCs) are not OER. However, depending on the intent of the educational intermediary, MOOCs could indeed be considered to be a category of OER. If one takes an open learning innovation perspective, we can state that MOOCs go beyond OER and they can be seen as enablers for innovative learning processes and experiences. In this understanding, MOOCs are indeed not resources but learning opportunities and environments for self-regulated as well as collaborative learning.
To summarize, the terms Open Education, MOOCs and OER need a clear basic definition of their meaning and perspective for their usage. We hope that this first article of our series on Open Education contributes to better understanding and broader application of MOOCs and OER and of Open Education in general to improve our future learning and education.
References
Abdolrasulnia, M., Collins, B.C., Casebeer, L., Wall, T., Spettell, C., Ray, M.N., ... & Allison, J.J. (2004). Using email reminders to engage physicians in an Internet-based CME intervention. BMC medical education, 4(1), 17.
Atkins, D.E., Brown, J.S., & Hammond, A.L. (2007). A Review of the Open Educational Resources (OER) Movement: achievements, challenges, and new opportunities. Menlo Park: The William and Flora Hewlett Foundation.
Bozkurt, A., Kilgore, W., & Crosslin, M. (2018). Bot-teachers in hybrid massive open online courses (MOOCs): A post-humanist experience. Australasian Journal of Educational Technology, 34(3), 39–59. https://doi.org/10.14742/ajet.3273
Butcher, N., & Moore, A. (2015). Understanding Open Educational Resources. Kingsway: Commonwealth of Learning.
Chi, M.T.H., & Wylie, R. (2014). The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes. Educational Psychologist, 49(4), 219–243.
Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., & Emanuel, E. (2013). The MOOC Phenomenon: Who Takes Massive Open Online Courses and Why? https://doi.org/10.2139/ssrn.2350964
Conole, G. (2015). Designing effective MOOCs, Educational Media International, 52(4), 239–252. https://doi.org/10.1080/09523987.2015.1125989 Conole, G., & Brown, M. (2018). Reflecting on the impact of the Open Education Movement. Journal of Learning for Development, 5(3), 187–203. Retrieved from https://jl4d.org/index.php/ejl4d/article/view/314/346
Cronin, C. (2017). Openness and praxis: Exploring the use of open educational practices in higher education. The International Review of Research in Open and Distributed Learning, 18(5). https://doi.org/10.19173/irrodl.v18i5.3096
Daniel, J. (2012). Making Sense of MOOCs: Musings in a Maze of Myth, Paradox and Possibility. Retrieved from http://sirjohn.ca/wordpress/wp-content/uploads/2012/08/120925MOOCspaper2.pdf
D'Antoni, S. (2009). Open Educational Resources: reviewing initiatives and issues, Open Learning: The Journal of Open, Distance and e-Learning, 24(1), 3–10. https://doi.org/10.1080/02680510802625443
Davidson, C. (2013). What Was the First MOOC? Retrieved from https://www.hastac.org/blogs/cathydavidson/2013/09/27/what-was-first-mooc Deming, W.E. (1982). Quality, productivity and competitive position. Cambridge, MA: MIT.
Deming, W.E. (1986). Out of the Crisis. Cambridge, MA: MIT.
Donabedian, A. (1980). The Definition of Quality and Approaches to Its Assessment [= Explorations in Quality Assessment and Monitoring, vol. 1]. Ann Arbor: Health Administration Press.
Downes, S. (1996). Stephen's Guide to the Logical Fallacies. Retrieved from: https://www.fallacies.ca/welcome.htm
Downes, S. (2007). Models for sustainable open educational resources. Interdisciplinary Journal of Knowledge and Learning Objects, 3. Retrieved from http://ijklo.org/Volume3/IJKLOv3p029044Downes.pdf
Dunbar, R. (1998). The Social Brain Hypothesis. Evolutionary Anthropology, 6(5), 178–190. https://doi.org/10.1002/(SICI)1520-6505(1998)6:5%3C178::AID-EVAN5%3E3.0.CO;2-8
Gaskell, A., & Mills, R. (2014). The quality and reputation of open, distance and e-learning: what are the challenges? Open Learning, 29(3), 190–205. https://doi.org/10.1080/026805
13.2014.993603 Hilton, J., III, Fischer, L., Wiley, D., & Williams, L. (2017). Maintaining momentum toward graduation: OER and the course throughput rate. The International Review of Research in Open and Distributed Learning, 17(6). https://doi.org/10.19173/irrodl.v17i6.2686 Hodges, C. (2008). Self-efficacy, motivational email, and achievement in an asynchronous math course. Journal of Computers in Mathematics and Science Teaching, 27(3), 265–285.
Jaffer, T., Govender, S., & Brown, C. (2017). "The best part was the contact!": Understanding postgraduate students' experiences of wrapped MOOCs. Open Praxis, 9(2), 207–221. https://doi.org/10.5944/openpraxis.9.2.565 Juran, J.M. (1992). Juran on quality by design. The new steps for planning quality into goods and services. New York: Free Press.
Juran, J.M. (Ed.) (1951). Quality Control Handbook. New York: McGraw-Hill.
Knox, J. (2013). The Limitations of Access Alone: Moving Towards Open Processes in Education Technology. Open Praxis, 5(1), 21–29. http://dx.doi.org/10.5944/openpraxis.5.1.36
Liyanagunawardena, T., Adams, A., & Williams, S. (2013). MOOCs: A systematic study of the published literature 2008–2012. The International Review of Research in Open and Distributed Learning, 14(3), 202–227. https://doi.org/10.19173/irrodl.v14i3.1455
McAndrew, P. (2010). Defining openness: updating the concept of 'open' for a connected world. Journal of Interactive Media in Education, 2010/10, 1–13.
Mulder, F. (2013). The logic of national policies and strategies for open educational resources. The International Review of Research in Open and Distributed Learning, 14(2), 96–105. https://doi.org/10.19173/irrodl.v14i2.1536
Nascimbeni, F. (2018). "The night of the living MOOCs": a feasible and high-impact proposal. Retrieved from https://education.okfn.org/the-night-of-the-living-moocs-a-feasible-and-high-impactproposal/ Nascimbeni, F., Burgos, D., Campbell, L.M., & Tabacco, A. (2018). Institutional mapping of open educational practices beyond use of Open Educational Resources. Distance Education, 39(4), 511–527. https://doi.org/10.1080/01587919.2018.1520040
Nyberg, D. (1975). The philosophy of open education. London: Routledge and Kegan Paul.
OpenEdOz (2016). Students, universities and Open Education, final report. Retrieved from http://openedoz.org/wpcontent/uploads/2017/02/ID14-3972_CSU_Wills_Final-Report_2016.pdf
Puentedura, R.R. (2013). SAMR and TPCK: An introduction. Retrieved from http://www.hippasus.com/rrpweblog/archives/2013/03/28/SAMRandTPCK_AnIntroduction.pdf Reich, J. (2015). Rebooting MOOC research. Science, 347(6217), 34–35. https://doi.org/10.1126/science.1261627
Shah, D. (2018, January 22). A Product at Every Price: A Review of MOOC Stats and Trends in 2017. Class Central. Retrieved from https://www.class-central.com/report/moocs-stats-andtrends-2017
Smith, C.D., Whiteley, H.E., & Smith, S. (1999). Using email for teaching. Computers & Education, 33(1), 15–25.
Stracke, C.M. (2015). The Need to Change Education towards Open Learning. In C.M. Stracke & T. Shamarina-Heidenreich (Eds.), The Need for Change in Education: Openness as Default? (pp. 11–23).
Stracke, C.M. (2017a). The Quality of MOOCs: How to improve the design of open education and online courses for learners? In P. Zaphiris & A. Ioannou (Eds.), Learning and Collaboration Technologies. Novel Learning Ecosystems. LCT 2017, Part I, LNCS 10295 (pp. 285–293). https://doi.org/10.1007/978-3-319-58509-3_23
Stracke, C.M. (2017b). Why we need High Drop-out Rates in MOOCs: New Evaluation and Personalization Strategies for the Quality of Open Education. In M. Chang, N.-S. Chen, R. Huang, Kinshuk, D.G. Sampson, & R. Vasiu (Eds.), The 17th IEEE International Conference on Advanced Learning Technologies (ICALT 2017) (pp. 13–15). IEEE: Computer Society. https://doi.org/10.1109/ICALT.2017.109
Stracke, C.M. (2018). Como a Educação Aberta pode melhorar a qualidade de aprendizagem e produzir impacto em alunos, organizações e na sociedade? In M. Duran, T. Amiel, & C. Costa (Eds.), Utopias and Distopias da Tecnologia na Educação a Distância e Aberta (pp. 499–545). Campinas: & Niterói: UNICAMP & UFF.
Stracke, C.M. (2019). Quality Frameworks and Learning Design for Open Education. The International Review of Research in Open and Distributed Learning, 20(2), 180–203. https://doi.org/10.19173/irrodl.v20i2.4213
Stracke, C.M., & Tan, E. (2018). The Quality of Open Online Learning and Education: Towards a Quality Reference Framework for MOOCs. In Rethinking learning in the digital age. Making the Learning Sciences Count: International Conference of the Learning Sciences (pp. 1029–1032). Retrieved from http://hdl.handle.net/1820/9909
Stracke, C.M., Tan, E., Texeira, A.M., Pinto, M., Kameas, A., Vassiliadis, B., & Sgouropoulou, C. (2018b). Gap between MOOC designers' and MOOC learners' perspectives on interaction and experiences in MOOCs: Findings from the Global MOOC Quality Survey. In Proceedings 18th IEEE International Conference on Advanced Learning Technologies (pp. 1–5). https://doi.org/10.1109/ICALT.2018.0000
Stracke, C.M., Tan, E., Texeira, A., Pinto, M., Vassiliadis, B., Kameas, A., Sgouropoulou, C., & Vidal, G. (2018a). Quality Reference Framework (QRF) for the Quality of Massive Open Online Courses (MOOCs). Retrieved from http://www.mooc-quality.eu/QRF
Tuomi, I. (2013). Open Educational Resources and the Transformation of Education, European Journal of Education, 48(1), 58–78. https://doi.org/10.1111/ejed.12019 UNESCO (2002). Forum on the Impact of Open Courseware for Higher Education in Developing Countries: Final report (CI-2002/CONF.803/CLD.1). Paris: UNESCO. Retrieved from http:// unesdoc.unesco.org/images/0012/001285/128515e.pdf UNESCO (2012). 2012 Paris OER Declaration (Paris, UNESCO). Retrieved from www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/Events/Paris%20OER%20Declaration_01.pdf
UNESCO (2017). Second world OER congress Ljubljana OER action plan. Retrieved from https://en.unesco.org/sites/default/files/ljubljana_oer_action_plan_2017.pdf
UNESCO (2019). Draft Recommendation on Open Educational Resources. UNESCO, Paris. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000370936.locale=en
United Nations (2015). Transforming our world: the 2030 Agenda for Sustainable Development. Washington: United Nations. Retrieved from http://www.un.org/ga/search/view_doc.asp? symbol=A/RES/70/1&Lang=E]
Veletsianos, G., & Shepherdson, P. (2016). A Systematic Analysis and Synthesis of the Empirical MOOC Literature Published in 2013–2015. The International Review of Research in Open and Distance Learning, 17(2), 198–221. https://doi.org/10.19173/irrodl.v17i2.2448
Weller, M., Jordan, K., DeVries, I., & Rolfe, V. (2018). Mapping the Open Education Landscape: Citation Network Analysis of Historical Open and Distance Education Research. Open Praxis, 10(2), 109–126. https://doi.org/10.5944/openpraxis.10.2.822
Wiley, D. (2007, August 8). Open Education License Draft [Blog post]. Iterating toward openness. Retrieved from https://opencontent.org/blog/archives/355
Wiley, D. (2014, March 5). The Access Compromise and the 5th R [blog post]. Iterating toward openness. Retrieved from https://opencontent.org/blog/archives/3221
Wiley, D., & Gurrell, S. (2009). A decade of development ... . Open Learning: The Journal of Open, Distance and e-Learning, 24(1), 11–21. https://doi.org/10.1080/02680510802627746
Zawacki-Richter, O., Bozkurt, A., Alturki, U., & Aldraiweesh, A. (2018). What Research Says About MOOCs – An Explorative Content Analysis. The International Review of Research in Open and Distributed Learning, 19(1), 242–259. https://doi.org/10.19173/irrodl.v19i1.3356
