Journal of Applied Learning & Teaching,
Dec 01, 2018
Abstract
The Canada School of Public Service (CSPS) leadership expressed a desire to modernise online programme development and delivery in order to more effectively meet the School's mandate. We conducted a review of modernised online programme development and delivery strategies, evaluated the School's current progress against those strategies, and recommended an integrated set of proposed activities that would augment online presence for learners, identify and deliver key learning and performance support needs, increase the effectiveness of online CSPS programs and services, integrate online CSPS services with those from other departments, and implement continuous review and improvement for CSPS online services
Introduction
The primary responsibility of the Canada School of Public Service (CSPS) is to provide a broad range of learning opportunities and establish a culture of learning within the public service as set out in the Canada School of Public Service Act (Government of Canada, 2017).
The School's mandate is to:
- encourage pride and excellence in the public service;
- foster a common sense of the purposes, values and traditions of the public service;
- support the growth and development of public servants;
- help ensure that public servants have the knowledge, skills and competencies they need to do their jobs effectively;
- assist deputy heads in meeting the learning needs of their organisation; and
- pursue excellence in public management.
The mandate as described in the Act therefore speaks to the role of the School not merely as a provider of learning and learning resources, but also to its role in developing a learning culture in the public service. In the last decade, and with the support of new technologies, corporate learning and performance support has been evolving from a model based on in-person and online classes to a broad-based strategy based on mobile support, personalisation and crowd-sourcing, and integration with online resources and communities.
The transformation at Accenture Connected Learning, provided to more than 370,000 employees worldwide, is a good example of this. According to Rahul Varma, Chief Learning Officer at Accenture: "Our field experts, our leaders, are actually curators. Within 18 months we've gone to over 1,000 learning boards that are accessed by more than 130,000 users, making it the fastest scalable learning vehicle we've ever known of" (Accenture, 2016).
The Government of Canada (GoC) is developing similar capacities. Launched in 2009, GCconnex provides social networking and online communities to the federal public service. It was complemented with the launch in 2017 of GCcollab, which extends similar services to provincial government employees and higher education institutions. The GoC also launched an online wiki encyclopedia, GCpedia, and GCdirectory, under the heading of GCTools. Concurrently, CSPS developed GCcampus to support social and collaborative online learning in the Canadian public service, expanding programme support beyond online courses in the institutional Learning Management System (LMS) to include online videos, case studies, job aids and videos. CSPS has also started offering learning events and distributes a quarterly newsletter.
The Government of Canada, and in particular the Canada School of Public Service, have been exploring the use of new technologies in order to support greater innovation and problem solving. It has identified social network technologies and crowdsourcing as a way to meet demographic and recruitment challenges (Robinson, 2014). These were targeted to help the public service compete with the private sector for the best talent and to engage employees "in order to empower them to contribute to their potential, to maintain their interest in working for the government, and to reach the government's goals" (Robinson, 2014).
Based on its own analysis of user requests, learning trends and client demand, CSPS identified five areas for programme expansion for GCcampus and the School. They were on a scale that required them to be treated as projects rather than areas for incremental improvement within the existing programmes. Those five areas were Mobile, Personalisation, Integration with GCTools, Crowdsourcing and Virtual library.
The National Research Council of Canada (NRC) was invited to provide expertise and assistance in scoping out these five areas. It performed an analysis of the School's existing service delivery and compared it to the state of the art in five major areas: mobile device support, personalisation, crowd-sourcing, virtual library and integration with other government services.
The NRC was selected because of its background and experience in the development of learning theory and applications supported by new media in education. In particular, the lead author is a leading proponen t of connectivism, a learning theory that describes how social media and learning networks can support online personal professional development (Siemens, 2005). Connectivism inspired the development of the massive open online course (MOOC), which in turn employed the technologies being contemplated by the School of Public Service in this project.
This work was done in early 2017. Since then the School reports that there have been many advancements in what CSPS has accomplished and is working towards. They now have much better data that shows increased access to GCcampus as well as increase in access of online courses. Satisfaction with online courses is now almost equal with classroom deliveries.
Literature Review: Research and Cases
Our literature review revealed widespread uptake and use of the five core technologies in society generally and in education in particular. We focused on the definition and educational uses of mobile learning, personalised learning, crowdsourcing, virtual libraries, and integrated systems. These were not intended to be all-encompassing, but rather to indicate major trends within instructional technology. The outcome of this work was the set of research questions guiding our interactions with CSPS staff
Mobile learning: this is most commonly defined with reference to the devices employed (Ally, 2008; JISC, 2011). For example, ISO describes it as "the use of mobile devices to support learning" (International Organization for Standardization (ISO), 2011). The adoption of mobile learning varies according to a number of factors. Empirical studies of technology acceptance show that major factors include ease of use, perceptions of usefulness, and attitudes toward mobile learning generally. These in turn are based on self-efficacy, relevance of the learning, system accessibility and social norms (Park, Nam, & Cha, 2012).
Personalised learning is described by the U.S. Department of Education National Education Technology Plan Update thus:
Personalised learning refers to instruction in which the pace of learning and the instructional approach are optimised for the needs of each learner. Learning objectives, instructional approaches, and instructional content (and its sequencing) may all vary based on learner needs. In addition, learning activities are meaningful and relevant to learners, driven by their interests, and often self-initiated (Office of Educational Technology, 2015).
Personalised learning is often based on a competency framework, which is a set of hypotheses about a person's performance, and mechanisms for testing and verifying them (Hirata & Laughton, 2012).
In the United States, these concepts are being expressed through an initiative called the 'Total Learning Architecture' by the U.S. Military's Advanced Distributed Learning program. Total Learning Architecture is "an evolving set of standardised Web service specifications to responsibly share essential learning data between applications using common API specifications and data models" (Advanced Distributed Learning, 2018). Analogous to the way health care providers need to be able to share personal healthcare data, "it is important to enable the responsible sharing of learning data between providers of education and training to improve learning outcomes" (Advanced Distributed Learning, 2018).
Competencies are described by ISO/IEC JTC1 SC 36 in terms of nine information object classes, including action, role, outcome, assessment process, method and criteria, and environment (International Organization for Standardization (ISO), 2014). As a part of its Total Learning Architecture, Advanced Distributed Learning is in the process of defining a Competencies and Skills Systems (CASS) programme (Advanced Distributed Learning (ADL), 2016). The purpose of CASS is to support competency portability, to support common reporting specifications, and to support resource alignment. By 'competency portability' we mean the ability to identify the skills or competencies being taught in a course, identify the same skills being taught in another course, and identify how this skill is applied in a profession or workplace (Robson, 2008). Javier Couto surveys the strengths and weaknesses of various chatbot platforms. "The chatbot ecosystem is moving very fast and new features are being released every day by the numerous existing platforms" (Couto, 2017).
Crowdsourcing: in education, crowdsourcing is the idea that learning "can be facilitated by connecting and empowering distributed communities of learners" (Maggio, Saltarelli & Stranack, 2016). It is based on the use of social networks in learning and especially content production (E-Learning 2.0, 2005). The attempt to crowdsource learning production and support gave rise to the first Massive Open Online Courses (MOOCs) in 2008. The synchronicity of networks, including social networks, is widely observed. The mechanisms of connectivity are described in graph theory (Bondy & Murty, 1976) and social network theory (Watts, 2003). Computationally, the effects can be observed in neural network software, a branch of theory called "connectionism" (Rumelhart & McClelland, 1986). Educational approaches based on this theory are classified under the heading of "connectivism" (Siemens, 2005).
Virtual libraries: these were found to be widely used in learning, and form the basis for work in the areas of learning resource metadata, access, copyright and licensing control. A virtual library is a collection of resources available on one or more computer systems, where a single interface or entry point to the collections is provided. A virtual library also provides user assistance services such as reference, interlibrary loan, technical assistance, etc. Terms such as Electronic Library and Digital Library are often used synonymously (Tella, 2016).
Integrated platforms: from the user perspective, it is a set of interface features. From a design perspective, it means managing (eg., Salesforce, 2012): web services, messaging, application programming interfaces (APIs) and associated toolkits, representational state transfer (REST) technologies, data aggregation and syndication using Rich Site Summary (RSS) or JavaScript Object Notation (JSON). These are machine- readable data formats that allow internet applications to work together and deliver more comprehensive services. It also means managing authentication and identity federation technologies that allow a person to use a single login to access multiple applications. The Canadian government's Blueprint 2020 (Clerk of the Privy Council, 2017) envisions a single integrated network of government services and including an open and networked environment.
Corporations are moving beyond courses and are supporting a full range of services. These new pedagogies are at the core of Visa's Corporate University, for example. Michael Ross, Visa's EVP of global HR, says: "Our goal is to foster a learning culture at Visa, where ideas... [include] everything from instructor-led training, virtual instructor-led training, performance support tools, e-Learning, simulations, gaming and interactive leaderboards" (cited in Kalra, 2016).
Companies and organisations are revising the traditional development cycle and model in order to respond to rapidly changing circumstances. The United States Department of Defense (DoD), for example, has created a comprehensive view of its 7 million personnel in order to deploy task- specific teams around the world on very short notice (Deloitte, 2017). This requires a system-wide understanding of the roles, functions and competencies of all staff, and a reorganisation of work.
Needs generation and performance assessment are at the core of modern workplace learning. Unilever, for example, in 2009 launched its Talent and Organisation Readiness Assessment Programme, concluding "skills need updating ever more rapidly so our learning strategy must deliver professional education that is mobile, engaging, easy to consume and on-demand" (Unilever, 2017). The company launched the Learning Hub in 2015. "The Hub uses digital technology and collaborative tools to meet the demands of modern, multilingual working" (Unilever, 2017).
Companies understand that the learning function needs to be supported with employee contributions and feedback, which in turn contributes directly to organisational performance. In 2016, for example, Ford incorporated employee polls, focus groups, and feedback strategies as part of its engagement program. It learned about areas where employees felt processes were overly administrative, and are "not working to create simpler, integrated customer- focused processes and tools" (Deloitte, 2017, p. 58).
Methodology
Research with the CSPS was conducted over three phases: first, a scan of literature relevant to the project scope; second, a review of the current state of the School with respect to trends discovered in the first phase; and third, development of recommendations for future work.
In the first phase project staff met with CSPS officials and determined the five areas of interest (mobile, personalised learning, crowdsourcing, virtual library, and integration). This was based on previous research by CSPS which was shared with project staff. These meetings determined the scope of these areas and the problems in these areas that were intended to address. The literature review combined formal and informal search. Using the five subject areas as search terms provided a list of relevant titles in each subject area. Additionally, informal research of 'grey literature' (Haddaway, Collins, Coughlin, & Kirk, 2015) was conducted. These included the use of Google Scholar and Google Search, as well as a review of Corporate Research reports available to the National Research Council Client Liaison Officer. Project staff also drew on existing expertise in the subject area. This research was conducted as a part of the Learning and Performance Support System programme (Learning and Performance Support Systems, 2013) and prior expertise created in the areas of open online courses and informal learning (Kop, 2012; National Research Council Canada, 2017).
The second phase focused on the current state of CSPS with respect to these five areas. It began with a review of internal documents provided by CSPS, including Usability studies, the GCCampus Business Plan, and CSPS Annual Reports. Project staff conducted 12 interviews averaging one hour each with 16 people. Most interviews were one-on-one, with two interviews grouping three people each. The sampling was purposive; participants were selected by School of Public Service management as responsible for various areas of CSPS administration: needs assessment, instructional design, data management, application and server support, course evaluation, operations, human resources, instructional technology and business intelligence, and subject matter experts capturing a wide representation of functions within the School.
Project staff also conducted an assessment of the CSPS website, including a review of videos and learning aids, enrolment and completion of several courses, and participation in an online event. The results of this investigation were compiled and presented in a second session with CSPS to confirm the completeness and reliability of the findings.
The third phase developed recommendations for the School to consider. Each of the five areas was mapped to the School's mandate and business objectives, resulting in a logic model describing the relation between the strategy and the goal (see Appendix 1). The logic model was contained in an editable format similar to Google Docs, with access granted to project and CSPS staff. The document underwent three formal revisions. As previously, the final result was presented and discussed with CSPS staff.
Government of Canada regulations on research ethics were followed throughout. The project plan was submitted to and approved by the NRC Research Ethics Board. Before interviews were conducted, participants read and signed consent forms. Individual interviews were recorded and transcribed; these transcriptions were combined into a set of anonymised documents, while original recordings and transcriptions were destroyed.
The content of this article corresponds with the three phases of the project. The sections on 'Research and Cases' and on methodology reflect the result of the first phase of the research. The sections titled 'Current State' and 'Perspective and Progress Toward New Technologies' correspond to the second phase that describes the result of the investigation of the current capacities of CSPS. Finally, the 'Discussion' section summarises the recommendations made in the final project report.
Study Results: Current State
The CSPS has focused traditionally on in-class learning. It is something the School understands and has developed expertise in. In recent years, online learning has increased dramatically and classroom-based learning has declined; recent numbers show that the School has gone from 90% classroom and 10% online to 90% online and 10% classroom courses. In addition to developing online learning, the School has been trying to modernise the classroom.
The School's business model was recently fundamentally changed. Originally, the School offered learning to other departments on a cost-recovery basis. Today, learning is offered to departments as a centralised service. This changes the way the School designs and offers courses. Whereas in the past it would develop custom learning for individual departments, today it is more focused on common learning offered to multiple departments. Fair and equitable access to core and common learning is provided at Journal of Applied Learning & Teaching Vol.1 No.2 (2018) 19 no cost to individual learners and in the case of the Aspiring Director Program where access is limited; organisations are responsible for allocating their seats via their own Talent Management Planning exercise.
GCcampus is the School's online presence. Drawing together some legacy systems and incorporating some new products, development began three years ago in order to help meet the School's new mandate. GCcampus itself can be accessed from the open internet without the need to be within a government intranet. A login is required using credentials provided by the School. On GCcampus (as of spring, 2017), there were 310 courses listed, 33 job aids, about 100 videos, two case studies and a couple dozen blog posts.
GCcampus consists of the following components:
• Saba learning management system (LMS) - the system is used to manage initial login and user accounts, registration, tracking and reporting on formal courses taken by public servants. It also hosts a number of School courses, for example, Security Awareness course. It is used by a large number of learning advisors who create formal learning products based on policy statements. The Saba system is a legacy system.
• Drupal 7 content management system (CMS) - this system provides indexing, filtered search, and some storage for learning resources provided by the School. The Drupal module has been customised with the installation of some modules, including custom modules. Drupal is an open source PHP application.
• Moodle LMS. Moodle is an open source LMS written in PHP. It is easier to operate than Saba for both teachers and students. Moodle is used to host a number of GCcampus courses and to support discussion groups in the Moodle forums.
• Kaltura online video platform (OVP). Another open source PHP application, Kaltura hosts the videos offered on the GCcampus platform.
• CSPS service bus. This is a custom-built service in GCcampus deploying a RedHat Fuse product with a JBoss server. The service bus manages the exchange of data from one GCcampus application to another, and enables (for example) single-sign on using the Shibboleth identity system.
GCcampus also offers access to School events via webcasting. Registration to the event is enabled with a single click within GCcampus. A company called CanWebCast has been broadcasting the events (CanWebCast has since merged with another company to become Collaborate.video).
The wider environment in which GCcampus operates consists essentially of two families of applications: GC Tools, including GCconnex, though there is no real connection with them yet; and Human Resource (HR) and Treasury Board tools and applications, which include services such as MyGCHR (Public Services and Procurement Canada, 2015). The richness of the data is on the HR side, but again there really is not a solid connection between them at this time.
Other tools employed by CSPS include an event scheduling system, survey functions (using Survey Monkey, which may be replaced with SimpleSurvey) and Cognos for reporting. MySchool News is a quarterly publication available by email subscription.
Questions were asked about the CSPS approach to assessment and quality control, and whether these processes would map to new technologies. Currently CSPS relies on course-completion surveys to judge course quality – several people mentioned Kirkpatrick's Level 1 assessment specifically (Kirkpatrick Partners, 2009). In some cases, 360-degree evaluations are conducted, whereby supervisors are also questioned, in order to determine the transfer of learning to performance (hence reaching Level 3).
On the question of quality control itself, responses were mixed, with some (apparent) satisfaction with the current process. The employment of Universal Design for Learning (UDL) and WETkit (which stands for 'Web Experience Toolkit', and is a design template all Government of Canada websites are required to use, in order to ensure ease of access and regulatory compliance (Government of Canada, 2012)) ensured quality, said some. However, others argued that training is more than just transmitting information and more than just compliance.
There was recognition that quality control needs to focus more on business goals and learner needs. And CSPS needs to consider in its assessment process what feedback it actually needs, and how to get that feedback. Study Results: Perspective and Progress Toward New Technologies
Mobile Technologies
Canadian government employees, especially at management level, have traditionally been supplied with Blackberry devices. The purpose of this section was to assess the impact of this policy and changes in the mobile device environment on expectations of device support by CSPS.
Nobody suggested standardising on Blackberry, and there was wide recognition that CSPS would need to be able to support a range of devices. Additionally, it was noted that government is trending toward 'Bring Your Own Device' (BYOD) and that therefore, non-standard device environments could be expected.
It was understood by participants that supporting BYOD raised policy issues connected to support, security and service delivery, including those concerns raised by the Office of the Privacy Commissioner (Office of the Privacy Commissioner of Canada, 2015).
Staff approached mobile technologies from several angles.
• Access - mobile learning means having access on their device, in any browser, at any time. It means learning anywhere you want, or being able to learn on the fly, learning as you go, using different devices.
• Device - there was some disagreement on the idea of device. Some people thought 'mobile' meant a focus on smartphones only, with tablets and laptops being more like desktop learning than mobile. Others included these, on the ground that they are mobile, in contrast to the desktop.
• Location - learners can learn outside their office using mobile devices. Not all civil servants are office-based, however, and many work in the field.
• Design - this is the idea of content specifically designed for mobile. For example, mobile could be like the YouTube or Google of learning: you have questions and you can get immediate answers. The applications have to be re-engineered, which means asking what the business need is and who the audience is.
People generally agreed that almost everybody has a mobile device, including especially smartphones. A significant number of public service employees had department-issued Blackberries. Meanwhile most people had personal devices, usually Androids or iPhones.
The market for mobile would be different than that for traditional learning content, and would not include all of GCcampus content. Some people mentioned videos, podcasts and online events explicitly. The primary existing use of mobiles was for communication, especially by phone, but sometimes by text (it was noted people use text messaging more in their personal life than they do for work).
People also mentioned the use of mobiles in classrooms (especially when tablets and laptops are included).
Personalised Learning
There was a relatively consistent understanding of personal learning based on adaptability and ease of use:
• Content-awareness – when I log on the system knows who I am, it knows what I'm looking for, it knows what my job is and what department I work for, and it pushes content accordingly.
• Broker – the system knows that I learn from a wide variety of courses, and brokers my access to them, according to my position and needs.
• Adaptability – the system adapts to my learning (what I've learned, how I learn) and adapts technologies to suit me, presenting only what I need.
Examples of personalisation mentioned by respondents included Duolingo, which tests for linguistic competence in a language and delivers lessons to help one learn the language accordingly, and Netflix, which recognises viewing patterns and recommends accordingly.
Personalised learning and performance support are linked. The service should be more like Google than anything, where it knows what you need and will help get it for you. At the executive level there have been many promotions in the system and the School hears a desperate need for 'show me how': - show me how to think strategically;
- show me how to reframe an issue;
- show me how to deal with the ambiguity of the world right now.
Related to this, several people expressed enthusiastic support for the online events being broadcast via webcasts (and which also could be distributed as podcasts). A specific block of time is allocated for them, which makes scheduling much less ad hoc, and they address current and timely issues, and support immediate feedback.
The discussion of personalised learning also raised discussion of competences, and the two concepts are closely linked. The Ministers and the public are keen for CSPS to look at personalised learning to support the transferability of learning when one enters or leaves the public sector. Their records should follow them, perhaps tied to Prior Learning Assessment / Recognition (PLA/PLR). But some of these records might be private or proprietary to the public service.
The Government of Canada employs a set of Key Leadership Competencies (KLC) (Treasury Board Secretariat, 2016) defined by the Treasury Branch. Leadership courses are built on the leadership competencies, but it is not a standardised approach. Respondents also referred to the "16 core competencies" as well as competencies for functional communities. For example, the IS group (IS1-IS6) has 20 or 30 competencies which are increased as you go along (in a grid).
There is perhaps a tension in different approaches to personalisation. GCconnex and GCcampus were developed using open source tools. By contrast, the government employs large enterprise systems such as Saba and PeopleSoft. The School could embrace the formal standards- based approach to learning inherent in these large systems. But personalised learning also supports the trends towards lifelong and informal learning.
Crowdsourcing
Crowdsourcing ties in well with mobile learning and supports the idea of social learning and collaborative learning. However, the question comes up immediately as to whether the information being uploaded by people can be trusted.
The idea of crowdsourcing can range from having a sharing space to encourage public servant contributions to orchestrated collaborative and social learning activities. Definitions included these and various shades in between:
• Course commentary – for example, a person could
write that they took a course, about how they applied it when they returned to work and make a selfie video about the results.
• User-generated content – for example, gathering content from users to create courses or other learning resources.
• External content – sourcing learning content from other departments, for example, a recent security course created through consultation with 24 departments, or job aids created in another department and posted in GCcampus.
• Content curation – large groups of people get together and share (and rate) resources from a variety of sources.
• Collaborative learning and social learning, where people would get together to solve problems, create resources, or evaluate policy.
• Alternative pedagogies – enabled through crowdsourcing methods. In addition to helping the School develop flexible responses to emerging issues, crowdsourcing supports hands-on experience-based learning, and it helps in the formation of personal networks and communities.
The voices in favour of crowdsourcing were very strongly in favour. They identified it as a mechanism to help with needs analysis, to obtain feedback and evaluation, to source new resources and new information, and to keep the School up to date with current approaches to teaching and pedagogy. Others could not imagine the idea of people uploading their own content to GCcampus, particularly if that content were to be used for learning.
The use of shared learning spaces and crowdsourcing also creates obvious privacy and security implications, especially in an environment like the Government of Canada. There is a need for technology developers to focus on providing tools for the employers that can help in mitigating disclosure risks for sensitive business information.
Moreover, the Treasury Board Secretariat (TBS) has a variety of rules governing accessibility, bilingualism, common look and feel, information management, the protection of personal privacy, and more. These regulations apply to services managed and offered by CSPS.
Having said that, after a three-month pilot, a new service called GCcollab has been launched by the Treasury Branch in early 2017. This service is similar to GCconnex, with the difference that members of the college and university community in Canada may also be members. GCcollab could be used by the School to illustrate what it takes to perform various jobs in the public service. They also felt that there might be a marketing and communications role.
Virtual Library
The original intent of plans to develop a virtual library was to offer online access to texts and resources employed in CSPS courses analogous to the manner in which the physical library offered resources to participants in in-class courses. The model would be to implement a library services- agreement with service providers for e-books, with all the functionalities.
But we can imagine a wider possibility. With all the departmental libraries closing there is a greater opportunity to do something meaningful. Is the Government of Canada maintaining EBSCO and for-fee academic databases? Perhaps there is an opportunity for all of the public service to be covered under a single service. This would go well beyond the mandate of the CSPS, however.
There is a large number of virtual libraries in other government departments, including, for example, the Federal Science Library and the libraries hosted by the National Research Council and the activities of Library and Archives Canada (LAC). And again there is possible interoperability via GCcampus. Some suggested the School could scan for and add resource listings, without any additional work – "but there would have to be some kind of caveat or waiver that we are not all-encompassing," said one respondent.
Any library initiative essentially entails the employment of resource metadata standards. Work on resource metadata standards has been an ongoing activity for the School for some time now and over the years, there have been a few groups working on this. Some staff were aware of initiatives such as Learning Object Metadata and Cancore (Cancore, 2006).
Respondents also described initiatives to employ metadata for learning resources (MLR; International Organization for Standardization (ISO, 2011). There was an effort to develop a taxonomy, and efforts to make metadata creation more integrated with content creation tools, so when staff enter resources, they have to tag them to get to the next page in the input process. The GCcampus operations process plays a significant role with respect to metadata: in order to upload content onto GCcampus, certain metadata fields have to be completed, for example, 'target audience'.
A common library function – virtual or otherwise – is content curation and materials assessment. The major quality control initiative is through the GCcampus onboarding process. For CSPS materials, translations / copyright / accessibility requirements are validated by the process and are based on Treasury Board of Canada Secretariat (TBS) standards. Staff generally felt that the School was very compliant with these regulations.
Integration with Other Platforms
The possibility of integration with GCTools creates business challenges for CSPS. First is the question of exactly what would be connected. Then there is the question of how deep the integration would go. Who would be responsible for technology development? What would branding look like? How would user access be managed?
What does 'integration' mean? Several scenarios were discussed:
• Linkage – each system would be represented with a tab or an icon on the other system.
• Single sign-on – GCcampus already has single sign- on within its own suite of tools. The Treasury Board Secretariat (TBS) has expressed interest in single sign on with GCcampus and GCTools.
• Common Services – CSPS services would be listed among the other services in a single Government of Canada employees' dashboard or menu.
• Extending the bus – user information and data would be exchanged between CSPS applications and other applications, especially GC TOOLs.
• Learning Tools Interoperability – GCTools services (and perhaps other services) would be launched using the Sharable Content Object Reference Model (SCORM; Rustici Software, 2017) or Learning Tools Interoperability (LTI; IMG Global, 2016) mechanisms and specifications.
• Full integration – CSPS resources are available throughout GCTools and vice versa. Thus, for example, GCconnex discussion groups could be created and accessed from courses, while courses (especially those on how to use GCTools) could be accessed directly from GCTools.
Respondents also addressed the business value of integration. A few things stood out. There was near universal support for a single sign-on mechanism, with respondents citing it as the most frequently sought-after improvement by learners. Additionally, respondents looked at integration as an excellent means to enable CSPS to deliver on its core mandate of offering training and support to the federal public service.
As mentioned above, single-sign on was the single most discussed issue in the entire consultation. Two major themes emerged:
• Everybody wants single sign-on, defined as "you sign on once to your government account, and then you have access to everything," and
• Multiple single sign-on projects exist in the Government of Canada. For example, CSPS has its own Shibboleth-based identity system (Shibboleth Consortium, 2017). There is also the Online Registration and Credential Administration (ORCA) MyKey initiative (Shared Services Canada, 2017). There is in addition the general desktop login people use in their own departments (note that we did not attempt a full survey of sign-on mechanisms).
In addition, several issues were raised. First, the level of security provided by (or required by) different sign-on systems varies. Examples include password change requirements, location (in or outside the intranet) requirements, and hardware requirements. Additionally, security needs vary department by department. Second, there is not a clear definition of the need for, or business value of, sign-on requirements. Part of the reason for this is record-keeping, so people can be tracked and recognised for the learning they do.
Discussion
The result of the review found that although CSPS had made significant progress extending its platform beyond the LMS, it offered limited support in each of the five major areas. It found that development and delivery services remained focused on online-course provision, and that the School's structural and service orientation was slowly transitioning to new technologies and pedagogies.
There is uncertainty in the School regarding instructional design and pedagogy. The in-person courses offer contemporary experience-based and discussion-based pedagogies. But the online courses employ a traditional 'presentation-and-test' pedagogical model. There is recognition in the School that this model needs updating and work is underway to address these areas.
It was the view of the analysts that CSPS technological progress in these areas should be incremental, rather than a rapid expansion of any individual service. The following considerations warrant this conclusion:
• The School has not yet had time to implement the new functionality that has already been installed. Many features (for example, ratings) are available but have not been switched on. Other features (for example, blogs) have been lightly used and would benefit from wider participation. Other features (for example, support for access to ebooks via EBSCO subscription) are in pilot mode or about to be piloted.
• Changes in the wider learning technology infrastructure environment will impact GCcampus. In particular, if a new LMS is acquired, the service bus linking GCcampus tools (Drupal, Kaltura, Moodle) will need to be updated. Adoption of single-sign-on by Shared Services Canada will also require that the service bus be updated. And the continuing expansion of capabilities in GCTools may pre-empt the need for a concurrent expansion in GCcampus.
• The School needs to build on and document experience and success using GCcampus tools. Most work, and most activity, centres around the online courses. The need, however, is for ongoing performance support in the form of communities, resource bases, events, and crowdsourcing, as well as the building of specific competencies to support performance and talent management. Academic and support staff at the School need to develop experience and skills in performance support as well as course-based learning.
• No single one of the services (mobile, personalisation, crowdsourcing, library, integration) can be expanded without impacting the remaining four. For example, mobile learning is not suitable for online classes, but rather, for context-specific performance support through access to a social network and relevant resources. An expansion of mobile would require an expansion of the other four areas. To a significant degree, they move in tandem.
The School has developed a used technological environment in GCcampus that will support future expansion in a number of areas. The primary imperative at this juncture, however, is to leverage that investment to become proficient in the use of these new technologies.
The outcome of the analysis was twofold: first, the need to grow expertise in the use of new learning technologies; and second, the recommendation that the expansion of these technologies focus on incremental improvements across all five themes.
In order to realise this mandate, and given the analysis of the CSPS Technology Integration Project to date, the following strategy is proposed:
• define and align to a contemporary model of online learning support;
• develop solutions incrementally rather than a rapid expansion of any individual service.
The project team thus made the following recommendations:
First, while personalisation based on adaptive systems supporting individual competencies is desirable, the existing resource base does not support such a programme, and the future LMS environment (if any) must be defined. In the meantime, concrete steps can be taken to prepare staff and students for the future of personalised learning, beginning with the development and distribution of notification and resources tailored to functional community, location and role.
Second, because crowdsourcing depends on the ability and willingness to integrate learner and third party feedback and contributions, and these are at a nascent stage of development, it is necessary to develop the School's capacity and comfort level with crowdsourcing before any large-scale development can be considered. It is also necessary to devise and embrace mechanisms that encourage participation and contributions to crowdsourcing initiatives.
Third, the integration with other government services is highly desirable but poses complex challenges. The primary target for integration is GCTools, given the potential for the GCTools environment to support personalisation and crowdsourcing services. It would also be desirable to integrate with human resources, competency and performance management systems. The single most-requested feature was single- sign-on. Beyond that, support for personalised learning will depend on integration, but in turn requires coordination with external services, some of which are not yet fully developed and implemented. The focus of the proposed activities is therefore to support loosely integrated interoperability with external services.
Fourth, we noted that most mobile devices are not suited to the delivery of online courses. Nor is it practical to develop platform-specific applications (such as an iPhone app). Mobile support in this context should focus on responsive and cross-platform design (for example, HTML5-based design) and on performance support resources, and in particular, the GCcampus website. Additionally, it should be recognised that 'mobile learning' entails support for learner mobility, and not merely support for mobile devices. The use of mobile devices should be considered in support of other initiatives, for example, mobile calendar notifications for learning events.
Finally, fifth, while it is desirable to provide access to eBooks to support online courses, and this is the intent of the virtual library, at the same time, library services are being offered in various departments, with some centralised functions (such as the Federal Science Library) being developed. It is not desirable to duplicate this service; the purpose of a CSPS virtual library is to support performance support (consider rephrasing – repetition of support: "support performance support"). There is a need to provide access to more than just eBooks; learners require access to videos, podcasts, learning objects, and other performance support resources. These should be both produced by CSPS and sourced from other departments.
Conclusion
This work examined five areas targeted for programme expansion by the Canada School of School Public Service: Mobile, Personalisation, Integration with GCTools, Crowdsourcing and Virtual Library. An analysis of the School's existing service delivery was compared with the state of the art in each area. Based on this work, a logic model was developed defining a roadmap for future work. This work was done in early 2017. Since then the School reports that there have been many advancements in what CSPS has accomplished and is working towards. They now have much better data that shows increased access to GCcampus as well as increase in access of online courses. They also reported that satisfaction with online courses is now almost equal with classroom deliveries.
In the months since this research was conducted, the School of Public Service has begun to move forward on most of the recommendations. Though the outcome of this research was the roadmap discussed above, the research underlines the need for broad-based consultation before the development of new learning technologies in an existing learning organisation can be accomplished. The work shows that implementing any of the five key technologies would have an impact that reaches into all areas of the organisation, and so it is necessary to have a clear understanding of what those areas are required to accomplish and how the deployment of new learning technologies impacts that mission.
On a more comprehensive note, this research also shows the relevance of recent research in advanced learning technologies to government and corporate learning. There is a greater acceptance of, and indeed, greater need for, approaches to learning that move beyond the courses and classrooms paradigm instantiated in the learning management system. Indeed, the question of whether the School should be moving in this direction was rarely raised; the concerns centred on how new learning technologies could most effectively meet the changing needs of the school. This serves to validate, to some degree, this recent research.
References
Accenture invests more than US$840 million in employee learning and professional development: Company reinvents training through digital learning environment. (2016, January 13). Accenture . Retrieved from hhttps://newsroom. accenture.com/news/accenture-invests-more-than-us- 840-million-in-employee-learning-and-professional- development.htm
Advanced Distributed Learning (ADL). (2016). CASS. Retrieved from CASS / Competency And Skills System: http://www.cassproject.org/
Ally, M. (2008). Mobile learning: Transforming the delivery of education and training. Edmonton, Canada: Athabasca University Press.
Bondy, J. A., & Murty, U. S. (1976). Graph theory with applications. New York, NY: Elsevier Science Publishing.
Canada. Clerk of the Privy Council. (2017). The next level: Normalizing a culture of inclusive linguistic duality in the federal public service workplace . Retrieved from https://www.canada.ca/en/privy-council/corporate/clerk/ publications/next-level.html
Canada. School of Public Service. (2017, June 19). Canada school of public service act . Retrieved from http://lois-laws. justice.gc.ca/eng/acts/C-10.13/FullText.html
Cancore. (2006). Cancore. Retrieved from Cancore: http:// cancore.athabascau.ca/en/ Chedemois, Y. (2006). Content construction kit (CCK). Drupal . Retrieved from https://www.drupal.org/project/cck
Couto, J. (2017, January 25). Building a chatbot: Analysis & limitations of modern platforms [Blog post]. Tryo Labs . Retrieved from https://tryolabs.com/blog/2017/01/25/ building-a-chatbot-analysis--limitations-of-modern- platforms/
Deloitte. (2017). Rewriting the rules for digital success: 2017 Deloitte global human capital trends. Deloitte University Press. Douglass, R. (2007). Apache solr search. Drupal. Retrieved from https://www.drupal.org/project/apachesolr
Downes, S. (2005, October). E-Learning 2.0. Elearn Magazine . Retrieved from http://elearnmag.acm.org/ featured.cfm?aid=1104968
Downes, S. (2013, December 4). Learning and performance support systems [Blog post]. Half an Hour . Retrieved from http://halfanhour.blogspot.ca/2013/12/learning-and- performance-support-systems.html
Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015, September 17). The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PLOS One, 10 (1371). doi:https://doi.org/10.1371/ journal.pone.0138237
Hirata, K., & Laughton, S. (2012). Quality management of learning content and assessment: An exploration of application methodologies based on a competency semantics information model. Information and Systems in Education, 11 (1), 14–23. Retrieved from https://www.jstage. jst.go.jp/article/ejsise/11/1/11_14/_pdf
International Organization for Standardization (ISO). (2011). ISO/IEC 19788-1:2011. Retrieved from International Organ ization for Standardization (ISO): https://www.iso.org/ standard/50772.html
International Organization for Standardization (ISO). (2011). ISO/IEC TS 29140-2:2011. Retrieved from https://www.iso. org/standard/52808.html
International Organization for Standardization (ISO). (2014). ISO/IEC 20006-1:2014. Retrieved from https://www.iso.org/ standard/57359.html JISC. (2011). Mobile Learning . Retrieved from JISC: https:// www.jisc.ac.uk/full-guide/mobile-learning
Kalra, A. S. (2016, September 1). Visa's first corporate university outside the US opens in Singapore. Human . Retrieved from http://www.humanresourcesonline. net/visas-opens-first-corporate-university-outside-us- singapore/
Kirkpatrick Partners. (2009). The Kirkpatrick Model . Retrieved from Kirkpatrick Partners: https://www. kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick- Model
Kop, R. (2012, January 5). Research publications on Massive Open Online Courses and Personal Learning Environments. Retrieved from Rita Kop: http://ritakop.blogspot. ca/2012/01/research-publications-on-massive-open.html
Learning Tools Interoperability. (2016). Retrieved from IMG Global: https://www.imsglobal.org/activity/learning-tools- interoperability Maggio, L., Saltarelli, A., & Stranack, K. (2016, March 21).
Crowdsourcing the curriculum: A MOOC for personalized, connected learning. EDUCAUSE Review . Retrieved from https://er.educause.edu/articles/2016/3/crowdsourcing- the-curriculum-a-mooc-for-personalized-connected- learning
National Research Council Canada. (2017, September 27). LPSS . Retrieved from NRC Research Publications Archive (NPARC): http://nparc.cisti-icist.nrc-cnrc.gc.ca/eng/ search/?q=LPSS&m=1
Office of Educational Technology. (2015). National Education Technology Plan. United States Government, Department of Education. Retrieved from National Education Technology Plan
Office of the Privacy Commissioner of Canada. (2015). Is a Bring Your Own Device (BYOD) Program the Right Choice for Your Organization? Retrieved from Office of the Privacy Commissioner of Canada: https://www.priv.gc.ca/en/ privacy-topics/technology-and-privacy/mobile-devices- and-apps/gd_byod_201508/
Park, S. Y., Nam, M.-W., & Cha, S.-B. (2012). University students' behavioral intention to use mobilelearning: Evaluating the technology acceptance model. British Journal of Educational Technology, 43 (2), 592–605. Retrieved from https://www.researchgate.net/publication/230551056_ University_students%27_behavioral_intention_to_use_ mobile_learning_Evaluating_the_technology_acceptance_ model
Public Services and Procurement Canada. (2015). Security, corporate and information services . Retrieved from Public Services and Procurement Canada: http://www.tpsgc- pwgsc.gc.ca/sc-cs/nsnnnr-ossr/2015-2016/page-10-eng. html
Rumelhart, D. E., & McClelland, J. L. (1986). Parallel distributed processing . MIT Press. Retrieved from https:// mitpress.mit.edu/books/parallel-distributed-processing
Rustici Software. (2017). What is SCORM? Retrieved from https://scorm.com/scorm-explained/
Salesforce. (2012). Integrating with the force.com platform . Retrieved from https://developer.salesforce.com/page/ Integrating_with_the_Force.com_Platform
Shared Services Canada. (2017). Welcome to ORCA . Retrieved from https://eajl-orca.securise-secure.gc.ca/O/ vw/bienvenue-welcome-eng.pub
Shibboleth Consortium. (2017). Shibboleth . Retrieved from https://www.shibboleth.net/
Siemens, G. (2005, January). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2 (1).
Tella, A. (2016). Information seeking behavior and challenges in digital libraries . IGI Global. Retrieved from https:// www.igi-global.com/book/information-seeking-behavior- challenges-digital/145469
Treasury Board Secretariat. (2016). The key leadership competency profile. Retrieved from Government of Canada: https://www.canada.ca/en/treasury-board-secretariat/ services/professional-development/key-leadership- competency-profile.html
Unilever. (2017). Developing & engaging our people. Retrieved from https://www.unilever.com/sustainable- living/the-sustainable-living-plan/our-strategy/ embeddingsustainability/developing-and-engaging-our- people.html
Watts, D. J. (2003). Six Degrees: The Science of a Connected Age. New York, NY: W W Norton. Acknowledgements Members of the National Research Council of Canada project team included Bruno Emond, Hélène Fournier, Irina Kondratova and Shirley MacLeod.
