Critical Facts

CRITICAL FACTS

Critical success factors in online education

Thierry Volery

Professor at EM Lyon, France

Deborah Lord

Manager, Curtin University of Technology, Perth, Australia

Keywords  Education, Learning, Distance learning, Internet

Abstract

The Internet is a major technological advancement reshaping not only our society but also that of universities

worldwide. In light of this, universities have to capitalise on the Internet for teaching, and one progressive development of this is the use of online delivery methods. This paper draws upon the results of a survey conducted amongst students enrolled in one online management course at an Australian university. Three

critical  success factors in online delivery are identified: technology, the instructor and the previous use of the technology from a student s perspective. We also argue that the lecturer will continue to play a central role in online education, albeit his or her role will become one of a learning catalyst and knowledge navigator.

The International Journal of Educational Management

Introduction

 

Online education has generated tremendous excitement both  inside and outside higher education. For  some, it offers  the potential to provide learning to new audiences; for others, it offers  the opportunity fundamentally to transform learning delivery and the competitive landscape (Poehlein, 1996). Among those institutions with better-defined reasons  for  embracing online education the rationales vary, but they  often  fall into one of four broad  categories:

1   Expanding access. Most  states need to expand  access to education in order to meet the education and training needs of state residents and companies and to educate  under-served populations. For many people in the past, academic programme calendars have not  matched work and family responsibilities, and programme offerings may  not  have met learner needs.

2   Alleviating capacity  constraints. There  has been a surge  of student numbers which current university facilities cannot accommodate any  more.  Some are hoping to leverage the scalability of online education to avoid overwhelming their bricks-and-mortar capacities (Weill and Broadbent, 1998).

3   Capitalizing on emerging market opportunities. The public's growing acceptance  of the value  of lifelong learning has fuelled an increased demand for  higher education services among people outside the traditional 18-24 age range.  Emerging student segments,  such as executives seeking further  education and working adults, may  be more lucrative than  traditional markets. By capitalizing on emerging market opportunities, many educational institutions hope to generate  significant revenue.

4   Serving as a catalyst for  institutional transformation. Many higher education institutions are being  challenged to adapt rapidly to a decrease in public funding and to an increasingly competitive environment (Whitty et al., 1998). Distance education can catalyse  institutional transformation.

The rapid expansion of the Internet as a potential course  delivery platform, combined with the increasing interest in lifelong learning and budget  restrictions, has created a significant incentive for  universities to develop  online programs. As the technology is now  available and relatively user-friendly, those universities which do not  embrace  it will be left  behind in the race for globalisation and technological development. If we want universities to make  the utmost use of the Internet, it is essential to identify and understand the critical success factors affecting the online delivery of education. Indeed,  if we continue to re-implement conventional models  borrowed from classroom-based or distance education focused  on passive  transmission, we can expect only marginal improvements and may well simply escalate  costs.

This paper aims to identify the key success factors of this new paradigm of education based on the case of a business course, which has been offered online for the past few years at an Australian university of technology. The first section will define the concept  of online education. In the second section a review of the literature will identify the key factors influencing the effectiveness of online delivery, and the third section will describe the method used to conduct this study. In particular, the features of the course offered online will be detailed. The last section presents the findings and a discussion.

Defining  online  education

14/5 [2000 J 216-223

The current  ssue and full  text  arch ve of th s journal   s ava lable at

The literature on online delivery in the field © MCB University Press [ISSN 0951-354XJ http://www.emerald-library.com of education has flourished since  the early 1990s with the rise  of the Internet. There  has Thierry Volery and Deborah Lord Critical success factors in online education.

The International Journal of Educational Management 14/5 [2000J 216-223 been a concomitant interest in a variety of issues linked to online delivery and numerous terms have emerged  in the literature such as distance learning, computer-based learning, distributed learning, and lifelong learning. It is therefore important to clarify the concept  of online delivery and to see how  it relates to these other terms. Distance learning is probably the oldest

and best-known concept.  It was originally intended to cater  particularly for  students disadvantaged by their geographical remoteness from university campuses.  The UK's  Open University and imitators in countries as different as India, Israel, and Australia have demonstrated, even without the benefit of the Internet, that  technology makes  it possible  to deliver a good (and relatively cheap) higher education beyond  a physical campus.  Course  materials and contact with instructors may  rely on traditional (such  as print and telephone) or newer technologies (such  as electronic communication). Hence, distance learning can be defined as any  approach to education delivery that  replaces  the same-time, same- place, face-to-face  environment of a traditional  classroom. Computer-based learning  was, until fairly

recently, limited mainly to technological fields such as mathematics, engineering and design. Now, however, computer laboratories are likely to be frequented by students in linguistics,  geography, history  or business. Computers provide an unparalleled capacity to manage  and access large  amounts of information, and present it in a novel  and interesting way.  Similarly,  computer-based education allows students to become active learners rather than  mere  passive  recipients of teaching (Candy  et al.,  1994). Computers are not  necessarily linked to a network: computer learning can be achieved by stand- alone  computers using a particular software stored  in the hard disk,  a floppy disk  or CD. 

Distributed learning refers  broadly to features of a learner-centred environment, which   integrates a number of technologies to enable  opportunities for  activities and interaction in both  asynchronous and real- time modes. The model is based on blending a choice  of technologies with aspects of campus-based  delivery and distance education'' (Reid,  1999, p. 4). Two  essential aspects therefore underpin the concept  of distributed learning: first, a heavy  reliance on technology, and second, self-learning. The latter implies that  the learner (or  student) assumes responsibility for  specifying individual learning needs, goals and outcomes, planning and organising the learning task,  evaluating its  worth and constructing meaning from it (Candy  et al., 1994, p. 128).

Lifelong learning refers  primarily to those forms of learning throughout life  that  are called  for  by social  and cultural change.  The rate  of social,  technical, economic and other change  is so great,  at least in industrialised countries, that  few people will hold  the same job throughout their lifetime. Similarly, recent  research on learning across the lifespan has shown that  people are not  only capable of, but actually engage in, continuing learning over  their active life  and beyond (Tuijnman and van  der Kamp,  1992). Although lifelong learning can be self- directed, a variety of agencies  (corporations, professional associations, unions, community groups) also represent an extraordinarily rich and diverse repository of learning opportunities. Drawing on the concepts  previously defined, online delivery is a form of distributed learning enabled  by the Internet. Uses may  include the provision of student access to learning resources, the facilitation of communication and collaborative working among  and between  students and academic staff,  the assessment  of individual students or groups of students, and the provision of administrative and student support. Online delivery goes beyond  traditional computer learning as it makes  full use of the Internet and other digital technologies. Online delivery can facilitate distance education by making course  material accessible  anytime anywhere. It provides substantial advantages over  traditional technologies, such as:

•        Collaborative tools  which offer  a rich,

shared,  virtual workspace in which interactions occur  not  between  an individual and technology, but  as many- to-many, interpersonal communication, among  students. The interaction can be synchronous (i.e. in real  time) with, for example, a chat  forum or video conferencing, or asynchronous.

•        Interactive tools  such as simulations or

self-administered quizzes  which allow the student to progress at his or her own  pace through required exercises and self- assessments. These collaborative tools are limited in that  they  do not  provide for interaction with other students or an instructor; the student interacts only with the technology.

Key factors in effective online delivery Effectiveness

Webster and Hackley (1997) remarked that students' performance, measured by their marks, represents a key  aspect of teaching effectiveness. However, several studies have shown that  there  is little or no difference in student performance between  educational television and face-to-face  instruction (Wetzel et al., 1994) or between  video instruction and face-to-face  instruction Thierry Volery and Deborah Lord

Critical success factors in online education.

The International Journal of Educational Management14/5 [2000J 216-223

(Storck and Sproull, 1995). Webster and Hackley (1997) further suggested that  the following dimensions can capture the concept of effectiveness: student involvement and participation, cognitive engagement, technology self-efficacy (i.e. the belief that one has the capability to interact with a given technology), perceived usefulness of the technology employed, and the relative advantage or disadvantage of online delivery.

According to studies conducted by Dillon and Gunawardena (1995) and Leidner and Jarvenpaa (1993), three  main variables affect the effectiveness of online delivery:

1   technology;

2   instructor characteristics; and

3   student characteristics.

Technology

The reliability, quality and medium richness are key  technological aspects to be considered (Sanders  Lopez and Nagelhout, 1995). In particular, the network set up should allow for  both  synchronous and asynchronous exchange;  students should have convenient access (e.g. through  a remote access); and the network should require minimal time for  document exchange. The quality of the interface also plays  a crucial role  (Trevitt, 1995). The literature concerning interface design  for online delivery ranges  from the highly artistic (e.g. Laurel, 1990) to highly technical (e.g. Blattner and Dannenberg, 1992). Reeves and Harmon (1993) presented a synthesis between  these two  tendencies and identify the following dimensions as being  important ones in the user  interface: ease of use, navigation, cognitive load,  mapping, screen design,  information  presentation, aesthetics, and overall functionality.

The perceived richness of the technology should also influence the effectiveness of online delivery. In medium richness theory (Daft  and Lengel,  1986), a rich medium is one that  allows for  both  synchronous and asynchronous communication and supports a variety of didactical elements (text, graphics, audio and video  messages). A central part of the medium richness relates to interactivity. Indeed,  McIntyre and Wolff (1998, p. 257) noted that:   One of the powers  of interactivity in a Web environment is the capability to engage by providing rapid, compelling interaction and feedback  to students.'' Engagement is also enhanced by problem-based presentation of educational material. An  engaged student is a motivated student (Neorman and Spohrer, 1996).

Instructor characteristics

Collis (1995, p. 146) remarked that  the instructor plays  a central role  in the effectiveness of online delivery:  It is not the technology but  the instructional implementation of the technology that determines the effects on learning.'' Webster and Hackley (1997) suggested that  three instructor characteristics influence learning outcomes:

1   attitude towards technology;

2   teaching style;  and

3   control of the technology.

Students attending a class with an instructor who  has a positive attitude towards distributed learning and who  promotes the technology are likely to experience more positive learning outcomes. In a distributed learning environment, students often  feel isolated since they  do not have the classroom environment in which to interact with the instructor (Serwatka, 1999). To overcome this feeling, instructors can provide various forms of office  hours and methods of contacts for  the students. Most  importantly, the instructor should exhibit interactive teaching styles,  encouraging interaction between  the students and with the instructor. Students in Internet distance learning courses often  face technical problems. It is therefore crucial that  the instructor has a good control of the technology and is able to perform basic  troubleshooting tasks  (e.g. adding a student at the last  minute, modifying students' passwords, changing the course  settings). Organisation skills go hand in hand with control of technology. Haynes  et al. (1997) remarked that  a designed instructor is essential for overall coordination and that, as the development of an online course  is labour intensive, both  faculty and technical resources must  be identified and committed to the schedule  at an early stage.

Student characteristics

A variety of characteristics with potential influence on online delivery can be identified in the literature. As maintained by Colley  et al. (1994), such variables as prior experience, having a computer at home,  and personality produce gender  difference towards computers. Reinen  and Plomp  (1993) found that  computer usage at school was dominated by males  in most  of the 21 countries they surveyed. Computer experience is another variable which can have an interaction with gender  (Kay,  1992).

In addition to gender,  other demographic characteristics are likely to impact on the effectiveness of online delivery. It is anticipated that  the programme in which the students are enrolled (e.g. Master of Business Administration, Master of International Business, Master of Electronic Commerce) will play  a role.  The enrolment interacts with computer experience: students enrolled in Curtin University's Master of Electronic Commerce take  all  their courses  online and are familiar with the technology. Another demographic variable to be considered relates to the country of origin of the student.

Thierry Volery and Deborah Lord Critical success factors in online education 

The International Journal of Educational Management 14/5 [2000J 216-223

Leidner and Jarvenpaa (1995) also suggested that  students lacking the necessary basic  skills and self-discipline may  do better in a traditionally delivered mode. Similarly, the brightest and most  motivated students may  prefer to learn in an individual competitive environment rather than sharing their knowledge with less motivated, less bright students in a traditional classroom setting.

WebCT: a Web publishing software to develop  online  courses

WebCT is an integrated Web publishing environment specifically tailored for  the design  and development of teaching and learning materials. WebCT is a tool  that facilitates the creation of sophisticated World Wide Web-based educational environments. It can be used to create entire online courses, or to simply publish materials that supplement existing courses.  WebCT requires minimal technical expertise on the part of the developer of the educational material, and on the part of the student. All content is accessible  via  a standard Web browser.

WebCT  was developed  in the Department

of Computer Science  at the University of British Columbia, in Canada.  Curtin University of Technology has purchased an unlimited licence for  the software so that  it can be used locally to produce Curtin courses. During 1998, the number of online courses being implemented by Curtin schools using WebCT grew  from under 50 to over  300.

The Global Business 650 course  was one of

the first courses  offered online at Curtin Business School.  One of the main drivers for online delivery was due to the fact that  this course  was also offered in Singapore within the Master of International Business, a challenging and flexible programme specifically designed  for  busy  executives. This programme features guided- independent study, student-centred learning and maximum use of information  and communication technology. Although the course  content is available online, a series of four intensive one-day seminars is conducted throughout the semester.  During these seminars, the instructor gives an overview of the different topics and students can interact in a classroom setting on case studies or students' presentations.

Global Business 650 is structured around 11 topics, which each have the following features:

    •       Summary. The summary covers  the essential concepts  for  each topic.

•       Slides. The slides  used by the instructor during the intensive seminars can be either viewed or downloaded.

•       Readings.  A series  of articles relating to the topic is available online or through  a link with another URL.

    •       Quiz.  Ten multiple-choice questions have been designed  to check  the understanding of the topic. Clicking on that  button presents the questions. WebCT automatically  marks student responses                     correct or incorrect.

The other features and tools  also available to students from the Global Business 650 Web page include:

•       Course conferencing system (bulletin board). This allows communication among all  course  participants. WebCT keeps track of which articles are read by each student and, by default, initially presents only unread articles. The conferencing system can be searched (new and old articles) for  content, sender,  date of sending, and more.

•       Electronic mail. An electronic mail facility can be added to a course  allowing one-to- one message transfer among  course participants. Like the bulletin-board, messages can be searched  for based on the sender,  content and the date of sending.

•        Virtual library. Several  links to the Curtin library, online journals and magazines, and international  organisations are provided. These online resources provide readily up-to-date  information when students complete their assignments.

Method

Data  were  collected through an anonymous questionnaire administered to the 47 students enrolled in Global Business 650 during the first semester  1999. The measurement of the variables was drawn from Reeves and Harmon (1992) and Webster and Hackley (1997):

•        Teaching effectiveness. The grades  of each

individual student were  not  available at the time the questionnaire was administered, so this dimension was not included in effectiveness. The following dimensions were  used to capture the concept  of effectiveness: student involvement and participation, cognitive engagement, technology self-efficacy (i.e. the belief that  one has the capability to interact with a given technology), perceived usefulness of the technology employed, and the relative advantage of online delivery. Seven items were developed  using a five-point Likert scale where the respondent had to indicate the extent of his  or her  agreement/ disagreement. A teaching effectiveness index was subsequently built  by adding the value  of the seven items.

Thierry Volery and Deborah Lord Critical success factors in online education

The International Journal of Educational Management 14/5 [2000J 216-223

•       Technology. A set of 11 items was developed  using a five-point Likert scale to capture the reliability,  quality, and medium richness of the technology. A technology index was subsequently built by adding the value  of the 11 items.

•       Instructor characteristics. A set of 12 items was developed  using a five-point Likert scale to capture the attitude towards technology, teaching style,  and control of the technology displayed by the instructor. An  instructor index was subsequently built by adding the value  of the 12 items.

•       Student  characteristics. In addition to a question relating to previous computer experience, the following demographic variables were  included in the analysis: access to the Internet at home, programme of study followed by the student, country of origin, and gender.

Findings

To determine the frequency of use of the various features of the course  available to students through WebCT, each feature was listed and students indicated their usage on a five-point Likert scale. Responses ranged from

1 =   not at all'' to 5 =   very often''. The least frequently used feature of the course was the video. A reason for the low  usage of this feature may have been that  students were able to attend face-to-face lectures that  covered  all topics, which eliminated the necessity to view the videos through the Internet. The next two features which students rated  as having medium usage were  the readings and the virtual library. The last four features of the course all rated  as often  being  used. These were  the summaries, slides, quizzes and the bulletin board. Of these four, the quizzes rated the highest, and were  also the most interactive, with students being able to submit the quiz  for immediate grading. The findings are illustrated in Figure 1.

The relationship between  teaching effectiveness and technology was tested by a Pearson  correlation (two-tailed). The Pearson correlation coefficient is 0.563 with a p-value of 0.001. Thus,  there  is a significant relationship between  teaching effectiveness and technology.

A factor analysis was further  conducted to identify the underlying factors in technology. The factors were  extracted with a principal component analysis and the factor matrix was rotated using the varimax method. Three factors emerged  from the factor analysis. Each factor displays a high level of reliability with Cronbach alpha  > 0.7. Overall, the three factors explain 70 per cent of the variance. The results of the factor analysis are summarised in Table  I.

Factor 1: ease of access and navigation This factor comprises the first five  variables of technology. These variables all  relate to the ease with which the students can access the site and the usability of the software in general. The instructor noticed through the student tracking facility that  students took full advantage of the access flexibility offered by WebCT.  For  example, they  logged in at any time during day and night. There  was no frustration experienced with access and navigation.

Factor 2: interface This factor comprises four variables of technology. These all  relate to the visual structure and design  of the Internet course. The Web page design  was perceived to be appealing and well structured. This ergonomic dimension was particularly important since  it was discovered that  some students could  spend up to two  hours at a time on the Web site.

Factor 3: interaction This factor comprises the final three variables of technology. These relate to the interactive abilities of the WebCT  course between  all  students and the instructor. The technology made it possible  to have a truly virtual classroom, as there  was no need to sit in a classroom to experience interaction with classmates and the instructor. The interaction dimension indicates that universities must  not  attempt to come to terms with the Internet in a fetish way, i.e. to require their lecturers to merely put  their lecture notes on the Web. The result of that approach is unfortunately too well-known: lectures become even more  rigid and boring. The relationship between  teaching effectiveness and instructor characteristics was tested by a Pearson  correlation (two- tailed). The Pearson  correlation coefficient is 0.594 with a p-value  of 0.001. Thus,  there  is a significant relationship between  teaching effectiveness and the instructor. Similar to technology, a factor analysis was conducted to detect  the underlying factors relating to the instructor. The results of the factor analysis are summarised in Table  II.

Factor 4: attitudes  towards students. This factor comprises the first five  variables of instructor characteristics. These variables all  relate to the instructor's personal approach and teaching manner, and their ability to motivate the students in a classroom setting during the intensive seminars. In other words, the instructor must show  some empathy towards students both  in a face-to-face setting and in cyberspace. This implies, among  other things, handling e-mail  queries rapidly and solving emerging problems efficiently in a remote fashion.

Thierry Volery and Deborah Lord 

Figure 1 Student usage of WebCT features (mean) Critical success factors in online education The International Journal of Educational Management 14/5 [2000J 216-223

Table I

Factor analysis of technology

 

Items

Factor 1

Factor 2

Factor 3

Easy access to Web site

0.640

 

 

Did not experience problems while browsing

0.809

 

 

Browsing speed was satisfactory

0.837

 

 

Overall, the Web site was easy to use

0.743

 

 

Web site was easy to navigate

0.586

0.677

 

Information was well structured/presented

 

0.838

 

I found the screen design pleasant

 

0.750

 

Web site contained useful features

Web site gave me direct/timely feedback

 

0.797

 

0.792

I could interact with classmates through Web

 

 

0.796

I could easily contact the instructor

 

 

0.690

Percentage of variance explained

44.420

16.619

9.287

Cumulative percentage of variance explained

44.420

61.039

70.326

Cronbach alpha

0.863

0.879

0.713

Note: Only loadings > 0.5 are shown

 

 

 

Factor 5: instructor technical competence This factor comprises four variables of instructor characteristics. These variables relate to the instructor's ability to use and promote the Internet technology effectively. The instructor vigorously encouraged the use of the technology and a lab session was organised at the first intensive seminar to familiarise students with the technology. In addition, the instructor was clearly able to handle the technology and believed in it. This dimension also suggests that  the lecturer is not  only the repository of knowledge but  he or she can play  an important role  alongside the Internet as a knowledge navigator.

Factor 6: classroom interaction This variable relates to the instructor's ability to encourage students to interact and participate in class and through the Internet. Students were  encouraged to participate and interact during the Saturday seminars. In fact, the instructor gave a participation mark for the interaction on the Web site and in the classroom. The Internet, however, allows a new level  of interactivity as it eliminates the temporal and spatial rigidity of office  hours or class meeting times. It will virtualise the walls of the university, creating  elsewhere'' learning.The relationships between  the various demographic variables relating to students and teaching effectiveness were tested with a Thierry Volery and Deborah Lord

Table II

Factor analysis of instructor characteristics

Critical success factors in                                                                                                                                                                                   

 online education

The International Journal of

Educational Management

14/5 [2000J 216-223

Items                                                                                            Factor 4              Factor 5               Factor 6

Instructor was enthusiastic about teaching the class                            0.694

Instructor s style of presentation held my interest                            0.846

Instructor was friendly towards individual students                           0.897

Instructor  had a genuine interest in students                                   0.845

Students felt welcome in seeking advice/help                                  0.683

Instructor encouraged student interaction                                        0.621

Instructor handled the Web technology effectively                            0.828

Instructor explained how to use the Web site                                   0.827

I feel the instructor was keen that we use the Web site                    0.809

We were invited to ask questions/receive answers                           0.647

We were encouraged to participate in class                                     0.818

I found the intensive seminars were useful                                      0.660

Percentage of variance explained                                                   29.547              23.703              17.075

Cumulative percentage of variance explained                                  29.547              53.250              70.326

Cronbach alpha                                                                             0.889                 0.837                0.684

Note: Only loadings > 0.5 are shown one-way  analysis of variance (ANOVA). The results are presented in Table  III. As can be seen in Table  III, the previous use of the WebCT  is the only student characteristic influencing teaching effectiveness. Demographic student characteristics such as country of origin and gender were not influencing factors, nor were the type  of programme being  undertaken or access to the Internet at home.

Conclusion

The Internet is a major technological advancement reshaping not  only our  society but also that  of universities worldwide. In the light of this, universities have to capitalise on the Internet for  teaching, and one progressive development of this is the use of online delivery methods. This paper identifies three  critical success factors in online delivery: technology (ease of access and navigation, interface design  and level  of interaction); the instructor (attitudes towards students, instructor technical competence and classroom interaction);  and the previous use of the technology from a student's perspective.

Table III

Student characteristics

Items                                 

F value                               Sig. Programme of study           0.298     0.827

Internet access at home                0.009          0.924

Previous use of WebCT                 5.612          0.001

Gender                                            1.755          0.192

In undertaking this research, the authors encountered only two  limitations to the study. The first was the small sample  size of 47 students, and the second that  student grades  were  not  available at the time of the data collection, to be included in the students performance measurement. In considering the reliability of the results to the evaluation of online education it must  be noted  that there  was no control group  to detect whether the pedagogy  and technology used significantly enhanced teaching effectiveness compared with more  traditional  methods. There  is also a need to replicate the study to gain  further insight from different teaching disciplines across the world. It should also be noted  that  the course  assessment  was not testing the acquisition of transferable skills in communication and information technology which are often  developed through the use of computer-aided learning in related courses.

Findings from this study indicate the Internet can be a powerful tool  in education. This tool  has the potential both  to support effective education programmes and to expose students to the implications of computer networks. It is evident, though, that lecturers need to upgrade their technical skills in order to keep in touch with the technological developments that  are undoubtedly taking place. As this study has illustrated, the Internet and traditional classroom teaching methods are not mutually exclusive, but  should be seen as an extra dimension in education which can facilitate the lecturer's task  while benefiting the students as well. More  importantly, the level  of interaction between  the students and the lecturer

Electronic commerce

students vs. others

0.313         0.579

appears predominant in online delivery. This calls  for a shift in the academic role  from the 

C  ountry of origin                        0.070          0.976

intellect-on-stage and mentor towards a