International Journal of Knowledge Content Development & Technology
1

Journal Archive

International Journal of Knowledge Content Development & Technology - Vol. 5 , No. 2

[ Article ]
International Journal of Knowledge Content Development & Technology - Vol. 5, No. 2, pp.103-122
ISSN: 2234-0068 (Print) 2287-187X (Online)
Print publication date Dec 2015
Received 23 Oct 2015 Revised 21 Nov 2015 Accepted 30 Nov 2015
DOI: https://doi.org/10.5865/IJKCT.2015.5.2.103

A Study on Evaluating Learning Effects Based on Analysis of Satisfaction in E-learning
Yeong-Ae Kwon* ; Younghee Noh**
*Researcher, Center for Teaching & Learning (CTL), Konkuk University, Korea (herayaa@kku.ac.kr)

Correspondence to : **Professor, Department of Library and Information Science, Konkuk University, Korea (irs4u@kku.ac.kr)


Abstract

This study examined student satisfaction with e-learning experiences in order to determine which factors had the greatest impact on reports of satisfaction among students at Konkuk University. We surveyed 4,889 students enrolled in e-learning courses and analyzed 830 completed questionnaires to identify factors that influence student satisfaction with e-learning. Results showed significant correlations between system factors and satisfaction (R2 = 0.577; p = 0.000). The system factor with the greatest impact on satisfaction was course attendance rate (0.224; p = 0.000).


Keywords: E-leaning, E-leaning Satisfaction, Learning Effect, Background Factor, System Factor

1. Introduction
1.1. Necessity and Purpose of Study

With the rapid development of information and communication technologies, and the resulting globalization of university education, tremendous changes have been made in the qualitative perspectives of students on their university educations. These changes have resulted in a paradigm shift in methods of teaching and learning in higher education, with a trend toward globalization of the university experience. Accordingly, education and learning paradigms have shifted to e-learning-based education, reflecting the advancements of the digital revolution and the need to integrate the global experience in higher education (National IT Industry Promotion Agency, 2013).

Therefore, e-learning has been actively adopted by many higher education institutions. Accompanying the universities’ self-efforts, the South Korean government also has established a comprehensive plan to promote informatization of the universities (E-campus vision 2007) and to encourage expansion of e-learning courses and access to e-learning (Ministry of Education & Human Resources Development 2006). With expansion of e-learning, students can strengthen self-directed learning activities without the limitations of time and space, they can perform iterative learning, and they can interact in new ways within the e-learning environment with teachers and other students (Choi & Jeong, 2006; Lim & Lim, 2004).

When experiencing these varied learning activities, the e-learning students tend to demand not only high quality educational content but also ease of use of the e-learning system environment. Discussions on the qualitative evaluations of e-learning are closely related to issues such as ‘learning effect’ and ‘education performance’. Our interests in the learning effect and education performance via e-learning originate a desire for in-depth understanding of the experience of the ‘students’. In other words, what the students, considered as the final consumer of e-learning, demand and their expectations and values on the e-learning are important and should be investigated (Lee, 2005).

To achieve these goals, we performed surveys to determine student satisfaction with the e-learning education at K University, targeting students who took e-courses. We examined and analyzed the background factors (gender, final grade, record of academic probation, record of leave of absence, and course type) that most influenced the reported satisfaction of the students. We also investigated the effects of sub-factors of the e-learning system (connection to the system, linkage of lecture contents, attendance via mobile devices, course attendance rate, and assignment upload) that were associated with reported satisfaction, and we determined which sub-factor of the e-learning system itself had the greatest impact on student satisfaction. We analyzed distinct characteristics of the students and the customized e-learning system environment to determine how best to provide high-quality education content and to suggest improvement plans for the e-learning process in higher education.

1.2. Research Questions

To achieve the goal of this study, the research questions were defined as follows, and the targets were students who took e-courses at K University. The results of the analysis will be discussed later in this paper.

  • (1) Which background factors (gender, final grade, record of academic probation, record of leave of absence, and course type) most influenced student satisfaction, as measured by a) comprehensive satisfaction, b) satisfaction with the learning environment, c) satisfaction with the system environment, and d) satisfaction with the teacher?
  • (2) Which sub-factors of the e-learning system (ease of connection to the system, organization and linkage of lecture contents, attendance via mobile devices, course attendance rate, and assignment upload) were correlated with student satisfaction?
  • (3) Which sub-factor of the e-learning system had the greatest impact on reported student satisfaction?

2. Theoretical Background

Satisfaction in learning was defined by Keller (1983) as being a student’s ability to succeed and a student's awareness of his/her own progress during the learning process (Keller, 1983). Satisfaction was the 1st step of the evaluation responses in the four stages of Kirkpatrick’s learning evaluation model (Choi, 2011). Satisfaction in e-learning offers an ‘immediate result’ for measuring e-learning performance outcomes. Therefore, the measure of student satisfaction in e-learning was used as a representative indicator for measuring e-learning performance, as it corresponded to the relationship between predetermined expectations of students who gained educational content from website and their recognition and satisfaction with the service provided. It implied the comprehensive satisfaction depended on the quality of the website in which the students took the courses. Providing the best service to the e-learning students and measuring the satisfaction with how the service satisfied the expectations and demands of the e-learning students might help to improve the learning effect of the e-learning students (Lee, 2010).

Jeong (2013) determined factors that affected satisfaction in e-learning, such as motivation to take a course, available learning support, lecture contents and methods, learning interactions, assessments and testing, satisfaction with lectures, and further improvements to analyze the students’ recognition of and demands for e-learning education at universities. Kim (2010) defined computer-related characteristics to include student attitude toward computer use, computer experience, computer application ability, academic achievement, and satisfaction in learning to influence the learning effect according to the student’s characteristics. Similarly, Lee (2001) identified influential factors for distance education at web-based virtual universities to include computer and internet use ability, attitude in lecture, psychological environment of course, lecture contents, screen composition of lecture, interaction, and satisfaction in lecture.

Gu (2006) indicated that factors such as student characteristics, learning environment, operator, program, improvement of IT application ability, field application, contribution to self-development were important measures for analyzing the effectiveness of distance education of informatization for faculty. Go (2005) determined the influential factors on learning effect in cyber education to include learning system, learning courses, students, motivations of the students, and learning effect. Park & Choi (2008) identified factors related to the learning effect of e-learning, including computer application ability, motivation of learning, self-regulated learning strategy, environment, teacher, and learning effect. Yoon (2011) defined the factors that affected learning effect of cyber universities, such as self-efficacy, teacher approval, immersion in learning, satisfaction of lecture, and intention for learning persistence, to verify the predictability of the motivations of students, teaching presence, and immersion in learning.

N. Kim (2009) investigated attitudes towards and ideas for online courses, learning content of online courses, designing content for online courses, organizational support for learning, immersion in learning, and comprehensive satisfaction in online courses to examine the structural correlations between motivation of learning, program, organization’s support and interaction, and immersion and learning performance in cyber education. S Kim (2009) identified factors such as current state of e-learning education, convenience of use of the e-course site, immersion in learning in the e-course, learning content and design of the e-course, student participation in the e-course, operation system of e-learning, support for e-learning teachers, assessment in e-learning, operation and support of e-learning education in their study on immersion and satisfaction in learning as perceived by e-learning students' recognized usefulness and convenience.

Jeong (2011) indicated that factors with an impact on e-learning statisfaction to include on the intention of persistence according in e-learning at a company, in the context of structural relationship such as motivation of learning, self-regulated learning, convenience of use, system quality, appropriate expression of information, teacher, reliability, support for administration, learning contents, interaction, assessment of learning, satisfaction, and intention of persistence. J. Kim (2007) found that factors such as inner motivation, convenience for time-saving, self-efficacy, environment of courses, interaction in e-learning, contents for learning, assessment in learning, satisfaction in learning, academic achievement, and immersion in learning all influenced satisfaction and performance in e-learning. Similarly, Kim (2011) identified factors such as computer application ability, motivation, self-regulated learning strategy, environment, teacher, and satisfaction in learning to perform the study as factors that influenced learning effect in e-learning at universities. These factors also predicted the intention of students to enroll in additional e-learning courses. J. Kim (2013) noted that factors such as sufficiency of information, objectivity of information, timely use of material, understandability, immersion, effectiveness and performance of learning, and integrity of information were useful to analyze the factors influencing student satisfaction in mobile learning at universities. Jeong, Seo and Cheong (2010) researched the effect of satisfaction according to elements of e-learning contents on the satisfaction with courses and recognized the importance of demographic factors of students at cyber universities in determining the importance of factors such as composition of learning contents, composition of screen, method of learning process, method of teaching and learning, inducing interaction, provision of material and support system, and convenience and stability of service.

Heo (2014) determined that factors such as learning effect of e-learning, satisfaction with course contents, system functions, and operation and support through confirmatory factor analysis in the study on influencing factors on learning effect of e-learning at universities using structural equation modeling. Moon & Nam (2006) performed a study on education effect of e-learning students using factors such as the environment of the course, experience with the course, fidelity to the course, level of understanding, satisfaction, and operation method of e-learning.

Ahn (2009) selected factors for analysis such as satisfaction in e-learning, e-learning content, and characteristics of e-learning system in a study on production types of e-learning course contents and their influence on satisfaction in learning. Yoo (2012) used factors such as motivation for e-learning and satisfaction in e-learning to perform a study on motivation of and satisfaction in e-learning among adult students. Jo (2012) analyzed the satisfaction and intention of persistence in learning according to students’ experiences of courses and the level of self-directed learning strategy in the environment of e-learning education at universities using factors of such as experience level of course, self-directed learning strategy, satisfaction in learning, and intention of persistence in learning.

The referenced question items from the preceding researches to measure the satisfaction in e-learning were summarized in the following Table 1. The refererenced survey tools developed in the preceding research investigations were modified and restructured to support the goals of the current study to identify the factors determining comprehensive satisfaction, learning, system usefulness, and the teacher effectiveness.

Table 1. 
Referenced Preceding Tools
Classification Referenced Preceding Tools Contents of Questions Number of Questions Reliability
Factor of Comprehensive Satisfaction N. Kim (2009) Additional course, intention for advanced course 5 0.883
J. Kim (2009) Comprehensive satisfaction, intention for recommendation
H. Kim (2011) Comprehensive satisfaction, intention for recommendation
Park and Choi (2008) Suitability of e-course
Suh (2001) Comprehensive satisfaction in course
Ahn (2009) Comprehensive satisfaction, intention for recommendation
B. Kim et al (2010) Comprehensive satisfaction in and intention for recommendation of e-course
Factor of Learning Lee (2001) Level of understanding in learning, inducing motivation of learning 5 0.873
N. Kim (2009) Systematization of learning, learning activities according to the goal of learning
S. Kim (2009) Learning activities according to the goal of learning, course evaluation for the content
J. Kim (2007) Achieving the goal of learning, systematization of learning, course evaluation for the content
H. Kim (2011) Level of understanding in learning, inducing motivation of learning
Lee (2010) Systematic composition of lecture content, suitability of learning activities
Factor of System Go (2005) Stability of system functions 5 0.774
S. Kim (2009) Stability of system functions, error of system
Lee (2010) Convenient submission of assignment, disconnection occurred during the course
Jeong (2011) Occurrence of system error, stability of system
Factor of Teacher Park and Choi (2008) Appropriate feedback by teacher 5 0.901
J. Kim (2009) Appropriate feedback by teacher
H. Kim (2011) Feedback by teacher, interactive exchange learning activities
Yoon (2011) Feedback by teacher, interaction between teacher and student
Jeong (2011) Feedback by teacher, level of support for learning process
Jeong (2012) Interaction between teacher and student


3. Research Methodology
3.1. Target

To measure the satisfaction in the e-courses at K University, we surveyed a target group of students who took e-courses in 2014. For the survey, we used the online survey tool of LMS (Learning Management System; Dec. 2014). A total of 4,889 students in twenty courses received survey requests through online contact such as e-mail and SMS; Of these, 830 students, or 17% of those surveyed, responded to the survey and provided survey data for analysis.

3.2. Research Tool
3.2.1. Validity and Reliability

To measure student satisfaction with e-courses at K University, various survey questionnaires that were used in previously described research were modified and restructured for this study. We utilized information from tools described by Suh (2001), Park and Choi (2008), N. Kim (2009), J. Kim (2007), H. Kim (2011), Lee (2010), J. S. Kim (2009), Lee, Kim and Kim (2010). We modified these tools to include the comprehensive satisfaction factors presented in survey tools by Lee (2010), Lee (2001), S. Kim (2009), J. Kim (2007), N. Kim (2009), and Kim (2011). We adapted the learning factors described by Go (2005), Jeong (2011), and Park and Choi (2008), and adapted our survey to include the system factors, and tools by J. Kim (2009), S. Kim (2009), Jeong (2012), Yoon (2011), Jeong (2011), and H. Kim (2011), were modified to contain teacher factors. Three specialists in the field of educational engineering verified the reliability of the modified measuring tool to insure its integrity and utility for the purpose intended.

The questionnaires were composed of the comprehensive satisfaction factor, learning factor, system factor, and teacher factor in 5-point Likert scale in which scores ranged from 5 points for “Strongly agree” to 1 point for “Strongly disagree”. We used the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) and Bartlett's Test of Sphericity to assess the validity of the survey. Results are reported in Table 2, with KMO>0.9 and significance for Bartlett test p<0.05, thus verifying that the factor analysis was appropriate.

Table 2. 
KMO and Bartlett's Test
KMO and Bartlett's Test
Kaiser-Meyer-Olkin Measure of Sampling Adequacy 0.939
Bartlett's Test of Sphericity Approx. Chi-Square 10410.884
df 190
Level of Significance 0.000

In Table 3, the rotated component matrix, 20 questions were extracted to 4 elements. Values in bold type values indicated the factor loading values of each individual variable, with factor loading values obtained using Varimax rotation. Element 1 was the teacher factor, Element 2 was the comprehensive satisfaction factor, Element 3 was the learning factor, and Element 4 was the system factor.

Table 3. 
Result of Exploratory Factor Analysis
Number of Questionnaires Element
1 2 3 4
Q17 .833 .269 .164 .162
Q19 .820 .329 .158 .134
Q18 .819 .204 .096 .096
Q16 .719 .211 .343 .095
Q20 .562 .272 .349 .267
Q3 .198 .781 .166 .097
Q5 .253 .759 .264 .114
Q1 .255 .741 .292 .282
Q2 .224 .711 .291 .279
Q4 .241 .670 .139 .046
Q6 .424 .438 .802 .238
Q7 .200 .244 .781 .177
Q8 .263 .280 .641 .162
Q9 .241 .408 .529 .269
Q10 .348 .410 .309 .261
Q11 .105 .092 .140 .799
Q12 .120 .074 .328 .761
Q15 .060 .150 .259 .683
Q13 .193 .343 -.164 .601
Q14 .432 .097 .106 .484
Extraction Method: Principal Element Analysis
Rotation Method: Varimax with Kaiser Normalization
a. Rotation converged in 6 iterations.

As the result of reliability analysis for each factor in the survey questionnaires in the Table 4, the reliability of the comprehensive satisfaction factor was 0.883, the learning factor was 0.873, the system factor was 0.774, and the teacher factor was 0.901 of Cronbach's alpha coefficient.

Table 4. 
Result of Reliability Analysis
( 0.935 Cronbach's Alpha Coefficient)
By Factors Contents Cronbach's Alpha Coefficient
Comprehensive Satisfaction Factor I am comprehensively satisfied with the e-courses. .883
I am willing to recommend the e-courses to other people.
I think that the e-courses are comprehensively related to my study.
I think that the e-courses are comprehensively appropriate for improving my ability.
I am willing to take additional and advanced classes related to the e-courses.
Learning Factor The e-courses suggest learning activities that are appropriate according to the goal of study. .873
The e-courses contain content for stimulating learning motivation.
The e-courses are systematically organized.
The e-courses are composed to be easily understandable.
The content of e-courses is appropriate for taking midterm/final exams.
System Factor During the e-courses, internet connection is not interrupted. .774
Loading the e-courses is enabled without problem.
Taking the e-courses on mobile devices is enabled without problem.
Course attendance rate is well-counted for the e-courses without errors.
Uploading assignment is exactly performed by learning management system.
Teacher Factor The teacher is enthusiastic about the e-courses. .901
The teacher gives feedback for the student’s learning activities at the appropriate time.
The teacher gives appropriate assignments.
The teaching process and notices by the teacher are helpful for the learning process.
The teacher communicates with the students through bulletin board, e-mail, and etc.

3.2.2. Data Processing and Statistics Method

Frequency analysis, analysis of variance (ANOVA), and T-tests were conducted using SPSS version 22.0 to process the data, and Scheffe's post hoc test was performed for significant data. In measuring variables of each item in the survey, valid factors were extracted by factor analysis with Varimax orthogonal rotation, and reliability analysis was carried out with the calculation of internal consistency coefficient to verify the reliability between the extracted items.

Regarding the process of research using the statistics program, the characteristics of students who responded to the survey were classified to conduct the frequency analysis; the difference in the satisfaction according to each group and correlations between the sub-factors of system and the satisfaction were analyzed, and then multiple regression analysis was conducted to investigate which sub-factor of system use had the greatest effect on satisfaction.


4. Result
4.1. Demographic Characteristics

Using survey responses of e-learning students at K university, this study analyzed how student satisfaction was differentiated according to the background of the students (gender, grade, record of academic probation, record of leave of absence, and course type), whether the sub-factors of system (connection to the system, linkage of lecture contents, attendance via mobile devices, course attendance rate, and assignment upload) were associated with student satisfaction, and which sub-factor related to system had the greatest impact on reported student satisfaction. Data of 830 respondents were statistically analyzed. Within this group, the number of male students, 492 (59.3%), was higher than the number of female students, 338 (40.7%). The number of the 1st grade students, 296 (35.7%), was the highest, and the number of the 3rd grade students, 130 (16.4%), was the lowest.

The number of students who were never placed under academic probation, 768 (92.5%), was substantially higher than the number of the students who had been placed under academic probation, 62 (7.5%); regarding the possible number of subjects for a semester, the number of respondents taking two classes, 420 (50.6%), was the highest, and those taking five classes, 52 (6.3%), was the lowest.

Regarding the subjects of the courses, the number taking liberal arts, 726 (87.5%), was higher than that taking major subjects, 104 (12.5%). The number of respondents who have never taken a leave of absence, 612 (73.7%), was higher than those who had taken a leave of absence, 218 (26.3%). Regarding the desired course of study, the number of desiring liberal arts subjects, 593 (71.8%), was higher than the number of desiring major subjects, 234 (26.2%). These demographic results are summarized in Table 5.

Table 5. 
Frequency According to the Background of Students
Background of Students Frequency (N=830) Ratio (%)
Male 492 59.3
Female 338 40.7
The 1st grade 296 35.7
The 2nd grade 208 25.1
The 3rd grade 190 22.9
The 4th grade 136 16.4
Placed under academic probation (Yes) 62 7.5
Placed under academic probation (No) 768 92.5
Took a leave of absence (Yes) 218 26.3
Took a leave of absence (No) 612 73.7
Subject (Liberal arts) 726 87.5
Subject (Major subjects) 104 12.5
Total 830 100.0

4.2. Analysis for Differences in Satisfaction by Groups
4.2.1. Difference in Satisfaction by Genders

We analyzed differences in student satisfaction by gender. Data are reported in Table 6. For female students, the learning factor ranked highest (3.3846), the system factor scored 3.2331, the teacher factor scored 3.2320, and the comprehensive satisfaction factor scored 3.2178. For male students, the learning factor again ranked highest (3.2935), the teacher factor scored 3.0846 and the system factor scored 3.0797. The comprehensive satisfaction factor of the female students was 3.2178, higher than that of the male students, which scored comprehensive satisfaction at 2.9366.

Table 6. 
Characteristics of the Sub-factors for Satisfaction in E-learning by Genders
Gender Comprehensive Satisfaction Learning System Teacher
Male Average 2.9366 3.2935 3.0797 3.0846
Variance .707 .550 .615 .644
Standard Deviation .84103 .74180 .78447 .80265
Ratio (%) 59.3% 59.3% 59.3% 59.3%
Female Average 3.2178 3.3846 3.2331 3.2320
Variance .991 .805 .688 .977
Standard Deviation .99538 .89708 .82934 .98856
Ratio (%) 40.7% 40.7% 40.7% 40.7%

Differences in the average satisfaction score by gender were analyzed using the two-tailed T -test, and the satisfaction scores by genders were analyzed using a 95% confidence interval (1.37, 1.44) (Table 7). The difference between the groups was statistically significant with t > 1.96 and at P = 0.000 level of significance (both tails).

Table 7. 
Post-hoc Analysis of Satisfaction in E-learning by Genders
Gender N t df Level of Significance (Both Tails) Difference in Averages Difference with 95% Confidence Interval
Lower Limit Upper Limit
830 82.467 829 0.000 1.407 1.37 1.44

4.2.2. Satisfaction by Grades

We also analyzed the differences by grade level. On the whole, the satisfaction of the 4th grade was the highest and the 2nd grade was the lowest. For the learning factor, the 4th grade scored 3.5912 and the 2nd grade scored 3.2462; it implied that lack of feedback in e-learning demanded a method to overcome the difference in the level of difficulty in the courses.

Table 8. 
Characteristics of the Sub-factors for Satisfaction in E-learning by Grades
Grade Comprehensive Satisfaction Learning System Teacher
The 1st Grade Average 2.9473 3.2892 3.0676 3.0770
Variance .660 .510 .487 .627
Standard Deviation .81257 .71413 .69772 .79166
Ratio (%) 35.7% 35.7% 35.7% 35.7%
The 2nd Grade Average 2.9308 3.2462 3.0942 3.0442
Variance .951 .876 .856 .907
Standard Deviation .97496 .93591 .92500 .95258
Ratio (%) 25.1% 25.1% 25.1% 25.1%
The 3rd Grade Average 3.0484 3.3011 3.1432 3.1937
Variance .917 .522 .533 .690
Standard Deviation .95783 .72265 .73010 .83086
Ratio (%) 22.9% 22.9% 22.9% 22.9%
The 4th Grade Average 3.4647 3.5912 3.3765 3.3765
Variance .760 .750 .797 1.004
Standard Deviation .87191 .86577 .89279 1.00224
Ratio (%) 16.4% 16.4% 16.4% 16.4%

The Table 9 shows the result of the Post-hoc test within 0.05 level of significance for the 1st grade, 2nd grade, and the 4th grade, to evaluate difference between the grades; the level of satisfaction ranking was, in order, 4th grade, 3rd grade, 1st grade, and 2nd grade.

Table 9. 
Post-hoc Analysis of Satisfaction in E-learning by Genders
(I) Grade Difference in Averages (I-J) Standard Error Level of Significance 95% Confidence Interval
Lower Limit Upper Limit
The 1st Grade The 2nd Grade .01642 .06364 .996 -.1618 .1947
The 3rd Grade -.07631 .06538 .715 -.2595 .1069
The 4th Grade -.35694* .07286 .000 -.5610 -.1528
The 2nd Grade The 1st Grade -.01642 .06364 .996 -.1947 .1618
The 3rd Grade -.09273 .07059 .631 -.2905 .1050
The 4th Grade -.37336* .07756 .000 -.5906 -.1561
The 3rd Grade The 1st Grade .07631 .06538 .715 -.1069 .2595
The 2nd Grade .09273 .07059 .631 -.1050 .2905
The 4th Grade -.28063* .07900 .006 -.5019 -.0593
The 4th Grade The 1st Grade .35694* .07286 .000 .1528 .5610
The 2nd Grade .37336* .07756 .000 .1561 .5906
The 3rd Grade .28063* .07900 .006 .0593 .5019
* The difference in averages was significant at the level of 0.05.

4.2.3. Satisfaction by record of Academic Probation

We analyzed the differences in student satisfaction by the record of academic probation; in this area, the satisfaction of students who had been placed under academic probation was higher than the students who had never been placed under academic probation. Students who had been placed on academic probation scored, in order, 3.4774 for the learning factor, 3.3935 for the teacher factor, 3.3097 for the comprehensive satisfaction factor, and 3.3135 for the system factor.

As the e-learning courses enabled iterative learning according to the student’s level of learning and desired time of learning, it appears to have helped students with a history of academic probation to improve their efficiency of learning. As the students should be faithful to their self-directed study, the satisfaction was reported as higher than that of students who were never been placed under academic probation.

Table 10. 
Characteristics of the Sub-factors for Satisfaction in E-learning by record of Academic Probation
Academic Probation Comprehensive Satisfaction Learning System Teacher
Yes Average 3.3097 3.4774 3.1355 3.3935
Variance .860 1.059 1.194 1.200
Standard Deviation .92731 1.02899 1.09291 1.09543
Ratio (%) 7.5% 7.5% 7.5% 7.5%
No Average 3.0302 3.3188 3.1427 3.1245
Variance .835 .622 .607 .747
Standard Deviation .91354 .78862 .77929 .86407
Ratio (%) 92.5% 92.5% 92.5% 92.5%

Differences in the satisfaction score by record of academic probation were analyzed using the two tailed T -test and a 95% confidence interval (1.91, 1.94). The difference between the groups was statistically significant with t > 1.96 and at P = 0.000 level of significance (both tails).

Table 11. 
Post-hoc Analysis of Satisfaction in E-learning by record of Academic Probation
Academic Probation N t df Level of Significance (Both Tails) Difference in Averages Difference with 95% Confidence Interval
Lower Limit Upper Limit
830 210.852 829 0.000 1.925 1.91 1.94

4.2.4. Satisfaction by Record of Leave of Absence

We analyzed the differences in student satisfaction by the record of leave of absence. In this area, students who took a leave of absence scored higher than the students who never took a leave of absence, scoring in order of 3.4477 for the learning factor, 3.3248 for the teacher factor, 3.2532 for the system factor, and 3.2330 for the comprehensive satisfaction factor. The system factor of the students who never took a leave of absence, scored 3.1026, lower than the students who took a leave of absence. As it reduced the satisfaction, the system should be improved.

Table 12. 
Characteristics of the Sub-factors for Satisfaction in E-learning by record of Leave of Absence
Leave of Absence Comprehensive Satisfaction Learning System Teacher
Yes Average 3.2330 3.4477 3.2532 3.3248
Variance .785 .684 .712 .883
Standard Deviation .88594 .82714 .84380 .93977
Ratio (%) 26.3% 26.3% 26.3% 26.3%
No Average 2.9863 3.2889 3.1026 3.0804
Variance .846 .639 .623 .734
Standard Deviation .91980 .79947 .78916 .85702
Ratio (%) 73.7% 73.7% 73.7% 73.7%

The differences in the satisfaction score by record of leave of absence were analyzed using T -test and a 95% confidence interval (1.71, 1.77). The difference between the groups was statistically significant with t > 1.96 and at P = 0.000 level of significance (both tails).

Table 13. 
Post-hoc Analysis of Satisfaction in E-learning by record of Leave of Absence
Leave of Absence N t df Level of Significance (Both Tails) Difference in Averages Difference with 95% Confidence Interval
Lower Limit Upper Limit
830 113.668 829 0.000 1.737 1.71 1.77

4.2.5. Satisfaction by Course Types

In the analysis of satisfaction by course types, the satisfaction of students who took courses in liberal arts was higher than that of students who took major subjects. This result implied that not only the characteristics of the liberal arts, but also the major subjects, should be better evaluated to imporve the e-courses.

Table 14. 
Characteristics of the Sub-factors for Satisfaction in E-learning by Course Types
Course Types Comprehensive Satisfaction Learning System Teacher
Liberal Arts Average 3.0457 3.4510 3.1642 3.2342
Variance .858 .692 .673 .819
Standard Deviation .92607 .83202 .82065 .90488
Ratio (%) 87.5% 87.5% 87.5% 87.5%
Major Subjects Average 3.0885 3.3231 2.9885 3.4192
Variance .729 .399 .462 .460
Standard Deviation .85392 .63203 .67971 .67839
Ratio (%) 12.5% 12.5% 12.5% 12.5%

Differences in the satisfaction score by course types were analyzed using T -test and a 95% confidence interval (1.10, 1.15); the difference between the groups was statistically significant with t > 1.96 and at P = .000 level of significance (both tails).

Table 15. 
Post-hoc Analysis of Satisfaction in E-learning by Course Types
Course Types N t df Level of Significance (Both Tails) Difference in Averages Difference with 95% Confidence Interval
Lower Limit Upper Limit
830 97.868 829 0.000 1.125 1.10 1.15

4.3. Analysis of Correlations between System Sub-Factors and Satisfaction

The correlations between the satisfaction and the sub-factors of system (connection to the system, linkage of lecture contents, attendance via mobile devices, course attendance rate, and assignment upload) were analyzed.

An analysis of correlations was conducted to determine whether the factors were independent or were correlated (0 < correlations ≤1) between factors using Pearson correlations analysis, and the results were significant at P = 0.01 level. The correlations between the satisfaction and connection to the system scored 0.534, linkage of lecture contents scored 0.593, attendance via mobile devices scored .503, course attendance rate scored 0.551, and assignment upload was positive at 0.544 (Table 16).

Table 16. 
Correlations between the Sub-factors of System and Satisfaction
Connection to the System Operation of Lecture Contents Attendance via Mobile Devices Course Attendance Rate Assignment Upload Satisfaction
Connection to the System 1
Linkage of Lecture Contents .611** 1
Attendance via Mobile Devices .366** .382** 1
Course Attendance Rate .371** .391** .303** 1
Assignment Upload .490** .509** .335** .298** 1
Satisfaction .534** .593** .503** .551** .544** 1
** Significant at the level of 0.01 (both tails).

4.4. Analysis of Influencing Relationship between System Sub-Factors and Satisfaction

Multiple regression analysis was conducted to investigate which sub-factor of system (connection to the system, linkage of lecture contents, attendance via mobile devices, course attendance rate, and assignment upload) most affected student satisfaction.

The value of significance probability was P = 0.000 to be significant in Table 17. The explanatory power between the sub-factors of system (connection to the system, linkage of lecture contents, attendance via mobile devices, course attendance rate, and assignment upload) and the satisfaction was R2 = 0.577 (57.7%). The value of Durbin-Watson was 1.995, very close to 2, implying independence among the sub-factors.

Table 17. 
Explanation of Factors
Summary of Model b
Model R R-squared Adjusted R-squared Standard Error of Estimate Values Durbin-Watson
1 .760a .577 .575 .46575 1.995
a. Predicted variables: (constant), connection to the system, linkage of lecture contents, attendance via mobile devices, course attendance rate, and assignment upload
b. Dependent Variable: Satisfaction

Table 18 shows the results of VIF (Variance Inflation Factor) values, used to evaluate the multicollinearity of sub-factors. As the VIF values of all items were less than 10, this confirmed multicollinearity. The unstandardized coefficients in the regression equation for the variable of connection to the system was 0.062, linkage of lecture contents was 0.142, attendance via mobile devices was 0.134, course attendance rate was 0.224, and assignment upload was 0.136; the multiple regression equation for satisfaction can be written as follows:

Satisfaction = 0.935 + 0.062 × connection to the system + 0.142 × linkage of lecture contents + 0.134 × attendance via mobile devices + 0.224 × course attendance rate + 0.136 × assignment upload.

The most influential factors on the satisfaction among the sub-factors of system was course attendance rate (0.224) according to the beta value of the unstandardized coefficients in Table 18.

Table 18. 
Influencing Relationship between the Sub-factors of System and Satisfaction [Std. error = Standard Error, LL = lower limit of confidence interval (CI), UL = upper limit of 95% confidence interval.]
Model Unstandard-ized
Coefficients
Standard-ized
Coefficients
t Level of Significance B with 95%
C.I.
Collinearity
Statistics
B Std. Error Beta LL UL Allow-able Error VIF
1 (constant) .935 .071 13.195 .000 .796 1.075
Connection to the System .062 .018 .103 3.391 .001 .026 .099 .561 1.782
Linkage of Lecture .142 .020 .220 7.117 .000 .103 .181 .538 1.860
Attendance via Mobile Devices .134 .016 .219 8.604 .000 .103 .164 .794 1.260
Course Attendance Rate .224 .019 .295 11.615 .000 .186 .262 .797 1.255
Assignment Upload .136 .017 .220 7.983 .000 .102 .169 .674 1.484


5. Conclusion and Future research
5.1. Conclusion

This study was aimed to suggest a direction for improving the operation of e-courses to increase learning outcomes and student satisfaction. Accordingly, online surveys were conducted with students who were enrolled in the e-courses. Student satisfaction with the e-learning experience was investigated and analyzed based on the data of 830 survey respondents. The results can be summarized as follows:

First, we analyzed the differences in satisfaction according to background factors such as comprehensive satisfaction, learning, system, and teacher. Satisfaction scores were significant according to the groups with differences by each sub-factor.

Jeong (2013), Lee (2010), Kim (2010), Park and Choi (2006) indicated that satisfaction by genders was differentiated according to the demographic characteristics for learning support, learning interaction, the comprehensive satisfaction factor, and system environment. Our study also revealed that the satisfaction in academic achievement rate and system environment was differentiated by genders. Jang (2010) researched the effect of satisfaction with elements of e-learning course contents on the satisfaction with courses and recognitions of effectiveness and importance with regard to demographic factors of students at cyber universities. According to the results of comparing satisfaction by grades, it showed higher satisfaction in the lower grades due to the convenience and stability of service. On the contrary, the current study revealed that the students of the lower grades could not overcome differences in the level of difficulty in the e-learning. To better understand this variable, additional research is required.

When we analyzed reports of student satisfaction by record of academic probation, the satisfaction of students who had been placed under academic probation at some point was higher. We expect that iterative learning enabled these students to improve their efficiency of learning, thereby increasing satisfaction with e-courses. The satisfaction of students who took courses in liberal arts subjects was higher than satisfaction reported for major courses, perhaps because the liberal arts topics satisfy student desires for more varied educational content. Especially in the differences in satisfaction by record of academic probation, the satisfaction of the students who were placed under academic probation was higher in their learning; thus, additional research focusing on the effect of e-learning for the depressed students could be useful. Also, satisfaction that was higher with the students who took the courses of liberal arts gives support for development of more varied course contents.

Second, we analyzed the correlations between the sub-factors of system (connection to the system, linkage of lecture contents, attendance via mobile devices, course attendance rate, and assignment upload) and satisfaction. The results were analyzed to be significant at the 0.01 level for the correlations; the correlations between the satisfaction and connection to the system scored 0.534, linkage of lecture contents scored 0.593, attendance via mobile devices scored 0.503, course attendance rate scored 0.551, and assignment upload scored 0.544 to be positive.

Third, the most influential sub-factors of system (connection to the system, linkage of lecture contents, attendance via mobile devices, course attendance rate, and assignment upload) on satisfaction was identified using multiple regression analysis; the value of significance probability was P = 0.000 to be significant. The explanatory power between the sub-factors of system and the satisfaction was R2 = 0.577 (57.7%), and the value of Durbin-Watson was 1.995, implying that it was independent; and as the VIF values of all items were less than 10, multicollinearity was confirmed. The most influential factor on the satisfaction among the sub-factors of system was the course attendance rate (0.224).


Fig. 1. 
Influencing Relationship between the Sub-factors of System and Satisfaction

5.2. Future research

Based on these results, numerous new directions for improving e-learning courses to increase the learning were revealed:

First, the differences by gender showed that the male students scored lower satisfaction than the female students. It implied that the male students were more sensitive to the motivation of learning, difficulty of the course, assessment, e-learning course system, etc. Thus, the content of e-courses and student support services could be updated to support these students.

Second, the analysis of differences by grades showed that the satisfaction of the higher grades were higher than the lower grades in the whole areas. Thus, e-learning content for the lower grades should be evaluated and higher-quality courses should be established to support the lower grades.

Third, e-learning enabled iterative learning according to the student’s level of learning and personal timing of learning to enhance efficiency in learning for the students who were ever placed under academic probation. Therefore, research should be conduct with a focus on the effect of e-learning on the depressed students to improve achievement.

Fourth, the satisfaction reported by students who took the courses of liberal arts was higher on the whole than the satisfaction of students who took the major subjects. Thus, the method to improve the characteristics major subjects in the e-courses, various e-learning content, and a model to overcome the individual characteristics of the courses targeting unspecified masses should be developed.

Finally, the course attendance rate (0.224), among all other sub-factors of system, was determined to have the greatest impact on the satisfaction. As the e-courses system determined 100% course attendance, the system did not effectively measure attendance. Therefore, the system should be upgraded to manage the e-learning content and determination of attendance more effectively in the learning management system.


References
1. Ahn, E., (2009), A study on the learner's satisfaction with e-learning on contents by different types of visual presentations, Doctoral dissertation, University of Hong-ik, Retrieved from http://www.riss.kr/link?id=T11709021.
2. Choi, B., (2011), A structural equation modeling among social presence, course satisfaction, and academic achievement of learners at a cyber university, A Master’s thesis, University of Soongsil, Retrieved from http://www.riss.kr/link?id=T12635150.
3. Choi, S., & Jeong, I., (2006), An effect factor analysis of open-online distance education, Journal of Educational Research, 37(1), p369-388.
4. Gu, G., (2006), A study on factors associated with the effectiveness of distance IT education for teachers, Journal of Lifelong Education, 12(10), p1-22.
5. Heo, G., (2014), A Study on the Structural Equation Modeling for the effect of e-Learning, Journal of Internet Computing and Services, 15(6), p77-84.
6. Jang, E., Seo, Y., & Cheong, H., (2010), The effects on course satisfaction, effectiveness awareness, and importance awareness from e-learning contents’ construct factors based on demographic factors of cyber university students, Journal of Educational Technology, 20(1), p57-85.
7. Jeong, K., (2011), The Structural relationship of Influencing factors of satisfaction and persistence in corporate e-Learning, Master’s thesis, University of Ewha Woman’s, Retrieved from http://www.riss.kr/link?id=T11551220.
8. Jeong, W., (2013), A study on the awareness of learners about university e-Learning and their Needs, Master’s thesis, University of Gyeongsang National, Retrieved from http://www.riss.kr/link?id=T13099385.
9. Jo, S., (2012), The Influence of University Students Online Course Experience and Level of Self-directed Learning Strategy on Learning Satisfaction and Persistence, A Master’s thesis, University of Korea National, Retrieved from http://www.riss.kr/link?id=T13091736.
10. Keller, J. M., (1983), Motivational design of instruction, In, (Ed.) C.M. Reigeluth, Instructional designing theories and models: An overview of their current status, Hillsdale, NJ: Lawrence Erlbaum Associates.
11. Kim, B., (2010), The Influence of Learner Characteristics on Learning Effect in University E-learning, A Master’s thesis, University of Sungshin Women’s, Retrieved from http://www.riss.kr/link?id=T12150525.
12. Kim, H., (2011), The structural relationship between presence and the effectiveness of e-Learning in the corporate setting, Doctoral dissertation, University of Konkuk, Retrieved from http://www.riss.kr/link?id=T11551220.
13. Kim, J., (2007), The Determinants of University Students’ Satisfaction and Performance in e-Learning Environment, Doctoral dissertation, University of Inje, Retrieved from http://www.riss.kr/link?id=T11077948.
14. Kim, J. S., (2009), The structural relationship between presence and the effectiveness of e-Learning in the corporate setting, Doctoral dissertation, University of Ewha Woman’s, Retrieved from http://www.riss.kr/link?id=T11551220.
15. Kim, N., (2009), The Structural Relationship among Academic Motivation, Program, Organizational Support, Interaction, Flow and Learning Outcome In Cyber Education, Doctoral dissertation, University of Ewha Woman’s, Retrieved from http://www.riss.kr/link?id=T11551318.
16. Kim, S., (2009), The Effect of Usability and Easiness Perceived by e-learners on the Flow and Satisfaction of Learning, A Master thesis, University of Sookmyung Women’s, Retrieved from http://www.riss.kr/link?id=T11731129.
17. Go, W., (2005), Research for studying effect factors of cyber education, A Master’s thesis, University of Dongguk, Retrieved from http://www.riss.kr/link?id=T10384005.
18. Lee, D., (2001), The Factors that affect educational effectiveness of distance education in web based virtual university, A Master’s thesis, University of Kongju, Retrieved from http://www.riss.kr/link?id=T8941056.
19. Lee, E., (2010), Analysis of the variables influencing user satisfaction in university e-learning, A Master’s thesis, University of Konkuk, Retrieved from http://www.riss.kr/link?id=T12088839.
20. Lee, O., Kim, B., & Kim, T., (2010), The Effect of Computer-Related Characteristics on University Students' Achievement in e-Learning Courses, Journal of Korean Association for learner-centered Curriculum and instruction, 11(2), p23-44.
21. Lee, S., (2005), An study on e-learning recognition and learning behavioral, Seoul: Korea Research Institute for Vocational Education & Training.
22. Lim, B., & Lim, J., (2004), Status and Activation Method of e-learning a Higher Education, Journal of Korea multimedia Society, 8(3), p16-30.
23. Ministry of Education Human Resources Development, (2006), Adapting education to the information age, Retrieved from http://english.keris.or.kr/whitepaper/WhitePaper_eng_2006.pdf.
24. Moon, S., & Nam, S., (2007), A Study on the Educational Effectiveness of e-Learning, The Journal of the Korea Contents Association, 7(1), p161-168.
25. National IT Industry Promotion Agency, (2013), e-Learning White Paper.
26. Park, H., & Choi, M., (2008), Relationships Between e-learning Effectiveness and Its Related Factors in Higher Education, Journal of Educational Technology, 24(1), p27-53.
27. Singh, K. P., & Chander, H., (2013), Professional inclination of library and information science (LIS) students of India, International Journal of Knowledge Content Development & Technology, 3(2), p5-27.
28. Suh, H., (2001), A study of the factors related learning outcome in the Web-Based lifelong learning program, Doctoral dissertation, University of Sookmyung Women's, Retrieved from http://www.riss.kr/link?id=T8163106.
29. Yoo, H., (2012), A Study on e-learning Motivation and Satisfaction of Adult learners, A Master’s Thesis, University of Chung-Ang, Retrieved from http://www.riss.kr/link?id=T12908869.
30. Yoon, S., (2011), Verification of the predictibility of learner’s motivation, teaching presence, learning flow on learning outcome in cyber university, Asian Journal of Education, 12(1), p141-166.

• About the authors:
‣ Yeong-ae Kwon

E-mail: herayaa@kku.ac.kr

- PUBLISHED PAPERS (2013~2015)

K. youngae, “Study on Real Time Digital Evidence Collection for Hidden Process Attack”, Korea Entertainment Industry Association, 2010, pp209-212

K. youngae (2011), "Modelling of Digital Forensic Evidence Collection Procedure for Detecting Hidden Process Attacks", Doctoral Degree, Chungbuk National University

- CAREER

MA, Computer Education,Graduate School of Education, Semyung University

phD, Department of Computer Engineering Graduate School, Chungbuk National University

researcher, Center for Teaching and Learning, Kunkuk University

‣ Younghee Noh

Email: irs4u@kku.ac.kr

Younghee Noh has an MA and a PhD in Library & Information Science from Yonsei University, Seoul. She has published more than 50 books, including 3 books awarded as Outstanding Academic Books by Ministry of Culture, Sports and Tourism(Government) and more than 120 papers, including one selected as a Featured Article by the Informed Librarian Online in February 2012.

She was listed in the Marquis Who's Who in the World in 2012-2016 and Who's Who in Science and Engineering in 2016-2017. She received research excellence awards from both Konkuk University (2009) and Konkuk University Alumni (2013) as well as recognition by “the award for Teaching Excellence” from Konkuk University in 2014. She received research excellence awards from ‘Korean Library and Information Science Society’ in 2014. One of the books she published in 2014, was selected as ‘Outstanding Academic Book’ by Ministry of Culture, Sports and Tourism in 2015. She received the Awards for Professional Excellence as Asia Library Leaders from Satija Research Foundation in Library and Information Science (India) in 2014. She has been a Chief Editor of World Research Journal of Library and Information Science in Mar 2013~ Feb 2016.

Since 2004, she has been a Professor in the Department of Library & Information Science at Konkuk University, where she teaches courses in Metadata, Digital Libraries, Processing of Internet Information Resources, and Digital Contents.