Co-Author: Harnoor Kaur
The unexpected lockdown announced by the Government of India on March 21, 2020 with an aim to control the COVID-19 pandemic has had several effects on the education of school and college-going students. In this paper, we examine the effectiveness of teaching and preparedness of the stakeholders at the University of Delhi, and the impact of the lockdown on students currently enrolled in the Undergraduate program at the University. Our analysis draws on a regression study, which uses the data of 181 undergraduate students collected through an online survey. We find that average student participation and teaching regularity drops significantly in an online setting. From the regression model established, efficient curriculum coverage implies a reduction in the probability of adoption of ineffective online methods by 25 percent. A majority of students would prefer on-campus classes, throwing light on the efficacy of existing educational infrastructure.
Introduction
The Coronavirus pandemic has forced the education sector to move entirely to an online setting without a warning. Although the teacher quality, course requirements and material remain identical in an offline and online setting, noticeable differences arise among the two, which are bound to have effects on class participation as well as class performance.
Studies conducted on classroom participation of students in each of the two settings reveal interesting insights. Galyon. Heaton. Best, et al (2015) studied group cohesion, class participation, and student performance in a physical classroom as well as a hybrid setting (which included online teaching and physical examinations). The study found that participation in both settings was high, but group cohesion and student performance were significantly lower in a hybrid setting. Despite identical course requirements, the participants favored a conventional classroom over a hybrid one. Caspi. Chajut. and Saporta (2008) found certain gender differences in participation in the two modes of teaching, where men were more interactive in a physical classroom setting whereas women were more active in participating through written communication in an online setting. Their study also concluded that all in all, an online environment was not conducive for either gender. However, there is a dearth of studies to measure the effectiveness of a sudden transition to the online mode of teaching, forced by the pandemic, in the University of Delhi.
The University of Delhi is a collegiate public central university which witnesses students from diverse backgrounds each year. It offers a plethora of courses, such as language courses, arts courses in subjects including, but not limited to Philosophy, Psychology, Economics, History, Geography, Political Science, Sociology and Journalism, and commerce based courses such B.Com, Business Economics and Business Studies. These majors contain little to no practical elements and can have a somewhat smoother transition to an online platform. For certain other subjects, like Mathematics and Statistics, an online transition is less feasible but can be made possible with proper tools such as an online interface to type mathematical equations and formulae. However, for most scientific disciplines, including but not limited to, Applied Science, Biomedical Science, Botany, Chemistry, Physics, Zoology and Radiography require exposure to experimental lab work and self-practice of the same. For subjects such as Computer Science, Fine Arts, Music and Dance, it is important for students to have the required tools such as software compatible computers along with required software, paints and canvas, musical instruments, and an open space to practice, as these components form an integral part of their course learnings. Subjects such as Physical Education, Vocational Studies and Social Work also struggle with similar issues. Apart from a diverse array of subjects, regional differences among students impact their learning outcomes, as the problem of access to these online facilities comes into.
In this paper, we focus our attention on the perceptions of undergraduate students at the University on teaching effectiveness and class participation before and during the lockdown. The main contribution of this paper is to assess the participation of students in current times along with measuring the effectiveness of online teaching and preparedness in handling the transition to an online teaching environment. This paper is the first step towards acknowledging the problems in the e-learning process, which highlights how this seemingly progressive transition to online learning might, in reality, be a move that takes us back insofar as gender, regional, and income inequalities are concerned.
Methodology
Data Collection
Primary data was collected via an online survey and circulated among students and peers using social media platforms such as WhatsApp and LinkedIn. The survey was split into two questionnaires, one aimed at students and the other towards teachers.. The questions were carefully designed for students to emphasize on environments before and during the lockdown using a linear scale of attitudes and agreement:
Basic description of college, course and year of study
Number of relevant technological devices owned and family members who use them
Pre-lockdown –
a. Participation in the classroom
b. Quality of teaching methodologies
c. Effectiveness of in-person classes
Quarantine –
a. Efficient curriculum coverage
b. Teaching regularity
c. Teaching methods
d. Participation in the online setting
e. Effectiveness of administration grievance counselling
f. Perception of online classes as a permanent substitute
g. Expectations of a future culture of education
h. The overall effectiveness of Online Classes
Other questions of interest included the merits and demerits of using online methods and possible health concerns, both physical and mental.
A crucial objective of designing the survey for teachers was to attempt an assessment of personal attitudes towards the online learning culture using qualitative, anecdotal comments. The following questions were included:
Rating personal comfort with regards to using digital platforms
Effectiveness of communication received from the administration and grievance counselling system
Issues faced with using online methods
Views on the preparedness of students for examinations
Expectations of future education culture
The overall effectiveness of online classes
Comments on a desired course of action to conduct semester examinations
Analysis of collected data includes a descriptive summary along with inferential statistics, and a Multinomial Logistic Regression Model fitted for the student sample using Python and Microsoft Excel.
Theoretical Construction
Inferential analysis of the data involved assessing sample means of ‘Participation in learning’ before and during the lockdown, under a null hypothesis that there is no change in average participation of students over the course of classroom teaching and online platforms, as stated below:
H0: m1 - m2 = 0
(No change in Average Participation)
HA: m1 - m2 ≠ 0
(Shift in Average Participation over the population), where
m1 = Average Participation before the lockdown
m2 = Average Participation during the lockdown
Various factors that shall have a hypothesized effect on the sample, such as stratification of respondents with respect to the course of study, college, year of study, and personal preferences over the college attended. Furthermore, financial and educational infrastructure of individual institutions concerning the objective of the study were not implicit in analysis owing to constraints of data collection brought on by government regulations of strict quarantine and social distancing, as an offline study of the aforementioned would be more effective. Said constraints also lead to the limited reach of the survey among the students of the University, raising questions about the sample chosen being truly representative. Thus, we do not have any assumed or known population variance of the variable for the hypotheses above. A Student’s t distribution was used to formulate the test statistic, with respective sample means as unbiased estimators of the variable under study, i.e. Participation.
Regression analysis as a Multinomial Logistic Model is made using Online Effectiveness as the dependent variable. The ordinal nature of this variable is assumed to have equal placement of the response categories and treated as nominal. Independent variables considered are summarized below:
Curriculum – Efficiency of curriculum coverage during the lockdown
Regularity – Regularity of teaching during the lockdown
Participation – Student participation using online methods
Grievance Counsel - Effectiveness of administration grievance counselling
Future Culture - Expectations of a future culture of education, treated
F, as a dummy variable for Future Culture - 1, if “On-campus classes”, and 0 Otherwise
The model used is a generalization of logistic regression in multiclass problems, where the dependent variable is categorically distributed with more than two responses, as is applicable in this study. The model predicts the probabilities of these categories, initially using a linear predictor function comprised of regression coefficients (weights) and respective independent variables
score (Xi, k) = bk. Xi , where
Xi is the vector of explanatory variables describing observation i, and bk is a vector of weights associated with the variables, and the score is associated with assigning observation i to category k. For all i ranging from 1 to N, there exists a set of M explanatory variables X1,i ….XM,i and a dependent variable Yi with k possible responses, assigning each with a number from 1 to K.
To arrive at the multinomial logistic regression equations for K categories, K-1 equations are formulated with one response category of Yi as the pivot, or reference category, and the remaining K-1 outcomes are individually regressed against the pivot. Thus, we have the following relative probabilities:
The sample was split into individual training and testing data sets to polish the regression model. A confusion matrix was formulated to obtain parameter counts such as true positives and false negatives. Accuracy, Recall, and Precision of the model were evaluated using the confusion matrix.
Results
Descriptive (Annexure 1 for a complete set of responses)
1. Overall Effectiveness – Average response of effective learning across the two periods shifts from “Somewhat Effective” (sd = 0.85) to “Somewhat Ineffective” (sd = 0.86). Regression results described below expand on the same.
2. Participation – Modal responses from classroom to lockdown conditions changed from “Somewhat Attentive” to “Somewhat Inattentive”, indicating a decline in attention span. Testing the hypothesis of unchanged average participation revealed high significance with reasonably low probability (t = 8.712, p ~ 0). Participation has significantly reduced due to a shift from pre-lockdown to online learning methods.
3. Curriculum Coverage – A majority of respondents agree with the absolute ineffectiveness of curriculum covered during the lockdown, which also conforms to a low frequency in the regularity of teaching using online methods as opposed to classrooms ( t = 13.25, p ~ 0).
4. Teaching Methodologies – A majority of lessons conducted are through assignments and projects, followed by self-compiled notes by the instructor and live lectures on online video conferencing applications.
5. Preferences – 76 percent of respondents do not prefer online learning as a substitute for classrooms, and about 59 percent would like on-campus classes in the future.
6. Features of online learning – Students agree on the characteristics of convenience (42%) and self-paced learning (36%). Respondents find difficulties in online methods with respect to a lack of interaction (70%) and issues with network and connectivity (60%). Responses also conformed to various health concerns, such as strain on eyesight (74%), headaches (53%), and physical inactivity.
Regression (Annexure 2 and 3 for the complete model)
The model predicted log odds coefficients for four regression equations, with the category “Extremely Ineffective” of the dependent variable, i.e. Online Effectiveness as a pivot. The regression equation for the probability of choosing “Extremely Effective” relative to the probability of falling in the pivot category is as follows:
An increase in participation and regularity increases the relative odds of Extreme Effectiveness by as much as 2 (~2.254) times and 8 (~8.364) times respectively. Effective grievance counselling raises the same odds by 4 (~4.356) times. Falling in the pivot category (Extremely Ineffective) reduces by 25 percent with an improvement in curriculum covered, as indicated by Marginal Effects calculated using the average of effects at each observation. Grievance counselling reduces the same probability by about 11 percent. The confusion matrix to assess the accuracy of the model shows precision as high as 72 percent and recall as much as 64 percent.
Note: The diagonals represent accurate predictions.
0: Extremely Ineffective
4: Extremely Effective
Discussion and Conclusion
Primary data collection from teaching staff (n = 11) reveals unanimous agreement on the lack of interpersonal connection with students to keep up with the overall progress of lectures imparted. Concerns are raised with respect to the nature and fairness of assignments and internal evaluation conducted, often as a proxy for curriculum coverage. Flaws in the evaluation aforementioned indicates a lack of uniformity to assess skills and preparedness of the student. Teachers report a consistent pattern of using unfair means to submit assignments and classwork, which leads to aggressive free riding. Comments recorded reflect a lack of institutional support as a two-way feedback expected to be facilitated by the administration is absent. As one respondent noted, “teachers are left at sea to devise their own”.
Responses concerning terminal semester examinations of the senior (graduating) Class of 2020 largely reflected an opposition towards online methods and agreed on past performance/internal assessment evaluation, postponement of examinations, or using conventional offline methods in differed periods with groups of candidates as more viable alternatives. The results enumerated above do not bear much significance due to a small sample size and lack of representation, but the conclusions drawn closely match the response set of students with respect to the agreeability of online learning methods independently employed in a state of haste.
The strong dissatisfaction with online learning is multifaceted. Social, pedagogical, and technical issues for the instructor can be solved with a domain of administrative/back-end support, along with facilitating a feedback loop with the learner, who faces issues broadly categorized in the area of knowledge generation and absorption, applied aptitude, and collaboration. A fragmented approach to continue teaching by instructors of the University is independently organized at present, with virtually no platform to assess any measurable outcomes of the preparedness of the student with respect to terminal examinations. The current pandemic shall lead to long term sustained efforts that restrict students from going back to physical classroom settings. Hence, at this point, we might benefit from further research on the assessment of accessibility and robustness of a digital infrastructural network available to the undergraduate students of the University. We also recommend that gender disparities in enrollment rates among different institutions be assessed since there might be changes owing to economic difficulties associated with the lockdown and the pandemic.
The commencement of a portal to conduct Online Open Book Examinations of the graduating class has received a less than lukewarm reception, which also opens a discussion on logistical capacities to manage such a behemoth task of evaluation. However, it is of utmost importance to initiate the development of a common portal that digitizes up-to-date learning resources and reading material for the upcoming semester. To resolve the ambiguity of measuring the participation and knowledge gained by the student, apart from the adjustment of curriculum according to the viability constraints posed by online methods, instructors must also employ collaborative, group activity oriented, and application based pedagogies to engage the (virtual) class.
Annexure
Descriptive Statistics Results
Multinomial Logistic Regression Summary, Pivot Category: Extremely Inefficient (1)*
(Using Python, JupyterLab)
* X1: Curriculum
X2: Regularity
X3: Participation
X4: Grievance Counsel.
X5: Future Substitution
Marginal Effects Summary (Using Python, JupyterLab)
