|
|
ENQUIRE PROJECT DETAILS BY GENERAL PUBLIC |
| Project Details |
| Funding Scheme : | General Research Fund | ||||||||||||||||||||||||||||||||
| Project Number : | 18600019 | ||||||||||||||||||||||||||||||||
| Project Title(English) : | Unpacking student self-assessment processes: A longitudinal naturalistic experiment | ||||||||||||||||||||||||||||||||
| Project Title(Chinese) : | 探究學生自我評估過程：一項自然情境下的縱向實驗研究 | ||||||||||||||||||||||||||||||||
| Principal Investigator(English) : | Prof Yan, Zi | ||||||||||||||||||||||||||||||||
| Principal Investigator(Chinese) : | 晏子 | ||||||||||||||||||||||||||||||||
| Department : | Department of Curriculum and Instruction | ||||||||||||||||||||||||||||||||
| Institution : | The Education University of Hong Kong | ||||||||||||||||||||||||||||||||
| E-mail Address : | zyan@eduhk.hk | ||||||||||||||||||||||||||||||||
| Tel : | 2948 6367 | ||||||||||||||||||||||||||||||||
| Co - Investigator(s) : |
|
||||||||||||||||||||||||||||||||
| Panel : | Humanities, Social Sciences | ||||||||||||||||||||||||||||||||
| Subject Area : | Education | ||||||||||||||||||||||||||||||||
| Exercise Year : | 2019 / 20 | ||||||||||||||||||||||||||||||||
| Fund Approved : | 733,312 | ||||||||||||||||||||||||||||||||
| Project Status : | Completed | ||||||||||||||||||||||||||||||||
| Completion Date : | 31-12-2021 | ||||||||||||||||||||||||||||||||
| Project Objectives : |
|
||||||||||||||||||||||||||||||||
| Abstract as per original application (English/Chinese): |
學生自我評估是目前評估改革的核心內容。許多國家包括美國、英國、澳大利亞和新西蘭等的教育政策都提倡學生自我評估，強調評估的形成性目的和技能的發展，以滿足未來社會的需求。香港現行的評核政策亦期望教師讓學生參與課堂評估活動。學生應該依靠自我評估，而不是頻繁的考試，去了解自己的學習需要、學習重點和優勢。研究指出，自我評估能提高學生的評估判斷能力、自主學習能力和終生學習能力。然而，有什麽具體課堂活動能促進學生自我評估，我們知之甚少。此外，對於自我評估過程與認知/元認知（例如，自主學習，學業表現）和學習的情感方面（例如，學業情緒）的動態交互作用，研究也較少。因此，本研究的目標是通過實驗來探究兩種不同的促進學生自我評估方法的效能，以及揭示自我評估過程如何影響學生學習的機制。通過在自然情境中進行研究，可以較好地確定這些方法的可行性和實用性。本研究的結果將有助理解學生自我評估的過程如何作為一種學習的策略，以致在政策和實踐方面被廣泛採用的潛力，還有其對促進學習評估的重要性。 在先導研究結果(Yan, Chiu, & Ko, under review)的基礎上，本研究採用準實驗設計，將參與者進行配對(King, Nielsen, Coberley, Pope, & Wells, 2011)，并收集時間序列數據 (Schmitz & Perels, 2011; Schmitz & Wiese, 2006)。本研究包括兩個實驗。實驗一將考察兩種介入措施（即是：自我評估日誌和自我評估訓練）的效果，並探究自我評估過程與學生學習的動態交互作用。參加者將包括同年級三個班的中學生（N=90）以及同年級三個班的小學生（N=90）。三個班級將會被隨機分配到兩個介入組或控制組。在實驗二中，我們將會檢驗自我評估過程中不同活動各自的效能。實驗組將實施一個綜合介入措施（即是：自我評估日誌加上自我評估訓練）。同年級兩個班的中學生（N=60）以及同年級兩個班的小學生（N=60）將會被隨機分配到介入組或控制組。在這兩個實驗中，所選的班級將根據學業能力進行配對，並且由同一名教師授課。實驗將安排在每周有頻繁課次和作業的科目（例如，核心課程語文、英文或數學），以便最大限度收集數據。 本研究的一個特點是期望能盡量發揮對教育政策和實踐的影響。在自然學習情境中進行研究，確保了自我評估為學生提供真實的學習機會，並能被教師有效運用於真實情境中。這些情境有助於揭示自我評估如何促進學生的自主學習、帶來更佳學業情緒和促進更好學習結果的機制。研究亦會考察自我評估的準確性，以及自我評估與學業情緒的關係。了解這種關係將有助於向教師提供教學建議，以幫助學生進行有意義的自我評估。更重要的是，本研究有可能進一步發展自我評估理論，幫助學生改善形成性評估以指導他們的學習，並為教育政策的制訂和家長角色的定位提供參考。 |
||||||||||||||||||||||||||||||||
| Realisation of objectives: | All three objectives have been 100% achieved. Two experiments using intact classes have been conducted. Both experiments were conducted in one secondary school and one primary school. Experiment 1 addressed Objectives #1 and #2, while Experiment 2 addressed Objective #3. Experiment 1: This experiment has two sub-experiments: one conducted on Primary 4 (Experiment 1-a) students and one on Secondary 5 students (Experiment 1-b). In Experiment 1-a and Experiment 1-b, three classes of Primary 4/Secondary 5 students, with comparable academic ability and taught by the same teacher, were recruited, respectively. They were randomly assigned to one of the three conditions: self-assessment diary condition (Nprimary4 = 34; Nsecondary5 = 33), self-assessment training condition (Nprimary4 = 38; Nsecondary5 = 31), and control condition (N primary4 =31; Nsecondary5 = 28). The total sample size is 195, while the proposed sample size is 180. Participants’ academic performance, self-assessment accuracy, self-assessment practice, self-regulated learning, and academic emotions were assessed through pre-, post-, and delayed post-tests/surveys. Objective #1: The results showed that the two different self-assessment interventions (i.e., self-assessment diary and self-assessment training) significantly enhanced students’ academic performance, self-assessment practice, and self-regulated learning for both primary and secondary students. However, the effects of self-assessment interventions on self-assessment accuracy and academic emotions were not significant. Objective #2: The diary data permit a time-series of students’ self-assessment data to uncover relationships among different learning variables. We conducted linear trend analyses with diary (time) as the predictor and self-assessment practice and self-assessment accuracy as dependent variables. We also tested for significant differences in the self-regulated learning processes, in particular Plan to Improve, across diaries and students. The results showed a significant positive linear trend over the intervention period for self-assessment practice, but not for self-assessment accuracy. Furthermore, students who were younger or used their own standards to self-assess were less likely than others to plan for learning. A journal article based on the findings of Experiment 1 is currently under preparation. Experiment 2: This experiment also has two sub-experiments: one conducted on Primary 6 (Experiment 2-a) students and one on Secondary 2 students (Experiment 2-b). In Experiment 2-a and Experiment 2-b, two classes of Primary 6/Secondary 2 students, with comparable academic ability and taught by the same teacher, were recruited, respectively. They were randomly assigned to one of the two conditions: experimental condition (i.e., self-assessment diary + training; Nprimary6 = 25; Nsecondary2 = 32), and control condition (N primary6 =23; Nsecondary2 = 35). The total sample size is 115, slightly lower than the proposed sample size 120. Participants’ academic performance, self-assessment accuracy, self-assessment practice, self-regulated learning, and academic emotions were assessed through pre-, post-, and delayed post-tests/surveys. Objective #3: The results of the experiment-control comparison were similar to experiment 1. The combined self-assessment interventions (i.e., diary + training) significantly enhanced students’ academic performance, self-assessment practice, and self-regulated learning, but not for self-assessment accuracy and academic emotions. Interrupted time-series analyses were applied to test the effect of each training session separately. The results showed that the “feedback-seeking” training session significantly increased students’ feedback-seeking behaviour; the “determining assessment criteria” and “self-reflection” training sessions did not bring significant changes. A journal article based on the findings of Experiment 2 is currently under preparation. Other outcomes In addition to the proposed experiments, we have extended our research agenda around the project’s overall theme. For example, we have conducted a comprehensive review of the effects of self-assessment and peer-assessment interventions. Two meta-analysis articles have been published, including “The effect of self-assessment on academic performance and the role of the explicitness: A meta-analysis” in Assessment & Evaluation in Higher Education (Yan, Wang, Boud, & Lao, 2021) and “Effects of self-assessment and peer-assessment interventions on academic performance: A meta-analysis” in Educational Research Review (Yan, Lao, Panadero, Fernández-Castilla, Yang & Yang, 2022). We also developed short self-assessment practice scale and published a journal article “Developing a short form of the Self-assessment Practices Scale: Psychometric Evidence” in Frontiers in Education (Yan, 2020). This instrument has greatly facilitated the implementation of the experimental studies. As the proposed study extensively investigated the relationship between self-assessment and feedback seeking, we have published a conceptual paper “Self-assessment is about more than self: The enabling role of feedback literacy” in Assessment & Evaluation in Higher Education (Yan & Carless, 2022). Furthermore, we extended this line of research by exploring self-assessment interventions other than diary. We have conducted an additional small experiment examing the effect of self-assessment mindmaps on student learning and published an article “Enhancing students’ self-efficacy in creativity and learning performance in the context of English learning: The use of self-assessment mind-maps” in Frontiers in Psychology (Yan, Lee, Hui, & Lao, 2022). | ||||||||||||||||||||||||||||||||
| Summary of objectives addressed: |
|
||||||||||||||||||||||||||||||||
| Research Outcome | |||||||||||||||||||||||||||||||||
| Major findings and research outcome: | The major findings generated from this project have the potential to inform theoretical development, classroom practices, and policy-making related to self-assessment. • Key finding 1 (effects of self-assessment interventions): Yan et al.’s (2022) meta-analysis of 227 effect sizes from 69 independent studies indicated that, in general, self-assessment intervention (g = 0.585) had meaningful effects on academic performance. However, self-assessment process is complex, and the positive impact comes only with appropriate design and implementation. Yan et al. (2021) highlighted the importance of making self-assessment explicit because self-assessment interventions involving explicit feedback from others on students’ performance had a significantly larger effect size (g = .664) than those without explicit feedback (g = .213). The two experiments further verified the positive impact of self-assessment interventions on students’ academic performance, self-assessment practice, and self-regulated learning. Significance: Despite the conceptual consensus on the benefits of self-assessment, empirical evidence is not conclusive. The published two meta-analyses and the two experimental studies (under preparation) offered a systematic account of the effect of self-assessment on student learning. More importantly, the design and successful implementation of an effective, low-cost intervention (i.e., self-assessment diary) shed light on how to create sustainable changes in classrooms. Nevertheless, the non-significant effect on self-assessment accuracy and academic emotions reminded us that these interventions could be further improved. • Key finding 2 (the dynamic relationship among students’ demographics, self-assessment practice, and self-regulated learning): The results showed students’ self-assessment practice could be enhanced through structured, explicit activities. But self-assessment accuracy appeared more difficult to be improved. Given that students’ academic performances have increased, this finding echoes the argument, including the PI’s studies, that engagement in self-assessment itself, regardless of its accuracy, can improve learning performance. We also found that students who were younger or used their own standards to self-assess were less likely than others to plan for learning. Significance: Most studies do not track student use of self-assessment throughout a course of learning, so the dynamic nature of students’ evolving self-assessment capabilities in light of task performance is unknown. This study explored the dynamic relationship between the self-assessment process and various learning variables including academic emotions, self-regulated learning, and academic performance. Our findings on the predictors of plan to improve, which is a crucial link of different self-regulation circles, show the importance of a complex model of demographics, assessment criteria, and self-knowledge for informing interventions to promote sustainable self-regulation. • Key finding 3 (the roles of different actions in self-assessment processes): Yan and Carless’s (2022) conceptual paper highlighted the important role of feedback-seeking in the self-assessment process. Experiment 2 further proved that feedback-seeking training had the most significant impact. Significance: Previous studies treat self-assessment, mostly as a whole, as the independent variable, without disentangling its component processes. Less is known about how different actions in the self-assessment process interact with learning processes and outcomes. Our findings filled this research gap and informed the design of self-assessment activities. | ||||||||||||||||||||||||||||||||
| Potential for further development of the research and the proposed course of action: |
This project has contributed to an in-depth understanding of the process and effect of self-assessment practices students engage in during learning. The findings of this study shed light on the mechanism by which the self-assessment process influences students’ learning outcomes and have the potential to inform the design of effective and sustainable self-assessment interventions in classrooms. There are three directions to develop this line of research further. First, the findings of this project justified the inclusion of self-assessment in schooling. Future studies can explore more diversified self-assessment strategies, instruments, and tools, taking into account student characteristics and contextual factors with an aim to maximise the positive impact of self-assessment on student learning. An initial attempt has been made by the project team. We have explored the effect of a new self-assessment intervention, i.e., mindmaps, on student learning and published a journal article in 2022. Second, most studies focus on the pedagogical aspects of the self-assessment process, leaving psychological antecedents and consequences of self-assessment processes largely understudied in literature. The non-significant impact of self-assessment on academic emotions revealed in this project should not discourage attention to this line of research. In contrast, more research is necessary for this direction to unpack the psychological mechanism of self-assessment. Third, as both teachers and students play important roles in the self-assessment process, it is crucial to develop the formative assessment literacy of teachers and students in order to optimise the impact of self-assessment activities. Thus, it will be meaningful to design teacher professional development programmes based on the findings of this project and evaluate the effectiveness of the developed programmes. | ||||||||||||||||||||||||||||||||
| Layman's Summary of Completion Report: | Student self-assessment has become a central agenda of assessment reform in the Hong Kong education system because it promises to enhance learner agency and self-regulated learning. This project has addressed three important issues: 1) investigating the effect of two self-assessment interventions on students’ learning outcomes; 2) exploring the dynamic relationship between the self-assessment process and various learning variables; and 3) disentangling the self-assessment process and examining the distinct effect of different actions in the processes. This project has contributed to an in-depth understanding of the process and effect of self-assessment practices students engage in during learning. The positive effect of self-assessment demonstrated in this study justified its inclusion in schooling at the classroom practice and policy-making levels. The findings of this study shed light on the mechanism by which the self-assessment process influenced students’ learning outcomes and laid a foundation for future research in this field. By informing the design of effective and sustainable self-assessment interventions in classrooms, this study can contribute to the long-term goal of worldwide assessment reform focusing on the formative purpose. | ||||||||||||||||||||||||||||||||
| Research Output | |||||||||||||||||||||||||||||||||
| Peer-reviewed journal publication(s) arising directly from this research project : (* denotes the corresponding author) |
|
||||||||||||||||||||||||||||||||
| Recognized international conference(s) in which paper(s) related to this research project was/were delivered : |
|
||||||||||||||||||||||||||||||||
| Other impact (e.g. award of patents or prizes, collaboration with other research institutions, technology transfer, etc.): |
|||||||||||||||||||||||||||||||||
| SCREEN ID: SCRRM00542 |