As the two core PG Cert units draw to a close, it’s an ideal time to look back at the online reflection that colleagues have participated in as part of the course. I’ll report on the outcomes first and then the process:
All those completing the first unit engaged to a satisfactory extent with the online reflective tasks, with the vast majority engaging at a high level. The tasks, with links to examples of posts and comments, are detailed here. Participation in the reflective tasks was measured through self and peer assessment and the spread of grades awarded was interesting; ‘B’ was by far the most commonly awarded grade. There were also several ‘A’s and a few ‘D’s. Hardly any ‘C’s were awarded. This supports my observation that once people began to engage in the tasks, they saw the benefits fairly swiftly and their engagement continued and deepened. The few who obtained ‘D’ grades generally did not engage until it was too late (I’ll post up a couple of graphic representations once I’ve got feedback out to the participants).
The quality of the posts exceeded my expectations. The quality of the comments was also very good, but in many cases these conversations did not take place when they were supposed to. While some groups maintained conversations throughout around the monthly topics, in many cases interactions were sparse until the few days preceding the terminal assessment, when they rose significantly. Some of these comments referred to posts made three or four months previously.
The self and peer assessment of participation demonstrated colleagues’ capacity for honest and constructive assessment of their own and their peers’ engagement in the tasks. It also revealed their understanding of the benefits of online reflection and the attitudes and approaches required to maximise these. The following is an example of anonymous peer feedback exchanged:
I believe that [x] is committed to the subject and is highly reflective, however there were some planning issues which meant I got to read some of the posts quite late. Some of the posts I do feel would have benefited from being a little more concise which would have helped with the articulation of some points. [x] shows a high level of self-direction in his/her blog posts and demonstrates a desire to improve his/her teaching practice throughout.
In terms of the impact online reflective practice has had on learning, I am certain of three things. First, the standard of the terminal assignments submitted for the first unit (i.e. after six months on the course and five day-long workshops), was significantly higher than those submitted by previous cohorts after 12 months on the course and 20 half-day workshops. Essentially, this cohort have achieved more in half the time. Second, in conversation with participants halfway through the unit, many told me that, were it not for the online tasks, they probably would not have read or written anything yet. And third, a small but significant proportion of those who have completed both units are continuing to use their blogs.
Feedback gathered from participants about the online reflective tasks revealed that the vast majority felt they should be retained as a fundamental part of the course. Some participants even felt that we should increase the weighting the tasks have on the unit grade (currently 10%), as such low stakes are disproportionate to the amount of work they put in to completing the tasks. A minority (albeit a significant one) stated a preference for using a less public forum for the tasks, or at least a system with simpler privacy settings.
Although engagement in online reflective practice was high among those who completed the unit, and appeared to enhance learning, participant drop-out/deferral rates were also significant. Minor factors contributing to this include improved flexibility of the programme, which makes it easier for participants to defer one or two units until the following year, and higher than normal redundancies. One major factor was the overlapping scheduling of the first two units, which increased the workload and the complexity of the programme; this has been resolved for 2012/13. Unfortunately it is very difficult to untangle this factor from the increased complexity and greater challenge inherent in the new curriculum. The other major factor was the continuous nature of the online reflective tasks. For the first time, participants were being required to produce something from the outset that would influence their grade for the course. This meant that those who were not engaging as they should were made aware of this – often through informal peer feedback – and forced to be pro-active in arranging deferral rather than waiting until the end of the course to submit a borderline assignment (or not).
Lessons learned about literacies for online reflective practice
Introducing online reflection as a compulsory element of a course has led to the development of the following elements of participants’ digital literacy:
Understanding the benefits and challenges of online reflective practice – as evidenced in terminal reflective assignments and ongoing exchange of feedback with participants.
Understanding the attitudes and approaches required to maximise the benefits of online reflective practice – as evidenced in the self and peer assessments.
Willingness to try new tools and processes – as evidenced on the blogs. Seeing group members posting images and videos often prompted participants to try this for themselves and/or ask each other for advice and support.
Willingness to find the answers oneself – whereas at the start of the course I would receive requests for assistance before a participant had even tried to find any information, by the end the requests I received were much more specific; participants had tried a number of avenues and had come to me as a last resort.
Greater tolerance of complexity – several participants who felt in the early stages that the course requirements were overwhelmingly complex seemed much more comfortable about the requirements in the latter stages. Four months in, we found we had to ask everyone to start logging in with their new student IDs to access their feedback, whereas we had assured them previously that they would be able to use their existing staff IDs. The response to this was nowhere near as negative as we expected – people seemed to just get on with it.
Tool-specific knowledge – e.g. how to add and edit blog posts and comments, how to create blog groups and subscribe to new posts, how to upload media and create hyperlinks.
The next phase will be to look more deeply at the data from the tasks, the assessments, and from forthcoming surveys and interviews, to get a better understanding of the role of compulsory, course-based online reflective activities in developing particular aspects of participants’ digital literacy. It will be important to contact colleagues who deferred their studies to gain a better understanding of the factors that influenced their decision. I’m not anticipating that this information will be of much use (previous studies have shown that withdrawing students often cite external pressures and are unlikely to connect their attrition with course design, student support, etc..), but it’s worth a shot. One strategy may be to ask them to speculate what they would have found most challenging had the external pressures (work, family etc.) not been as significant. My major concern is that the online reflective activities were actually a little too challenging, and that this contributed to participant withdrawal and delayed engagement. If this is the case, I need to identify how best to address this; through greater applicant awareness, longer participant induction, different support structures, etc..
I am also interested in looking at the other end of the spectrum; those participants who are now continuing to engage in online reflective practice through their blogs now the units have ended. What are their motivations for doing this and how did these come about? Was it a particular aspect of the course that helped them over a particular threshold, and is this something that can be developed, or given more emphasis, for future cohorts? Was it a particular experience they had, and can this be replicated for others?