Online Reflective Practice group: Update

As the two core PG Cert units draw to a close, it’s an ideal time to look back at the online reflection that colleagues have participated in as part of the course. I’ll report on the outcomes first and then the process:

Outcomes

All those completing the first unit engaged to a satisfactory extent with the online reflective tasks, with the vast majority engaging at a high level. The tasks, with links to examples of posts and comments, are detailed here. Participation in the reflective tasks was measured through self and peer assessment and the spread of grades awarded was interesting; ‘B’ was by far the most commonly awarded grade. There were also several ‘A’s and a few ‘D’s. Hardly any ‘C’s were awarded. This supports my observation that once people began to engage in the tasks, they saw the benefits fairly swiftly and their engagement continued and deepened. The few who obtained ‘D’ grades generally did not engage until it was too late (I’ll post up a couple of graphic representations once I’ve got feedback out to the participants).

The quality of the posts exceeded my expectations. The quality of the comments was also very good, but in many cases these conversations did not take place when they were supposed to. While some groups maintained conversations throughout around the monthly topics, in many cases interactions were sparse until the few days preceding the terminal assessment, when they rose significantly. Some of these comments referred to posts made three or four months previously.

The self and peer assessment of participation demonstrated colleagues’ capacity for honest and constructive assessment of their own and their peers’ engagement in the tasks. It also revealed their understanding of the benefits of online reflection and the attitudes and approaches required to maximise these. The following is an example of anonymous peer feedback exchanged:

I believe that [x] is committed to the subject and is highly reflective, however there were some planning issues which meant I got to read some of the posts quite late. Some of the posts I do feel would have benefited from being a little more concise which would have helped with the articulation of some points. [x] shows a high level of self-direction in his/her blog posts and demonstrates a desire to improve his/her teaching practice throughout.

In terms of the impact online reflective practice has had on learning, I am certain of three things. First, the standard of the terminal assignments submitted for the first unit (i.e. after six months on the course and five day-long workshops), was significantly higher than those submitted by previous cohorts after 12 months on the course and 20 half-day workshops. Essentially, this cohort have achieved more in half the time. Second, in conversation with participants halfway through the unit, many told me that, were it not for the online tasks, they probably would not have read or written anything yet. And third, a small but significant proportion of those who have completed both units are continuing to use their blogs.

Feedback gathered from participants about the online reflective tasks revealed that the vast majority felt they should be retained as a fundamental part of the course. Some participants even felt that we should increase the weighting the tasks have on the unit grade (currently 10%), as such low stakes are disproportionate to the amount of work they put in to completing the tasks. A minority (albeit a significant one) stated a preference for using a less public forum for the tasks, or at least a system with simpler privacy settings.

Process

Although engagement in online reflective practice was high among those who completed the unit, and appeared to enhance learning, participant drop-out/deferral rates were also significant. Minor factors contributing to this include improved flexibility of the programme, which makes it easier for participants to defer one or two units until the following year, and higher than normal redundancies. One major factor was the overlapping scheduling of the first two units, which increased the workload and the complexity of the programme; this has been resolved for 2012/13. Unfortunately it is very difficult to untangle this factor from the increased complexity and greater challenge inherent in the new curriculum. The other major factor was the continuous nature of the online reflective tasks. For the first time, participants were being required to produce something from the outset that would influence their grade for the course. This meant that those who were not engaging as they should were made aware of this – often through informal peer feedback – and forced to be pro-active in arranging deferral rather than waiting until the end of the course to submit a borderline assignment (or not).

Lessons learned about literacies for online reflective practice

Introducing online reflection as a compulsory element of a course has led to the development of the following elements of participants’ digital literacy:

Understanding the benefits and challenges of online reflective practice – as evidenced in terminal reflective assignments and ongoing exchange of feedback with participants.
Understanding the attitudes and approaches required to maximise the benefits of online reflective practice – as evidenced in the self and peer assessments.
Willingness to try new tools and processes – as evidenced on the blogs. Seeing group members posting images and videos often prompted participants to try this for themselves and/or ask each other for advice and support.
Willingness to find the answers oneself – whereas at the start of the course I would receive requests for assistance before a participant had even tried to find any information, by the end the requests I received were much more specific; participants had tried a number of avenues and had come to me as a last resort.
Greater tolerance of complexity – several participants who felt in the early stages that the course requirements were overwhelmingly complex seemed much more comfortable about the requirements in the latter stages. Four months in, we found we had to ask everyone to start logging in with their new student IDs to access their feedback, whereas we had assured them previously that they would be able to use their existing staff IDs. The response to this was nowhere near as negative as we expected – people seemed to just get on with it.
Tool-specific knowledge – e.g. how to add and edit blog posts and comments, how to create blog groups and subscribe to new posts, how to upload media and create hyperlinks.

Next steps

The next phase will be to look more deeply at the data from the tasks, the assessments, and from forthcoming surveys and interviews, to get a better understanding of the role of compulsory, course-based online reflective activities in developing particular aspects of participants’ digital literacy. It will be important to contact colleagues who deferred their studies to gain a better understanding of the factors that influenced their decision. I’m not anticipating that this information will be of much use (previous studies have shown that withdrawing students often cite external pressures and are unlikely to connect their attrition with course design, student support, etc..), but it’s worth a shot. One strategy may be to ask them to speculate what they would have found most challenging had the external pressures (work, family etc.) not been as significant. My major concern is that the online reflective activities were actually a little too challenging, and that this contributed to participant withdrawal and delayed engagement. If this is the case, I need to identify how best to address this; through greater applicant awareness, longer participant induction, different support structures, etc..

I am also interested in looking at the other end of the spectrum; those participants who are now continuing to engage in online reflective practice through their blogs now the units have ended. What are their motivations for doing this and how did these come about? Was it a particular aspect of the course that helped them over a particular threshold, and is this something that can be developed, or given more emphasis, for future cohorts? Was it a particular experience they had, and can this be replicated for others?

 

This entry was posted in Activity & progress, DIAL pilot Groups, DIAL updates, Online Reflective Practice (group). Bookmark the permalink.

4 Responses to Online Reflective Practice group: Update

  1. Thanks Lindsay fascinating stuff, lots of great evidence for open reflective practice supporting progressive and sustainable practice (lots to unpack and follow up on!) – could the google doc of tasks be CC licensed? – and added as an attachment / link to the post – http://process.arts.ac.uk/content/cltad-teaching-development-projects

    Is there a compiled list of all the blog post and the authors, could they start to build their profiles on their process.arts posts and link the two like here – http://process.arts.ac.uk/user/35/contents (students could introduce highlight their blogs by adding a process.arts post about their blog, if they all tagged they could cluster) Also good to see some are using their blogs still, be good to see what are the factors for students to continue with their blogs after the PGCert ?
    Did students talk about how these tasks impacted on their normal schedules, did they find it pervasive presence in their personal/social space, is that a problem long term, were the implications of this discussed?
    Looking forward to hearing more about the next steeps and ‘motivations’ John and I could come do a session on open practice and try and address some of the concerns and ways forward for those with the bug.

    @lindsayjordan @DIALProject “You want some stuff about the vids too?” – Yes please (a video ☺)’ ….it would be good to have some info about the video project could the work/research Laura’s group is doing crossover in some way and again John and I would like to do a focus group/ feedback session?

  2. Thanks for these super suggestions Chris – I have added a CC license to the G-doc (at least I think I have – I just inserted a 3.0 Attribution logo). I will post this to Process Arts but will actually post up a different one within the TDP group; there was a second det of activites that supported the Teaching Development project unit.

    The best way of anyone seeing a list of all the PG Cert bloggers is through the PG Cert group activity page – this isn’t an exhaustive list as I only ‘strongly encouraged’ participants to add themselves to the group: http://myblog.arts.ac.uk/groups/pg-cert-lt-1011/

    I can definitely encourage colleagues to expand their presence on Process Arts by linking to their blogs etc; my jurisdiction over them is fading rapidly now they’ve completed both the units I lead, but it’s something we can build into the new unit in September. Thanks so much for the pointers re: questions to ask – this is exactly the kind of assistance I need 😉

    …and I will try to get something together about the videos. It’s hard because every time I look at them I start sobbing with relief and to be honest it’s getting embarrassing… 😉

  3. Oh – and I’m definitely planning to get Laura or Chitra involved with a video skills workshop as part of the Teaching Development Project unit… I was thinking, we get so many cinimatography & film production types on the courses, I should ask *them* to design & run the workshop for the group…

    Last time I ran an activity where everyone folded up a sheet of paper and drew a ‘record’ button on it, then held it up in front of their partner and made them talk at it. Almost as effective as real Flipcams and SO much cheaper…

Leave a Reply

Your email address will not be published. Required fields are marked *


*