5.3.3: The prime directives for an ePortfolio

Link: Tolley, R.J. (2008). The prime directives for an ePortfolio.

---

The past few posts on this blog have benefitted from the comments of Ray Tolley, whose work I link to here. Ray's ten 'prime directives' resonate strongly with me - and, oddly(?) enough, they are reflected in the design brief we gave for the development of Mahara. As we consider how ePortfolios might thrive outside of the classroom, criteria such as Ray's will no doubt be discussed more widely.

5.3.3: ePortfolios and what we need to know

Article: Yancey, K.B. (2009). Electronic portfolios a decade into the twenty-first century: What we know, what we need to know. Peer Review 11(1), 28-32.

---

Yancey's (2009) brief article is, in some ways, disappointing - it does not draw deeply from the vast literature on ePortfolios, instead opting to treat the book Electronic Portfolios 2.0 as its major source. Now, I have only read the first book the author was involved with, called Electronic portfolios, so I cannot comment on the worth of the 2.0 version. But this article does not fully deliver what it's title promises, and it is disappointing not to see some of the very valuable UK-based research directly cited (this despite the author's own admission that "our knowledge-base is both wide and culturally complex", p.32). Anyway, there is still ample value in the article!

Among Yancey's (2009) what we know's:

• Student engagement is a critical aspect of ePortoflio development
• Student engagement is, in part, a result of students having control over their ePortfolios and not having to work too much in a prescripted environment
• Students using ePortfolios tend to perform better than those who do not (though the possibility that this is because only more successful students would tend to use them is not acknowledged)
• The choice of ePortfolio platform is important; "the ways the technology is programmatically formative" (p.29) should be considered.
  
• Reflection across courses - 'meta-reflection' is a good (my) term for it - is proving highly valuable. Typically assignments require students to reflect on what they have learned within each course; ePortfolios broaden the scope of reflection, enabling "a shift from discrete courses to a larger frame of reference" (2009, p.30).
  
• A Skills Matrix (like a table that students populate with artefacts to demonstrate their attainment of various skills) can help to guide meta-reflection.
• Reflection is a valuable exercise encouraged by the effective use of ePortfolios; the process is more important than the outcome. Yancey (p.32) notes that "Many colleges and universities... found that helping students develop a 'capability to reflect' is a critical educational outcome, in and of itself".
  

What we need to know:

• The influence on ePortfolio tools to student outcome.
  
• The nature of reflection in the development of ePortfolios.
  

Not a bad list - but far from comprehensive. I do agree wholeheartedly though that the 'What we need to know' areas are very rich ones for further research, particularly the latter one on reflection. The first, relating to ePortolio tools, is also important - but I think the current case study research speaks for itself on the matter of tool choice (see previous comments relating to Swan's article).

5.3.3: Key findings from Swan (2009)

Article: Swan, G. (2009). Examining barriers in faculty adoption of an e-portfolio system (full text). Australasian Journal of Educational Technology 25(5), 627-644.

---

Swan (2009) reports on an 'exploratory implementation' of ePortfolios in a teacher education programme. ePortfolios were used in particular for student placement reflections and evaluations.

Swan performed in-depth interviews with faculty and university supervisors for some insight into their experiences (n=15) as evaluators of ePortfolios. Interviews took between 45 and 75 minutes.

Key findings:

• Faculty tended to rely on their existing means of evaluating students. Student ePortfolios provided additional insight into student experiences. Faculty were used to reviewing the reports of cooperating teachers and were content with the feedback on those reports, so seldom made use of the additional information available in student ePortfolios. They also felt tha they knew their students well enough to not need to consult the eportfolios. Those supervising larger numbers of students werre more likely to make use of the student reports in their ePortfolios.
• Supervisors liked the data storage capability of the ePortfolio. The fact that all information was availle through the same paperless interface was perceived as a plus.
  
• Providing feedback to students through the ePortfolio system was perceived as 'redundant'. Supervisors met with their students on a regular basis anyway, so they did not se the point of repeating themselves when giving written feedback on the ePortfolio.
  
• Accreditation (final marking) can be messy. Respondents indicated that they found the final marking (or 'accreditation') of the ePortfolio difficult, because information was so spread and marking is often left to the last moment. I suspect that the problem in this case was to do with the ePortfolio platform itself (Open Portfolio) and the requirements of the accrediting body.
  

Swan (2009) concludes with the observation that, in his case, "the culture of assessment in this research setting is far from ideal for meeting the loftier goals of an e-portfolio system" - a vital lesson from the 'coal face'. Conversations, it seems, be centred on assessment systems, practice and aspirations rather than 'which tool should we use' or 'which course should we try it on'. Swan (2009) talks in terms of gaining faculty 'consensus' (a common theme in ePortfolio literature), and 'centrifugal force' surrounding implementation. As Swan hinted at earlier (see previous post), implementation involves challenge:

It may be that project leadership must not shy away from conflict and possibly be courageous enough to actually initiate and see it through to resolution.

This certainly resonates with experience! There are a number of implicit lessons from Swan's work:

• Take systematic change seriously. It is not enough to add an ePortfolio into a workflow; it must be embedded within systems. This, of course, requires change management, intensive conversation and negotiation, listening, changing...
  
• Choose the tool carefully. I have not examined the Open Portfolio tool, but from the article screenshots it looks highly customised. This can be problematic, as highly customised tools lack flexibility and can serve to reinforce older systems rather than provide opportunity for new ones.
  
• Understanding faculty workflows is vital. After all, it is faculty who will ultimately determine the effectiveness of implementation. If all an ePortfolio does is add an optional extra to their responsibilities, it is no surprise that they may not use it at all. On the other hand, an effective and smooth-flowing system that saves time and improves their ability to interact with students' ideas is likely to achieve its own success.
  
• Get students to create their own final submissions. Swan (2009) found that student information tended to be difficult to compile when needed for accreditation; why not get stduents to do this themselves, according to clear guidelines? The problem Swan encountered may be to do with the constraints of a highly customised system however transferring aggregating work to the stduent seems a very clear and valuable exercise - both for faculty, and for the students themselves. It seems Swan is working on further customising the ePortfolio platform to permit better summaries of data... (see second bullet above).
  

A very useful article, with some very important lessons for implementing ePortfolios at a programme level. In the days ahead I hope to draw additional ePortoflio-related lessons from literature together.

5.3.3: ePortfolios: More research needed at the coal face

Article: Swan, G. (2009). Examining barriers in faculty adoption of an e-portfolio system (full text). Australasian Journal of Educational Technology 25(5), 627-644.

---

I'm beginning a focus on e-portfolios, because I have a keynote to present in late April and I need to get back in touch with the literature!

Swan (2009) reports on the implementation of an e-portfolio system in a teacher preparation programme in Kentucky. Swan (2009, p.629) begins with the need for more work in the area of "the schisms created by the intersection of faculty practice and system design"... in other words, more work is needed on how ePortfolios might be effectively integrated within the systems and structures of academic programmes, and the faculty responsible for them. Institutional studies, Swan suggests, are numerous... but there are not many "at the user-utility, program assessment level" (ibid.) How do we effectively operationalise ePortfolios within programmes of study?

Swan answers the question of effective operationalisation with reference to:

1. the emergent perspective, that is, from the understanding that the interface between people and technologies is symbiotic and unpredictable; and
2. interaction resistance theory, which holds that the differences between "designer intention and user perception cause implementation problems" (2009, p.629).

So, the greater the change, the more implicit resistance to that change and the more potential there is for problems. Swan (2009, p.670) rightly observes that "Adaptation is necessary by both the developer of the system and the members of the organisation to facilitate adoption... a high level of conflict management is crucial to achieve success" (ibid.)

Swan provides an excellent platform for further studies in ePortfolio implementation/operationalisation at the user-level. His introduction of the emergent perspective and interaction resistance theory ring true to my own work in institutional change and development in e-learning where leadership, conversation and improvement (I was going to write 'compromise' as the result of conversation, but in fact it is ultimately improvement) form a tri-unity of absolute importance.

More on the findings of Swan's study tomorrow...

5.1.1: The realities of Web 2.0

Source: Boyd, D. (2009). "Streams of Content, Limited Attention: The Flow of Information through Social Media" Web2.0 Expo. New York, NY: November 17.

---

A good, honest and interesting look at the phenomenon of Web 2.0. The "Four Core Issues" are very thought-provoking in terms of calls for Education 2.0!

This quote seems a useful (part) mission statement for higher education:

...the tools that consumers need are those that allow them to get into flow, that allow them to live inside information structures wherever they are, whatever they're doing. The tools that allow them to easily grab what they need and stay peripherally aware without feeling overwhelmed.

Now, I'm not sure of the 'grab what they need' part, but I like the 'getting them into flow' (citing Csikszentmihalyi) - and the 'staying peripherally aware without feeling overwhelmed' neatly describes my own appreciation of e-learning (even having completed the e-primers). Reading between the lines (and even on them), this article provokes serious challenge to those suggesting an egalitarian, open and networked Education 2.0.

5.3.2: Student behaviour in wikis

Article: Meishar-Tal, H., and Gorsky, P. (2010). Wikis: what students do and do not do when writing collaboratively. Open Learning 25(1), 25-35.

---

The article describes the editing behaviour of n=60 graduate students required to contribute to a wiki containing course concepts. The article demonstrates good practice in wiki design:

• A compulsory task
• A clear objective (building a glossary of key terms)
  
• Beginning with an existing set of definitions
• Recognition of the complexity of the task
• Specific instructions.
  

The article cites previous findings indicating that students have a reluctance to edit one another's work or even to criticise; one study reported that a completed collaborative wiki resembled a threaded discussion "that lacked integration and unity" more than a polished document.

The authors constructed a taxonomy of actions that would be useful for further studies into categorising wiki use by students. All 60 students edited the wiki, with 2986 editorial changes made. The most common activity was adding sentences: "Additions occurred three times more than deletions and 4.3 times more than moving entire sentences" (p.31). However, findings indicated an uneven level of activity:

• About two-thirds of all sentence deletions were carried out by two students.
  
• 33% of all grammatical changes were made by one student.
  
• One student carried out 478 of the total editorial actions (16% of the total).
  
• About 10% of students were "extremely dominant" (p.32) in the activity.
  

These dominant students are further described:

The dominant students were ‘specialists’ who created near ‘monopolies’ on certain kinds of editorial actions: one ‘mover’, two ‘deleters’, one ‘stylist’ as well as one student who served as ‘formatter’ and ‘linker’. These roles and behaviours were assumed without any direct instruction, apparently quite spontaneously.

The authors suggest that editing behaviour might be correlated with student traits... far more interesting to me is how they graded the exercise, and whether the 'dominance' of some students served to alienate their peers and shape the overall outcome so that it was, perhaps, less representative of the class. It is interesting to note that these findings relate to graduate students... I wonder how editorial behaviour might change among undergraduate students?

A very interesting study... it's great to see some primary analysis. However the findings are, for me, a bit disturbing. Participation was far from equal... was the outcome, then, far from optimal? Might a better level of student activity resulted had the assignment required students to prepare (say) a set of short-answer definitions individually, marked by the lecturer and returned with individualised feedback?

5.1.3, 5.3.1: The non-bloggin' generation

A recent Pew Internet Project report, "Social Media and Young Adults" (PDF), examines how those in the teens and early adulthood years (18 to 29) interact with technology. One of the interesting finds: blogs are not a high-use, highly interactive medium for young people. Bloggers in the age group went from 24% in 2007 to just 15% in 2009. Among adults, well, about 11% of users over 30 maintain a blog (the "Are blogs for old people?" news item cites the Pew report). The attention of young people is shifting to social networking. Twitter, the report found, has a 37% use among online users aged 18-24, 25% use among those 25-29, and 22% use among those aged 30-49; 19% of all online adults use Twitter. Virtual worlds (it is not clear if this statistic includes virtual worlds such as World of Warcraft) are used by about 4% of online users.

The report contains lots of useful information relating to internet uptake and connectivity, gadget ownership (cell phones, computers, mp3 players, gaming devices) and overall trends in internet use. I found this take on ReadWriteWeb thought-provoking:

But blogging? Passé, says the report. The medium once used for sharing either news and/or personal thoughts and feelings is no longer popular among teens. The why is simple: Facebook. With the ability to update your status on social networking sites, the need to communicate using long-form mechanisms like sentences and - ugh! - paragraphs is no longer necessary. Instead of summarizing a day's events via blog post for example, a teen may simply update Facebook multiple times throughout the day with the details as to what's happening in their life at the moment...

Longer forms of communication are being sacrificed for smaller, more convenient, bite-sized pieces. What will this mean for a generation already struggling with general literacy? I think the "young people are using it so we should" argument starts to rapidly fall down here... the genre of use for social networking is immediate, brief, and more descriptive than reflective.