For quite some time now we have been talking about the idea of integrating some form of peer assessment and feedback into Webmaker, as a way to level up an individual user's craft and community. Today I want to share some ideas that Chloe Varelidi and I have been tinkering with and starting to actually prototype with Atul Varma. I am going to do a little walkthrough of a potential user experience. Keep in mind that some of this is probably wrong, but I want to put it out there so that we can work on polishing those bits.
Imagine that you came to Webmaker and you were looking for a way to learn or gain specific skills. You click on the "Skills" tab (I'm sure there are tons of other names that could be more appropriate for this navigation item). On the skills page you see all of the possible badges that you can apply for - and these badges in the future will not just be individual small badges, but represent larger learning pathways.
If a user clicks on a badge - they will learn about the badge through a playful interactive displayer and have the ability to immediately apply to earn that badge.
When the user clicks "Apply" they are given the apply interface. This reminds the user what the criteria is for the badge, gives them the ability to upload evidence and write reflective thoughts on their process. There are several things that we could do here, one idea was to have the user create a "how to" showing how they made their project, which eventually could be integrated into some kind of maker gallery. What's great about this approach to "badge pledge" is that it is a useful interaction in its own regard, and not existing solely for assessment/recognition purposes - that way there's both extrinsic and intrinsic incentives involved.
The user then submits their badge application. There are several directions that we could go from here. One option is to have all of the entries go into a queue that is accessible for mentor reviews from a secure interface that they will be able to access by logging into the queue through a mentor username and password to the site.
Another option might be to completely make this public and leverage a service like twitter. I imagine that the interaction here (after a user signs in and then clicks submit) could be similar the Army of Awesome (see below) where a the applications compile in a tweet queue and then peers pick up the individual applications to review. If we went in this direction, some more sophisticated flow will need to be developed so that there is a way to in essence vet the quality of peer review.
The actual application review for a mentor might be something like this:
Here, the reviewer can see the badge applicant's evidence and reflection and then provide feedback as well as recommend the user for a badge. One of the ideas that we had for this particular touchpoint is to create a feedback widget that encouraged constructive feedback. Taking a cue from the Lifelong Kindergarten Lab Chloe actually helped us to work on our internal tools for feedback at Mozilla meetings. Since we are often typing on etherpads during calls, we organize our feedback into categories. We added the blue one to the model to encourage collaborative making.
- Green (what's awesome)
- Yellow (what I am unsure about)
- Red (what's not working)
- Blue (an idea that I have)
Here is a screenshot of one that we recently did:
Since this has been a successful prototype for us internally, we thought that we could take this concept to a feedback widget. We were thinking of something like disqus that could be modified by different users to suit their own needs and own kinds of evaluations (imagine evaluating using just emoji's ?!!!) The idea is that a mentor is able to click on the different colors and then is given prompts to help them to construct their feedback. In this moment they are able to recommend the badge as well. There are individual rubrics that we could create for the individual assessments as well, which is not shown. Here is the sketch for this general idea:
After this, let's imagine that the user was recommended for a badge. They are then sent a link (via twitter, via their Webmaker account -- depending on what direction we take above) that would look something like this:
What is worth noting here is that we include options for the user to send thanks or to rebound the feedback. This sets us up for beginning a dialog as well as using this touchpoint to evaluate the evaluator. This is a crucial step to the user flow, since it helps mentors improve their feedback, which is a skill in and of itself. There is some work that still needs to be figured out here in terms of what the interaction is if a user is not given the badge. We want that to be a positive experience.
Chloe made this great stop motion video to quickly go over the overall interaction that I described:
Wondering what the next steps are here? Well, I am happy to tell you:
1. Prototype: We are going to use the Chicago Summer of Learning work to prototype a version of some of this interaction and then iterate on that to develop some of this flow out for Webmaker.
2. Identify mentors: we need to identify how mentors are going to play into this work flow for the Summer campaign. I know that we are going to have an active, dedicated community, however the decisions that we make around their involvement will impact some of the choices that we make moving forward with the assessment interaction design.
3. Work with the Webmaker, Mentor and Badges teams to make sure that we are all collaborating on this work.
