Design Feedback for Badge Systems

Last week I participated in a workshop for Digital Media and Learning grantees. Everyone at the workshop presented their progress on creating open badge systems for their organization. It was my role during the workshop to act as a design expert, to give feedback on the projects and help facilitate a conversation about the design elements of their work.  Over the course of 2 days, I listened to detailed presentations about badge system design, learning theory and communication plans and realized that some themes were emerging. For today's post, I thought that I would share my 5 most common bits of feedback that I gave out to the grantees.

 

1.    Get to know your end-user

Take the time to think through your user persona and user flows. Remember that you have a specific human using your product and they are receiving the badges that you create. Get really specific here and come up with a user story and test that against your designs. 

For example: My user's name is Adrian, he is 18 years old and enjoys reading Science Fiction books, riding his bike and exploring New York City. He is participating in this after school program because he has a passion for math and has felt relatively uninspired by his in-school options. He does this program on Wednesdays after school and has already earned badges for peer mentoring, leadership and problem solving. These are particularly meaningful for him because he wants his peers to know that he is not only a smart guy, but really has street cred in the math community. He posts his badges both in his school LMS and on his personal Facebook page.

From my example, we can already learn a few things that will help inform some design decisions. We know he is doing this work as an extracurricular activity, so we want to frame the work as informal and we know that he is more interested in the social capital that he gains from this experience outside of the classroom and school community, so he is invested in the badges being flexible and shareable. 

You probably will have 3 or 4 types of users for any given product. Embrace that and come up with unique user stories that are based on real life use cases.

2.    Badges and identity

Think about how your badges communicate or enhance an end-users identity. This might seem like an obvious one, but it's crucial to think about how the end-user relates to the badge on a personal level and how that might inform their identity, both within the context of your tool or badge system design and outside of that framework in the real world. On a simple level, think about how might be naming the badges and ask yourself - does this have resonance with the badge recipient? What is more appealing "CSS 101 badge" or "CSS Super Styler"? Think about how this relates to the visual design as well, are you offering something that would have more significance to your organizational brand than to your users brand or identity? If so, then I would recommend reviewing the design and iterating.

3.    Badge design 

It's important to remember that 90% of the badge system design is not visual. Take the time to go and think through all of the touchpoints for badges and ask yourself "are they meaningful experiences?" and if they are meaningful - who are they meaningful for? Once you past that test, then you can move on to visual design. I gave a webinar on this about a month ago and put the slides up on a Google Doc here. 

The top level things to consider are legibility, scale and typography. Open badges are relatively small graphics, so think about that when you are designing them. Reduce the amount of graphic complexity as much as humanly possible. Ask yourself if you even need to incorporate text into the badge - because this adds an additional layer of content that a user needs to digest. I pull this point out separately because I don't think that text is needed because the backpack and/or system distribution could include that information alongside the badge, giving you the opportunity to free up your design. Badges are like icons, you need to attempt to communicate as much as possible with the fewest elements as possible.

4.    User Testing 

How can you engage your community in user testing and analysis? Can you think of ways to work in a more agile way- incorporating feedback and testing so that you are putting out releases that build on your learning and findings from fieldwork and testing?  All of the questions that you have about your design will be acknowledged the second that you have someone who is unfamiliar with your project review your work.

And, as part of the DML/ MacArthur/ HASTAC / Mozilla community- you can ask us for help. Yeah you have a grant deliverable, but you have a community right here who are asking themselves the same design questions and are a perfect user tester. That said, I would take the time to mix up your user tests - have friends and family, but also include complete strangers. 

Remember, user testing can be as informal as throwing two badge designs in front of a user and asking them which one they prefer and why. Don't be intimidated by the term user testing - in reality, what is more intimidating is putting out a product that will be used by hundreds of people that has not been tested.

5.    Metrics and dashboard Data

My final bit of feedback has to do with how you are valuing success but more importantly about how your end user values success, learning and skill acquisition. Many of the dashboard type tools that I saw were actually speaking to how an institution or organization values success within a Learning Management System (or badge system, or tool etc.) but not how the end - user values success. So for example, the tool was designed for the high school teacher instead of the high school student. Remember that success and metrics is something that can be shared and communicated to users as part of their experience within the product offering and it is an opportunity to help a user challenge themselves to level up or exceed both your and their expectations. In terms of design, this might mean that you should consider surfacing some of this data. 

Design needs to constantly be iterated upon, however, if done correctly it will be a huge contributor to the success of your badge system design. So go forth, and design with courage. Use this top 5 list as tools in your arsenal to help you craft a meaningful user experience.

Drumbeat

"No one in this house is going to pick up the phone! ... and if you do pick up the phone, I am NOT giving you dinner!"

I remember the feeling of great joy and satisfaction that I had when my brother and I not only showed my mother how we would connect to the seemingly mythical Bulletin Board System in the early 90s- but how we would get her buy in to collaborate with us to make the experience last just a little bit longer than possible without her. We started up the computer, booted up DOS, ran our dial in program to access the modem, typed in the command line and told it to dial some magical number-then the fun of running around the house like a maniac ensued. We needed to tell everyone not to pick up the phone- out of penalty of death. When we finally did connect to some service, we would type in the user name and password combination and enter this secret world of the Bulletin Board System.

Everyone who was in this sacred space had some interest or skill in common, but for the most part, we would join the community without knowing we were talking to, or what we were going to be talking about. It took a leap of faith to enter this community, and an even larger level of courage and creativity to publicly put your thoughts out there into this void. But, when you did take that leap- it was exhilarating! Here I was, a 10 year old girl, talking to people about computing, comics and just anything that came to my mind. I felt connected to something that was much larger than my living room in Queens- now, I was talking to people all over the world and, even more amazing than that was the fact that they were talking to me! At the end of the day, we would part ways and possibly never talk to each other again, but we had a moment of connection. My family had this moment. These moments changed my perception of the world and allowed me to conceive of the potential for the future internet and my future life in computing.

Since the early 90s, I have grown up a little bit and become a full fledged designer. I now work at the Social Science Research Council on a project supported by the MacArthur Foundation called the New Youth City Learning Network. It was through this position that I was given the opportunity to go to the Mozilla Drumbeat Festival in Barcelona this Fall. Like my childhood experience of entering the Bulletin Board System, I came to the festival with little understanding of who I would be meeting, and quite honestly what exactly I would be doing. Channeling the courage of my 10 year old self- I entered this non- conference like space, and participated in a variety of projects and conversations with people from all over the world.

Since no one knew who I was coming into any given workshop or project, my actions defined me as a "hacker" or "maker". I became one of those people who was using her hands to figure things out. It's funny, because I guess in the Bulletin Board days, I assumed roles or identities as no one was going to remember my name. In the 90s, I was really activated by the ASCII art and making words into visual forms (while my brother found himself entwined in code). Today, I am a designer (and my brother is a programmer) and took on that persona at Drumbeat. The unique thing about Drumbeat was that it was designed to be almost a physical realization of the collaborative communities of the web. Here, programmers, designers, educators and enthusiasts work together to ponder the future of the open web.

I attended a jam session building temperature sensors with blinking lights with the arduino- while thinking about the classroom and educational opportunities for teens and physical computing. It was fun, empowering and super hands-on geekertainment.


I then attended a conversation with Dale Dougherty, founder of Make magazine, and Mr. Maker himself. It was a really interesting talk because we discussed making communities all over the world. I had been familiar with Dale and his DIY and maker enthusiasm from my work with NYCLN and his maker posse at Maker Faire NYC, however we ended up having a great conversation about the potential for maker or fabrication spaces. What kind of affordances do these spaces provide being outside of the school environment? Our group (who were mostly not from the U.S.) wondered if this need for alternative art spaces was an American problem. However, after listening to the full conversation- I believe that this is not about the lack of arts in schools but about forming your own identity through informal learning opportunities. Informal here, being out of the classroom and self-motivated- I like to compare this to the idea of a community garden.


So here I was, in the beautiful tactile Gaudi- crafted city of Barcelona, feeling my way through physical computing projects by hacking into the arduino with a gang of new friends- wondering how I was going to bring it all back to my real world and work in New York. I then attended the session that NYCLN and the Chicago Learning Network was running on local storytelling. It was an awesome session, with a mixture of people from our two networks as well as locals from Barcelona, and other educators from around the world. The premise of the session was simple- go out onto the streets of Barcelona- as a team or by yourself and come back and tell a story that represents your experience of the city. We left the interpretation open and the results were pretty awesome. One group made a poignant 45 second film of children playing soccer, another group used a google map to embed images and stories of their journey, and another team did a he said/ she said story of their time on the streets using twitter and facebook- which culminated in a crowdsourced experience of members from the drumbeat community helping one of the lost participants back to conference.

What was then an interesting turn of events- we then combined with a session on developing badges to discuss how badges could be applied to the activity. So, they used the twitter story to basically "badge".

While this was an interesting experience to be having a conversation as both a maker, as well as someone who supports learning in some form, I have to admit it felt really weird. I had just had this amazing experience crafting the story using casual skills in a non-formal setting and then, it felt like they were just trying to formalize my experience and "grade it" in some form. I am willing to keep playing along with these nascent badge ideas but, as Tim Gunn would say, "I'm dubious."

Regardless, our twitter story won the "multi modal" badge amongst other things. I feel that badges are useless unless they are put out there into the world in some kind of social context. I think that the work that P2PU is doing in now developing out a universal open source badge backpack is really going to help this concept along and I am excited to see where that goes.

Since Drumbeat, I have incorporated this local learning "incubator" activity into one of the youth oriented design jams for our network. This jam is going to be hosted by the New York Hall of Science. Chris Lawrence, the Director of Formal and Informal Education at NYSCI helped to develop the initial concept for the incubator, so taking it back into our own local setting - in Queens! seems not only appropriate, but important.




Another way that we at NYCLN and CLN are bringing the Drumbeat experience back to our local community and world is by developing out a project called Hackasaurus. The Network's involvement with Hackasaurus was really birthed at Drumbeat. The goal behind this work is to create tools to help teens hack the web, and as a by-product learn web design and development. This is a project that is near and dear to my heart because experiences that you have as a teenager- connecting with the world through a medium that engages you, helps you to not only define your identity- but help establish your identity within the context of the world. This was true for me in the 90s, using the BBS, and even more true now with the web. We are working in collaboration with Mozilla, the New York Public Library and YouMedia Chicago, to develop the tools and the complimentary curriculum so that this experience can be easily implemented by any teacher, mentor, parent of teen. The goal for us is to be transparent about our process so that we can engage whoever from the community is interested in collaborating.



At this point we have user-tested in four hack jams in New York and Chicago and developed a small team of teen designers and developers who are going to do user testing as well as development with us. The jams have been an interesting learning experience and I will try to write about them more in another post.

Drumbeat was an engaging and eye-opening experience for me as well as my colleagues in the New York and Chicago Learning Networks, but really what it did was bring back the 10 year old spirit in me. It reminded me of the importance of ingenuity, collaboration and learning by getting messy and jumping in. However, it also reminded me of how as a family, or a community- learning can be even more fun and powerful. My brother and I learned to become designers and developers over time by engaging with the web through meaningful identity building experiences. It is my hope that with the projects that we are developing through the Learning Networks in conjunction with Mozilla that we can inspire a new generation of learners, ones who connect remotely with collaborators, take risks and inform the world of their unique existence using this flexible and definable medium of the web.

7 Scenes Test

A few weeks back I participated in a 7 Scenes test which was part of a MacArthur grant for a project that asked high schools to take part in making design decisions about the neighborhood surrounding their school. I really love the idea of this project. The test that I took part in was really just familiarizing the students (and some of the faculty) at Taft High School in the Bronx with the technology. We walked around with the kids in groups and showed them how to operate the 7 Scenes program on the Nokia N95 cell phones and then we took a walking tour around the school grounds. Prior to the test, the teachers programmed a 'scene' for the students to try out which included some interesting historical tid-bits about their school and the goal of the test was to get the students to think about how they could enhance the scene, and make it more into a game format. The group of students who I lead were extremely articulate and, exceptionally well versed in critiquing technology.


Projects like this are exciting because they use mobile technology to augment the natural environment so that students can see it with a critical eye. I would like to continue to think about ways to implement this kind of project with students in Far Rockaway. It would be fantastic.