Showing posts with label user testing. Show all posts
Showing posts with label user testing. Show all posts

Thursday, March 6, 2014

Designing BadgeKit

After several months of hard work by the Open Badges team, we are announcing that BadgeKit is  available for access to Private Beta. This means that BadgeKit is now available in two forms:  a hosted version of Mozilla BadgeKit available in private beta for select partner organizations that meet specific technical requirements, and anyone can download the code from GitHub and implement it on their own servers. 

BadgeKit is a set of open, foundational tools to make the badging process easy. It includes tools to support the entire process, including badge design, creation, assessment and issuing, remixable badge templates, milestone badges to support leveling up, and much more. The tools are open source and have common interfaces to make  it easy to build additional tools or customizations on top of the  standard core, or to plug in other tools or systems.

From a design perspective, this milestone represents refinements in user research and testing, user experience, user interface and branding. 

We did user testing with members of the Hive in Brooklyn.
In preparation for this release, we conducted extensive user research to define the needs and goals for badge issuers. This work, led by Emily Goligoski, helped to define requirements for the BadgeKit offering as well as inform the user experience. The research was done using a variety of methodologies, however, it is worth noting that all of this work was done in the open. Emily organized distributed user testing in key markets such as New York, Chicago and Toronto to do everything from needs analysis to accessibility and functionality testing. The Open Badges weekly community calls were leveraged to pull in input from the highly motivated research and practitioner cohorts. Much of the work is documented both on her blog and in github. We paired every implementation milestone with some form of user testing and iteration. While this may sound obvious, it was a new way of working for our team, and I can unequivocally say that the product is better because of this practice. User research and testing did not happen in a bubble, but rather it became completed integrated with our design and implementation cycle. As a result, developers and designers became comfortable making informed iterations on the offering, as developers, designers and team researchers all participated in some form of user testing over the past three months. 

As a direct result of the extensive research and testing, the user experience for the entire BadgeKit offering was deeply refined. This work, led by Matthew Willse introduced some new features, such as badge “templates” which give the ability for any badge issuer to clone a badge template and remix it. This gives us the unique ability to offer template packages based on common badge requests from the community, as well as eventually to empower the large Open Badges ecosystem to develop badge templates of their own (and perhaps explicitly state how they are comfortable with their content being shared and remixed). One component of this work that evolved as a direct result of testing, was the increased attention to copy. Sue Smith led this work, which entailed everything from tool tip development and a glossary to API documentation. Considering that BadgeKit takes an issuer from badge definition



and  visual design



 to assessment and issuing,

designing the user experience was no small effort and the attention to detail combined with designing in the open - proved to be a solid approach for the team. 

Perhaps the most obvious design component of this release is the user interface design and brand definition. Adil Kim kicked off this work with an exploration of the brand identity. BadgeKit is under the parent brand of OpenBadges, which sits under the even larger parent brand of Mozilla - which gave us the constraints of designing within the brand guidelines. After exploring options to represent the visual metaphor for this modular system, here is the new logo:



The logo is meant to evoke the imagery of both a badge as well as a tool in one glance. For the untrained craftsperson (ahem) - while gazing into the mark - you will see a bolt . This connotes that BadgeKit is a tool, something that allows you to dive into the details and construct a badge, and a system for your community. The logo incorporates the palette from Mozilla Open Badges, in a playful mobius - at once implying that while this is a handcrafted experience, it is also a seamless one. This logo nicely fits into the larger brand family while reading on it’s own, as if to say, “hey, BadgeKit is the offering for badge MAKERS, dive in and get your hands dirty!” 

The brand is in turn extended to user interface design. The overall art direction here was that this needs to be clean, yet approachable. We know that many organizations will not be using all of the components in the interface directly on badgekit.org, however, the design needs to take into account that everything needs to be accessible and read as remixable. Some details to note here are the simplified navigation, the palette and subtle details like the ability to zoom on hover over thumbnails. 

It’s worth noting that while Emily, Matthew, Sue and Adil , as well as Carla, Meg, Erin, Jade, Sabrina Ng, Chloe and Sunny were invested in much of this design work, there was an intentional yet organic partnership with the developers (Zahra, Erik, Andrew, Chris, Mavis Ou, Mike and Brian + many, many community contributors) who were doing the implementation. We had weekly critiques of the work and often engaged in conversation about design as well as implementation on github. 



Another component of this work is looking ahead towards future features. Chloe Varelidi lead work here thinking through the potential for badge and skill discovery. Under a grant from The Bill & Melinda Gates Foundation, Chloe and her team are thinking through ways to represent earner pathways. This eventually will be leveled up into the core BadgeKit offering, but you can start to dip your toes into those features by checking out the work here.

And the good news is that design never ends! Design isn’t just a destination, it’s an invitation to a conversation. Check it out, let us know what’s working and importantly, what’s not.

Monday, April 9, 2012

Beta Testing Kitchen Tables

Imagine sitting in the kitchen with your friends, hanging out, chilling and making some cool stuff on the web. That's the idea behind the Kitchen Table event type that Mozilla is in the process of Beta testing for release in conjunction with the Summer Campaign.   This past week, I did my dogfoodin' and  Beta tested the Kitchen Tables event type with my family.

The Beta testing was scheduled for a two week window, that really coincided with Passover, so I decided to take this as a positive thing and do a "Hacking the Holidays" themed kitchen table. I had done that concept as a Hack jam with Mouse, but this was more challenging because it was with- the parentals.

kitchentable

THE INVITE:
I asked my parents if they would be up for spending some time and making some things online with me when I visited. I created a little url based on some discussions I had with other user testers to send out as an invite. I said it would only take 30 minutes and we could have tea while we were doing it. This was a difficult sell. My parents, both retired teachers, and honestly very supportive of the work that I do and are the first people on the soapbox about teaching youth to learn to code, were very resistant. My mother, who is a web and computer enthusiast agreed to do it, and my dad said he would watch and asked if there was anything with paper that he could do. I followed all of the steps in the wiki, but I frankly think that a lot of Mozilla's efforts do not reach this older audience. In fact, I would go so far as to say that Mozilla appeals to 30 somethings - teens but does not really reach out to 40 + users. I think that my sell only worked because I forced my parents to do it.

kitchen table beta 2


THE KITCHEN TABLE:
So, as I said, this was pre- Passover. My parents sat with me and Adrian (the boyfriend) and I took them to my kitchen table website and decide to teach them how to hack that. It was a bit crazy because I had to move fast to keep them focused. They really weren't super into it- particularly my dad. I don't think that there was a real value add for him in terms of hanging out and doing this, other than spending some time with me. Perhaps if he had a grandchild to do this with -- ahem, not suggesting ANYTHING! -- he would be more willing because it would be an activity that they could do together. We slowly lost him and just ended up with my mom.


The session went alright, we essentially made a goal of hacking the "kitchen table" website to be a Passover table. The hardest concept for my mom and Adrian was to understand that every piece of content needed to be on the web to  show up. For example, they needed to put images up on flickr, or pull from an existing website. My mom really wanted to just upload an image to the editor and be done with it. This is a really important concept that I think that we need to do a better job communicating.

After about 25 minutes of tinkering and joking- my Mom came up with this:

 

All in all- it was a success! My mom shared her link with friends and family. My dad, asked if I had something for him to read with, but I have a hunch my Mom will convince him to try it out on their own time. 

WHAT WOULD I DO DIFFERENT/ NEXT STEPS:
  • I would have a better ask
  • I would pitch the activity more casually while I was already spending time with them
  • I would bring another laptop- I forgot that my parents only had one
  • I would find webpages/ things that they are already interested in and hook them there
  • I would give my dad something with paper to do- already in the works for the Missions project
  • I am going to send out my Mom's website to her temple community and suggest that we do a bigger meet up to teach others how they can make their own webpages




Wednesday, February 8, 2012

Open News Design Sprint Day 3: Meet the Journalists


After two days of doing a design sprint brainstorming about a tool that could help journalists learn webmaking, we decided to go right to the source for help. We invited about a dozen journalists to join us for a conversation today. We lucked out and got a fantastic group of professionals with a variety of different skill sets and backgrounds. Journalists showed up representing tons of different orgs, including: New York Times, Forbes, Gotham Schools, NYU Journalism Institute, Mashable, NYU ITP and the Huffington Post.


The main question that we asked this group was: "What do they you to make on the web?"We had a range of responses to this question.  There we are a lot of answers in the categories of data manipulation, data visualization, interactives, and text based interactives.
"There are two categories: a special project that's a one-off that blows people away and wins awards. But the thing that's more interesting are the things that can sustain and help a community. A database of property transfers that is maintained and is there for someone to access when they want to find out about it. Something that continues, is sustained, and is part of the fabric of info flow in a community. Data that comes in and continues to live." (note that all quotes in this post are paraphrased from the session)
We delved into an interesting conversation about what journalists need to know in terms of learning about the underpinnings of webmaking. Many participants talked about wanting to learn skills that actually fall into the category of "design thinking."
"Journalists don't have the knowledge of when to use a map vs not use a map. Or from a story to getting more interactive. They need to understand that a story isn't just an article, that it can be a whole bunch of different things. design thinking methodology"
"When you get down to it, we have a history of journalists as writers. But the last thing you want is a situation when all the journalists become coders. The more powerful thing is to have more of that process--the methodology/training/etc. To run a group of coders, or talk to coders. To know enough about your topic to come up with ideas and know if something is feasable or not. Not to be condescending, but 'skills are for interns' they are important to know to do, but nobody's going to make work-changing this just coding CSS. Skills are great and they're important, but it's about getting to understand the bigger picture. We spend a lot of time on, what do you call it, agile journalism."
I explained to the group that I personally have no interest in forcing anyone to learn how to code, but I was interested in learning about what would empower journalists to take an interest in webmaking.

"It's about showing them good work, and then showing them easy things--things that are very much within reach. It's about motivating them and liberating them."

"A big problem is internal. Which traffic cop gives the most tickets per parking meter: you dont' want to wait for six weeks for the data guys to come back to you with the answer. It's about being able to quickly test a hypothesis that could be interesting. There could be a story here--well, you want to know that quickly. Maybe the distribution of skills in the newsroom is 200 writers and 2 coders--it's going to be a long time before that. How much coding do you need to be able to play and test those hypothesis?"

"It's the bridge/demystification thing. There's a big wall--it feels like it. But then you go to hackjams and it feels attainable. It demystified it. When you realize those tools are attainable."

Mentorship and peer based learning was another topic that came up. One participant admitted to the group that he had signed up for CodeAcademy because of "social peer pressure" but then is now in the position where he has 6 untouched lessons in his inbox. This is because the great myth about something like CodeAcademy is that you are doing it with your friends. You signed up because your friends tweeted about it and you got all pumped up and signed up together- but then you are at home by yourself at the end of the day- not coding. Another participant talked about how she and a friend got a book and together followed all the steps to learn Python. After a month or so, they hired a "geek" to ask questions. We talked about how there are online communities for that like Stack Overflow or Quora- but these are often intimidating communities where a journalist would never think to go.

We asked for some initial feedback on the prototypes that we are working on. Here were some of their responses:

1) On the sequencing of learning content: Is the tag flashy enough to be the first thing that is taught in the instructional overlay?
  • some say yes, some say no. The yes came from a not super tech savy journalist "I don't want big blocks of text"

2)  On the editing window: I see a lot of use of the immediacy of the left/right pane as tool
  • everyone agrees
  • "there is no internal demo tool other than building the page

3) On if this is useful: Yes: even myself, on my Tumblr, if I want to make a change, etc, I have to publish it, look at it, reload, etc, that's incredibly frustrating.
  • A bigger use-case  for this is internal training, self training, etc. I'm throwing in on being the weekend home page editor.
4)  On Badges: I can totally see only entertaining resumes from people with that badge. And if you don't have it, spend a weekend to earn it. 
At the end of the session- Carl Lavin was generous enough to sit with me and tell me-- if we were going to do webmaking 201, 301 etc for journalists what topics would he want covered. He wrote out these cards for me:
I think these will be really useful when we start to build out learning pathways for our Mozilla learning offerings.
I spent a good deal of time picking the journalists brains as well about the topic that I have been struggling with this week- about framing this as a project about storytelling or something specifically for journalists- and many participants told me that they prefer the storytelling angle because it is something that can appeal to a larger audience. Some of the journalists told me that that "journalism" means different things to different people, but at the end of the day everyone has a story to tell. And that is something that I can bake into the prototype!

       

Thursday, May 27, 2010

CUNY on Wheels, PS 43, Rockaway Waterfront Alliance and... OCEANLAB

Yesterday we had a great day at the beach with students from PS43Q. I ran 4 Oceanlab workshops and had the help of my mother, Rose Klein- DOE educator. The workshops went well. Each time I ran the workshop, I became a little bit more comfortable working with students and I felt that much more confident presenting the material. Not being an educator myself, I truly admire the way that teachers can calmly talk to children - without talking down to them, or being intimidated by them. Regardless, I introduced the oceanlab website to approximately 100 students and got some great feedback. I learned that students in this school tap into the social networks of facebook, myspace and twitter. Many use the cell phone as means to search the internet throughout the day.

In a formal setting- such as this classroom on wheels- students were most concerned about learning where they could go to learn how to swim. This makes me think that I need to add more links and informal "lessons".

I am thinking of modifying the Oceanlab site to include areas for research and representation of the informal activities- but then again, I could rely on a facebook page to communicate that content as well.

Here are some photos from the day. I really enjoyed the experience of this mobile learning vehicle and am already talking more with CUNY on Wheels to plan our next event.
Look how neat the bus was!


Water safety learning in action.
We parked the bus right next to the boardwalk.

This is Norma Coleman James, Assistant Director of CUNY on Wheels.

Monday, March 29, 2010

CUNY on Wheels has signed on!

A brief bit of exciting news- CUNY on Wheels will be donating a bus for the Earth Day launch. This is exciting, because the Earth Day celebration will be happening totally on the beach, so we are doing some mock workshops that will be on the beach but then we will also be user testing the website on the CUNY buses. These buses are fully equipped with computers and movie screens so it is quite an interesting opportunity. I mean, check out the bus in this picture- it's legit.

7 Scenes Test

A few weeks back I participated in a 7 Scenes test which was part of a MacArthur grant for a project that asked high schools to take part in making design decisions about the neighborhood surrounding their school. I really love the idea of this project. The test that I took part in was really just familiarizing the students (and some of the faculty) at Taft High School in the Bronx with the technology. We walked around with the kids in groups and showed them how to operate the 7 Scenes program on the Nokia N95 cell phones and then we took a walking tour around the school grounds. Prior to the test, the teachers programmed a 'scene' for the students to try out which included some interesting historical tid-bits about their school and the goal of the test was to get the students to think about how they could enhance the scene, and make it more into a game format. The group of students who I lead were extremely articulate and, exceptionally well versed in critiquing technology.


Projects like this are exciting because they use mobile technology to augment the natural environment so that students can see it with a critical eye. I would like to continue to think about ways to implement this kind of project with students in Far Rockaway. It would be fantastic.

Wednesday, November 4, 2009

PS3 User Testing/ Research

The classroom was really awesome. The kids had their own domain maps!







Note the "screen" that we had for watching a video as a class.
Said screen ran out of power, but I had a back up.