1. Get to know your end-user
Take the time to think through your user persona and user flows. Remember that you have a specific human using your product and they are receiving the badges that you create. Get really specific here and come up with a user story and test that against your designs.
For example: My user's name is Adrian, he is 18 years old and enjoys reading Science Fiction books, riding his bike and exploring New York City. He is participating in this after school program because he has a passion for math and has felt relatively uninspired by his in-school options. He does this program on Wednesdays after school and has already earned badges for peer mentoring, leadership and problem solving. These are particularly meaningful for him because he wants his peers to know that he is not only a smart guy, but really has street cred in the math community. He posts his badges both in his school LMS and on his personal Facebook page.
From my example, we can already learn a few things that will help inform some design decisions. We know he is doing this work as an extracurricular activity, so we want to frame the work as informal and we know that he is more interested in the social capital that he gains from this experience outside of the classroom and school community, so he is invested in the badges being flexible and shareable.
You probably will have 3 or 4 types of users for any given product. Embrace that and come up with unique user stories that are based on real life use cases.
2. Badges and identity
Think about how your badges communicate or enhance an end-users identity. This might seem like an obvious one, but it's crucial to think about how the end-user relates to the badge on a personal level and how that might inform their identity, both within the context of your tool or badge system design and outside of that framework in the real world. On a simple level, think about how might be naming the badges and ask yourself - does this have resonance with the badge recipient? What is more appealing "CSS 101 badge" or "CSS Super Styler"? Think about how this relates to the visual design as well, are you offering something that would have more significance to your organizational brand than to your users brand or identity? If so, then I would recommend reviewing the design and iterating.
3. Badge design
It's important to remember that 90% of the badge system design is not visual. Take the time to go and think through all of the touchpoints for badges and ask yourself "are they meaningful experiences?" and if they are meaningful - who are they meaningful for? Once you past that test, then you can move on to visual design. I gave a webinar on this about a month ago and put the slides up on a Google Doc here.
The top level things to consider are legibility, scale and typography. Open badges are relatively small graphics, so think about that when you are designing them. Reduce the amount of graphic complexity as much as humanly possible. Ask yourself if you even need to incorporate text into the badge - because this adds an additional layer of content that a user needs to digest. I pull this point out separately because I don't think that text is needed because the backpack and/or system distribution could include that information alongside the badge, giving you the opportunity to free up your design. Badges are like icons, you need to attempt to communicate as much as possible with the fewest elements as possible.
4. User Testing
How can you engage your community in user testing and analysis? Can you think of ways to work in a more agile way- incorporating feedback and testing so that you are putting out releases that build on your learning and findings from fieldwork and testing? All of the questions that you have about your design will be acknowledged the second that you have someone who is unfamiliar with your project review your work.
And, as part of the DML/ MacArthur/ HASTAC / Mozilla community- you can ask us for help. Yeah you have a grant deliverable, but you have a community right here who are asking themselves the same design questions and are a perfect user tester. That said, I would take the time to mix up your user tests - have friends and family, but also include complete strangers.
Remember, user testing can be as informal as throwing two badge designs in front of a user and asking them which one they prefer and why. Don't be intimidated by the term user testing - in reality, what is more intimidating is putting out a product that will be used by hundreds of people that has not been tested.
5. Metrics and dashboard Data
My final bit of feedback has to do with how you are valuing success but more importantly about how your end user values success, learning and skill acquisition. Many of the dashboard type tools that I saw were actually speaking to how an institution or organization values success within a Learning Management System (or badge system, or tool etc.) but not how the end - user values success. So for example, the tool was designed for the high school teacher instead of the high school student. Remember that success and metrics is something that can be shared and communicated to users as part of their experience within the product offering and it is an opportunity to help a user challenge themselves to level up or exceed both your and their expectations. In terms of design, this might mean that you should consider surfacing some of this data.
Design needs to constantly be iterated upon, however, if done correctly it will be a huge contributor to the success of your badge system design. So go forth, and design with courage. Use this top 5 list as tools in your arsenal to help you craft a meaningful user experience.