Working in the acceleration team of a venture builder means helping ventures build and validate their Minimal Viable Products. That’s exactly how I started working with Route3. It is a platform for managing application specific blockchains. By the time I stepped into the picture, they had a figma prototype and were done with the 1st version of production. I started by learning as much as I could about the platform and its features and I was full of questions why something works this way, why we designed it this way, how this and this flow has been imagined. It was difficult to get clear answers because everyone on route3’s team is a technical expert, they were planning the platform based on their technological innovations rather than the needs and problems of their users.
Soon it started to be apparent that we were working mostly based on assumptions, so logically we needed to start turning towards our users. Since we already had a finished product I thought conducting usability tests was the first step in understanding our users.
I needed to get buy in from the CEO so I shared the benefits of conducting usability tests with him:
• Identifying usability problems
• Validating our prototypes
• Learning about the behaviors and preferences of our users
• Building empathy
• Uncovering opportunities to improve
• Confirming our product meets user expectations
I decided to go for moderated remote tests. Moderated because I wanted to have an opportunity to talk with users and ask follow up questions. I knew that by interacting with users while they are using the app we could gain more insights on what they think and what their unmet needs are.
Don’t make me think: A Common Sense Approach to Web (and Mobile) Usability by Steve Krug
Before conducting the tests, I sat down and re-read Don’t make me think. The book is a must read for anyone doing usability testing. Even though it was last updated 10 years ago it gives advice that’s no less valuable today. Steve also offers some useful resources including test scripts, instructions for observers, scripts for facilitators and checklists.
Design thinking
A nice reminder how empathizing with users and testing solutions are non negotiables when developing new ideas.
Practical advice from the best
The Nielsen Norman group strikes again with brilliant and concise videos.
Our objective was to test the onboarding process and users first interactions and impressions with the app. The CEO and I decided that because the platform is relatively limited for now, we can test all live features during the Usability testing sessions. All together we had 5 tasks.
I was the facilitator and responsible for setting up the calls, preparing necessary materials like wallet addresses and login details for users as well as setting up the equipment needed for usability testing. The CEO of route3 was the note taker and there to help with any questions at the end of each session.
This was an easy task, given that our users are developers, the founder and the team are developers and we work inside a bigger ecosystem with even more developers. Also lucky me, we already had two clients that haven’t seen the platform so they were ideal subjects for the tests as well.
I don’t want to bore you with the script, but if you’re curious check it out here.
The trickier part is what happens off script.
Thankfully, Steve Krug wrote a guide on how to talk like a therapist during these sessions and I will just paste it in here.
We compiled all our observations on sticky notes and sorted them initially on the feature it was concerning.
After that we sat down and realised we can sort the issues based on how easy they were to solve:
1. Fast fixes- issues or bugs that would only require a couple of hours of work to fix
2. Big issues that still need some thinking
After sorting our bigger issues we were able to agree on inputs we were getting from our users and figure out solutions to the problems they faced.
Problem: The design seems more made for end users than for developers.
Solution: Since a similar product to route3 in the web2 world is Amazon web services(AWS) and every developer is accustomed to it, we decided to give route3 a bit of a facelift, make the design tighter, more squared off and professional looking.
Problem: Waiting for something to load with no clarity is always frustrating.
Solution: Clearly communicate what’s happening and what are expected waiting times. We added an activity feed where we communicate the state of each action that is asynchronous and an extra step in the confirmation process where we communicate the action is permanent.
Problem: Features are not fully understood and help is hard to find.
Solution: We added an explainer on each feature page in case less experienced users aren’t 100% what it does and tech documentation next to videos, because most users prefer to read at their own pace and get more details.
Problem: The table on the Smart Contracts page doesn’t offer enough options.
Solution: We redesigned the table, added more information and options our users were requesting.
Problem: Users wished that they had a quick way of getting all information they need to use outside our app like RPC link, chain ID, block explorer link.
Solution: We created an extra widget on the dashboard that offers quick actions to users.
Just go out and do it: I am glad I didn’t let uncertainty discourage me from doing these tests. Were they thought out perfectly? Not fully, but even this way we found out what our users thought, where we can improve and what we lack to make an excellent product for them. After the usability test we were trying to make sense of user inputs we realised that we don’t have clear use-cases for our app and that we essentially let users play around in it, at our cost. We will continue defining the use-cases that will hopefully get us closer to defining a product-market fit for Route3.
If something feels off, it’s probably for a reason: In the beginning of the process of working with route3 I believed I was not a good enough designer or didn’t have enough experience and that’s why my design intuition was not working. That was not the case. I couldn’t make decisions because I was lacking critical information - who am I designing this for and what does the future entail for route3. And instead of waiting for someone to put it in my lap, I went searching for it.
