What We Learned Through Conducting Hundreds of Usability Tests

Every Friday at JotForm, we sit down as a company for a few hours and have a “Demo Day.” In addition to showcasing what each team has accomplished that week, we also collectively watch a handful of usability tests. Using a third-party service, we record people interacting with JotForm’s online form builder for the first time, ask them to speak out while using the product and see what we can learn. When we started doing this, we noticed a trend.

People were still having trouble creating the forms they needed. With all of the buttons on the interface, users didn’t know where to start or where to go next once they did start. Forms took a long time to create. People who used their mobile device to create a form were also hitting obstacles. Not only did the form builder display horribly on a mobile screen, it made it hard to access certain features.

We needed to address this.

So over the next one and a half years, we completely rebuilt our form builder. Since it’s harder to make a desktop version fit mobile, we adopted a mobile-first approach for our redesign. From there, the redesign took on a life of its own. In order to be fully mobile-friendly, that meant incorporating offline capabilities. After all was said and done, we built JotForm 4.0, the first fully functional online form builder that works on any device, including desktop, smartphone, and tablet.

Here are the four biggest lessons we learned along the way.

1. Get as much data as you can before you make a decision

You may be inspired after seeing a user test perform an unexpected behavior. With the energy and motivation to make changes, it’s easy to want to run with the new information. However, you should keep in mind that one new use case shouldn’t dictate an overhaul, or even a feature update. Instead, have an honest discussion about what that threshold should be before it becomes a significant issue. If a large enough percentage of your user tests follow the same pattern, then make a plan for steps for change. And if your data doesn’t show any significant patterns, conduct more tests.

2. Know when to stop

We made continual improvements to the form builder and regularly conducted user tests on the newest version. Even when we reached our goal of creating a fully mobile-friendly form builder, we still had a list of improvements that we had yet to incorporate into the product. However, at a certain point, we had to make the decision to stop and move onto QA. For product teams, it can be hard to finally step away. They always want to add a new feature, or make an adjustment to another. It finally reaches the point where you have to stop developing new features and ship the product. So after a year and a half of work, we decided that features we had brainstormed would simply go into the next version, and we would release the current version to our users.

3. Outdated doesn’t mean it was wrong

As a small company with team members who are involved in almost every aspect of the process, it’s easy to become emotionally attached to previous versions and want to fight to keep them. However, realizing that your product needs an overhaul should never be a personal issue, nor an insult. JotForm 3.0 was built for a reason, and at the time of its launch, it resonated with users because it was ahead of its time. It was a clearly a standout option compared to other existing form builders. But that was seven years ago. Coming to terms with the need for a change should be seen as liberating. It gives your design and product teams the freedom to come up with new ideas and once again, stay ahead of the curve in your industry.

4. Usability testing is a never ending quest

Every time we run usability tests, we find new problems to fix. When you fix larger problems, things start going smoothly, but inevitably, you start noticing smaller, less obvious problems. Perfection is impossible to reach since people will always be very different. When you optimize your product for one group of people, it might still not be perfect for another type of user. But, you can still come pretty close to perfection if you accept usability testing as a continuous process. That’s why every Friday afternoon, we all sit down together and watch usability tests in front of a large screen.

We’re taking away many lessons with us from this experience. As much as they are helpful, usability tests are also very humbling. They force you to take a hard look at your product and make tough decisions. At the end of the day, you have to push aside your personal preferences and make sure your decisions are in the best interest of your users. The end product will be one that you and your users both appreciate.

Guest article written by: Marina Lau is the Senior Marketing Manager at JotForm, a popular tool for easily building online forms. JotForm users can create forms with just a few clicks and analyze data without writing a single line of code. JotForm is the ideal solution for online payments, contact forms, lead collection, surveys, registrations, applications, online booking, event registrations, and more.