What do we mean when we say enterprise software is bad? Sure, terrible hacks and technical debt could be weighing the code down. Maybe it’s a nightmare to integrate and administer. But, for most of us, it’s all about the experience of using the software. In the early 2000s, agile began to tackle the problems of organizing software projects. Around the same time, usability specialists such as Jakob Nielsen and Don Norman were popularizing ideas of user-centered design. One aspect they focused on was user research. First, observe how people try get things done. Then, build software that solves the problems they have rather than relying on assumptions, hearsay and management misunderstandings.
Just as today agile is pretty much everywhere, it’s now commonplace for consumer-oriented software companies to have at least one user researcher. Mozilla, the organization behind Firefox, has a team dedicated to user research and one of its leaders is Bill Selman.
Let the User Be...Researched
I asked Bill to tell me about the specifics of what his team works on.
“We work with the product and UX teams to answer research questions that come up. So, on the one hand, we do lots of summative research: things like user testing, usability testing for prototypes, for new features, for current features.”
Summative research tests things after they’ve been built. That doesn’t mean it deals only in finished products. It could also be a wireframe of a prototype user interface or perhaps an early working beta. It could also be looking at finished products from elsewhere.
“Sometimes,” Bill told me, “we do competitive research on products that have features we're considering implementing. In those cases, we’re looking to understand the users’ perspective—their mental models—to learn how well a product works with users in the wild.”
Summative research is great for course correction and confirming assumptions. However, a lot of user research happens far earlier in the product creation process.
Bill explained, “Our other angle is formative research. It's basically going out and trying to study a problem and understand different users' perspectives.”
User Research Is a Philosophy
If user research has a role right through the lifecycle of a product, then what specifically do user researchers do?
“User research is an interesting hybrid of activities from many different disciplines in the same way that UX is this multidisciplinary approach to designing products. User research borrows a lot, obviously, from human-computer interaction, but also from other fields like anthropology, sociology and cognitive psychology. “
Bill sees user research as a philosophy for designing and building better products. It’s less about the techniques, he says, and more about user-centered mindset. To get a firmer idea of the practicalities of user research, I asked Bill to take me through some of those techniques.
“We employ a lot of different methods. Surveys are one of them, but it's honestly one that we avoid when possible. But a lot of the research that we do is in person, observing how people are using a prototype or using our product.
“Sometimes we do task-oriented tests, where we ask participants to complete a series of tasks so that we can understand how well they did them and how easily. Then there are eye tracking studies where we're able to understand not just how people are completing tasks, but what is the focus of their visual attention during a task.
“And, of course, we do a lot of in-person interviews with participants. We think they're particularly important because you get a lot more valuable information when you understand a user’s context.”
We’re Here to Observe: The Impact of User Research
Understanding how people might use a new feature is interesting by itself but user research is useful only if it can influence the product. Bill and his team work to make sure that everyone involved in a product is engaged with the research and the findings it produces.
“So one thing,” Bill said, “is that we include as many observers from our teams as possible in the research in order to have people really see firsthand the problems that users are encountering.
“And so we do a study and we come away with two basic kinds of insights. One is, ‘Hey, this prototype has these problems. If we change these things, then this will make it easier to use.’ Another kind of finding would more fundamental, such as, 'Hey, this is a problem that we haven't thought about.' When that happens, we present it to the team to say, ‘This is something that we haven't thought of before. It seems like a really interesting feature, a new product. Let's go do that.’”
The Influence of Culture
Mozilla is a global organization that makes tools, including Firefox and Thunderbird, used around the world. That puts a particular burden on user research. It’s not enough to gather a handful of people who live near your office and ask them to try your latest build. People living on the same few streets will have many of the same cultural assumptions and biases, even if their own backgrounds are varied.
“One project that we did two years ago was understanding how users approach their tasks on multiple devices. So, for example, writing an email or saving a piece of information to revisit later. And what's interesting about Mozilla for this kind of research is two things. One is we think of ourselves as a global corporation and a global product and globally focused set of services.
“For us, it's really important to do user research in different places around the world. We consider how important localization is but also how different culture and context really impact how people use our products and services. For this project, we did research in the U.S. and Canada, but also in Japan and Taiwan. There were some major differences in how people approached the tasks.
“The second thing is that Mozilla is fundamentally an open source organization. So, we tend to share a lot of this data on our blog with the public at large.”
Enterprise Software: User Research to the Rescue?
Mozilla’s Firefox browser has a notoriously large and complex codebase. Complexity is one reason often given to explain enterprise software’s lack of finesse—or, in some cases, outright user hostility. So, what does Mozilla do differently from typical enterprise software vendors?
“We build software that's user-centered, that puts users first and puts users' needs and goals first. If you're thinking about making products easier to use, and making people more productive and ultimately more successful, then it seems to me that everything would follow from placing the greatest priority in your software development process on understanding users' needs and goals.”
Before working at Mozilla, Bill spent some time working with enterprise software shops. What he saw was an indication of why user research had less impact in those enterprise environments.
“I worked for almost two years for a consulting company that primarily did enterprise software. In my experience, it was not software designed for users in mind.
“It was designed for salespeople and people who are doing requisition for software; directors who are responsible for making purchase decisions; and the concerns of the IT department about security policies or privacy policies. And certainly, all of those perspectives are important. You don't want to get ripped off. You don't want software that is not secure.
“On the other hand, the whole point of software is to improve and increase productivity, and the people who are responsible for the productivity aren't necessarily part of the purchase equation. I worked on a few projects that were successful but the biggest challenge with enterprise software is not starting with the users’ goals and needs.
“Some projects were completely sales-driven: the sales person was not incentivized to think about the quality of the product or the people using it.”
Give Me the Scope
So, what scope is there in such projects for user research to have an impact?
“It was done mostly just to provide a sheen. There would be line items in the contract for user research, for design, but not a reasonable amount of time to even do the basics.
“We had this one project that was really interesting. It was a completely sales-driven process but it was also just as much the client’s fault because they wanted to pay as little as possible. The client had offices all over the world and the product was really complicated. The point of it was to facilitate collaboration among people from different cultures and different contexts. You know, that's a huge problem to solve.
“The client said to me, ‘The salesperson told me that you've done these kinds of products before with global companies, so I just want you to give me what you gave them.’ I proposed doing some small user research tasks and the customer said, ‘Yeah, we're not gonna do that. That sounds like a science project. I just want you to give me what you gave them.’”
Is There Any Hope, Then, for User Experience in Enterprise Software?
“It’s becoming easier and easier to do user research as part of your software development process. I mean, it's not cheap, yeah. But it is becoming less expensive to do quality user research studies before you ever write a line of code, before you ever design a wireframe or an interface. From my perspective that's a very positive trend for enterprise software.
“The way that I tried to solve that process was to become as much a part of the sales process as I could be. Ultimately, your reputation is built on the quality of the products that you deliver. And if your competitors are providing these kinds of services and they're having outcomes that improve the quality of the product, that's a pretty compelling argument that user experience should be part of the sales process.”
User research stops stupid mistakes
Who’d have thought that taking the time to understand people’s needs would be so effective? It’s not enough to ask a disconnected manager or to make assumptions based on what worked elsewhere.
As Bill says, user research tools and practices are now commonplace. Sure, there’s still the expense of hiring people to conduct the research and draw insightful conclusions. But, if you’re building enterprise software that does a bad job of solving the problem, then perhaps it’s worth spending a little extra in order to deliver the right product.
Next time I’ll be speaking to Mark Tareshawty, one of the engineers at GitHub, to get an idea of the developer’s perspective on delivering quality software.