Reflections on User Research Smoke & Mirrors article

The article “User Research Smoke & Mirrors” by Christopher Fahey is an insightful article that has perspectives and issues which I have never thought in depth about.

Indeed we have to acknowledge that less-rigorous techniques such as “card sorting, focus groups, guerilla usablity testing and user personas” are helpful methodologies that can aid the user experience designer to know more about the users. These techniques enable us to understand the users’ behaviour and reactions, though the results are subjective in nature. It is important to note that the ultimate aim of conducting user research is to design effectively for the users, who will have subjective views of their own.

Interpretation of data by non-experts and even experts is subjective and differs from individual to individual. A subjective data analysis should not be intentionally presented to appear objective. The seemingly scientific research will mislead others into thinking that the “scienticfic” research is impartial and credible.  

The “Persona Rooms” mentioned in the article actually sounds pretty interesting because the design team can actually get a feel of the persona by entering the room with a particular set up. This no doubt brings the design process to a higher level. However, I also agree with the author that having professional set designers to build these living room spaces does inccur a significant amount of money. The amount of money spent on such rooms may not even be necessary in the first place. Hence, I feel that setting up the persona rooms shouldn’t be a priority in user research.  In fact, it should be of low importance even though it is an innovative idea.

The eyetracking example mentioned by the author reminded me of a guest lecture in one of the modules I took last time, where there was a live demo of the eyetracking system. Volunteers sat in front of the computer and took a test, where they had to see pictures or find a particular piece of information from a website. Heatmaps were generated according to how long the eye looked at a particular area on the computer. Graphs and data can also be generated from the results of the tests taken. 

I find that such innovation shows the technological progression in our society as we may never have thought that such an eyetracking system is possible. I have to admit that, just as the author pointed out, the results stated the obvious like what I would have expected. For example, users took longer to find a particular piece of information on websites that are more wordy and cluttered. I agree that good user interface designers would probably come up with the same conclusions as compared to the analyses obtained from eyetracking tests. Probably, the only difference is that the eyetracking tests have numbers, figures and graphs to backup their conclusion while user interface designers do not have such data as evidence. 

Conducting such research “can’t hurt” and it can be an addition to the current research findings, but the amount of money required to conduct such eyetracking research is an important consideration. After all, the conclusion arrived from eyetracking research does not differ much from that of user interface designers. In addition, as the whole idea of interpreting data is subjective (even if it’s done by experts), there is a risk of misinterpreted results as the eyetracking tests do not question why the user looks at a particular point.     

The author also explained that user research is useful when explaining good design decisions to people (or higher authority) without deep design skills or instincts. Having concrete data of research findings makes it easier for these people to be convinced that certain design decisions are good or that a particular bad design needs to be changed. Hence, even if the design problem is obvious to designers right from the start, user research may be necessary in such cases.