Being an information architect can be stressful. There are certain points in the design process that are more stress-inducing than others. I'm not talking about normal wipe your brow with a handkerchief stress. Think peptic ulcer bordering on active volcano.

Imagine, for example, that you're the lead information architect for a four person team tasked with conducting research and developing an information architecture strategy for the web site of a Fortune 100 corporation.

Your team has spent ten intense weeks analyzing content, interviewing stakeholders, and observing users. You've learned a great deal about this information ecology and you're ready to apply that knowledge towards development of the information architecture.

But first, you must present the results and analysis of your research to the web strategy team, a group of high-powered web site managers and content stakeholders. You begin the presentation, things are going smoothly, and then it happens, someone invokes the fearsome Q words.

"It appears that your research has been purely Qualitative rather than Quantitative. So, the results aren't statistically meaningful or scientifically valid. I'm not sure we can trust an information architecture strategy that's based on this research."

You are now pinned to the wall like a doomed butterfly, fluttering helplessly before the concerned stares of the web strategy team. You have been exposed as a practitioner of black magic, a participant in that which is not science. Beads of sweat form on your brow. Your ulcer begins to pulse.

You believe your research effort made sense and that the results can help you design a better information architecture, but how can you prove it? How can you defend your qualitative approach against charges of statistical irrelevance?

Statistics Can Lie

You'll want to begin by dispelling the myth that quantitative studies and scientifically-derived statistics are always messengers of the Truth.

First of all, any study, quantitative or qualitative, that is based on a false assumption can lead you astray. Consider for a moment, our friend, the ulcer. Until fairly recently, the medical community (and the rest of us) believed that ulcers were caused solely by stress and spicy foods.

Pharmaceutical firms made huge investments in quantitative studies (probably involving highly stressed rats and large volumes of Szechuan spicy beef) aimed at curing ulcers with antacid drugs. Then, in 1982, the connection between a bacteria (H pylori) and peptic ulcers was discovered. It became evident that an antibiotic was needed to cure this disease. Scientists had been barking up the wrong tree by missing the root cause.

Second, interpretations of quantitative data can be (accidentally or purposefully) misleading. From sample selection to analysis and presentation of results, there are ample opportunities to introduce bias. There's even a great how-to manual on the topic, How to Lie With Statistics.

Making the Case for Qualitative Research

From anthropology to zoology, professionals in academia and business have had to make the case for qualitative research methods. Let's explore some of the key reasons why qualitative approaches are critical to information architecture design.

Learning to Count to One

The field of information architecture is new and relatively immature. We're still all trying to understand the variables and define the metrics. Before we begin quantitative studies involving hundreds of users and large investments of time and money, we need to learn to count to one. We need to make sure we've identified the right business strategy, content, and audiences.

For example, before you perform quantitative studies aimed at optimizing your web site for speed of navigation (measuring time to find or numbers of clicks), ask whether learning is an important component of your site. Is there value in your customers learning about additional products or services that could be useful to them? Might optimizing for speed mean that you're actively reducing support for associative learning?

Putting The Quality In Qualitative

Qualitative research studies can and should be designed and implemented in a rigorous manner. By putting careful thought into the sampling approach, data gathering methods, and the interpretation of results, you can minimize bias and maximize actionable learning.

Note that with qualitative studies, the responsibility for data analysis falls more heavily on the researcher than with quantitative studies. As Marie Hoepfl notes, "the credibility of a qualitative research report relies heavily on the confidence readers have in the researcher's ability to be sensitive to the data and to make appropriate decisions in the field."

You must also be careful not to obscure the qualitative nature of the data. For example, if you interview 10 users and 8 prefer one organization scheme, use the language "8 out of 10 preferred X" rather than "80% preferred X."

A Balanced Approach

I don't mean to suggest that quantitative studies have no place in information architecture design. On the contrary, a mix of qualitative and quantitative studies (known as triangulation) can be the best approach over the course of a project.

Qualitative studies early in the process can provide a direction and framework for subsequent quantitative research. These qualitative studies (e.g., user interviews, card sorting) are exploratory in nature. They allow you to learn why users do the things they do by talking directly with them. These studies can be done quickly and inexpensively with small user groups. The research methodology can be modified based on results, providing an important degree of agility within a dynamic environment. You can get tremendous bang for your buck out of this iterative design and testing approach.

Quantitative studies can then be employed to refine the strategy and to settle ties. For example, you can quantitatively test the predictability of three different labeling schemes across a large number of users and select the best labeling scheme with a high degree of confidence.

Ultimately, information architecture design involves a balance of art and science. You can't design information architectures in the real world based purely on scientifically-defensible quantitative research.

As an information architect, you will always be faced with the stress-inducing challenges of taking risks and making difficult decisions based on incomplete information. This is why you get paid the big bucks. Just stay away from the spicy beef and the H. Pylori and you'll be ok.

End Notes

Did this article give you an ulcer?

Please send your rants and raves to Peter Morville.

Subscribe to our bi-weekly newsletter for notification of new articles.

If you'd like to bookmark this column use this and if you'd like to bookmark this article use that.