Mark: I have bachelor's and master's degrees in computer science from MIT, with some graduate work at the MIT Media Lab. In the spring of 1995, after graduation, I went to work for Yoyodyne Entertainment, an online game company and one of the first pure-play Internet companies. I was one of the first employees hired as the chief game designer. Eighteen months later, I departed Yoyodyne to start Creative Good in my apartment, where I still live, in the suburbs of New York City. (Yoyodyne was later bought by Yahoo.) The mission of Creative Good was, and still is, to make the Web easier to use -- so from day one I've tried to help companies make their web sites easier, simpler, more useful, more relevant, more valuable to their customers.

I single-handedly took Creative Good from, in 1997, a one-person company with no money, to, in 1998, a one-person company with almost no money. ;) But in those first eighteen months I had accomplished a few things: I had refined my vision of customer experience, gained an unlikely amount of press, established a happy client base, and partnered with Robert Seidman on the writing of "In Search of E-Commerce," my first report evaluating e-commerce sites. That report is now available free here: www.goodexperience.com/reports/isoe/

Around that time I met Phil Terry, who was at McKinsey, and who joined Creative Good in February '99 as my partner and the CEO of Creative Good. Eighteen months later (that time period again), Phil and I have grown Creative Good to 30 people in two offices (New York and San Francisco), 30,000 subscribers to our weekly e-mail newsletter (e-mail [email protected] to sign up), three free reports on e-commerce with tens of thousands of downloads, a popular site with resources and commentary, and a large and happy client base. And most importantly, we have hired and grown the most capable team I've ever seen in the Net industry.

So that's where Creative Good is today, in August 2000.

Lou: Tell us a little more about Creative Good; what kinds of work does the company do? What are some of Creative Good's success stories?

Mark: The mission of Creative Good is to make the Web easier to use. Our longer-term mission is to apply our vision to any and all digital technologies. We're already doing work in wireless, and we plan to do work in other technologies as opportunities arise. But the goal is always the same: simplify the technology, make it easy for the customer, and always create something good.

We have a particular term for what we do: we say that Creative Good creates a good "customer experience." The customer experience is the combination of everything the customer sees, touches, reads, interacts with, or otherwise engages. The customer experience is a holistic combination of everything on a web site (or wireless app, or PDA interface, or whatever).

By taking a holistic view of the experience, our work is extremely effective; way more effective than if we just focused on one aspect of the experience, like the "usability" or, if you'll allow me, just the "information architecture."

In our projects, we advise clients how to improve their e-business by improving the customer experience. We first create a strategy, informed by industry analysis, organizational work, and user tests (what we call "listening labs"), then finish by creating prototypes for the new site. We do no implementation, instead leaving that to the client's internal resources or a third-party agency.

A Creative Good project typically raises the client's numbers by 100%. So we usually double the sales, or conversion rate, or whatever it is the client wants to improve. I don't know of any other methodology, or any other company, that can deliver such consistently dramatic improvements.

Lou: I want to probe further about customer experience, but if I don't challenge your claim of 100% improvement first, the readers will never forgive me: How exactly is it that you know you double your clients' numbers? What sorts of science do you use to measure and validate your results? It seems like you face the nearly impossible task of quantifying or completely removing other factors, such as the exponential growth of Internet users or the results of a good marketing campaign.

Mark: Results that dramatic, tied to the day the changes launch, are hard to ignore. One recent client was monitoring a particular conversion rate on their site. For three months straight it held steady at a certain percentage. Marketing programs came and went, and the conversion rate remained the same. (Remember that Internet growth and marketing campaigns won't change the conversion rate, since if the on-site experience is bad, the same percentage of people will abandon the site.)

So Creative Good finished the project and delivered the prototypes, and the client's design team got to work implementing. A month later, the changes went live on the site -- and THAT VERY DAY the conversion rate doubled. Not a tiny five percent jitter -- but a huge step increase by 100%. During the months before the relaunch, the conversion rate was one constant number. The day our changes went into place, and during the following weeks, the conversion rate held steady at double the old rate. That's a pretty strong case for Creative Good -- which is why the client's CEO thanked us for doubling the number and promptly hired us again.

Lou: OK, back to customer experience: it sounds like one of those new-fangled "disciplines," like information architecture. How is it really any different from established fields like market research or, for that matter, usability engineering?

Mark: Apples and oranges. Market research takes surveys; there's certainly a place for that in the product development cycle. Some companies in the industry make plenty of money conducting online surveys and feeding clients interesting numbers. I don't think that methodology has much depth or value to it, but sure, once in a while a survey is interesting.

Usability engineering, having grown up in the age of software, incrementally improves the tactics of the technology interface -- which is a fine process, but more suited to software than to web sites or other experience-based technologies.

And information architecture... well, I guess we'll get to that one later in the interview. Flame suit on!

Customer experience, as I said before, recognizes the customer's engagement with Web technology as a holistic combination of factors. Our customer experience work is created for the Net only, not for consumer products or software or libraries or anything else. The other methods you listed were not born on the Net and therefore tend to be less effective.

Lou: But how can your work be holistic if it's geared toward the Net only? It seems that a holistic approach by definition would address other factors of the customer's experience. For example, I might use the LL Bean site with their catalog in hand. Or consult the guy in the next cube to help understand a benefits policy that I'm reading on the company intranet. Does customer experience acknowledge the broader array of actors that make up an information ecology?

Mark: My point in saying "holistic" is to differentiate customer experience from the other methods you mentioned. Usability engineering tends to be more narrowly oriented to the TASKS that the user might perform on the site; information architecture tend to be more narrowly oriented around the INFORMATION the user might try to find, and how that information is organized.

Customer experience focuses on everything the customer sees, touches, etc... what I mentioned above. That means we look at tasks, we look at information on the site, but we also look at the marketing and messaging, the merchandising, the use of graphics and other technologies, the relationship of the site to any competitors' customers may have seen -- and about twenty-two other factors. That's what I mean by holistic. Customer experience is broader than other methods, and (since the Web is a multi-faceted medium) it fits better on the Web than other methods do.

Lou: Is the field of customer experience an art or science? Or both? And a question similar to one we information architects constantly face: how do you measure improvements in customer experience? Can you point us to any tools or techniques?

Mark: I think the question gets at the old discussion between qualitative and quantitative work. The short answer is, customer experience is bigger than that question. It's like asking, "is running a company qualitative or quantitative?" It's both. Yes, there are qualitative methods to guide us: past experience, proven heuristics, things like that. And yes, there are quantitative tools: "running the numbers," monitoring certain metrics. But customer experience isn't defined by its tools. It's a way of seeing, a way of working, that draws on many different ideas and tools.

Incidentally, it's not important whether we call it "customer experience," or "user experience," or "the holistic view of customers and technology," or "flying squid." So long as we don't use words (like "usability" or perhaps "information architecture") that are already known to describe something different, the words aren't important. What's important is not the term but the thing it points to: the perspective that we take to creating technology.

This is what I'm trying to get across to the industry: the perspective that the industry takes to technology is mostly misguided, and customers suffer, and ultimately the technology industry suffers greatly in the long term. There is a better perspective that is GOOD for customers, and ultimately GOOD for the industry, that's new, that's better, that's sometimes painful to adapt to, but that creates good technology in the end. Whatever new label one chooses, this is the perspective that Creative Good stands for, and which only a few people in the industry seem to be able to see today.

Lou: When you spoke at last April's "Defining Information Architecture" conference in Boston, some of us came away with the feeling that you see customer experience as much broader than more recognized fields like usability engineering and information architecture. But if that's true, doesn't that mean a site can have lousy usability or a poor architecture and still provide good customer experience?

Mark: The question doesn't compute. The only gauge of success for a web site is, do customers have a good enough experience that they'll return, and encourage their friends to come, too? Usability and information architecture are wrapped up in that experience; but so is the marketing, the messaging, the graphics, the layout, the business plan, the tone of the writing, everything. This isn't software, where usability -- the efficiency of a user's task -- is the key determinant of success. This is the Web, where everything is wrapped up into one package! So if you ask, "If I take out usability, what happens to the experience?", you're viewing the site through the old lens of usability again. Instead, view the site from a different perspective -- from the angle of the customer experiencing it in all its aspects, all its contexts, and THEN ask, "Is this a good experience?" After you answer that, you can try to understand how usability and information architecture contribute to that experience.

Lou: I guess I'm still struggling to make that connection -- I'm trying to get a sense of how any sort of specialization fits into your broader practice of customer experience (assuming it does at all). Do you analyze a site from a CE perspective and then say "OK, here's an information architecture problem that we'll fix; there's a usability problem..." and so on? Do you have IA, UE and other professionals on staff to do such specific work? And what then would a customer experience professional actually do for a client?

Mark: Creative Good almost never hires people with formal training in information architecture or usability, unless they show an eager willingness to learn a new way to look at technology. No, we don't break things down into information architecture and usability problems; everything we see is a customer experience issue. We simply teach our employees to see technology through the eyes of a customer. Once our consultants go through our customer experience training program, they're ready to handle just about any important issue on a site. Given some time to gain experience, our team members handle these issues as well or better than formally trained information architects or usability professionals.

Lou: OK, here's a long (and even more contentious) question, so please bear with me. We both recently participated in a CHI-WEB thread that debated the value of how tabs are labeled on the Toys'r'Us web site.

The CHI-Web thread debated whether or not Pokemon and X-Men should have been included in the tabs. Yes, we all really do need to get a life...

I argued that mixing apples and oranges in the tabs probably confused users. You said that "groupings, like many information schemes, can be helpful, but they must be subservient to getting the right things in front of the user in ANY order, whatever order is best for the customer's goals." I don't know that many information architects would disagree, so I was surprised with your later contribution to the thread:

"Many information architects tend to base their work on a strict view of navigation elements-period. It's much more effective to take a holistic view of the customer experience, so that yes, we pay some attention to information architecture, but we're also flexible to take into account the merchandising strategy, or the marketing, or any number of inputs that IAs don't pay enough attention to."

Ouch! Is it really your experience that information architects are inflexibly focused on navigation at the expense of everything else? And if so, how is it that we can be so blind to all the other factors that impact users' ability to find the information they need?

Mark: What do you mean, "I don't know that many information architects would disagree"? ;) You disagreed explicitly in your first comment on the thread:

"We group things for a reason. But by mixing apples and orange so much, the Toys'r'Us main page seems to risk losing the value in any of its groupings... My gut is that this is a classic case of calling a bug a feature; I have huge respect for Creative Good, but I've got to disagree with them here."

Your point, as you put it, was that the page risks "losing the value in any of its groupings." Maybe so, but I'll sacrifice a grouping's value for the benefit of the customer's value. My point was that groupings aren't important, if there's some other organization (or disorganization) that makes the site better for the customer. In other words, the customer, not the information architecture, is the key.

Lou: Without any data to go on, I'm still not convinced that the Toys'r'Us approach really helps users, especially those who come to the site actually looking for stuff -- I'd bet that such users are better served by a more consistent "finding-oriented" architecture. But back to that thread: you seemed to imply that information architects follow (sometimes blindly) some set of rules. What the heck are those rules anyway, and why can't I find them?

Mark: Apparently, "don't risk the value of groupings" is one of those rules! ;)

More to the point, let's go to the source, the best book I know that describes information architecture online: "Information Architecture for the World Wide Web," by you and Peter Morville. Go ahead, link to that Amazon page!

As I said, this is a great book about information architecture, and I enjoyed reading it back in 1998. In particular, Chapters 7 and 8 contain helpful methods and ideas for site development. However, I don't agree with everything the book says. One quote in particular exemplifies the disconnect that information architecture has with the customer experience. This is from page 37:

"Because of this pervasiveness of hierarchy, users can easily and quickly understand web sites that use hierarchical organization models. They are able to develop a mental model of the site's structure and their location within that structure."

I respect you and Argus, but Lou, that just isn't true! I've sat through hundreds of user tests, and users do not form mental models of the site's hierarchical structure. Users aren't concerned with how a site is constructed, but instead are interested in how to fulfill THEIR OWN GOAL. Guess what that means: where appropriate, sites should break their hierarchy to fit whatever organization that customers find most useful. The customer experience, not the information architecture, is the ultimate guide to building a site.

If we take page 37 at its word, then on Toys'r'Us we group the tabs into all the nice and neat groupings; after all, that's what customers really like, to understand how the architects organized all the information, right? Nope. Good information architecture, but not the best customer experience.

On the other hand, if we base our site development on a customer experience-based perspective, sure, we'll base the site on a well-organized model, but we'll be more concerned in the end with what the customer will find most useful.

Lou: That reasoning (and, really, customer experience as a whole) seems to work for sites where experience is the goal, such as entertainment sites or retail sites in highly-competitive niches. But is such a customer-centered approach always appropriate? For a site that's research-oriented, users aren't always the best judges of whether or not they've found the content they need. They may be happy with what they found, but don't realize that they've missed some truly relevant (and possibly more accurate) content. But your customer-centered approach seems to suggest that it's OK for ignorance to be bliss.

Mark: The opposite case is more likely. I've seen how task-oriented usability works, and I've seen how our customer-oriented method works, and it's usability that often mistakes customers' failure for success. Here's how: the usability test moderator asks the user to complete Task A. The user completes Task A successfully, and so it goes down as a positive mark for the site. A customer experience-focused listening lab would have shown that the user didn't need or WANT to do Task A in the first place!

We've seen this in client engagements, too. One recent client conducted three, count them THREE, batteries of usability tests on its site. Those task-oriented tests showed that the site was performing well and in good shape. We ran one day of listening labs, where the customers set the context of the test, and immediately found strategic problems with the site. The client was surprised and happy to find that there were much bigger improvements to make, which traditional usability tests had completely overlooked.

Before I get flamed too badly, let me say that I do respect usability tests. After listening labs, usability tests are the most effective way to get customer-driven feedback on the site. I'd choose traditional usability tests before I'd run focus groups (which are almost completely ineffective for on-site feedback) or online surveys (which aren't much better than focus groups).

Lou: You've described the "page paradigm" as the most consistently accurate predictor of customer behavior on an e-commerce site. This sounds like an incredibly important factor; would you give us an overview of the concept and let us know how we can learn more?

Mark: This is a nice follow up to the previous question. The "page paradigm" -- again, call it what you want, but what's important is the thought behind it -- came from our user tests in preparation for our Holiday '99 report (free download at www.creativegood.com/holiday99). We noticed in tests that most online customers have a particular goal in mind, and they click on anything on a page that appears to take them closer to fulfilling the goal. If nothing on the page appears to do that, they click the Back button on the browser.

The implications of the page paradigm are fairly simple. Figure out what's important to your customers on each individual page, and then construct each page in a way that the most important things are most easily accessible. And delete everything that's not important. It's a great way to focus and simplify a site, page by page.

The page paradigm is also discussed in our most recent report, the Dotcom Survival Guide, which is also a free download, available here: www.creativegood.com/survival.

Lou: How do you see customer experience evolving in the coming years?

Mark: The Web-via-PC will become less important, relative to the rise in devices (cell phones, PDAs, and others) that access the Net. The customer experience will be increasingly holistic, made up of many more different interfaces and devices. The only constant in all this change will be the bit. The technology customer experience for the next ten years will be defined by the bit: its inherent properties, its seepage into all areas of life, its increasing numbers, and people's ability to engage all of that. I have developed a new perspective that allows people to engage and thrive within the bits: it's called "bit literacy." We have a training program at Creative Good that teaches bit literacy, and I plan to write a book on it within the next year.

Finally, if people want to stay abreast of developments in customer experience (and bit literacy), they should sign up for the free e-mail newsletter I send out every week or so: just e-mail [email protected]. We also publish much of our customer experience knowledge base, including my daily commentary on customer experience, at http://www.goodexperience.com.

Lou: Thanks very much Mark, I hope this was a good experience for you and for the readers as well. I'm looking forward to your book, not to mention our next CHI-Web wrasslin' match...

Mark: Thank you, Lou! Here, let me get you out of this headlock... ;)

Lou: (...mmmph...)