way? SSA will also give you tips on how to shore up your site’s navigation and metadata. (We cover this in Chapter 9.)
Content: For example, you can study queries that retrieve zero results. Is this because there isn’t content on the topic? Should there be? Or is the relevant content mistitled? Or poorly written? SSA will help you determine what content is missing and what to do to existing content to make sure it gets found. (We cover this in Chapter 10.)
Whatever design challenges you face, SSA—like any other data analysis—will back up your design decisions with actual facts.
Of course, as much as you’d like to make users happy, you also have to make your employers happy. They have goals—for your organization and for the site itself. (They ought to, at least.) These can be expressed and measured as KPI—Key Performance Indicators. The types of search-related metrics that you saw in Chapter 1 can serve as components to these KPIs—in fact, many organizations that are otherwise sophisticated in their measurement of performance often fall down when it comes to measuring findability. In Chapter 3, we’ll help you do what John Ferrara did: use goal-based analysis to measure, monitor, and optimize performance, again and again.
Finally, there are some other important ways to analyze search data:
Pattern analysis: What patterns emerge when you “play” with the data? Can you use those patterns to determine what types of metadata and content are the most important to your searchers? Can you detect changes in seachers’ behavior and needs that are seasonal? Do you also find instructive surprises and outliers? (We cover this in Chapter 3.)
Failure analysis: When searches return no results—or poor results—what can we learn? And what can we do to fix those problems and improve performance? (We cover this in Chapter 4.)
Session analysis: What happens during a specific search session? How do searchers’ needs and understanding of the content change as they search? (We cover this in Chapter 5.)
Audience analysis: How might we uncover the differences between audience segments and their information needs? And how might we better address those differing needs? (We cover this in Chapter 6.)
What Gets in the Way of SSA?
So you’re wondering: if SSA is so valuable, why don’t you hear more about it? And why haven’t you been taking advantage of it?
There are a few predictable and mostly mundane reasons, such as the following:
Lack of awareness: The idea has been around for years, but so was the Web before it took off. There’s simply a lack of critical mass behind SSA getting more attention; hence this book.
Technical hurdles: Your IT people might be too busy to write the scripts to parse your log files or even provide you with access to large and unwieldy data files. This is becoming less of an issue as organizations move toward using analytics applications to access the data; still, you might need a developer’s help in writing ad hoc queries.
Political hurdles: Your IT people might be too busy (or instructed not) to answer your phone calls. Or they might feel that anything related to search is their and only their responsibility (because many equate search with a search engine). There’s no simple solution here. Often, your best and only approach will be patience and persistence—just keep trying.
Legal hurdles: Lawyers often freak out any time someone wants access to user data—even if it’s for internal use—and issue blanket denials to requests for access. If you can get the attention of your legal department’s representative for even 30 seconds, explain to that person that you’re interested in analyzing the collective behavior of your site’s users, rather than digging into the habits of individuals.
Lack of data: Many sites—your personal blog, for example—likely don’t generate enough search activity to merit studying. In fact, they probably aren’t creating any sort of behavioral stream worth analyzing at all. That said, it’s still won’t hurt to poke into even a small data set, given that...
Lack of tools: ...the price of analytics tools is coming down. Way down. Like, free, thanks to Google Analytics (though you won’t be able to use a hosted service for your intranet). It’s not perfect, but it’s pretty useful, especially given the price. And if you’re working with simple data, Excel will do in a pinch.
But these barriers to taking advantage of SSA don’t explain why it’s still something of an unknown in most circles. So why has SSA fallen through the cracks?
Who Is Responsible for SSA?
Frankly, in smaller, less advanced organizations, SSA receives little or no attention. It’s just one of a few dozen non-urgent aspects of maintaining a Web presence—like meeting accessibility standards or keeping content fresh—that often gets pushed aside as assorted fires get put out. And when it does get done in these settings, it’s by a webmaster who already wears nine other hats.
In more advanced settings, where there are entire business units devoted to web analytics and user research, SSA still falls through the cracks. That’s because when SSA comes up, it seems just different enough from each unit’s existing daily responsibilities to assume that it’s someone else’s job. Why is that? It comes down to what people are comfortable with, and usually we’re comfortable with the familiar.
For example, web analytics people tend to prefer analyzing “cleaner” types of data—like conversion data—that have a more clear impact on the bottom line. (Monitor the Web Analytics Forum Yahoo Group for a week or two, and you’ll see what we mean.)[6] The successful conversion of a search is far more difficult to determine, much less measure, as language (and therefore searching) is so ambiguous. So, in a sense, the semantic richness of search query data is a double-edged sword—while the data might be quite interesting, it can be relatively difficult to analyze.
User experience people, on the other hand, tend to be less comfortable with numbers in general and data analysis in particular. They more typically rely upon qualitative analyses, where there are fewer expectations of conclusive, measureable outcomes and more is open to interpretation. And they may assume that analyzing data requires sophisticated expertise in statistical analysis. So, for UX people, SSA is usually on someone else’s table.
Let’s face it, in most situations today, SSA is no one’s job, but it should belong to someone (hence this book). Whatever your perspective—whether you’re a web analytics expert, a UX researcher, or a wearer of nine hats—you’ll want to have a clear picture of those top most common queries and how well your site is performing. And you’ll want to have that clear picture this month, next month, next season, and next year. Seeing SSA as part of your ongoing work (for example, 5% of your normal week) rather than as a one-off project (for example, a 12-hour assignment) will enable you to continually improve your site and make sure that it keeps up with the changes in its environment. The world around it changes, and like a living organism, your site must
