Who Uses the Keywords in Metadata?

We often hear that keywords are important to help readers find and discover books. But what does that mean, and do keywords actually make a difference? In this post, we look at how keywords are used to search book websites (in particular, online booksellers), and their adoption by publishers. For this investigation, I had help from Pat Payton (Bowker) and Catherine Toolan (Firebrand). We set out to answer the following questions:

• Are publishers adding keywords to book metadata?
• Are they providing quality keywords?
• Do online booksellers use keywords in their search engines?

In this post, “keywords” refer to consumer-oriented terms to describe a book that are added to an ONIX feed and sent to third parties. These terms aren’t seen by the public and are primarily used for search indexing. Conversely, web search engines (such as Google) don’t make use of ONIX keywords, but analyze the text of public webpages to create search indexes. As book content isn’t public, search providers rely on metadata to help consumers locate books.

Keywords Help Consumers Find Books

Most retailers solve the simple use cases of finding a book by title, author or category. Many searches, however, are comprised of natural language queries that describe different elements of a book, such as its setting, characters, theme or an emotional response to its content. Keywords were designed to fill this gap, by allowing people knowledgeable of the book to specify additional terms by which to find it.

Books are multi-dimensional, complex products that are typically highly nuanced and represent multiple buy trigger points for different types of consumers. Books have much more depth than, say, a kettle or a toothbrush, and determining the best keywords is therefore proportionally complex.

Note that extracting keywords from the book’s text is a naïve approach to solving this problem. The most effective keywords relate to a reader’s experience with a book, and the language she uses to describe it.

Are Publishers Adding Keywords to Their Books?

Bowker analyzed the keywords added to ONIX files from roughly 150,000 publishers, which included reprint and self-publishing service providers to university presses, trade, school and audio publishers. Of these publishers, about 23,000 (15.3 percent) had added keywords to at least one book. And of these, smaller publishers (less than 100 titles) typically had a higher percentage of keyword coverage than did larger publishers.

Over the past 10 years, though, publishers have increased the number of titles with keywords from approximately 25,000 to approximately 114,000, in 2015. But this number is still a very small proportion of all books available.

How Sophisticated Are Publishers’ Efforts to Choose and Maintain Keywords?

While keywords have been part of the ONIX standard for many years, they definitely rose in importance around 2013. As publishers had whole backlists without keywords, obtaining coverage was (and still is) a resource-intensive task. In order to achieve high coverage of keywords across a catalog, many publishers undertook a stopgap approach, adding other metadata to keywords (from title/subtitle, subject codes, contributors, product format, and audience), which are already available to search providers, and therefore are unlikely to help with search visibility. To improve keyword quality and to recommend against practices such as keyword stuffing, the Book Industry Standard Group (BISG) published the “Best Practices for Keywords in Metadata,” in 2014, to guide publishers on choosing effective keywords.

Keyword quality is still low today, though. One example from Bowker shows the use of the keyword “audiobook” (relating to form, not content) in just about 12,000 of the approximately 114,000 titles sampled from 2015.

Do Online Retailers Use Keywords?

Every book search implementation is proprietary, so the exact use of keywords is generally not public knowledge. It is possible, however, to determine whether keywords, when used as search queries, return the books they’re associated with in ONIX.

Kadaxis tested 13 websites that consume ONIX and provide book search, and found that only Amazon showed books returned in search results for keywords attributed to the book in ONIX.

Keywords are central to Amazon’s search capability across all its product lines. The site receives keywords of wildly varying quality from a huge number of product suppliers (from individuals to large companies), which means its capability for filtering, cleaning and incorporating keywords into a search index and mapping these to consumer search queries is sophisticated.

As the quality of keywords provided by publishers is generally low, it is a challenging endeavor for other websites, without this history and experience, to use the data as extensively.

Are Keywords Worth the Investment?

From the research above, Amazon is the only online bookseller making use of keywords today. If increasing sales of books on Amazon is important, then investing in keywords may be worthwhile. As most publishers aren’t adding keywords to their titles (and of those that are, the quality is typically low), there also appears to be a window of opportunity in which publishers can gain a ranking advantage in Amazon’s search by adding keywords to titles.

Conclusion

While some publishers (see here and here) are quietly providing effective, consumer-oriented keywords, most aren’t investing significant resources. But doing so might represent a low cost, low risk investment for a potentially strong, recurring return. At least until a better solution is created, that takes the onus of keyword curation away from publishers and authors.

Additional thanks to Chris Saynor from OnixSuite.

This article originally appeared on the DBW blog March 4, 2016