Why a 500 keyword character limit is costing you book sales.

I bet that if you’re reading this, you’ve always accepted 500 characters as the gold standard for maximizing book visibility through search - work your way up to 500 characters and you’ve joined the metadata elite. You've surpassed the majority of publishers who add only two or three hundred keywords, or gasp, don’t add keywords at all. Well, dear reader, if your ONIX makes its way to Amazon, and Amazon is an important retailer for your sales, I have some news for you:

If you only send 500 keyword characters to Amazon, you’re probably missing out on sales.

Contrary to what you may have been told, Amazon accepts and uses far in excess of 500 characters per book. In this post we’ll uncover the origin of the 500 character limit, and provide actual hard data that shows Amazon does indeed accept keywords far beyond the 500 character limit. (In case you’re wondering why more keywords are better, the short answer: more keywords = more chances for a book to be found in search = more sales opportunities. A longer explanation is available here).

The "Best Practices For Keywords" Recommendation

The BISG’s metadata keyword best practice standard states a recommended limit of 500 characters. The guide is a fine document, and I highly recommend you give it a read in case you haven’t come across it before. I was a member of the committee who authored the document and contributed to it's content and discussions. If you’re tasked with “creating keywords for online retailers”, you should absolutely use this resource as a guide. When it comes to keywords destined to Amazon, it’s worth noting that: Amazon wasn't involved in authoring the standard, and the “best practices” are a general recommendation to the industry as a whole. They’re not prescriptive, and retailers can implement anything they wish to. In fact the committee, correctly, ensured they weren’t too Amazon-centric in their recommendations. Long story short - even though 500 characters is the “recommended” limit, it doesn’t require Amazon or any other retailer, to strictly abide by the recommendation.

Where Did The 500 Character Limit Come From?

I asked this question on the committee and the best answer I received was: “it’s always been in the ONIX standard”. As someone who has lead teams to build book search engines, I wanted to understand what the technical rationale might be - was there some data science based research that underpinned this widely accepted “best practice”? My search led me to the good folks over at EDItEUR who oversee the ONIX standard globally. From my exchanges with these publishing metadata veterans, I learned that:
- The keyword character limit as it’s defined in ONIX, has always been a “suggested limit”.
- The ONIX standard places no limit on how many keyword characters an ONIX sender can transmit.
- Publishers should rely on receivers (such as retailers) accepting at least 500 characters.
- The limit used to be 100 characters and was then raised to 250 before the current suggestion of 500.
- The 500 limit considers old library systems that may not have the technological resources of retailers, and therefore are limited in what they can receive.
Again, a sensible limit to accommodate myriad publishing systems of various sophistication levels and vintage. But also, still nothing prescriptive about what a receiver (retailer) should do with a keyword field, other than accept at least 500 characters.

It's Easy To Test The Limit, But You Have To Do It Properly

Search technology is complex and doesn’t operate in the same manner as say, a bank transaction system. If you login to your online banking website and send money to someone, they better receive it. Every cent needs to be managed, accounted for and then auditable by all parties involved.  Search engines are somewhat different. When a search engine indexes data, while it may read everything available to it, the use and visibility of the source data comes down to a whole host of factors which is far beyond the scope of this post. Suffice to say that if you supply Amazon with say 100 keywords, it will read all of them, but won’t necessarily use all of them for your book (understand more about this here). If your character length test involves typing in each of your keywords into Amazon search, paging through countless results to check for your book, then using the absence of your book in results as evidence that the keyword wasn’t indexed - you’re doing it wrong. (We haven’t even touched on partial term matches). Each keyword is an opportunity, not a guarantee.

Cold, Hard, Data: How Many Keyword Characters Should I Provide?

Running a test to correctly identify whether a keyword improves search rank, involves analyzing hundreds of books, removing words found in the title, author names and category (BISAC/browse node) names from test queries and also searching for combinations of individual keyword terms (as a metadata expert, you already know that exact keyword matches are the tip of the search iceberg). We did all of this and here’s what we found:

Amazon indexes at least 1500 keyword characters from the keyword field in an ONIX file (or uploaded directly using Amazon's internal tools).
Kadaxis clients receive up to 1500 keyword characters, so this is the limit we tested. (On average, our clients receive an average of 1000 keyword characters per book). We found books matching search queries all the way up to the high 1400s character count after stripping away other metadata data we know is indexed. This approach gives us confidence that the book’s presence in search is attributed to the keyword and not other sources (like BISAC names).

30-50% of searches matched keywords were found in the 500-1500 character range
Said another way: if you’re only adding 500 keyword characters, your book is missing out on matching to 30-50% of search queries, than if you’d used 1000+ characters.
 
Books with 1000-1500 keyword characters match 67% more search queries, compared to books with 500 characters or less.
One of our tests compared 100 books from two trade publishers - one with Kadaxis keywords, and one who used an alternative service that maxes out at 500 characters. Both sets included a mix of good selling fiction and non-fiction titles and were run through identical measurement systems. The Kadaxis publisher had an average of 1098 keyword characters (max 1500) which matched to 67% more search queries than the publisher who added an average of 446 keyword characters (max 500).

Let's look at a couple of examples to illustrate further:

Title: Medical-Surgical Nursing Made Incredibly Easy (Incredibly Easy! Series)

This medical text matched numerous keywords in search, but one example of note is the keyword phrase "advanced pathophysiology". Neither of these terms are found anywhere in the book's metadata (as an aside, while this keyword is also not in the description text, know that the description isn't indexed for Amazon search, but that's a post for another day).

The keyword itself is present in the keyword field at character position 964 (out of 1079 total keyword characters). We can find the book in the Books search engine on Amazon for the search query "advanced pathophysiology, wedged between two other pathophysiology books. Note, the term is relevant for people interested in the topic, as evidenced by review mentions.

Let's take a look at one more example:

TITLE: The Greatest Story Ever Told--So Far: Why Are We Here?

Again, this title matches numerous keyword derived search queries, but we'll focus on one keyword: "heisenberg uncertainty", which refers to Werner Heisenberg's Uncertainty Principle. This keyword isn't mentioned anywhere in the book's metadata, but is mentioned several times by readers, including examples where "The Greatest Story Every Told" helped readers to better understand Heisenberg's Uncertainty Principle.

The keyword "heisenberg uncertainty" is present in the book's keyword field at character position 1101, and the book is found in the search results among related titles.

A few other examples, from hundreds in our set:

• Blockchain Revolution: How the Technology Behind Bitcoin Is Changing Money, Business, and the World. Keyword with search match: "smart contracts", character position 1142.
• The Perfect You: A Blueprint for Identity. Keyword with search match: "neuroscience", character position 1267 (actually matches 5 search queries about neuroscience).
• Darkfever (Fever Series, Book 1). Keyword with search match: "male characters", character position 1095.

What does this all mean? More keywords equals better search visibility, which equals more chances to sell (after all, most books are sold through Amazon search).

Keyword ROI: It’s Worth It

1000 characters is a lot of work, 1500 characters even more so, but compared to the rest of the effort and expense that goes into making a book a success, it’s a comparably small investment for the potential upside - especially when you consider the compounding value better search visibility earns a book. If you have questions about how we conducted our tests, how you can replicate our results or to learn more about our keyword services, please get in touch via the Contact form.

Thank you to the folks at Firebrand (Catherine Toolan, Steve Rutberg and Joshua Tallent) for their help with this article.

Measuring Keyword Effectiveness on Amazon

So you've found the perfect keywords for a book, how do you know if they're effective? Off-page keywords aren't visible to potential customers, so assessing whether they'll work or not takes a completely different approach to assessing visible metadata (title, description, etc.) about a book. The purpose of a keyword is to help a search engine return a book to a customer in a set of search results, in response to a search query. If a keyword doesn't achieve this one task, then it is of no value to you. It doesn't matter how beautifully descriptive or categorically accurate a keyword is, if it doesn't help your book show up in search, then it is worthless. Public data, such as categories or subtitles are dual purpose - they can improve your book's search presence and also help to convince a searcher to buy your book.

With an understanding of an off-page keyword's purpose - how do measure it's effectiveness on Amazon search? Amazon doesn’t share much data in the way of search metrics and even the data in Amazon's Retail Analytics (ARA) is pretty limited. But it’s still possible to measure whether a book has poor or good search visibility by analyzing public data. The most rudimentary method is to type every keyword into search, and see if the book appears, or ranks, in the search results. If it doesn’t show up in the results, then it has no visibility for that search query and it's effectiveness is nil. It’s pretty simple.

If a book does show up for the keyword, where does it rank? By rank, we mean the position it holds in the search results. Is it number one? Number 5? Does it show up on the first page, second page? Search rank can have a huge impact on conversion. Customers are significantly more likely to click on the first few results of a search query, than further down the page. Techniques such as Discount Cumulative Gain to measure search quality are predicated on this assumption.

Beyond simple one-to-one keyword to search query testing, to get a true appreciation of a keyword’s impact, you also need to test for derived keyword combinations. As an example take these three key phrases: “ya romance”, “contemporary romance” and “thriller”. These keywords will also match to the search queries: “ya contemporary romance”, “romance thriller”, “contemporary romance thriller” and so forth. Books are also matched against search queries that only partially match keywords. Using the same example, the book might also match “romance suspense” and “ya paranormal romance” even if “suspense” and “paranormal” weren’t specified as keywords for the book. To truly figure out how well a book ranks in search, requires figuring out all of these combinations, then running searches for each of them.

Once you find a match for your book, you then need to calculate how valuable the search query is, as they’re not all equal. Ranking in the top 5 for “romance” will generate more traffic for a book than ranking in the top 5 for “cozy beach romance set in florida”. The latter search query is more specific and while it receives lower search volume, it may have higher conversion potential. Long-tail searches are more granular, specific and easier to rank for, and are likely to generate better leads. Publishers almost never include them.

On Amazon, the majority of customer searches are long-tail - in fact a large percentage of searches have only been seen a couple of times or less. A significant percentage of these search queries on Amazon each month, have never been previously observed.

With the knowledge above, it's quite easy, albeit time consuming, to measure the effectiveness of a keyword for a book on Amazon (or any other search engine for that matter).

How A Keyword Sells A Book On Amazon

In order for a keyword to create a sale through Amazon search, two events need to occur:
1. The book needs to be present in the search results (or rank) for search queries matching the keywords added to the book.
2. The book needs to convince customers (or convert them) to click on a search result, add the book to their cart, then check out (or, make the purchase).

Each keyword creates an opportunity for sales to be made, as each keyword provides the Amazon search engine with an additional way to surface a book to customers. The more keywords assigned to a book, the more data the search algorithms have to work with. Not every assigned keyword will be matched to customer search queries, but you can think of each keyword as asking the search engine: have you thought about showing my book to customers who search for this query? It's up to the search engine to decide, but the more questions you ask, the more likely it will find more ways to return a book.

Behavioural Influence On Search Rank

Once keywords are ingested by Amazon, they’re subjected to a whole lot of parsing and processing, and testing with customers.

First, let’s remember that the goal of a retailer search engine is to maximise revenue, not help people find information (like on Google). This means that on Amazon, discoverability and sales are tightly linked. The rationale behind this is pretty straight-forward - if you show the books most likely to sell,  more often, you’re more likely to sell more books.

Every time Amazon displays a book to a customer in search, the customers’ action is logged. If the action is positive, such as a click on a link to view the product page, a cart add, a purchase or a read on Kindle, it positively impacts the books search rank. The more positive actions a book accrues, the more visibility it’s rewarded with. 

Conversely, if customers don’t respond in a positive manner to a book in a search result, then the book will eventually be outranked by other books that customers respond more positively to.

For example, if a book is shown in search, and the books above and below it are clicked, and not the book itself, this is regarded as a negative signal and will mean the book-query pair will be weighted downward. The book’s search rank for the query hasn’t encouraged positive behaviour so it is penalized.

A simpler way to think of this is - if book A sells more than book B when shown in search, keep showing book A ahead of book B, because book A is making more money. Let’s also try putting book C ahead of book B, and see if that book makes more money instead.

The ability to maintain and grow search visibility is dependent on the book converting through this funnel of customer actions. Sales are the ultimate goal for publishers and retailers, and are a strong search magnifier, but these sales are dependent on conversion.

All of these scores attributed to customer behaviour decay over time, which means momentum matters. It’s easier to improve the discoverability of a book that is already selling and converting well, using keywords, than it is to improve a book starting from low sales.

Conversion And Social Proof

We've all searched for books on Amazon before - you type in a search query and are presented with a list of books. Perhaps you were looking for an exact title, or perhaps you were searching for books about a topic - in either scenario, you might have been swayed to click on a search result if it looked interesting to you.

Think about the factors that persuaded you to click into the book's product page from search. If you were encountering the book for the first time, the only information you would have at that point was the data visible to you in the search result - cover, title, subtitle, price and importantly: the review count and average rating. These last two elements capture the social proof of the expected reading experience.

Social proof is one of the most powerful on-page converters, as customers often trust each other more than the organization trying to sell to them. Reviews are a book’s public record - they convey more value to a reader than whether book it has sold well, and let them know what other people thought about their experience (good or bad) and why.

Reviews are a permanent, mostly unbiased and irrefutable history of a book by it’s readers. And unlike search rank, the social proof from reviews is cumulative and compounds over time - it doesn’t really decay.

Once you have 500 reviews you’ll always have at least that many reviews. With an established positive star rating across a large base of reviews, it’s also highly likely future reviews and ratings will follow a similar pattern, further strengthening the book’s ability to convert.

In a future post, we'll analyse some data to see how social proof relates to search visibility on Amazon, including examples where Top and Recent customer reviews have been impactful. But in the meantime, thinking about a keyword's place and ability to influence a sale in the search funnel, will help you to focus on which element of metadata to prioritize to maximize sales potential.
 

How Do Keywords Impact Sales?

The question I receive most often from publishers is: “How do keywords impact sales?” While adding keywords to book metadata is considered best-practice, publishing businesses are naturally more interested in whether the practice will increase revenue. Keywords in this context are ‘off-page’ keywords, which are sent to retailers in an ONIX feed or added to a book via KDP, Amazon’s dashboard for Kindle books. Keywords aren’t visible to customers, but are indexed directly by retailer search engines (such as on Amazon), and allow publishers and authors to influence how readers find their books online.

At Kadaxis, we’ve added keywords to thousands of books, on behalf of a wide variety of publishers, and while some titles have seen significant short-term sales improvement, in most cases, publishers observe an average overall increase across a portfolio of titles over time. In this post we’ll cover the relationship between search traffic and sales, and outline how the title selection component of a keyword strategy can have an impact.

Keywords Direct Online Shoppers To Books

When purchasing a book online, a customer, can take many paths in a session of book browsing. We’ve isolated one path for discussion. A typical path a customer might follow involves:

• Typing a search query
• Viewing a list of search results
• Clicking on a book
• Viewing the book’s product information
• Making a purchase

Keywords can assist at the start of this flow, by helping books to appear in search results more frequently. But getting customers from search result to purchase is dependent on their previous exposure to the book, product information and other factors. Readers need to discover a book three times before they’re ready to buy, says Peter Hildick-Smith of the Codex Group, and ranking in search presents them with that option.

But the final responsibility to sell the book sits with the book’s product page. The stronger this page is the higher the likelihood of converting search traffic to sales. Some factors include and appealing title and cover, well-written descriptions, and positive customer reviews. A book with a bland, wordy description and a low count of negative reviews is unlikely to yield much return from adding keywords.

Sales Leads to Discoverability Leads to Sales

From a publisher’s perspective, a keyword’s core utility is to direct search traffic to books in the hope of selling more copies. If excellent, reader-focused keywords are assigned to a book, these keywords will only serve their function if the book appears in the search results of customers searching for books by those keywords. If the book doesn’t rank for those keywords, they are of no value.

So how do you determine whether a book will rank for its assigned keywords? The best predictor is sales. We consistently see a correlation between sales and the number of keywords a book ranks for: higher selling books also rank higher in search results. Generally, the more a book sells, and the more recently those sales occurred, the more discoverable it will be.

It can be insightful to examine the intent of different search providers when understanding how search works. Ecommerce retailers, such as Amazon, use search to sell products, whereas search companies, like Google, use search to help people find content. The focus on selling in retailer search can strongly influence how discoverable books become. (See also: How do Amazon and Google use my book metadata in search?).

For many reasons, products that have sold well in the past have a high chance of selling well in the future. Amazon exploits this phenomenon in search (and across their site), by boosting the visibility of higher selling books in an attempt to maximize sales. They understand that the odds of a sale are higher if a customer is presented with a popular item, so search results are reordered based on sales data (and other signals, such as page views and conversion rate). This means even the most well reasoned keywords might not have any impact for some books, but for others, they’re afforded the opportunity to rank for disproportionately more search queries.

Maximizing Return Through Title Selection

The myriad factors influencing search visibility, conversion and buyer sentiment, make it challenging to determine which books will benefit most from keywords. But since the endeavor is relatively low cost compared to rewriting jacket copy or updating a cover, and the possible return is high, the most prudent strategy to maximize ROI is to add keywords to a number of high potential titles.

Tying the concepts above together, this means selecting titles with:

• A high chance of converting: books with good publisher-provided metadata (to assist customers in their buying decision) and customer-created reviews and ratings (social proof).
• A high chance of ranking in search: typically books with a solid sales history, ideally performing above the competition, with recent sales valued more highly (or pre-promotion).

Recurring ROI

Titles that respond positively from keywords will experience increased sales over time, while maintaining search visibility and accumulating social proof, criteria which positively reinforce each other. But this can take time to build, and the rate of improvement varies for different genres, audiences, titles, and is heavily influenced by the prevailing zeitgeist of the moment. It’s not uncommon for titles to “tip” after several months of gradual improvement, which is why it’s best to adopt a medium to long-term outlook for any keyword strategy. But once the right keywords take effect, the return can persist long after the keywords were put in place.

As with most sound marketing strategies, keywords aren’t a silver bullet to an overnight improvement in sales. But when applied strategically across a quality catalog, they can significantly impact discoverability, leading to an ongoing recurring increase in sales over time.

This article originally appeared on the DBW blog May 22, 2017

What are off-page keywords?

In the world of publishing metadata, when we talk about keywords, we’re talking about structured off-page keywords, often sent in an ONIX file, from a publisher to a retailer like Amazon. The retailer indexes the keywords and matches them against customer search queries, in order to display relevant books to them. Keywords are made up of phrases used to describe a book and their purpose is to give a search engine clues about how to show a book to consumers. We call them "off-page", because the retailer uses them directly, and doesn't show them to customers, like they do with other book metadata such as the title or description.

Web search engines, such as Google, determine what content such as a web page is about, and also how people might search for the content. Off-page keywords put this burden on publishers or authors, who have the complex task of trying to understand how readers might search, then how a search engine will use the provided keywords.

A typical book search engine, that reads ONIX, will index various metadata fields, like the title, author, categories and so forth, data who’s primary purpose is to inform consumers about the book - it’s public data. It needs to be appealing and be constructed in a way that is optimized for a search engine to work with.

Conversely, the primary purpose of off-page keywords is to directly inform a search engine how to match a book against search queries. The intended audience is a machine, and the data is hidden from consumers - it is "off" the product "page". This private nature gives publishers a lot of freedom to test and experiment.

Here's an official, dry, textbook definition of keywords in publishing:
“Keywords are words or phrases to describe the theme or content of a book. They are assigned by the metadata creator to supplement title, author, description or other consumer facing data.”

While accurate, it leaves out the motivation behind why we use keywords at all.

On the surface, keywords are just a metadata element. But used properly, they can be a powerful discovery mechanism to capture a reader’s experience with a book, in a way that facilitates sharing that experience with others.

Creating effective keywords is an exercise in studying reader psychology and linguistics, requiring empathy and insight into how people communicate about books with each other. If you’re able to think and talk like your audience, you’re more likely to reach them.

Keywords are used to sell all kinds of products online, but creating them is probably toughest for publishers, as books are far more complex and subjectively experienced than other products, like toothbrushes or hair dryers. So figuring out which elements to express can be challenging.

How do search engines use keywords?

Search engines are just computer programs written to find information for us. We type a query, and the engine thumbs through large swathes metadata to decide what books to display.  The richer the metadata, the more search queries the book might match to.

A book with only basic metadata (title and author and so forth) will show up in fewer search results than the same book with 100 or even 50 good keywords. Every keyword you add is an opportunity to widen the search funnel, letting you suggest to the search engine another way consumers can find your book.

Most books are sold online, and most people find books through search (per Amazon). If you can improve a book’s visibility in search, you improve it’s likelihood of selling more copies. A recent study by Recode, a tech news website, found that more shoppers begin their product search on Amazon (55%) than Google (28%).

9 Common Keyword Mistakes

Coming up with relevant and effective keywords is hard! Keeping them up-to-date and optimized for then number of sales your book is currently making is even harder. Here are common mistakes we see authors make when implementing their keyword strategy on Amazon:

1.  Choosing keywords that are too broad

2. Not validating that a keyword is commonly used by customers

3. Choosing keywords without much traffic

4. Not monitoring keyword progress (checking search rank for a book)

5. Leaving keywords unchanged for a month or longer

6. Choosing keywords that are too competitive for their book

7. Repeating terms across keywords

8. Not aligning keyword strategy with external marketing activities (to capitalize on sales rank increases)

9. Not having a keyword strategy!