Searching is a very important elements in many of today’s website.  Almost every corporate website has some form of searching capability.  Tyndale’s corporate site is no exception.  Our Tyndale.com product search has come a long way.  The original search method was a very basic 1-dimensional search known as a stem searching using nothing but a “LIKE” query on our product title, product description, and author tables.  So for example a search for “Car” would yield the following results:

Baby and Child Care
Bible Quiz Cards
Mission: Caring 101
Christ in the Carols
The Scarlet Thread

As you can see, it wasn’t the most precise algorithm for a search, especially if you were trying to find a word that could be a sub-word within a larger word.  It worked fine as long as you search on a word or combination of words that were unique enough.  But this basically required the user to know exactly what they are searching for and typing in most if not the entire author name or title of the book.

I took the original algorithm and built on top of it by adding a few more logic turning it into a stem and exact match search engine.  The first thing I did that made a dramatic difference in terms of improving relevancy was by tweaking the part of the algorithm that diced up a search phrase and performed a stem search on each diced up word.   So a search for “Christ in the Carols” would be broken up into each word and a search would be performed on:  “Christ” “In” “The” “Carol” along with the entire phrase “Christ In The Carol”.

You can probably see the issue with this algorithm.  If you look closely, it is performing a stem search on two of the most common words and/or letter combination in the English language:  “in” and “the”.   This basically yielded back 99.5% of all our products.  And since a stem search is being performed on a description of the book, you can bet that the SERP (search engine result page) return all sort of unrelated products.  So delicate tweaks of when to break up words when searching, when to do a stem and/or exact search, and which database fields to searched on were made to the original algorithm and greatly improved relevancy and user expectations.

The other major tweak that was done to the algorithm is how to order the results in a way that the user would expect it.  To get this right, you would have to get into your user’s head and figure out why they are using your search (or at least the majority of them) and what kind of results they would expect back.  This is easier said than done.  Without spending a whole year and throw a room full of developers at it; the following assumptions were made that would at least satisfy the 80/20 rule.
-People would come to the site and have some knowledge basis of Tyndale’s product and type in a keyword containing partial author name, title, subtitle, or genre.

-People coming to the site would be searching new or upcoming products that they heard or saw from a marketing campaign.

-Most likely, people come to the site searching for a popular (bestsellers or hot title) book or an author that are currently or has been ranked on the New York Times rather than an old C-list title and an unknown author.

Using this assumption, books that have a bestseller indicator would naturally be floated to the top of the page, while books that are out-of-print would be near the bottom.   In addition, books that have a release date of within the last 6 months would be naturally higher than those that were released 6 years ago.  The fields to search on would be title, author, genres, ISBN, and series thus more logic were written to put search on those fields and eliminate search on product description and subtitle which caused a lot of irrelevant products to show up due to the fact that these fields containing a lot of common and irrelevant keywords which diluted the precision of the algorithm.
And now on to today’s Tyndale.com search.  The search you are using today was written from the ground up with features that aids in the user’s experience and improvement in the relevancy of the SERP.

1. Weight system that puts emphasis (“weight”) on certain keywords and database fields allowing for more accurate results.
2. Allow searches on our digital assets such as eBooks and downloadable audio (which is a nightmare since the metadata and backend systems to organize these products aren’t as sophisticated as their physical counterparts.
3. Keywords
4. Advanced Filters (Scale down results based on price, full-color, format, bindings, language, etc..
5. Paging with user friendly pagination system (The numbers at the bottom of a SERP allowing for easy page flipping)
6. Search based by language.
7. Bi-lingual.  You can search in Spanish if you want too!

The all-time favorite feature of the new search is the introduction to keywords.  The new Tyndale.com search allows users to type in keywords or phrases that do not exist in the product’s meta info such as author information, title, subtitle, description, etc..

For example you can misspell an author such as “brock taney” and get products written by Tyndale author “Brock Thoene.”  Or typing “christian football coach” and getting the upcoming Tony Dungy book “The Mentor Leader.”  At the time of this writing, Tyndale’s keyword database has over 55,000 keywords/phrases spanning almost 2,000 products. Thank you for those who have contributed to this system!

Another feature that was added to the search which is the core reason why the search returns much more relevant is what we coined as the weight system.  Basically in a nutshell, certain matches (stem or exact) yields are given preference over others.  For example, a match on ISBN, title, or author name would yield preference over a match on subtitle or general keywords.   This concept is simple in theory but very difficult to implement since there are countless of fields to set weights on, and then tweaking the weights so that each weight is proportionally set according to how relevant the field is according to the assumptions we made for the 80/20 rule that we set earlier.  I can write a 5 page paper on this feature alone, but it would be pretty boring, so I’ll save you the time.

So that’s basically the jist of the history of the Tyndale.com product search.  To give you an idea of the sophistication of the Tyndale.com search algorithm below is are some numbers to bring things into perspective.

Version 1 560 lines Self contained into 1 flat file No JavaScript or graphics. Queries 1 database

Version 2 803 lines 117 lines of JavaScript Self contained into 1 flat file Queries 1 database

Version 3 2715 lines 1098 lines of JavaScript (not including Spry Framework) Separated into 8 files + 1 XML file Queries 7 tables from 3 different database

Sarn
31 May, 2012 at 9:18 am
That’s ok with us!