No 1, April 2008
Welcome to the first edition of the Funnelback bi-annual newsletter. In each newsletter we will aim to provide news about the latest Funnelback developments as well as some tips on how to further enhance search within your organisation. In this issue, Geoff Dibley of DAFF shares his experiences in building a highly effective intranet search implementation and we also provide insight into the very exciting and soon to be released Funnelback 8.
Brett Matson, CEO
Optimise your site to help searchers
Author: Dr. David Hawking, Funnelback's Chief Scientist
Follow the steps below to enable searchers to find what they're looking for.
Make it easy for search engines to crawl your site. Ensure all search-useful content is reachable via simple HTML links. (If it makes sense to have links in JavaScript, Flash, PDF, Word, etc., make sure they are all listed again in a simple HTML site map.). Use a robots.txt file to avoid search engines wasting their time crawling content with no search value.
Prevent your web server generating error pages such as "Your browser doesn't support JavaScript" (or frames etc.) when accessed by web crawlers. Instead, generate a text version of the page to ensure that your content is indexed.
Encourage people to link to your pages (those with search value). Publish stuff that people will want to link to (if possible). Only publish each thing once, using a single consistent URL (every time you change a URL, all the external links to it break). Use simple, meaningful URLs (other web authors don't like linking to long, complex URLs, particularly when they contain punctuation and parameters). Ask appropriate other sites to create links to your pages.
Encourage linking sites to use anchor text which matches queries which are important to your business. For example, 'ACME plastic cups' rather than just ACME or click here.
Make sure your web content complies with W3C standards. Use validation tools to check this.
Choose good titles for your pages - e.g. ACME plastic cups - about us.
Make sure that the language you use in titles, content, and anchor text matches the language which people are likely to use as queries. Governments often use more sophisticated names for things than do the searching public. For example, "permit to acquire a longarm" rather than "gun licence".
Monitor your query and web server logs to find out what queries people most commonly use.
Use thesaurus expansion and search short-cuts on your local site search but note that they are no help to external engines.
Improving Internal Site Search for Users
Author: Geoff Dibley
Web Systems Section, Dept. of Agriculture Fisheries & Forestry
Introduction
The redesigned Department of Agriculture, Forestry and Fisheries (DAFF) website (http://www.daff.gov.au) was released in mid 2007. The site uses the Funnelback Agency Search facility to provide an internal search facility across the entire DAFF website. During the later part of 2007, a full review of internal site search was conducted, using information derived from the Funnelback Agency Admin reporting facility.
Methodology
Monthly reports of internal search queries were extracted and analysed from the Agency Admin reporting tool for the period 1 July 2007 to 30 September 2007. This included information on top queries, and top queries returning zero search results.
The top 500 internal search queries were categorised into website topic areas. The top 40 search terms for each month were examined for changes over time.
A search "success" rate was compiled, based on the one hundred most popular query terms. The most relevant content page for each query was identified, and then tested using the Agency Search to see whether that page was returned as one of the top three search results.
Results
The number of search queries was found to be remarkably consistent from month to month (around 37,000). About 18,000 search queries resulted in someone clicking on a search result link (roughly half).
Most search queries correlated well with website analytics information (ie. the most popular searches related to the most popular content areas). However, there were some surprises with employment related searches (eg. jobs, positions vacant, vacancies etc.) making up over a quarter of the top 500 internal search queries.
Most search queries were simple - the overwhelming majority of the top 500 queries consisted of a single word.
The internal search success rate (ie. where the most relevant content page appeared in the top three search results) was around 60%.
The top search results were very consistent from month to month. New queries appeared in response to highly publicised national and international news events eg. equine influenza outbreak in Australia, foot and mouth disease outbreak in the UK.
Queries returning no results particularly related to spelling errors and content "gaps". In some cases, users were searching for information that was not present on the DAFF website.
A large portion of content on the DAFF website is stored as PDF or Word documents. These documents are often titled poorly (eg. "Appendix A"), making it hard for users to determine the nature of their content in the search results.
A relatively small number of queries account for a large proportion of the overall number of queries made (eg. the top 100 search queries account for 21% of all searches made).
Search Improvements
We changed from a single Agency Funnelback profile (with multiple scopes) to multiple profiles based on the key topic areas on the website (eg. Quarantine and Exports, Animal and Plant Health etc.). This allows users to continue to perform scoped searches on the DAFF website, but importantly allows DAFF to report individually on each profile.
In addition, it allows us to make customisations (eg. best bets), with users being directed to different content based on the topic area they are searching from. Early results indicate that around 20% of internal searches are based on a scoped search.
We implemented Best Bets (Featured Pages) for our top 100 search queries and also for those search queries that consistently returned zero search results. People won't trawl through more than 1 or 2 pages of results, nor will they reformulate query more than once or twice. Early results indicate that around 10% of users are now using the Featured Pages.
Query Expansions were implemented to correct common misspellings (eg. "empolyment" to employment). We are also following up with departmental areas to ensure that content gaps are addressed, and therefore reduce zero result search outcomes.
Internal search requires ongoing monitoring and reporting across the organisation, particularly in response to major events. Internal search reporting has been integrated with our other monthly website reporting mechanisms, allowing us to quickly communicate search information to the different organisational units, and ensure that search (and content) improvements (eg. new best bets) are added in a timely manner.
In summary, the Funnelback Agency search has allowed DAFF to easily report on and monitor internal site search, and has enabled us to make a number of adjustments to improve search for users on our website.
New Feature Highlight
If you're interested in any of the following:
- A fuller, richer TRIM search,
- Geospatial search - providing spatial content to search queries,
- e-commerce search using Funnelback faceting,
then contact sales@funnelback.com
New features in Funnelback 8.0
Author: Francis Crimmins, Manager, Research and Development, Funnelback
Funnelback v.8.0 is due for release later in the year.
Faceted Navigation - Funnelback 8.0 allows users to navigate search results using "facets", helping them refine their search and drill down to the content they need. The Funnelback 8.0 administration interface allows an administrator to define facets based on an organisation's pre-defined categories (e.g. fields in a database record), as well as metadata fields, URL structure and pre-defined queries. It acts as a complement to Funnelback's existing Contextual Navigation technology, which can be used when no pre-existing categories are present. Search the CareerOne website for examples of Faceted Navigation.
Improved Database Search - Database collections in Funnelback 8.0 now support incremental updates, which result in faster updates by avoiding gathering database content which hasn't changed. If the database has a single table or view with a primary key that increments in sequential order then Funnelback can request only new records that have been added. An alternative mechanism requires "triggers" to be set up in the database, which populate a table with a list of all the additions, updates and deletions to the database. Funnelback can use this table to update its own index.
Compressed storage is now available as an option for database collections, which can reduce storage requirements when indexing very large databases. The database collection now supports custom processing of individual database records during the update process, by allowing a developer to define a custom Java class which implements the processing.
Basic CJKT Support - This release of the product adds basic support for Chinese, Japanese, Korean and Thai (CJKT) character sets.
Feeds - The addition of "feeds" support allows other programs to "feed" data into the Funnelback index. This feature makes use of HTTPS POST requests, with feeds defined in an XML format. A program or script can add and delete content in the index, as well as adding data "inline" (i.e. including document content in the XML feed itself).
Crawl content behind web forms - The Funnelback 8.0 webcrawler has the ability to access content protected by HTML form authentication. For example, a university might have online course content which uses a HTML form to log students in. The webcrawler can be configured with appropriate access details and once the form has been submitted can then access the content for indexing and searching. An additional improvement is better support for extracting links from Javascript content in web pages. This will result in better coverage of sites which make heavy use of Javascript for navigation elements etc.
User Tagging - Organisations can allow users to "tag" search results in Funnelback 8.0 - Figure 3 shows an example result with tags which have previously been applied to this result. Clicking on the "tag" link will take a user to a page where they can login and then tag the page themselves. Clicking on a tag shows all pages to which this tag has been applied.
Author: Product & Services Team, Funnelback
Funnelback 7 has a range of powerful features to help make your search application more effective.
Tips for Administrators
Here are a couple of simple tools for customising Funnelback for your search needs. For more detailed information on any of these tips, please refer to your help documentation which is accessible from the Funnelback web administration interface.
Featured Pages allow you to quickly and easily specify particular results to be presented for specific search queries. This can be used to ensure important content appears at the top of the results, to highlight promotions or present alternative information.
Note: You can also customise how your Featured Page results will appear; as a highlighted call-to-action, appearing within your natural results or anything in-between.
Query Expansions can help you ensure that your search users are seeing all the information that you have available. Expansions allow you can set additional terms that will automatically be appended to the search phrases you nominate. For instance, if a user searched for "lawyer", their query could be seamlessly transformed to "lawyer barrister solicitor".
Note: You may like to view the reports on "Query terms that return zero full results" and create query expansions for commonly misspelled terms.
Tips for Search Users
Search for an exact phrase: Put quote marks ("") around your query. E.g. "telstra tower"
Search for a document title: Start your query with "t:" (without the quote marks) to apply it to titles. E.g. t:"To Kill A Mockingbird"
Use the plus operator "+" (without the quote marks) followed by the terms to return results with the term included. For example, search for +news +canberra to search for documents with both terms 'news' and 'canberra'.
Exclude results: Usethe negation operator"!" (without the quote marks) followed by the terms to exclude. For example, search for !news to search for documents without the term 'news'. Useful tip: Find the total document count in the index by using the negation operator (!) followed by text that does not exist in the index (e.g. !asdfgh).
If you have any Funnelback tips or tricks you'd like to share, please email us with your suggestions: media@funnelback.com
We hope you've enjoyed our 1st e-newsletter.
If you'd like to suggest how we can improve the Funnelback product or, you're interested in a search tuning and upgrade please call Ph:1300 65 58 52 or email: media@funnelback.com. We're always happy to hear from you.
Sincerely, The Funnelback Team
