Lets begin

The Apache Solr 4.0 has changed slightly how fl parameter can be used. In 4.0 this parameters can be added to the query multiple times and Solr will take all the values into consideration. Sometimes it will be useful, at least in my case.

Custom field names

In Solr 4.0 you will be able to rename fields that are returned in the results. Imagine that, depending on the query we would like rename fields like price_en, price_pl or price_fr to price. In Solr 4.0, we can do this by placing the following query:

fl=price:price_pl

This will cause the field price_pl to be returned as price.

All fields with a common name start

If we want Solr to return the all fields whose name starts with the word price (useful for dynamic fields) all we will need to do is add the following parameter to the query:

fl=price*

Returning function values

The last functionality, which we will look at today, is the ability add the result of the function, as a field in the documents returned by Solr. Thus, in Solr 4.0 will have the option to add such values as sum of prices, or calculated distance between two points. Quite useful. To use this functionality You will have to add the appropriate function call to the fl parameter for example:

fl=*,stock:sum(stockMain,stockShop)

The above will result in Solr returning all the fields for the document (value *) and a field named stock, which will be the sum of two fields: stockShop and stockMain.

A few words at the end

In addition to the new features mentioned above, there is one more thing that is connected to the parameter fl – the DocTransformer. However I decided to leave it for a separate blog post.

Let’s start

The first example that comes to mind, perhaps because of the project on which I worked some time ago, it’s sorting on the basis of distance between two geographical points. So far, to implement such functionality was needed changes in Solr (for example, LocalSolr or LocalLucene). Using Solr 3.1 and later, you can sort your search results using the value returned by the defined functions. For example, is Solr, we have the dist function calculating the distance between two points. One variation of the function is a function accepting five parameters: algorithm and two pairs of points. If, using this feature, we would like to sort your search results in ascending order from the point of latitude and longitude 0.0, we should add the following sort parameter to the Solr query:

...sort=dist(2, geo_x, geo_y, 0, 0) asc

I suspect that the most commonly used values of the first parameter will be:

• 1 – calculation based on the Manhattan metrics
• 2 – calculation of Euclidean distance

A few words about performance

Everything is fine till now, but how it looks like in terms of performance ? I’ve made a two simple tests.

During the first test, I indexed 200 000 documents, every one of them consisted of four fields: identifier (numeric field), description (a text field) and location (two numeric fields). In order not to obscure the test results for sorting, I used one of the simplest functions currently available in the Solr – the sum function which sums two given arguments. I compared the query time of the default sorting (by score) with the ones which used the value of the function. The following table shows the results of the test:

Another test was based on a comparison of sorting by a string field to sort using function. The test was almost identical to the first test. I’ve indexed 200,000 documents indexed (with additional field: name_sort – type string) and used the sum function. The following table shows the results of the test:

Above test shows that sorting using the sort function is much slower than the default sort order (which you’d expect). Sorting on the basis of function value is also slower than sorting with the use of string based field, but the difference is not as significant as in the previous case.

A few words at the end

Of course, the above test just glides through the topic of sorting efficiency using Solr functions, however, shows a direct relationship. Given that, in most cases, this will not be the default sort method and giving us a really powerful tool it seems to me that this is a feature worth remembering. It will definitely be worth using when the requirements says that we have to sort on the value that depends on the query and index values – as in the case of sorting by distance from the point specified by the user.