Solr 4.0: Realtime GET

The next functionality I decided to look at, from the upcoming Solr 4.0, is the so called “Realtime Get”. It allows you to see the data even though it was not yet added to the index, thus before the commit operation being sent to Solr. Let’s see how it works.

Some theory

Data update in Lucene and Solr has one disadvantage – when you submit index updates they can’t be seen until commit operation is run. The problem is that commit is costly in terms of performance and intense commiting may cause performance problems. So, when you need your data to be visible right after being change you may be forced to choose – either performance, or fast updates. In order to address that Lucene and Solr are working towards enabling Near Real Time (NRT) searching. In Lucene we have that possibility, in Solr 4.0 we will also be able to use that and not only that.

Configuration

In order to use Realtime Get functionality we need to configure the following Solr features:

Transaction log

The first thing to configure is the transaction log writing. In order to do that you need to add the following to your updateHandler configuration:

The above entry says, that the directory holding transaction log will be located in the same directory where the index directory is located.

Realtime Get handler

The second thing that needs to be done, to see the Realtime Get in action, is the appropriate handler configuration (or adding component to your already defined handler). To do that add the following to your solrconfig.xml file:

The above entry it’s nothing unusual – it just add a new request handler implementing solr.RealTimeGetHandler class, which enables checking the transaction log.

Action

To check how Realtime Get works I decided to do a simple test. The first thing I did is indexing one file (from the ones that are available in the exampledocs directory) with the use of the following bash command:

Of course I did not send the commit operation after indexing. As we could expect the following query:

didn’t return search results. So let’s check, if the handler registered as /get will be able to get us some results. In order to do that I send the following query:

And in result I got the following document:

So Solr returned the result that wasn’t added to the index – nice !

Usage possibilities

You probably noticed, that in order to fetch a document with /get handler I needed to provide it’s unique identifier (or identifiers list). That’s true, Realtime Get doesn’t support searching, because it was not created to support full searching. This functionality is able to show us the updates of the documents which identifiers are known (so for example the ones in the index) – in example by adding the component used in solr.RealTimeGetHandler to any of your defined handler. And the good news is – you don’t have to worry update performance – solr.RealTimeGet is very fast. So, if one of your problems is frequent updated you can look in the future with a smile 🙂

Last few words

The Realtime Get functionality brings new possibilities when it comes to Solr and also on the road to the SolrCloud. With the use of transaction log one can implement automatic cluster node restore or instance NRT instance updates. As you can see Solr 4.0 is not only about search, but also about data store and bringing Solr closer to NoSQL solutions.

One thought on “Solr 4.0: Realtime GET

  • 23 December 2012 at 06:06
    Permalink

    does this mean I can access prior versions of a document? So say I have
    ID:1234
    Price:100 and then next month there is a price drop
    ID1234
    Price:80
    I can somehow access (and show visitors) that 1 month ago this product was 20$ more expensive?

    That would be great. Thanks.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.