{"id":527,"date":"2013-01-07T12:20:42","date_gmt":"2013-01-07T11:20:42","guid":{"rendered":"http:\/\/sematext.solr.pl\/?p=527"},"modified":"2020-11-12T12:22:27","modified_gmt":"2020-11-12T11:22:27","slug":"solr-4-1-solrcloud-multiple-shards-on-the-same-solr-node","status":"publish","type":"post","link":"https:\/\/solr.pl\/en\/2013\/01\/07\/solr-4-1-solrcloud-multiple-shards-on-the-same-solr-node\/","title":{"rendered":"Solr 4.1: SolrCloud – multiple shards on the same Solr node"},"content":{"rendered":"

We would like to discuss another new feature that will be a part of upcoming Solr 4.1 – the ability to place more than one shard of a given collection on a single Solr instance. As you may know this is not possible currently. So, lets look how this works by comparing Solr 4.1 to 4.0.<\/p>\n\n\n\n\n\n

In order to illustrate how this feature works I decided to see how the process of creating a new collection looks like. We will use a single Solr instance and our collection with be built of two shards.<\/p>\n

Solr 4.0<\/h2>\n

Solr.xml<\/h3>\n

What we need to do first is clean the solr.xml <\/em>file, so it doesn’t have any information about cores. Of course we should do that if we migrate from the earlier Solr version.<\/p>\n

Starting Solr<\/h3>\n

Now we need to run a single Solr instance with embedded ZooKeeper. We do that, by running the following command:\n<\/p>\n

java -DzkRun -jar start.jar<\/pre>\n
Preparing configuration<\/h3>\n
Before creating our collection we need to send all the needed configuration file to ZooKeeper. Assuming that we have Solr installed in \/home\/solrpl\/solr\/<\/em> directory and that we have our configuration files stored in  \/home\/solrpl\/configs\/collection1\/conf<\/em> directory I run the following script that is distributed with Solr 4.0:\n<\/p>\n
\/home\/solrpl\/solr\/cloud-scripts\/zkcli.sh -cmd upconfig -zkhost localhost:9983 -confdir \/home\/solrpl\/configs\/collection1\/conf\/ -confname collection1<\/pre>\n
Creating the collection<\/h3>\n
We should have our configuration files stored in ZooKeeper so now we can use the collections API to create our collection. In order to do the we run a query to Solr to the \/solr\/admin\/collections<\/em> endpoint with the action=CREATE<\/em> parameter that tells Solr that we want to create new collection. We also need to provide the name of the collection by adding the name=collection1<\/em> parameter. In addition to that we inform Solr that we want to have our collection divided into two shards (numShard=2<\/em>) and we don’t want any replicas (replicationFactor=0<\/em>). So the full request looks like this:\n<\/p>\n
curl 'http:\/\/localhost:8983\/solr\/admin\/collections?action=CREATE&name=collection1&numShards=2&replicationFactor=0'<\/pre>\n
Solr administration panel view<\/h3>\n
If you would repeat the above steps and look at the cloud view in Solr administration panel you would see something like this:
\n\"Solr<\/p>\n
Comments<\/h3>\n
As you can see Solr 4.0 didn’t place both shards on a single machine. The shard named shard1<\/em> was placed on the xubuntu-virtual<\/em> node, but the one called shard2 <\/em>was not assigned. Of course that would change if we had more nodes forming the cluster, but that’s not the point of this entry.<\/p>\n
Solr 4.1<\/h2>\n
Solr.xml<\/h3>\n
Similar to what we did with Solr 4.0 we start with cleaning the solr.xml <\/em>file. Of course we should do that if we migrate from the earlier Solr version.<\/p>\n
Starting Solr<\/h3>\n
We do exactly the same when starting Solr 4.1, so we run the following command:\n<\/p>\n
java -DzkRun -jar start.jar<\/pre>\n
Preparing configuration<\/h3>\n
Similar to what we did with Solr 4.0, we need to send our configuration to ZooKeeper. We do that by running exactly the same command as we did before:\n<\/p>\n
\/home\/solrpl\/solr\/cloud-scripts\/zkcli.sh -cmd upconfig -zkhost localhost:9983 -confdir \/home\/solrpl\/configs\/collection1\/conf\/ -confname collection1<\/pre>\n
Collection creation<\/h3>\n
Creating our collection will be a bit different this time. We send the same values of parameters like action<\/em>, collection<\/em> and numShards<\/em>. However we add a new parameter the maxShardsPerNode <\/em>one that specifies the maximum number of shards that can be placed on a single Solr instance (by default this value is set to 1<\/em>). In our case we want to have two shards on a single Solr node so we set this parameter to 2. In addition to that Solr forces us to have at least a single replica, so we need to set the replicationFactor<\/em> parameter to 1<\/em>. The whole query looks like this:\n<\/p>\n
curl 'http:\/\/localhost:8983\/solr\/admin\/collections?action=CREATE&name=collection1&numShards=2&replicationFactor=1&maxShardsPerNode=2'<\/pre>\n
Solr administration panel view<\/h3>\n
After all the above steps the cloud view in Solr administration panel looks like this:
\n\"solr_4.1_cloud\"<\/p>\n
Comments<\/h3>\n
As you can see, with Solr 4.1 we were able to create collection built of two shards and place both of them on a single Solr node. So if you need to have this kind of functionality you can wait for Solr 4.1 and be sure that it will be working.<\/p>","protected":false},"excerpt":{"rendered":"
We would like to discuss another new feature that will be a part of upcoming Solr 4.1 – the ability to place more than one shard of a given collection on a single Solr instance. As you may know this<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[27],"tags":[],"class_list":["post-527","post","type-post","status-publish","format-standard","hentry","category-solr-en"],"_links":{"self":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts\/527","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/comments?post=527"}],"version-history":[{"count":2,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts\/527\/revisions"}],"predecessor-version":[{"id":529,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts\/527\/revisions\/529"}],"wp:attachment":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/media?parent=527"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/categories?post=527"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/tags?post=527"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}