{"id":554,"date":"2013-05-20T13:58:51","date_gmt":"2013-05-20T11:58:51","guid":{"rendered":"http:\/\/sematext.solr.pl\/?p=554"},"modified":"2020-11-12T13:59:24","modified_gmt":"2020-11-12T12:59:24","slug":"solr-4-2-index-structure-reading-api","status":"publish","type":"post","link":"https:\/\/solr.pl\/en\/2013\/05\/20\/solr-4-2-index-structure-reading-api\/","title":{"rendered":"Solr 4.2: Index structure reading API"},"content":{"rendered":"

With the release of Solr 4.2 we’ve got the possibility to use the HTTP protocol to get information about Solr index structure. Of course, if one wanted to do that prior to Solr 4.2 it could be achieved by fetching the schema.xml<\/em> file, parsing it and then getting the needed information. However when Solr 4.2 was released we’ve got a dedicated API which can return the information we need without the need of parsing the whole schema.xml<\/em> file.<\/p>\n\n\n\n\n\n

Possibilities<\/h3>\n

Let’s look at the new API by example.<\/p>\n

Getting information in XML format<\/h4>\n

Many Solr users are used to getting their data in the XML format, at least when using Solr HTTP API. However, the schema API uses JSON as the default format. In order to get the data in the XML format in all the below examples, you’ll need to appeng the wt=xml<\/em> parameter to the call, for example like that:\n<\/p>\n

$curl 'http:\/\/localhost:8983\/solr\/collection1\/schema\/fieldtypes?wt=xml'<\/pre>\n
Defined fields information<\/h4>\n
Let’s start by looking at how to fetch information about the fields that are defined in Solr. In order to do that we have the following possibilities:<\/p>\n
    \n
  1. Get information about all the fields defined in the index<\/li>\n
  2. Get information for a one, explicitly defined field<\/li>\n<\/ol>\n
    In the first case we should use the following command:\n<\/p>\n
    $curl 'http:\/\/localhost:8983\/solr\/collection1\/schema\/fields'<\/pre>\n
    In second case we should add the \/<\/em> character and the field name to the above command. For example in order to get the information about the author<\/em> field we should use the following command:\n<\/p>\n
    $curl 'http:\/\/localhost:8983\/solr\/collection1\/schema\/fields\/author'<\/pre>\n
    Solr response for the first command will be similar to the following one:\n<\/p>\n
    {\n  \"responseHeader\":{\n    \"status\":0,\n    \"QTime\":1},\n  \"fields\":[{\n      \"name\":\"_version_\",\n      \"type\":\"long\",\n      \"indexed\":true,\n      \"stored\":true},\n    {\n      \"name\":\"author\",\n      \"type\":\"text_general\",\n      \"indexed\":true,\n      \"stored\":true},\n    {\n      \"name\":\"cat\",\n      \"type\":\"string\",\n      \"multiValued\":true,\n      \"indexed\":true,\n      \"stored\":true},\n    {\n      \"name\":\"category\",\n      \"type\":\"text_general\",\n      \"indexed\":true,\n      \"stored\":true},\n    {\n      \"name\":\"id\",\n      \"type\":\"string\",\n      \"multiValued\":false,\n      \"indexed\":true,\n      \"required\":true,\n      \"stored\":true,\n      \"uniqueKey\":true},\n    {\n      \"name\":\"url\",\n      \"type\":\"text_general\",\n      \"indexed\":true,\n      \"stored\":true},\n    {\n      \"name\":\"weight\",\n      \"type\":\"float\",\n      \"indexed\":true,\n      \"stored\":true}]}<\/pre>\n
    On the other hand the response for the second command would be as follows:\n<\/p>\n
    {\n  \"responseHeader\":{\n    \"status\":0,\n    \"QTime\":0},\n  \"field\":{\n    \"name\":\"author\",\n    \"type\":\"text_general\",\n    \"indexed\":true,\n    \"stored\":true}}<\/pre>\n
    Getting information about defined dynamic fields<\/h4>\n
    Similar to what information we can get about the fields defined in the schema.xml<\/em> we can get the information about dynamic fields. Again we have to options:<\/p>\n
      \n
    1. Get information about all dynamic fields<\/li>\n
    2. Get information about specific dynamic field pattern<\/li>\n<\/ol>\n
      In order to get all the information about dynamic fields we should use the following command:\n<\/p>\n
      $curl 'http:\/\/localhost:8983\/solr\/collection1\/schema\/dynamicfields'<\/pre>\n
      In order to get information about a specific pattern we append the \/ <\/em>character followed by the pattern, for example like this:\n<\/p>\n
      $curl 'http:\/\/localhost:8983\/solr\/collection1\/schema\/dynamicfields\/random_*'<\/pre>\n
      Solr will return the following response for the first query:\n<\/p>\n
      {\n  \"responseHeader\":{\n    \"status\":0,\n    \"QTime\":2},\n  \"dynamicfields\":[{\n      \"name\":\"*_coordinate\",\n      \"type\":\"tdouble\",\n      \"indexed\":true,\n      \"stored\":false},\n    {\n      \"name\":\"ignored_*\",\n      \"type\":\"ignored\",\n      \"multiValued\":true},\n    {\n      \"name\":\"random_*\",\n      \"type\":\"random\"},\n    {\n      \"name\":\"*_p\",\n      \"type\":\"location\",\n      \"indexed\":true,\n      \"stored\":true},\n    {\n      \"name\":\"*_c\",\n      \"type\":\"currency\",\n      \"indexed\":true,\n      \"stored\":true}]}<\/pre>\n
      And the following response will be returned for the second command:\n<\/p>\n
      {\n  \"responseHeader\":{\n    \"status\":0,\n    \"QTime\":1},\n  \"dynamicfield\":{\n    \"name\":\"random_*\",\n    \"type\":\"random\"}}<\/pre>\n
      Getting field types<\/h4>\n
      As you probably guess, in a way similar to the above describes examples, we can also get the information about the field types defined in our schema.xml<\/em> files. We can fetch the following information:<\/p>\n
        \n
      1. All the field types defined in the schema.xml<\/em> file<\/li>\n
      2. A single type<\/li>\n<\/ol>\n
        To get all the defined field types we should run the following command:\n<\/p>\n
        $curl 'http:\/\/localhost:8983\/solr\/collection1\/schema\/fieldtypes'<\/pre>\n
        The get information about a single type we should again add the \/<\/em> character and append the field type name to it, for example like this:\n<\/p>\n
        $curl 'http:\/\/localhost:8983\/solr\/collection1\/schema\/fieldtypes\/text_gl'<\/pre>\n
        Solr will return the following information in response to the first command:\n<\/p>\n
        {\n  \"responseHeader\":{\n    \"status\":0,\n    \"QTime\":3},\n  \"fieldTypes\":[{\n      \"name\":\"alphaOnlySort\",\n      \"class\":\"solr.TextField\",\n      \"sortMissingLast\":true,\n      \"omitNorms\":true,\n      \"analyzer\":{\n        \"class\":\"solr.TokenizerChain\",\n        \"tokenizer\":{\n          \"class\":\"solr.KeywordTokenizerFactory\"},\n        \"filters\":[{\n            \"class\":\"solr.LowerCaseFilterFactory\"},\n          {\n            \"class\":\"solr.TrimFilterFactory\"},\n          {\n            \"class\":\"solr.PatternReplaceFilterFactory\",\n            \"replace\":\"all\",\n            \"replacement\":\"\",\n            \"pattern\":\"([^a-z])\"}]},\n      \"fields\":[],\n      \"dynamicFields\":[]},\n    {\n      \"name\":\"boolean\",\n      \"class\":\"solr.BoolField\",\n      \"sortMissingLast\":true,\n      \"fields\":[\"inStock\"],\n      \"dynamicFields\":[\"*_bs\",\n        \"*_b\"]},\n    {\n      \"name\":\"text_gl\",\n      \"class\":\"solr.TextField\",\n      \"positionIncrementGap\":\"100\",\n      \"analyzer\":{\n        \"class\":\"solr.TokenizerChain\",\n        \"tokenizer\":{\n          \"class\":\"solr.StandardTokenizerFactory\"},\n        \"filters\":[{\n            \"class\":\"solr.LowerCaseFilterFactory\"},\n          {\n            \"class\":\"solr.StopFilterFactory\",\n            \"words\":\"lang\/stopwords_gl.txt\",\n            \"ignoreCase\":\"true\",\n            \"enablePositionIncrements\":\"true\"},\n          {\n            \"class\":\"solr.GalicianStemFilterFactory\"}]},\n      \"fields\":[],\n      \"dynamicFields\":[]},\n    {\n      \"name\":\"tlong\",\n      \"class\":\"solr.TrieLongField\",\n      \"precisionStep\":\"8\",\n      \"positionIncrementGap\":\"0\",\n      \"fields\":[],\n      \"dynamicFields\":[\"*_tl\"]}]}<\/pre>\n
        In response to the second command Solr will return the following:\n<\/p>\n
        {\n  \"responseHeader\":{\n    \"status\":0,\n    \"QTime\":2},\n  \"fieldType\":{\n    \"name\":\"text_gl\",\n    \"class\":\"solr.TextField\",\n    \"positionIncrementGap\":\"100\",\n    \"analyzer\":{\n      \"class\":\"solr.TokenizerChain\",\n      \"tokenizer\":{\n        \"class\":\"solr.StandardTokenizerFactory\"},\n      \"filters\":[{\n          \"class\":\"solr.LowerCaseFilterFactory\"},\n        {\n          \"class\":\"solr.StopFilterFactory\",\n          \"words\":\"lang\/stopwords_gl.txt\",\n          \"ignoreCase\":\"true\",\n          \"enablePositionIncrements\":\"true\"},\n        {\n          \"class\":\"solr.GalicianStemFilterFactory\"}]},\n    \"fields\":[],\n    \"dynamicFields\":[]}}<\/pre>\n
        As you can see, the amount information is nice as we are getting all the information about the field types and in addition to that the information which field are using give field (both dynamic and non-dynamic.<\/p>\n
        Retrieving information about copyFields<\/h4>\n
        In addition to what we’ve discussed so far we are able to get information about copyFields section from the schema.xml<\/em>. In order to do that one should run the following command:\n<\/p>\n
        $curl 'http:\/\/localhost:8983\/solr\/collection1\/schema\/copyfields'<\/pre>\n
        And in response we will get the following data:\n<\/p>\n
        {\n  \"responseHeader\":{\n    \"status\":0,\n    \"QTime\":1},\n  \"copyfields\":[{\n      \"source\":\"author\",\n      \"dest\":\"text\"},\n    {\n      \"source\":\"cat\",\n      \"dest\":\"text\"},\n    {\n      \"source\":\"content\",\n      \"dest\":\"text\"},\n    {\n      \"source\":\"content_type\",\n      \"dest\":\"text\"},\n    {\n      \"source\":\"description\",\n      \"dest\":\"text\"},\n    {\n      \"source\":\"features\",\n      \"dest\":\"text\"},\n    {\n      \"source\":\"author\",\n      \"dest\":\"author_s\",\n      \"destDynamicBase\":\"*_s\"}]}<\/pre>\n
        The future<\/h3>\n
        In Solr 4.3 the described API was improved and is now being prepared to enable not only reading of the index structure, but also writing modifications to it with the use of HTTP requests. We can expect that feature in one of the upcoming versions of Apache Solr, so its worth waiting in my opinion, at least by those who needs it.<\/p>\n
        W Solr 4.3 opisywane API zosta\u0142o usprawnione oraz jest przygotowywane do umo\u017cliwienia zmian w strukturze indeksu za pomoc\u0105 protoko\u0142u HTTP. Mo\u017cemy zatem spodziewa\u0107 si\u0119, i\u017c w jednej z kolejnych wersji serwera wyszukiwania Solr otrzymamy mo\u017cliwo\u015b\u0107 \u0142atwej zmiany struktury indeksu, przynajmniej takich, kt\u00f3re nie b\u0119d\u0105 powodowa\u0107 konflikt\u00f3w z ju\u017c zaindeksowanymi danymi.<\/p>","protected":false},"excerpt":{"rendered":"
        With the release of Solr 4.2 we’ve got the possibility to use the HTTP protocol to get information about Solr index structure. Of course, if one wanted to do that prior to Solr 4.2 it could be achieved by fetching<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[27],"tags":[514,513,195,182,515,178,164,201],"class_list":["post-554","post","type-post","status-publish","format-standard","hentry","category-solr-en","tag-4-2-2","tag-api-2","tag-index-2","tag-schema-2","tag-schema-api","tag-schema-xml-2","tag-solr-2","tag-structure"],"_links":{"self":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts\/554","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/comments?post=554"}],"version-history":[{"count":1,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts\/554\/revisions"}],"predecessor-version":[{"id":555,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts\/554\/revisions\/555"}],"wp:attachment":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/media?parent=554"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/categories?post=554"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/tags?post=554"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}