Continuing the overview of the filters included in Solr today we look at the PatternReplaceCharFilter.<\/p>\n
As you might guess the task of the filter is to change the matching input stream parts that match the given regular expression.<\/p>\n\n\n\n\n\n
You have the following parameters:<\/p>\n
The use of a filter is simple – we add its definition to the type definition in schema.xml file, for example:\n<\/p>\n
<fieldType name=\"textCharNorm\" class=\"solr.TextField\">\n <analyzer>\n <charFilter class=\"solr.PatternReplaceCharFilterFactory\" \u2026\/>\n <charFilter class=\"solr.MappingCharFilterFactory\" mapping=\"mapping-ISOLatin1Accent.txt\"\/>\n <tokenizer class=\"solr.WhitespaceTokenizerFactory\"\/>\n <\/analyzer>\n<\/fieldType><\/pre>\nPoni\u017cej przyk\u0142adowe definicji dla r\u00f3\u017cnych przypadk\u00f3w.<\/p>\n
Below are examples of definitions for different cases.<\/p>\n
Cut pieces of text<\/h3>\n
You just need to specify, in the pattern attribute, what we want to cut. Example:\n<\/p>\n
<charFilter class=\"solr.PatternReplaceCharFilterFactory\" pattern=\"#TAG\" \/><\/pre>\nwhich will suppress the content of the data elements: “#TAG”<\/p>\n
Text fragments replacement<\/h3>\n
A similar case to the one above, but we want to convert text to another.\n<\/p>\n
<charFilter class=\"solr.PatternReplaceCharFilterFactory\" pattern=\"#TAG\" replacement=\"[CENZORED]\"\/><\/pre>\nChanging patterns<\/h3>\n
The two above cases were trivial. What is the strength of this filter is handling regular expressions. (You use regular expressions, right?) The following example is simple – it hides all the numbers by turning them into stars. It also handles the numbers separated by hyphens, treating them as a single number.\n<\/p>\n
<charFilter class=\"solr.PatternReplaceCharFilterFactory\" pattern=\"(\\\\d+-*\\\\d+)+\" replacement=\"*\"\/><\/pre>\nText Manipulation<\/h3>\n
The replacement doesn’t have to be plain text. This filter supports references which allow you to refer to parts of the matched pattern. For details, refer to the documentation of regular expressions. In the following example, all multiplied characters are replaced with a single sign.\n<\/p>\n
<charFilter class=\"solr.PatternReplaceCharFilterFactory\" pattern=\"(.)\\\\1\" replacement=\"$1\"\/><\/pre>\nAdvanced Parameters<\/h2>\n
So far I have not mentioned the following parameters: blockDelimiters <\/em>and maxBlockChars<\/em>. If you look at the source code you would see that those parameters are related to the way the filter is implemented. CharFilter <\/em> operates on a single character, and pattern matching requires an internal buffer to read more characters. MaxBlockChars <\/em>allows you to specify the size of the buffer. You do not have to worry about it, if the pattern you defined, does not match piece of text larger than 10k characters). BlockDelimiters <\/em>can further optimize filling of the buffer. It can be used if the information in the analyzed field is somehow divided into sections (eg, it is a CSV, sentences, etc.). It is a text that informs the scanner, that a new section starts, therefore, parts matched in the previous section are no longer useful.<\/p>\n
Limits<\/h2>\n
An important limitation of the filter is that it directly manipulates the input data and does not keep information related to the original text. This means that if the filter removes a portion of the string, or add a new fragment, tokenizer will not notice that and the location of tokens in the original box will not be saved properly. You should be aware of that when using queries that operate on the relative positions of tokens or if you use highlighting.<\/p>","protected":false},"excerpt":{"rendered":"
Continuing the overview of the filters included in Solr today we look at the PatternReplaceCharFilter. As you might guess the task of the filter is to change the matching input stream parts that match the given regular expression.<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[27],"tags":[180,208,181,290,164],"class_list":["post-266","post","type-post","status-publish","format-standard","hentry","category-solr-en","tag-analysis","tag-configuration","tag-filter","tag-filtering","tag-solr-2"],"_links":{"self":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts\/266","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/comments?post=266"}],"version-history":[{"count":1,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts\/266\/revisions"}],"predecessor-version":[{"id":267,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/posts\/266\/revisions\/267"}],"wp:attachment":[{"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/media?parent=266"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/categories?post=266"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/solr.pl\/en\/wp-json\/wp\/v2\/tags?post=266"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}