Bas is a DZone MVB and is not an employee of DZone and has posted 11 posts at DZone. View Full User Profile

Solr select query GET vs POST request

01.10.2012
| 7545 views |
  • submit to reddit
In most cases a GET request is used to send select queries to Solr. This is how it’s done in most examples, it’s easy to test in the browser and easy to implement. However, Solr also supports POST requests for select queries. This can for instance be useful if you use complex Solr queries with lots of  facets and/or filterqueries where the total length of your querystring can exceed the limits of servlet containers. But how does GET compare to POST in Solr? First of all I want to check performance. I’m not expecting a big difference here, but I want to be sure. So I’ve made two simple test scripts, one for a short query and one for a longer one. The short query:
facet=true&sort=price+asc&f.price.facet.range.gap=100&facet.range={!key%3Dpriceranges}price&hl.simple.pre=<b>&hl.fl=name,features&wt=json&hl=true&rows=20&fl=id,name,price&start=2&q=*:*&f.price.facet.range.end=1000&hl.simple.post=</b>&facet.field={!key%3Dstock}inStock&fq=price:[1+TO+300]&f.price.facet.range.start=1

And the longer one:

facet=true&sort=price+asc&f.price.facet.range.gap=100&facet.range={!key%3Dpriceranges}price&hl.simple.pre=<b>&hl.fl=name,features&wt=json&hl=true&rows=20&fl=id,name,price&start=2&q=*:*&f.price.facet.range.end=1000&hl.simple.post=</b>&facet.field={!key%3Dstock}inStock&fq=price:[1+TO+300]&fq=price:1+OR+price:2+OR+price:3+OR+price:4+OR+price:5+OR+price:6+OR+price:7+OR+price:8+OR+price:9+OR+price:10+OR+price:11+OR+price:12+OR+price:13+OR+price:14+OR+price:15+OR+price:16+OR+price:17+OR+price:18+OR+price:19+OR+price:20+OR+price:21+OR+price:22+OR+price:23+OR+price:24+OR+price:25+OR+price:26+OR+price:27+OR+price:28+OR+price:29+OR+price:30+OR+price:31+OR+price:32+OR+price:33+OR+price:34+OR+price:35+OR+price:36+OR+price:37+OR+price:38+OR+price:39+OR+price:40+OR+price:41+OR+price:42+OR+price:43+OR+price:44+OR+price:45+OR+price:46+OR+price:47+OR+price:48+OR+price:49+OR+price:50+OR+price:51+OR+price:52+OR+price:53+OR+price:54+OR+price:55+OR+price:56+OR+price:57+OR+price:58+OR+price:59+OR+price:60+OR+price:61+OR+price:62+OR+price:63+OR+price:64+OR+price:65+OR+price:66+OR+price:67+OR+price:68+OR+price:69+OR+price:70+OR+price:71+OR+price:72+OR+price:73+OR+price:74+OR+price:75+OR+price:76+OR+price:77+OR+price:78+OR+price:79+OR+price:80+OR+price:81+OR+price:82+OR+price:83+OR+price:84+OR+price:85+OR+price:86+OR+price:87+OR+price:88+OR+price:89+OR+price:90+OR+price:91+OR+price:92+OR+price:93+OR+price:94+OR+price:95+OR+price:96+OR+price:97+OR+price:98+OR+price:99+OR+price:100+OR+price:101+OR+price:102+OR+price:103+OR+price:104+OR+price:105+OR+price:106+OR+price:107+OR+price:108+OR+price:109+OR+price:110+OR+price:111+OR+price:112+OR+price:113+OR+price:114+OR+price:115+OR+price:116+OR+price:117+OR+price:118+OR+price:119+OR+price:120+OR+price:121+OR+price:122+OR+price:123+OR+price:124+OR+price:125+OR+price:126+OR+price:127+OR+price:128+OR+price:129+OR+price:130+OR+price:131+OR+price:132+OR+price:133+OR+price:134+OR+price:135+OR+price:136+OR+price:137+OR+price:138+OR+price:139+OR+price:140+OR+price:141+OR+price:142+OR+price:143+OR+price:144+OR+price:145+OR+price:146+OR+price:147+OR+price:148+OR+price:149+OR+price:150+OR+price:151+OR+price:152+OR+price:153+OR+price:154+OR+price:155+OR+price:156+OR+price:157+OR+price:158+OR+price:159+OR+price:160+OR+price:161+OR+price:162+OR+price:163+OR+price:164+OR+price:165+OR+price:166+OR+price:167+OR+price:168+OR+price:169+OR+price:170+OR+price:171+OR+price:172+OR+price:173+OR+price:174+OR+price:175+OR+price:176+OR+price:177+OR+price:178+OR+price:179+OR+price:180+OR+price:181+OR+price:182+OR+price:183+OR+price:184+OR+price:185+OR+price:186+OR+price:187+OR+price:188+OR+price:189+OR+price:190+OR+price:191+OR+price:192+OR+price:193+OR+price:194+OR+price:195+OR+price:196+OR+price:197+OR+price:198+OR+price:199+OR+price:200+OR+price:201+OR+price:202+OR+price:203+OR+price:204+OR+price:205+OR+price:206+OR+price:207+OR+price:208+OR+price:209+OR+price:210+OR+price:211+OR+price:212+OR+price:213+OR+price:214+OR+price:215+OR+price:216+OR+price:217+OR+price:218+OR+price:219+OR+price:220+OR+price:221+OR+price:222+OR+price:223+OR+price:224+OR+price:225+OR+price:226+OR+price:227+OR+price:228+OR+price:229+OR+price:230+OR+price:231+OR+price:232+OR+price:233+OR+price:234+OR+price:235+OR+price:236+OR+price:237+OR+price:238+OR+price:239+OR+price:240+OR+price:241+OR+price:242+OR+price:243+OR+price:244+OR+price:245+OR+price:246+OR+price:247+OR+price:248+OR+price:249+OR+price:250&f.price.facet.range.start=1

Normally a range would be used instead of that very long filterquery, but it’s an easy way to create a long request URI.

Each query is executed 100 times, and the total time measured. And this is repeated 25 times to check for consistency. Both scripts are tested this way once with GET and once with POST.

The results:

  • 100 short query GET requests take an average total time of 0.16 seconds. No noticeable difference by using POST.
  • 100 long query GET requests take an average total time of 0.22 seconds. No noticeable difference by using POST.
  • The timings are very consistent.

So, performance is not really a factor in this. How about logging? This depends on your setup, but with a default Solr setup the logging for a query received by GET or POST is the same.

Conclusion

Using GET or POST is doesn’t really make a big difference in a standard Solr setup.
However a POST request will usually have a higher limit, depending on your servlet container. In Jetty the default ‘headerBufferSize’ is 4kB. Tomcat has a similar setting ‘maxHttpHeaderSize’, also 4kB by default. This limit applies to all the combined headers of a request, so it’s not just the querystring. This might seem more than enough, but I have already seen two cases where a query with many filterqueries and facets reached this limit.
In comparison, the default for POST data in tomcat (‘maxPostSize’) is 4MB. Jetty uses a ‘maxFormContentSize’ setting with a lower default value of 200kB, but still way higher than the header limit and more than enough for even the biggest queries.

So, if you plan on constructing very big queries POST might be a safe choice. In other cases a GET request will be just fine.
And, there is always the option to simple raise your servlet container limits instead of switching to POST.

While writing this post I’ve started to wonder if moving query parameters to the Solr config by using a custom request handler would make a difference. I’ll test that in a new post soon.


Source: http://www.raspberry.nl/2011/12/22/solr-select-query-get-vs-post-request

Published at DZone with permission of Bas De Nooijer, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Afandi Merathi replied on Sun, 2012/03/18 - 7:15am

“And, there is always the option to simple raise your servlet container limits instead of switching to POST.”

This would pose a problem with certain browsers, since they enforce a maximum size for the URL (up to 2K characters is considered ‘safe’).

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.