solrq documentation - read the docs · 2019-04-02 · chapter 1 solrq solrqis a python solr query...
TRANSCRIPT
![Page 1: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/1.jpg)
solrq DocumentationRelease 1.1.1
Michał Jaworski
Mar 27, 2017
![Page 2: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/2.jpg)
![Page 3: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/3.jpg)
Contents
1 solrq 1
2 usage 32.1 quick reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3 contributing 7
4 testing 9
5 Detailed documentation 115.1 API reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
6 Indices and tables 17
Python Module Index 19
i
![Page 4: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/4.jpg)
ii
![Page 5: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/5.jpg)
CHAPTER 1
solrq
solrq is a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reservedcharacters. solrq is has no external dependencies and is compatibile with python2.6, python2.7, python3.3, python3.4, python3.5, pypy and pypy3. It might be compatibile with other python releases/implentationsbut this has not been tested yet or is no longer tested (e.g python3.2).
pip install solrq
And you’re ready to go!
1
![Page 6: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/6.jpg)
solrq Documentation, Release 1.1.1
2 Chapter 1. solrq
![Page 7: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/7.jpg)
CHAPTER 2
usage
Everything in solrq is about Q() object. Drop into python repl and just feed it with bunch of field and search termsto see how it works:
>>> from solrq import Q>>> # note: all terms in single Q object are implicitely joined with 'AND'>>> query = Q(type="animal", species="dog")>>> query<Q: type:animal AND species:dog>
>>> # ohh, forgot about cats?>>> query | Q(type="animal", species="cat")<Q: (type:animal AND species:dog) OR (type:animal AND species:cat)>
>>># more a cat lover? Let's give them a boost boost>>> Q(type="animal") & (Q(species="cat")^2 | Q(species="dog"))<Q: type:animal AND ((species:cat^2) OR species:dog)>
But what to do with this Q? Simply pass it to your Solr library of choice, like pysolr or mysolr. Most of python Solrlibraries expect simple string as a query parameter and do not bother with escaping of reserved characters so you musttake care of that by yourself. This is why solrq integrates so easily. Here is an example how you can use it withpysolr:
from solrq import Qimport pysolr
solr = Solr("<your solr url>")
# simply using Q objectsolr.search(Q(text="easy as f***"))
# or explicitely making it stringsolr.search(str(Q(text="easy as f***")))
3
![Page 8: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/8.jpg)
solrq Documentation, Release 1.1.1
quick reference
Full reference can be found in API reference documentation page but here is a short reference.
boosting queries
Use python ^ operator:
>>> Q(text='cat') ^ 2<Q: text:cat^2>
AND queries
Use python & operator:
>>> Q(text='cat') & Q(text='dog')<Q: text:cat AND text:dog>
OR queries
Use python | operator:
>>> Q(text='cat') | Q(text='dog')<Q: text:cat OR text:dog>
NOT queries
Use python ~ operator:
>>> ~ Q(text='cat')<Q: !text:cat>
ranges
Use solrq.Range wrapper:
>>> from solrq import Range>>> Q(age=Range(18, 25))<Q: age:[18 TO 25]>
proximity searches
Use solrq.Proximity wrapper:
>>> from solrq import Proximity>>> Q(age=Proximity("cat dogs", 5))<Q: age:"cat\ dogs"~5>
4 Chapter 2. usage
![Page 9: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/9.jpg)
solrq Documentation, Release 1.1.1
safe strings
All raw string values are treated as unsafe by default and will be escaped to ensure that final query string will not bebroken by some rougue search value. This of course can be disabled if you know what you’re doing using Valuewrapper:
>>> from solrq import Q, Value>>> Q(type='foo bar[]')<Q: type:foo\ bar\[\]>>>> Q(type=Value('foo bar[]', safe=True))<Q: type:foo bar[]>
timedeltas, datetimes
Simply as:
>>> from datetime import datetime, timedelta>>> Q(date=datetime(1970, 1, 1))<Q: date:"1970-01-01T00:00:00Z">>>> # note that timedeltas has any sense mostly with ranges>>> Q(delta=timedelta(days=1))<Q: delta:NOW+1DAYS+0SECONDS+0MILLISECONDS>
field wildcard
If you need to use wildcards in field names just use dict and unpack it inside of Q() instead of using keywordarguments:
>>> Q(**{"*_t": "text_to_search"})<Q: *_t:text_to_search>
2.1. quick reference 5
![Page 10: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/10.jpg)
solrq Documentation, Release 1.1.1
6 Chapter 2. usage
![Page 11: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/11.jpg)
CHAPTER 3
contributing
Any contribution is welcome. Issues, suggestions, pull requests - whatever. There are no strict contribution guidelinesbeyond PEP-8 and sanity. Code style is checked with flakes8 and any PR that has failed build will not be merged.
One thing: if you submit a PR please do not rebase it later unless you are asked for that explicitely. Reviewing pullrequests that suddenly had their history rewritten just drives me crazy.
7
![Page 12: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/12.jpg)
solrq Documentation, Release 1.1.1
8 Chapter 3. contributing
![Page 13: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/13.jpg)
CHAPTER 4
testing
Tests are run using tox. Simply install it and run:
pip install toxtox
And that’s all.
9
![Page 14: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/14.jpg)
solrq Documentation, Release 1.1.1
10 Chapter 4. testing
![Page 15: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/15.jpg)
CHAPTER 5
Detailed documentation
API reference
class solrq.Proximity(raw, distance, safe=False)Bases: solrq.Value
Wrapper around proximity value searches.
Parameters
• raw (str) – string of words for proximity search.
• distance (int) – distance between words.
Examples
>>> Proximity('foo bar', 4)<Proximity: "foo\ bar"~4>>>> Proximity('foo bar', 4, True)<Proximity: "foo bar"~4>
Note: Proximity will in fact accept any type as a raw value that has __str__ method defined so it isdeveloper’s responsibility to make sure that raw has a reasonable value.
class solrq.Q(children=None, op=<bound method type.and_ of <class ‘solrq.QOperator’>>, **kwargs)Bases: object
Class for handling Solr queries in a semantic way.
Parameters
• children (iterable) – iterable of children Q objects. Note: can’t be used with kwargs.
• op (callable) – operator to join query parts.
11
![Page 16: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/16.jpg)
solrq Documentation, Release 1.1.1
• kwargs (dict) – list of query parts. Note: can’t be used with children.
Examples
>>> Q(foo="bar")<Q: foo:bar>>>> str(Q(foo="bar"))'foo:bar'
>>> Q(text="Skyrim")<Q: text:Skyrim>
>>> Q(language="EN", text="Skyrim")<Q: ...>
>>> ~(Q(language="EN", text="cat") | Q(language="PL", text="dog"))<Q: !((... AND ...) OR (... AND ...))>
Note: it is possible to specify query params that are not valid python argument names using dictionary unpack-ing e.g.:
>>> Q(**{"*_t": "text_to_search"})<Q: *_t:text_to_search>
compile(extra_parenthesis=False)Compile Q object into query string.
Parameters extra_parenthesis (bool) – add extra parenthesis to children query.
Returns compiled query string.
Return type str
Examples
>>> (Q(type="animal") & Q(name="cat")).compile()'type:animal AND name:cat'>>> (Q(type="animal") & Q(name="cat")).compile(True)'(type:animal AND name:cat)'
class solrq.QOperatorBases: object
Simply a namespace for handling Q object operator routines.
classmethod and_(qs_list)Perform ‘and’ operator routine.
Parameters qs_list (iterable) – iterable of “compiled” query strings.
Returns query strings joined with Solr AND operator as single string.
Return type str
12 Chapter 5. Detailed documentation
![Page 17: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/17.jpg)
solrq Documentation, Release 1.1.1
classmethod boost(qs_list, factor)Perform ‘boost’ operator routine.
Parameters
• qs_list (iterable) – single element list with compiled query string
• factor (float or int) – boost factor
Returns compiled query string followed with ‘~’ and boost factor
Return type str
Note: this operator routine is not intended to be directly used as Q object argument but rather as acomponent for actual operator e.g:
>>> from functools import partial>>> Q(children=[Q(a='b')], op=partial(QOperator.boost, factor=2))<Q: a:b^2>
classmethod not_(qs_list)Perform ‘not’ operator routine.
Parameters qs_list (iterable) – single item iterable of “compiled” query strings.
Returns
string with containing Solr ! operator followed by query. string.
Return type str
Note: qs_list must be a list despite ‘not’ operator accepts only single query string here, to avoid morecomplexity in Q objects initialization.
classmethod or_(qs_list)Perform ‘or’ operator routine.
Parameters qs_list (iterable) – iterable of “compiled” query strings.
Returns query strings joined with Solr OR operator as single string.
Return type str
class solrq.Range(from_, to, safe=None, boundaries=’inclusive’)Bases: solrq.Value
Wrapper around range values.
Wraps two values with Solr’s [<from> TO <to>] syntax (defaults to inclusive boundaries) with respect torestricted character escaping.
Wraps two values with Solr’s [<from> TO <to>] (defaults to inclusive boundaries) syntax withrespect to restricted character escaping.
Parameters
• from (object) – start of range, same as parameter raw in Value.
• to (object) – end of range, same as parameter raw in Value.
5.1. API reference 13
![Page 18: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/18.jpg)
solrq Documentation, Release 1.1.1
• boundaries (str) – type of boundaries for the range. Defaults to 'inclusive'.Allowed values are:
– inclusive, ii, or []: translates to [<from> TO <to>]
– exclusive, ee, or {}: translates to {<from> TO <to>}
– ei, or {]: translates to {<from> TO <to>]
– ie, or [}: translates to [<from> TO <to>}
Examples
Simpliest range that matches all documents with some field set:
>>> Range('*', '*', safe=True)<Range: [* TO *]>
Note that there are shortucts already provided:
>>> Range(ANY, ANY)<Range: [* TO *]>>>> SET<Range: [* TO *]>
Other data types:
>>> Range(0, 20)<Range: [0 TO 20]>>>> Range(timedelta(days=2), timedelta())<Range: [NOW+2DAYS+0SECONDS+0MILLISECONDS TO NOW]>
To use exclusive or mixed boundaries use boundaries argument:
>>> Range(0, 20, boundaries='exclusive')<Range: {0 TO 20}>>>> Range(0, 20, boundaries='ei')<Range: {0 TO 20]>>>> Range(0, 20, boundaries='[}')<Range: [0 TO 20}>
Note: We could treat any iterables always as ranges when initializing Q objects but “explicit is better thanimplicit” and also this would require to handle value representation there and we don’t want to do that.
BOUNDARY_BRACKETS = {‘exclusive’: ‘{}’, ‘ei’: ‘{]’, ‘inclusive’: ‘[]’, ‘ee’: ‘{}’, ‘{]’: ‘{]’, ‘[]’: ‘[]’, ‘ii’: ‘[]’, ‘[}’: ‘[}’, ‘{}’: ‘{}’, ‘ie’: ‘[}’}
class solrq.Value(raw, safe=False)Bases: object
Wrapper around query values.
It allows easy handling of character escaping and further query value translations.
By default it escapes all restricted characters so query can’t be easily broken with unsafe strings. Also it recog-nizes timedelta and datetime objects so they can be represented in format that Solr can recognize (usefulwith ranges, see: Range)
Parameters
14 Chapter 5. Detailed documentation
![Page 19: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/19.jpg)
solrq Documentation, Release 1.1.1
• raw (object) – raw value object. Must be string, datetime, timedelta or have __str__method defined.
• safe (bool) – set to True to turn off character escaping.
Examples
In most cases you will pass string:
>>> Value("foo bar")<Value: foo\ bar>
But it can be anything that has __str__ method:
>>> Value(1)<Value: 1>>>> Value(timedelta(days=1))<Value: NOW+1DAYS+0SECONDS+0MILLISECONDS>>>> Value(Value("foo"))<Value: foo>
To get final query string just make it str:
>>> str(Value("foo bar"))'foo\\ bar'
Note that raw strings are not safe by default:
>>> Value('foo [] bar')<Value: foo\ \[\]\ bar>>>> Value("foo [] bar", safe=True)<Value: foo [] bar>
ESCAPE_RE = <_sre.SRE_Pattern object>
TIMEDELTA_FORMAT = ‘NOW{days:+d}DAYS{secs:+d}SECONDS{mills:+d}MILLISECONDS’
5.1. API reference 15
![Page 20: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/20.jpg)
solrq Documentation, Release 1.1.1
16 Chapter 5. Detailed documentation
![Page 21: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/21.jpg)
CHAPTER 6
Indices and tables
• genindex
• modindex
• search
17
![Page 22: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/22.jpg)
solrq Documentation, Release 1.1.1
18 Chapter 6. Indices and tables
![Page 23: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/23.jpg)
Python Module Index
ssolrq, 11
19
![Page 24: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/24.jpg)
solrq Documentation, Release 1.1.1
20 Python Module Index
![Page 25: solrq Documentation - Read the Docs · 2019-04-02 · CHAPTER 1 solrq solrqis a Python Solr query utility. It helps making query strings for Solr and also helps with escaping reserved](https://reader033.vdocuments.us/reader033/viewer/2022050522/5fa62c755e7cf26fde22d116/html5/thumbnails/25.jpg)
Index
Aand_() (solrq.QOperator class method), 12
Bboost() (solrq.QOperator class method), 12BOUNDARY_BRACKETS (solrq.Range attribute), 14
Ccompile() (solrq.Q method), 12
EESCAPE_RE (solrq.Value attribute), 15
Nnot_() (solrq.QOperator class method), 13
Oor_() (solrq.QOperator class method), 13
PProximity (class in solrq), 11
QQ (class in solrq), 11QOperator (class in solrq), 12
RRange (class in solrq), 13
Ssolrq (module), 11
TTIMEDELTA_FORMAT (solrq.Value attribute), 15
VValue (class in solrq), 14
21