"machine learning"~2 — phrase and proximity search in Whoosh

When users enclose words in quotes, they're indicating an exact phrase. Machine learning should not match a page containing the individual words "machine" and "learning" across different paragraphs. Phrase and proximity queries in Whoosh are necessary to capture this intent. To enable phrase matching, the field must store term positions, which is set by specifying phrase=True when defining the field.

To search for exact phrases, simply enclose the desired terms in quotes. For example, searching for "machine learning" will only return documents containing those two terms adjacent and in the same order. In the provided example, only the document with id 'x' matches this query, as it contains the exact phrase "machine learning" without any intervening words.

Proximity queries allow for flexibility in word order by permitting a specified number of words to sit between the targeted terms. This is achieved by appending ~N (where N represents the maximum number of intervening words) to the closing quote. In the example, machine learning ~2 searches for the phrase "machine learning" with up to two words between them.

This results in matching both document 'x' (which contains "machine learning is powerful") and document 'z' (which contains "machine models for learning"), as both have "machine" followed by "learning" with two words in between.

The Phrase query object in Whoosh can also be constructed programmatically, providing an alternative approach to querying. This is particularly useful when terms come from structured input, such as tags or product names, and direct query construction is preferred over using the parser. The Phrase query accepts the field, a list of ordered terms, and an optional slop parameter. In the example, a Phrase query for "machine learning" with a slop of 2 returns the same results as the quoted phrase search.

Exact phrase queries are most appropriate for names, error messages, quoted titles, and other code identifiers where word order carries significant meaning. Proximity queries, on the other hand, are better suited for concept searches where the words are related but can be separated by varying phrasing. A starting point for proximity is typically around ~2 or ~3, and the value should be adjusted based on real-world queries to achieve optimal results.

In general, plain AND/OR queries with BM25F scoring are sufficient for searches involving words in any order, as phrase queries are stricter and slightly more resource-intensive. One important consideration is to ensure the field is indexed with positions (phrase=True) to avoid silent failures; fields without position data (such as KEYWORD or ID fields) cannot support phrase queries.