Wiki source code of Using the CQL Query Module
Last modified by superadmin on 2026/10/14 18:44
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | {{warning}} | ||
| 2 | This document should probably be split into smaller, more focused documents. | ||
| 3 | {{/warning}} | ||
| 4 | |||
| 5 | The Confluence Query Language (CQL) is used in Confluence to build complex document queries. | ||
| 6 | |||
| 7 | For a description of how this module was designed, please see [[Design>>doc:.design.WebHome]]. | ||
| 8 | |||
| 9 | {{warning}} | ||
| 10 | This module is still in a very early stage in its life. Expect changes, missing features and surprising behavior. | ||
| 11 | {{/warning}} | ||
| 12 | |||
| 13 | {{info}} | ||
| 14 | This module is intended at developers of code providing compatibility and interoperability with Confluence. Anybody else is urged to use native features of XWiki. End users are invited to use the [[Solr Search Application>>doc:extensions:Extension.Solr Search Application]] and developers the [[extension:Extension.Solr Search Query API]]. | ||
| 15 | {{/info}} | ||
| 16 | |||
| 17 | == Basic Usage == | ||
| 18 | |||
| 19 | To query XWiki using CQL, you can create a query and execute it the familiar way, using "cql" as the query type. The CQL query is translated to Sorl's standard syntax and executed using Solr. The results returned by the ##execute## function are the same as for a Solr query: results are found in the ##results## property of the first and only returned row and which type is ##org.apache.solr.common.SolrDocumentList##. There's no wrapping or abstraction of the Solr result part in the CQL Query Module. | ||
| 20 | |||
| 21 | In Velocity: | ||
| 22 | |||
| 23 | {{code language="velocity"}} | ||
| 24 | {{velocity}} | ||
| 25 | #set ($query = $services.query.createQuery("creator = currentUser() and lastmodified > startOfWeek() order by title", "cql").setLimit(5)) | ||
| 26 | #set ($results = $query.execute()[0].results) | ||
| 27 | #foreach ($result in $results) | ||
| 28 | * $result.id | ||
| 29 | #end | ||
| 30 | {{/velocity}} | ||
| 31 | {{/code}} | ||
| 32 | |||
| 33 | In Groovy: | ||
| 34 | |||
| 35 | {{code language="groovy"}} | ||
| 36 | {{groovy}} | ||
| 37 | def query = services.query.createQuery("creator = currentUser() and lastmodified > startOfWeek() order by title", "cql").setLimit(5) | ||
| 38 | def results = query.execute()[0].results | ||
| 39 | for (def result : results) { | ||
| 40 | println "* " + result.id | ||
| 41 | } | ||
| 42 | {{/groovy}} | ||
| 43 | {{/code}} | ||
| 44 | |||
| 45 | In Java: | ||
| 46 | |||
| 47 | {{code language="java"}} | ||
| 48 | import java.util.List; | ||
| 49 | |||
| 50 | import javax.inject.Inject; | ||
| 51 | import javax.inject.Singleton; | ||
| 52 | |||
| 53 | import org.xwiki.component.annotation.Component; | ||
| 54 | import org.xwiki.query.QueryException; | ||
| 55 | import org.xwiki.query.QueryManager; | ||
| 56 | |||
| 57 | @Component (roles = DefaultScratch.class) | ||
| 58 | @Singleton | ||
| 59 | class DefaultScratch | ||
| 60 | { | ||
| 61 | @Inject | ||
| 62 | private QueryManager queryManager; | ||
| 63 | |||
| 64 | public List<Object> example() throws QueryException | ||
| 65 | { | ||
| 66 | return queryManager.createQuery("title = 'Main'", "cql").setLimit(1).execute(); | ||
| 67 | } | ||
| 68 | } | ||
| 69 | {{/code}} | ||
| 70 | |||
| 71 | === Sorting === | ||
| 72 | |||
| 73 | It is possible to order results by specifying an ##order by## clause in the CQL statement. The order by clause will need to follow the name of the fields in Confluence and will be converted to the corresponding XWiki Solr fields. XWiki Solr fields cannot be used directly in a CQL expression. | ||
| 74 | It is also possible to do so by binding a sort parameter. The sort parameter will then override whatever ##order by clause## might be present in the CQL query. In the sort parameter, the XWiki Solr fields need to be used directly, Confluence fields cannot be used there: | ||
| 75 | |||
| 76 | {{code language="velocity"}} | ||
| 77 | #set ($query = $services.query.createQuery("...", "cql").sort("creator_display asc")) | ||
| 78 | {{/code}} | ||
| 79 | |||
| 80 | === Supported syntax === | ||
| 81 | |||
| 82 | See the [[documentation of the supported parts of the CQL Syntax and how it is translated>>xwiki:documentation.extensions.dev.cql.syntax.WebHome]]. | ||
| 83 | |||
| 84 | === Debugging a query that doesn't work as expected === | ||
| 85 | |||
| 86 | A query that doesn't work as expected, apart from being caused by unexpected results and unexpectedly present / absent documents in the wiki, will most likely be caused by: | ||
| 87 | |||
| 88 | * a parse error | ||
| 89 | * a conversion error | ||
| 90 | * an incorrect translation to Solr | ||
| 91 | |||
| 92 | ==== Parse and Conversion errors ==== | ||
| 93 | |||
| 94 | A parse error would happen if something is wrong with the syntax of the query. A conversion error would happen if something is wrong with how it's "understood" or if something else prevent the query to be converted to Solr, for instance a missing feature, after the parsing is done. | ||
| 95 | |||
| 96 | They will both raise exceptions so it should be easy to get to the root cause quite rapidly. Care has been taken to make error messages as understandable as possible, with precise pointing of the error: | ||
| 97 | |||
| 98 | {{image reference="parseerror.png"/}} | ||
| 99 | |||
| 100 | {{image reference="parseerrordetailed.png"/}} | ||
| 101 | |||
| 102 | ==== Incorrect translation to Solr ==== | ||
| 103 | |||
| 104 | It can happen that the translation to Solr is incorrect due to a bug in the translator, or in a component that extends the translation coming from elsewhere (as described below). | ||
| 105 | |||
| 106 | In this case, it can be useful to enable ##DEBUG## log level for the ##org.contrib.cql## package. The queries and their Solr translations are printed each time a query is executed. | ||
| 107 | |||
| 108 | Spot lines like: | ||
| 109 | |||
| 110 | {{{2024-05-31 11:50:42,651 [qtp1991294891-812 - http://localhost:8080/xwiki/bin/view/Main/WebHome] DEBUG o.x.c.c.q.i.CQLQueryExecutor - CQL Statement [(label = "hello" OR label = "world") AND type = "page"] converted to Solr query [((property.XWiki.TagClass.tags:hello) OR (property.XWiki.TagClass.tags:world)) AND (type:DOCUMENT AND -class:Blog.BlogPostClass)], sort parameter [] for execution | ||
| 111 | }}} | ||
| 112 | |||
| 113 | === Translating CQL to Solr queries without querying === | ||
| 114 | |||
| 115 | You may want to convert CQL statements to Solr statements, for instance if you want to provide a bridge between some Confluence feature to an equivalent Solr-based feature. | ||
| 116 | |||
| 117 | This is quite straight forward. You need to parse the CQL statement using [[AQLParser>>doc:documentation.extensions.dev.cql.aql-parser.WebHome]], and call the ##getSolrStatement## and ##getSolrSort## methods of the ##org.xwiki.contrib.cql.query.converters.CQLToSolrQueryConverter## component. | ||
| 118 | |||
| 119 | {{code language="java"}} | ||
| 120 | import java.io.IOException; | ||
| 121 | import java.util.List; | ||
| 122 | |||
| 123 | import javax.inject.Inject; | ||
| 124 | import javax.inject.Singleton; | ||
| 125 | |||
| 126 | import org.xwiki.component.annotation.Component; | ||
| 127 | import org.xwiki.contrib.cql.aqlparser.AQLParser; | ||
| 128 | import org.xwiki.contrib.cql.aqlparser.ast.AQLStatement; | ||
| 129 | import org.xwiki.contrib.cql.aqlparser.exceptions.ParserException; | ||
| 130 | import org.xwiki.contrib.cql.query.converters.CQLToSolrQueryConverter; | ||
| 131 | |||
| 132 | @Component (roles = DefaultScratch.class) | ||
| 133 | @Singleton | ||
| 134 | class DefaultScratch | ||
| 135 | { | ||
| 136 | @Inject | ||
| 137 | private CQLToSolrQueryConverter cqlConverter; | ||
| 138 | |||
| 139 | public void example() throws ParserException, IOException | ||
| 140 | { | ||
| 141 | AQLStatement cql = AQLParser.parse( "title = 'Main'"); | ||
| 142 | String solr = cqlConverter.getSolrStatement(cql); | ||
| 143 | String sort = cqlConverter.getSolrSortParameter(cql); | ||
| 144 | // do something with solr and sort | ||
| 145 | } | ||
| 146 | } | ||
| 147 | {{/code}} | ||
| 148 | |||
| 149 | == Advanced topics == | ||
| 150 | |||
| 151 | This section goes beyond basic usage and is intended to developers seeking to improving or altering the behavior of the CQL query to Solr translator. | ||
| 152 | |||
| 153 | Although improving the CQL to Solr translation should ideally be done in the cql module itself, it's not always desirable or possible. In which case, the CQL Query module offers way to extend the translation at different levels. | ||
| 154 | |||
| 155 | === Common stuff === | ||
| 156 | |||
| 157 | In what follows: | ||
| 158 | |||
| 159 | * throw a ##ConversionException## when there is an obvious error somewhere, or when you called code that throws | ||
| 160 | * return null when you didn't find something or you don't know how to convert something | ||
| 161 | |||
| 162 | For instance, if you couldn't resolve document with id 42 or space DEMO, return null. But if the query is wrong (for instance ##lastmodified > "hello bad world"##), you can throw a ##ConversionExpression##. However, it would be recommended to null in this case too, so the default converter or another converter can handle the expressions you wouldn't expect but was legit after all, and throw if appropriate. | ||
| 163 | |||
| 164 | === Providing Confluence IDs to the translator === | ||
| 165 | |||
| 166 | Some CQL expressions like `id = 42` require knowing which XWiki document corresponds to the given Confluence ID (here 42) to be translated to Solr. | ||
| 167 | |||
| 168 | When migrating content from Confluence to XWiki using [[Confluence XML>>doc:documentation.extensions.user.confluence.WebHome]], it is possible to enable the ##Store Confluence details## parameter. When doing so, ##Confluence.Code.ConfluencePageClass## objects will be added to each document, with some confluence metadata like their IDs. | ||
| 169 | |||
| 170 | The translator will try to probe the wiki and find such ##Confluence.Code.ConfluencePageClass## objects to convert such CQL expressions. | ||
| 171 | |||
| 172 | However, the ##Store Confluence details## parameter isn't enabled by default and the ##Confluence.Code.ConfluencePageClass## object corresponding to a particular Confluence ID might not be there. | ||
| 173 | |||
| 174 | It is possible to help the translator find documents corresponding to a given ID by providing a component implementing the [[##org.xwiki.contrib.cql.query.converters.ConfluenceIdResolver##>>https://github.com/xwiki-contrib/cql/blob/master/query/src/main/java/org/xwiki/contrib/cql/query/converters/ConfluenceIdResolver.java]] interface. Please give the component a name using the ##@Named## annotation. | ||
| 175 | |||
| 176 | For convenience, here is a copy of this interface, but please actually refer to the newest version of the linked file. | ||
| 177 | |||
| 178 | {{code language="java"}} | ||
| 179 | package org.xwiki.contrib.cql.query.converters; | ||
| 180 | |||
| 181 | import org.xwiki.component.annotation.Role; | ||
| 182 | import org.xwiki.contrib.cql.aqlparser.ast.AbstractAQLRightHandValue; | ||
| 183 | import org.xwiki.model.reference.EntityReference; | ||
| 184 | |||
| 185 | /** | ||
| 186 | * Find a document from its Confluence ID. | ||
| 187 | * @version $Id$ | ||
| 188 | * @since 0.0.1 | ||
| 189 | */ | ||
| 190 | @Role | ||
| 191 | public interface ConfluenceIdResolver | ||
| 192 | { | ||
| 193 | /** | ||
| 194 | * @return the document in XWiki, or null if the document is not found. | ||
| 195 | * @param node the CQL node containing this id, or null if not applicable | ||
| 196 | * @param id the Confluence ID of the document | ||
| 197 | * @throws ConversionException when something bad happens | ||
| 198 | */ | ||
| 199 | EntityReference getDocumentById(AbstractAQLRightHandValue node, long id) throws ConversionException; | ||
| 200 | } | ||
| 201 | {{/code}} | ||
| 202 | |||
| 203 | Here's the provided implementation, which uses ##ConfluencePageClass## objects: https://github.com/xwiki-contrib/cql/blob/master/query/src/main/java/org/xwiki/contrib/cql/query/converters/internal/ConfluencePageClassConfluenceIdResolver.java | ||
| 204 | |||
| 205 | The translator will try to resolve IDs using all the registered components, until one returns a non-null entity reference. | ||
| 206 | |||
| 207 | === Providing the migrated location of Confluence spaces to the translator === | ||
| 208 | |||
| 209 | Some CQL expressions like `space = DEMO` require knowing which XWiki document corresponds to the given Confluence ID (here 42) to be translated to Solr. | ||
| 210 | |||
| 211 | By default, the translator will blindly convert a Confluence space key to the entity reference of the root space of the same name (i.e the space key DEMO is translated to xwiki:DEMO). This is a fallback mechanism which works most of the time. | ||
| 212 | |||
| 213 | However, there are some situations where this doesn't work. For instance, for Confluence spaces that were migrated with a non empty "Root space" parameter of [[Extension.Confluence.XML~]. It is possible to help the translator find spaces corresponding to a given space key by providing a component implementing the ~~~[~~~[##org.xwiki.contrib.cql.query.converters.ConfluenceIdResolver##>>https://github.com/xwiki-contrib/cql/blob/master/query/src/main/java/org/xwiki/contrib/cql/query/converters/ConfluenceSpaceResolver.java]] interface. Please give the component a name using the ##@Named## annotation. | ||
| 214 | |||
| 215 | For convenience, here is a copy of this interface, but please actually refer to the newest version of the linked file. | ||
| 216 | |||
| 217 | {{code language="java"}} | ||
| 218 | package org.xwiki.contrib.cql.query.converters; | ||
| 219 | |||
| 220 | import org.xwiki.component.annotation.Role; | ||
| 221 | import org.xwiki.contrib.cql.aqlparser.ast.AbstractAQLRightHandValue; | ||
| 222 | import org.xwiki.model.reference.EntityReference; | ||
| 223 | |||
| 224 | /** | ||
| 225 | * Find a space from its Confluence key. | ||
| 226 | * @version $Id$ | ||
| 227 | * @since 0.0.1 | ||
| 228 | */ | ||
| 229 | @Role | ||
| 230 | public interface ConfluenceSpaceResolver | ||
| 231 | { | ||
| 232 | /** | ||
| 233 | * @return the space in XWiki, or null if the document is not found. | ||
| 234 | * @param node the CQL node containing this id, or null if not applicable | ||
| 235 | * @param spaceKey the Confluence ID of the document | ||
| 236 | * @throws ConversionException when something bad happens | ||
| 237 | */ | ||
| 238 | EntityReference getSpaceByKey(AbstractAQLRightHandValue node, String spaceKey) throws ConversionException; | ||
| 239 | |||
| 240 | /** | ||
| 241 | * @return the space reference (of type EntityType.SPACE) to the root of the migrated Confluence space in which the | ||
| 242 | * current document is. | ||
| 243 | * @param node the CQL node requesting the current space (likely a 'currentSpace()' call) | ||
| 244 | * @throws ConversionException when something bad happens | ||
| 245 | */ | ||
| 246 | EntityReference getCurrentConfluenceSpace(AbstractAQLRightHandValue node) throws ConversionException; | ||
| 247 | } | ||
| 248 | {{/code}} | ||
| 249 | |||
| 250 | The translator will try to resolve space keys using all the registered components, until one returns a non-null entity reference. | ||
| 251 | |||
| 252 | === Adding or modifying the translation for a particular field === | ||
| 253 | |||
| 254 | The provided translation is not complete, and CQL is also extensible. It is possible to add or modify the existing translation for a particular field by implementing the [[##org.xwiki.contrib.cql.query.converters.CQLToSolrSortFieldConverter##>>https://github.com/xwiki-contrib/cql/blob/master/query/src/main/java/org/xwiki/contrib/cql/query/converters/CQLToSolrSortFieldConverter.java]] interface, copied here for your convenience but please refer to the latest version of this file: | ||
| 255 | |||
| 256 | {{code language="java"}} | ||
| 257 | package org.xwiki.contrib.cql.query.converters; | ||
| 258 | |||
| 259 | import org.xwiki.component.annotation.Role; | ||
| 260 | import org.xwiki.contrib.cql.aqlparser.ast.AQLAtomicClause; | ||
| 261 | import org.xwiki.stability.Unstable; | ||
| 262 | |||
| 263 | /** | ||
| 264 | * CQL to Solr Atom Converter. | ||
| 265 | * @since 0.0.1 | ||
| 266 | * @version $Id$ | ||
| 267 | */ | ||
| 268 | @Role | ||
| 269 | @Unstable | ||
| 270 | public interface CQLToSolrAtomConverter | ||
| 271 | { | ||
| 272 | /** | ||
| 273 | * @return the given CQL atom converted to a Solr atom, as string | ||
| 274 | * @param atom the atom to convert | ||
| 275 | * @throws ConversionException if something wrong happens | ||
| 276 | */ | ||
| 277 | String convertToSolr(AQLAtomicClause atom) throws ConversionException; | ||
| 278 | |||
| 279 | /** | ||
| 280 | * @return the name(s) of the fields handled by this converter, as a String equal to the name or a Pattern | ||
| 281 | * matching the name. If null, the name of the component will be used. | ||
| 282 | * @since 0.2.0 | ||
| 283 | */ | ||
| 284 | default Object getHandledFields() | ||
| 285 | { | ||
| 286 | return null; | ||
| 287 | } | ||
| 288 | } | ||
| 289 | {{/code}} | ||
| 290 | |||
| 291 | Your component should have a name (hint) specific to the name of the field(s) it handles. If the field has a simple name, the hint is sufficient. If you must handle fields that don't have a fixed name (there's some dynamic part in the field), Implement the getHandledFields(). It should return a String equal to the name of the field, or a Pattern that matches the field. An example of this is https://github.com/xwikisas/xwiki-pro-macros/blob/04c586092e308cb86d32441febeb54adbece535c/xwiki-pro-macros-confluence-bridges/xwiki-pro-macros-confluence-bridges-api/src/main/java/com/xwiki/macros/confluence/internal/cql/MetadataFieldCQLToSolrConverter.java#L44-L58 | ||
| 292 | |||
| 293 | Your component should take the Abstract Syntax Tree node corresponding to an atomic CQL expression, and return the corresponding Solr code. | ||
| 294 | |||
| 295 | As this is not a trivial task and it would be undesirable to reimplement all the logic that is already implemented in the CQL Query module, it is advised to extend [[##org.xwiki.contrib.cql.query.converters.DefaultCQLToSolrAtomConverter##>>https://github.com/xwiki-contrib/cql/blob/master/query/src/main/java/org/xwiki/contrib/cql/query/converters/internal/AbstractIdCQLToSolrAtomConverter.java]] instead of writing your component from scratch. | ||
| 296 | |||
| 297 | ##DefaultCQLToSolrAtomConverter## implements ##convertToSolr(AQLAtomicClause atom, TYPE right) throws ConversionException## methods for all supported CQL types. | ||
| 298 | |||
| 299 | You can then: | ||
| 300 | |||
| 301 | * override ##List<String> getSolrFields(AQLAtomicClause atom) throws ConversionException## to provide the Solr field(s) in which to search | ||
| 302 | * override the specific method of the types you want to support and you only need to return the converted value | ||
| 303 | |||
| 304 | For instance: | ||
| 305 | |||
| 306 | * override ##String convertToSolr(AQLAtomicClause atom, AQLNumberLiteral expression) throws ConversionException ## to implement a numerical field | ||
| 307 | * override ##String convertToSolr(AQLAtomicClause atom, AQLDateLiteral expression) throws ConversionException ## to implement a date field | ||
| 308 | * override ##String convertToSolr(AQLAtomicClause atom, AQLBooleanLiteral expression) throws ConversionException## to implement a boolean field | ||
| 309 | * override ##String convertToSolr(AQLAtomicClause atom, AQLStringLiteral expression) throws ConversionException## to implement a string field | ||
| 310 | |||
| 311 | If you don't specifically want to have to deal with specific types of values, you can alternatively override ##String convertToSolr(AQLAtomicClause atom, AbstractAQLAtomicValue right) throws ConversionException## and call super, like this: | ||
| 312 | |||
| 313 | {{code language="java"}} | ||
| 314 | //... | ||
| 315 | @Override | ||
| 316 | protected String convertToSolr(AQLAtomicClause atom, AbstractAQLAtomicValue right) throws ConversionException | ||
| 317 | { | ||
| 318 | String value = super.convertToSolr(atom, right); | ||
| 319 | // do something to value, which is the default escaped Solr value | ||
| 320 | return value; // beware, this should be solr-escaped | ||
| 321 | } | ||
| 322 | |||
| 323 | @Override | ||
| 324 | protected List<String> getSolrFields(AQLAtomicClause atom) throws ConversionException | ||
| 325 | { | ||
| 326 | return List.of("title_sort"); // return solr fields corresponding to this atomic expression | ||
| 327 | } | ||
| 328 | |||
| 329 | //... | ||
| 330 | } | ||
| 331 | {{/code}} | ||
| 332 | |||
| 333 | There is an example of this at https://github.com/xwiki-contrib/cql/blob/master/query/src/main/java/org/xwiki/contrib/cql/query/converters/internal/AncestorCQLToSolrAtomConverter.java, which extends https://github.com/xwiki-contrib/cql/blob/master/query/src/main/java/org/xwiki/contrib/cql/query/converters/internal/AbstractIdCQLToSolrAtomConverter.java. |
