Version 2.1.1-beta, 20 feb 2013
Major new features
- added query profiling (SET PROFILING=1 and SHOW PROFILE statements)
- added AOT-based Russian lemmatizer (morphology={lemmatize_ru | lemmatize_ru_all}, lemmatizer_base, and lemmatizer_cache directives)
- added wordbreaker, a tool to split compounds into individual words
- added JSON attributes support (sql_attr_json, on_json_attr_error, json_autoconv_numbers, json_autoconv_keynames directives)
- added initial subselects support, SELECT * FROM (SELECT ... ORDER BY cond1 LIMIT X) ORDER BY cond2 LIMIT Y
- added bigram indexing, and phrase searching with bigrams (bigram_index, bigram_freq_words directives)
- added HA/LB support, ha_strategy and agent_persistent directives, SHOW AGENT STATUS statement
- added RT index optimization (OPTIMIZE INDEX statement, rt_merge_iops and rt_merge_maxiosize directives)
- added wildcards support to dict=keywords (eg. “t?st*”)
- added substring search support (min_infix_len=2 and above) to dict=keywords
New features
- added --checkconfig switch to indextool to check config file for correctness (bug #1395)
- added global IDF support (global_idf directive, OPTION global_idf)
- added “term1 term2 term3”/0.5 quorum fraction syntax (bug #1372)
- added an option to apply stopwords before morphology, stopwords_unstemmed directive
- added an alternative method to compute keyword IDFs, OPTION idf=plain
- added boolean query optimizations, OPTION boolean_simplify=1 (bug #1294)
- added stringptr return type support to UDFs, and CREATE FUNCTION ... RETURNS STRING syntax
- added early query termination by predicted execution time (OPTION max_predicted_time, and predicted_time_costs directive)
- added index_field_lengths directive, BM25A() and BM25F() functions to expression ranker
- added ranker=export, and PACKEDFACTORS() function
- added OPTION agent_query_timeout
- added support for attribute files over 4 GB (bug #1274)
- added addr2line output to crash reports (bug #1265)
- added OPTION ignore_nonexistent_columns to UPDATE, and a respective UpdateAttributes() argument
- added --keep-attrs switch to indexer
- added --with-static-mysql, --with-static-pgsql switches to configure
- added double-buffering for RT INSERTs (bug #1200)
- added --morph, --dumpdict switch to indextool
- added support for multiple wordforms files, comment syntax, and pre/post-morphology wordforms
- added ZONESPANLIST() builtin function
- added regexp_filter directive, regexp document/query filtering support (uses RE2)
- added min_idf, max_idf, sum_idf ranking factors
- added uservars persistence, and sphinxql_state directive (bug #1132)
- added POLY2D, GEOPOLY2D, CONTAINS functions
- added ZONESPAN operator
- added snippets_file_prefix directive
- added Arabic stemmer, morphology=stem_ar directive (bug #519)
- added OPTION sort_method={pq | kbuffer}, an alternative match sorting method
- added SPZ (sentence, paragraph, zone) support to RT indexes
- added support for upto 255 keywords in quorum operator (bug #1030)
- added multi-threaded agent querying (bug #1000)
New SphinxQL features
- added SHOW INDEX indexname STATUS statement
- added LIKE clause support to multiple SHOW xxx statements
- added SNIPPET() function
- added GROUP_CONCAT() aggregate function
- added GROUPBY() builtin function
- added iostats and cpustats to SHOW META
- added support for DELETE statement over distributed indexes (bug #1104)
- added EXIST('attr_name', default_value) builtin function (bug #1037)
- added SHOW VARIABLES WHERE variable_name='xxx' syntax
- added TRUNCATE RTINDEX statement
Major behavior changes and optimizations
- changed that UDFs are now allowed in fork/prefork modes via sphinxql_state startup script
- changed that compat_sphinxql_magics now defaults to 0
- changed that small enough exceptions, wordforms, stopwords files are now embedded into the index header
- changed that rt_mem_limit can now be over 2 GB (bug #1059)
- optimized tokenizer (upto 1.25x indexing and snippets speedup)
- optimized multi-keyword searching (added skiplists)
- optimized filtering and scan in several frequent cases (single-value, 2-arg, 3-arg WHERE clauses)
