开发者关系通讯 – 2024 年 8 月_AI阅读总结 — 包阅AI

Logs data stream and LogsDB index mode: A logs data stream is a specialized data stream type designed for more efficient storage of log data, similar to the time series data stream for metrics. In our benchmarks it can reduce the disk space usage by 2.5x over regular data streams, though the exact savings depend on the specific data set.

Key features of a logs data stream include:

• Synthetic source: The _source field occupies a significant amount of disk space. Instead of storing the documents exactly as requested, Elasticsearch can reconstruct source content from indexed fields on the fly upon retrieval. Synthetic source is now also available on all field types.

• Index sorting: Reduces storage footprint by sorting indices by host.name and @timestamp at index time.

• Space-efficient compression: Applies more efficient compression for fields with doc_values enabled.

Elasticsearch’s native Learning to Rank (LTR) is generally available: Learning to Rank is a technique that improves relevance by using a machine learning (ML) model to re-rank search results. Typically, LTR is applied after an initial retrieval stage, refining the results based on a search context and a judgment list. The search context includes user queries and potentially other user-related data while the judgment list is a data set of query-document pairs labeled with relevance scores, which is used to train the model. The ML model, often a Gradient Boosted Decision Tree (GBDT) like LambdaMART, ranks documents based on features extracted from the query and documents.