Es

异常检测之指数平滑(利用elasticsearch来实现)

指数平滑法是一种特殊的加权平均法,加权的特点是对离预测值较近的历史数据给予较大的权数,对离预测期较远的历史数据给予较小的权数,权数由近到远按指数规律递减,所以,这种预测方法被称为指数平滑法。它可分为一次指数平滑法、二次指数平滑法及更高次指数平滑法。 关于指数平滑的得相关资料: ES API接口: > https://github.com/IBBD/IBBD.github.io/blob/master/elk/aggregations-pipeline.md https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-pipeline-movavg-aggregation.html 理论概念 > http://blog.sina.com.cn/s/blog_4b9acb5201016nkd.html ES移动平均聚合:Moving Average的四种模型 simple 就是使用窗口内的值的和除于窗口值,通常窗口值越大,最后的结果越平滑: (a1 + a2 + … + an) / n curl -XPOST 'localhost:9200/_search?pretty' -H 'Content-Type: application/json' -d' { "size": 0, "aggs": { "my_date_histo":{ "date_histogram":{ "field":"date", "interval":"1M" }, "aggs":{ "the_sum":{ "sum":{ "field": "price" } }, "the_movavg":{ "moving_avg":{ "buckets_path": "the_sum", "window" : 30, "model" : "simple" } } } } } } ' 线性模型:Linear 对窗口内的值先做线性变换处理,再求平均:(a1 * 1 + a2 * 2 + … + an * n) / (1 + 2 + … + n)

继续阅读

Elasticsearch-DSL部分集合

ELK是日志收集分析神器,在这篇文章中将会介绍一些ES的常用命令。 点击阅读:ELK Stack 从入门到放弃 DSL中遇到的错误及解决办法 分片限制错误 Trying to query 2632 shards, which is over the limit of 1000. This limit exists because querying many shards at the same time can make the job of the coordinating node very CPU and/or memory intensive. It is usually a better idea to have a smaller number of larger shards. Update [action.search.shard_count.limit] to a greater value if you really want to query that many shards at the same time.

继续阅读