ES 按照每隔几分钟,几小时,几天统计折线图

前端之家收集整理的这篇文章主要介绍了ES 按照每隔几分钟,几小时,几天统计折线图前端之家小编觉得挺不错的,现在分享给大家,也给大家做个参考。

公司应用es有一段时间,今天接触了一个相对复杂的业务,针对每隔几分钟,几小时,几天进行统计折线图,具体逻辑如下图:


如图,系统必须要支持查询,每小时(每隔10分钟),每日(每隔4小时统计),每周(每隔1日),每月(每隔5日)进行统计,找到最大值显示到折线图上。

首先4张图像使用term聚合,每张图像上有两条线,表示cpu和内存,也属于term聚合方式,整个折线图采用dateHistogram聚合方式。

使用语句如下:

  1. GET /system-audit1/auditEvent/_search
  2. {
  3. "aggs": {
  4. "sales": {
  5. "terms": {
  6. "field": "psName.keyword"
  7. },"aggs": {
  8. "type": {
  9. "terms": {
  10. "field": "type.keyword"
  11. },"aggs": {
  12. "staticTime": {
  13. "date_histogram": {
  14. "field": "statisticTime","interval": "4h"
  15. },"aggs": {
  16. "maxValue": {
  17. "max": {
  18. "field": "value"
  19. }
  20. }
  21. }
  22. }
  23. }
  24. }
  25. }
  26. }
  27. }
  28. }
执行结果:

  1. "aggregations": {
  2. "sales": {
  3. "doc_count_error_upper_bound": 0,"sum_other_doc_count": 0,"buckets": [
  4. {
  5. "key": "192.168.1.241:es","doc_count": 7516,"type": {
  6. "doc_count_error_upper_bound": 0,"buckets": [
  7. {
  8. "key": "cpu","doc_count": 3763,"staticTime": {
  9. "buckets": [
  10. {
  11. "key_as_string": "2018-01-05T16:00:00.000Z","key": 1515168000000,"doc_count": 2067,"maxValue": {
  12. "value": 23.100000381469727
  13. }
  14. },{
  15. "key_as_string": "2018-01-05T20:00:00.000Z","key": 1515182400000,"doc_count": 132,"maxValue": {
  16. "value": 22.799999237060547
  17. }
  18. },{
  19. "key_as_string": "2018-01-06T00:00:00.000Z","key": 1515196800000,"doc_count": 0,"maxValue": {
  20. "value": null
  21. }
  22. }...

java代码实现:

  1. List<SystemDistribution> list = new ArrayList<>(); //统计最终的数据
  2. BoolQueryBuilder boolQueryBuilder=QueryBuilders.boolQuery();
  3. boolQueryBuilder.must(QueryBuilders.rangeQuery("createTime").lte(endTime).gt(startTime)); //createTime是YYYYMMDDHHMMSSSSS格式字符串
  4. DateHistogramInterval dateHistogramInterval=getDateHistogramInterval(timeType); //聚合时间类型
  5. TermsAggregationBuilder termAggregation=AggregationBuilders.terms("psName").field("psName.keyword"); //服务器名称聚合
  6. TermsAggregationBuilder typeAggregation=AggregationBuilders.terms("type").field("type.keyword");
  7. AggregationBuilder timeAggregation =
  8. AggregationBuilders
  9. .dateHistogram("agg")
  10. .field("statisticTime")//统计时间聚合
  11. .dateHistogramInterval(dateHistogramInterval);
  12. MaxAggregationBuilder maxAggregation = AggregationBuilders.max("maxValue").field("value");//最大值聚合
  13. timeAggregation.subAggregation(maxAggregation);
  14. typeAggregation.subAggregation(timeAggregation);
  15. termAggregation.subAggregation(typeAggregation);
  16. SearchResponse response = client.prepareSearch(INDEX_NAME).setTypes(TYPE)
  17. .setQuery(boolQueryBuilder).addAggregation(termAggregation).execute().actionGet();
  18. Terms genders = response.getAggregations().get("psName");
  19. for (Terms.Bucket entry : genders.getBuckets()) {
  20. SystemDistribution systemDistribution=new SystemDistribution();
  21. String psName=entry.getKey().toString();
  22. systemDistribution.setHostName(psName);
  23. Terms typeTerm = entry.getAggregations().get("type");
  24. List<RiskStatisticsVo> memRiskStatistics=new ArrayList<>();
  25. List<RiskStatisticsVo> cpuRiskStatisTics=new ArrayList<>();
  26. for (Terms.Bucket entry1 : typeTerm.getBuckets()) {
  27. String type = entry1.getKeyAsString(); // Key as String 2017-12-27T00:00:00.000Z
  28. Histogram histogram=entry1.getAggregations().get("agg");
  29. for(Histogram.Bucket entry2 : histogram.getBuckets()){
  30. RiskStatisticsVo riskStatisticsVo=new RiskStatisticsVo();
  31. riskStatisticsVo.setRiskType(type);
  32. String statisTime=entry2.getKeyAsString();
  33. Max max=entry2.getAggregations().get("maxValue");
  34. Double maxValue=max.getValue();
  35. if(maxValue.equals(Double.NEGATIVE_INFINITY)){ //如果为无穷大,赋值为0
  36. maxValue=0.0;
  37. }//-Infinity
  38. riskStatisticsVo.setStatisticTime(formatReturnTime(statisTime,timeType));//2018-01-08T11:00:00.000Z
  39. riskStatisticsVo.setCount(maxValue.toString());
  40. if("mem".equals(type)){
  41. memRiskStatistics.add(riskStatisticsVo);
  42. }else{
  43. cpuRiskStatisTics.add(riskStatisticsVo);
  44. }
  45. }
  46. }
  47. systemDistribution.setcpuStatisticList(cpuRiskStatisTics);
  48. systemDistribution.setEmeStatisticList(memRiskStatistics);
  49. list.add(systemDistribution);
  50. }
  51. return list;
获得时间类型:

  1. private DateHistogramInterval getDateHistogramInterval(String dateType) {
  2. if(StatisticTimeTypeEnum.HOUR.getName().equals(dateType)){
  3. return DateHistogramInterval.minutes(10);//统计一个小时内数据,每隔10分钟一个显示
  4. }else if(StatisticTimeTypeEnum.Day.getName().equals(dateType)){
  5. return DateHistogramInterval.hours(4); //统计每日,每隔4小时统计
  6. }else if(StatisticTimeTypeEnum.WEEK.getName().equals(dateType)){
  7. return DateHistogramInterval.days(1); //每周,统计每天的数据统计
  8. }else{
  9. return DateHistogramInterval.days(5); //每月,每隔5天一个统计数据
  10. }
  11. }
格式化返回时间:

  1. private String formatReturnTime(String time,String dateType){
  2. if(StatisticTimeTypeEnum.HOUR.getName().equals(dateType)){
  3. return time.substring(11,16);
  4. }else if(StatisticTimeTypeEnum.Day.getName().equals(dateType)){
  5. return time.substring(8,10)+"日"+time.substring(11,13)+"时";
  6. }else{
  7. return time.substring(8,10)+"日";
  8. }
  9. }

猜你在找的设计模式相关文章