.NetCore使用Docker安裝ElasticSearch、Kibana 記錄日誌

2021 年 7 月 1 日
筆記
.NET Core, docker, elasticsearch, Kibana

前言

最近園子里看到一篇<.Net Core with 微服務 – Elastic APM> 的文章(主要用於對接口的調用鏈、性能進行監控),非常實用,這裡講解.NetCore將日誌寫入ElasticSearch,並用Kibana搜索日誌

部署環境

centos

依賴環境

docker
docker-compose
.net core 3.1

使用部署

1.創建 docker-compose.yml 文件

注意事項:

設置es內存,java程序一般很吃內存,根據服務器配置進行調優 – “ES_JAVA_OPTS=-Xms512m -Xmx512m”
kibana漢化(7.0以上版本),根據個人情況決定 – output.i18n.locale=”zh-CN”
安裝apm_server,這個模塊非必須項,不需要的可以去掉,感興趣可以參考 <.Net Core with 微服務 – Elastic APM>
拉取鏡像失敗,由於網絡原因,可能存在拉取鏡像失敗,網絡較差建議單獨拉取鏡像,再運行

version: '3.1'
services:
  elasticsearch:
   container_name: elasticsearch
   hostname: elasticsearch
   image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
   restart: always
   ports:
    - 9200:9200
    - 9300:9300
   volumes:
    - elasticsearch-data:/usr/share/elasticsearch/data
   environment:
    - xpack.monitoring.enabled=true
    - xpack.watcher.enabled=false
    - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    - discovery.type=single-node
  kibana:
   container_name: kibana
   hostname: kibana
   image: docker.elastic.co/kibana/kibana:7.9.2
   restart: always
   ports:
    - 5601:5601
   depends_on:
    - elasticsearch
   environment:
    - ELASTICSEARCH_URL=//localhost:9200
  apm_server:
    image: docker.elastic.co/apm/apm-server:7.9.2
    restart: always
    container_name: apm_server
    hostname: apm_server
    command: --strict.perms=false -e
    environment:
      - output.elasticsearch.hosts=["localhost:9200"]
      - output.i18n.locale="zh-CN"
    ports:
      - 8200:8200
    depends_on:
      - kibana
      - elasticsearch
volumes:
  elasticsearch-data:

2.運行 docker-compose.yml 文件

注意事項:

容器啟動後,kibana需要一段時間初始化
本次需要開放的端口 es:9200,9300 kibana:5601 apm-server:8200

docker-compose up -d    //構建啟動容器  -d 後台運行
docker-compose down     //停止up 命令所啟動的容器，並移除網絡
docker-compose stop     //停止容器
docker-compose start    //啟動容器
docker-compose restart  //重啟容器

3.啟動kibana web界面,瀏覽器輸入://[ip]:5601/app/home#/ [ip]為服務器ip地址或域名

能正常的打開界面,則ElasticSearch、Kibana、Apm 部署完成

4.使用 ASP.NET Core 和 Serilog 記錄到 Elasticsearch

1.新建一個.net core web程序(.net core3.1)

2.將以下 Serilog 包添加到項目中

Serilog.AspNetCore
Serilog.Enrichers.Environment
Serilog.Sinks.Debug
Serilog.Sinks.ElasticSearch

3.刪除 appsettings.json 中的 Logging 部分並將其替換為以下配置,如果是dev環境,則修改appsettings.Development.json文件

{
  "Serilog": {
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Information",
        "System": "Warning"
      }
    }
  },
  "ElasticConfiguration": {
    "Uri": "//localhost:9200" //這裡填寫es地址
  },
  "AllowedHosts": "*"
}

4.在 Program.cs 中配置登錄
注意在es中創建的索引,這裡的索引是it-tibos-api,可以根據業務進行修改

 public class Program
    {
        public static void Main(string[] args)
        {
            //configure logging first
            ConfigureLogging();

            //then create the host, so that if the host fails we can log errors
            CreateHost(args);
        }

        private static void ConfigureLogging()
        {
            var environment = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");
            var configuration = new ConfigurationBuilder()
                .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
                .AddJsonFile(
                    $"appsettings.{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT")}.json",
                    optional: true)
                .Build();

            Log.Logger = new LoggerConfiguration()
                .Enrich.FromLogContext()
                .Enrich.WithExceptionDetails()
                .Enrich.WithMachineName()
                //.WriteTo.Debug()
                .WriteTo.Console()
                .WriteTo.Elasticsearch(ConfigureElasticSink(configuration, environment))
                .Enrich.WithProperty("Environment", environment)
                .ReadFrom.Configuration(configuration)
                .CreateLogger();
        }

        private static ElasticsearchSinkOptions ConfigureElasticSink(IConfigurationRoot configuration, string environment)
        {
            return new ElasticsearchSinkOptions(new Uri(configuration["ElasticConfiguration:Uri"]))
            {
                AutoRegisterTemplate = true,
                IndexFormat = $"it-tibos-api" //索引
            };
        }

        private static void CreateHost(string[] args)
        {
            try
            {
                CreateHostBuilder(args).Build().Run();
            }
            catch (System.Exception ex)
            {
                Log.Fatal($"Failed to start {Assembly.GetExecutingAssembly().GetName().Name}", ex);
                throw;
            }
        }

        public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)
                .ConfigureWebHostDefaults(webBuilder =>
                {
                    webBuilder.UseStartup<Startup>();
                })
                .ConfigureAppConfiguration(configuration =>
                {
                    configuration.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);
                    configuration.AddJsonFile(
                        $"appsettings.{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT")}.json",
                        optional: true);
                })
                .UseSerilog();
    }

5.在action里記錄日誌,並運行程序

    _logger.LogInformation($"這是一條測試日誌,發送時間{DateTime.Now}");
    _logger.LogWarning($"這是一條測試警告日誌,發送時間{DateTime.Now}");
    try
    {
        throw new Exception("Some bad code was executed");
    }
    catch (Exception ex)
    {
        _logger.LogError($"這是一條異常日誌,發送時間{DateTime.Now}");
    }