.Net Crank性能測試入門
Crank 是微軟新出的一個性能測試框架，集成了多種基準測試工具，如bombardier、wrk等。
Crank通過統一的配置，可以轉換成不同基準測試工具命令進行測試。可參考Bombardier Job實現。
安裝Crank
運行如下兩個命令分別安裝Crank的cli(Controller)和Agent。
dotnet tool update Microsoft.Crank.Controller --version "0.2.0-*" --global
dotnet tool update Microsoft.Crank.Agent--version "0.2.0-*" --global
需要.Net SDK 5.0環境
安裝完成後執行命令crank，會列印出如下可以配置的參數和介紹，或者github查看相關參數介紹。
PS C:\Users\Stack\Desktop> crank
Crank Benchmarks Controller
The Crank controller orchestrates benchmark jobs on Crank agents.
Usage: Crank [command] [options]
Options:
-?|-h|--help Show help information
-c|--config Configuration file or url
-s|--scenario Scenario to execute
-j|--job Name of job to define
--profile Profile name
--script Execute a named script available in the configuration files. Can be used multiple times.
-j|--json Saves the results as json in the specified file.
--csv Saves the results as csv in the specified file.
--compare An optional filename to compare the results to. Can be used multiple times.
--variable Variable
--sql Connection string of the SQL Server Database to store results in
--table Table name of the SQL Database to store results in
--session A logical identifier to group related jobs.
--description A string describing the job.
-p|--property Some custom key/value that will be added to the results, .e.g. --property arch=arm --property
os=linux
執行crank-agent，啟動基準測試所需的代理服務。github查看相關參數介紹。
PS C:\Users\Stack\Desktop> crank-agent
Hosting environment: Production
Content root path: C:\Users\Stack\.dotnet\tools\.store\microsoft.crank.agent\0.2.0-alpha.21567.1\microsoft.crank.agent\0.2.0-alpha.21567.1\tools\net5.0\any\
Now listening on: //[::]:5010
創建Crank配置文件
配置文件參考官方hello.benchmarks.yml示例
示例文件中引入了bombardier.yml，由於大環境對githubusercontent.com域名不太友好，可以考慮將bombardier.yml下載到本地，imports引入本地路徑或者直接將文件內容加入到新建的配置文件。
imports:
-//raw.githubusercontent.com/dotnet/crank/main/src/Microsoft.Crank.Jobs.Bombardier/bombardier.yml
生產中使用推薦imports文件的方式，crank配置文件中做少量配置就可以完成基準測試，並且可以引入不同的Microsoft.Crank.Jobs.XXX/XXX.yml,基於不同的基準測試工具進行測試。
其他測試工具配置文件都在//github.com/dotnet/crank/blob/main/src/Microsoft.Crank.XXX下。
variables:
headers:
none: ''
plaintext: '--header "Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7" --header "Connection: keep-alive"'
html: '--header "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" --header "Connection: keep-alive"'
json: '--header "Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7" --header "Connection: keep-alive"'
connectionclose: '--header "Connection: close"'
presetHeaders: none
jobs:
bombardier:
source:
repository: //github.com/dotnet/crank.git
branchOrCommit: main
project: src/Microsoft.Crank.Jobs.Bombardier/Microsoft.Crank.Jobs.Bombardier.csproj
sourceKey: bombardier
noBuild: true
readyStateText: Bombardier Client
waitForExit: true
variables:
connections: 256 #設置連接數
warmup: 15 #設置測試預熱次數
duration: 15 #設置測試時間
requests: 0 #設置測試請求實例數
rate: 0 #設置每秒請求頻率
transport: fasthttp # | http1 | http2 設置使用golang的fasthttp庫發送http請求
serverScheme: http
serverAddress: localhost
serverPort: 5000
path:
bodyFile: # path or url for a file to use as the body content
verb: # GET when nothing is specified
customHeaders: [ ] # list of headers with the format: '<name1>: <value1>', e.g. [ 'content-type: application/json' ]
arguments: "-c {{connections}} -w {{warmup}} -d {{duration}} -n {{requests}} --insecure -l {% if rate != 0 %} --rate {{ rate }} {% endif %} {% if transport %} --{{ transport}} {% endif %} {{headers[presetHeaders]}} {% for h in customHeaders %}{% assign s = h | split : ':' %}--header \"{{ s[0] }}: {{ s[1] | strip }}\" {% endfor %} {% if serverUri == blank or serverUri == empty %} {{serverScheme}}://{{serverAddress}}:{{serverPort}}{{path}} {% else %} {{serverUri}}:{{serverPort}}{{path}} {% endif %} {% if bodyFile != blank and bodyFile != empty %} -f {{bodyFile}} {% endif %} {% if verb != blank and verb != empty %} -m {{verb}} {% endif %}"
onConfigure:
# - job.timeout = Number(job.variables.duration) + Number(job.variables.warmup) + 10;
server:
source: #指定需要測試的項目，本文直接使用本地路徑
localFolder: .
project: crank_demo.csproj
readyStateText: Application started.
# source: 指定測試項目的遠程倉庫地址，並通過branchOrCommit指定分支
# repository: //github.com/dotnet/crank
# branchOrCommit: main
# project: samples/hello/hello.csproj
# readyStateText: Application started.
scenarios: #配置基準測試場景
crank_demo: #定義方案名，執行crank命令時指定該名稱
application:
job: server # 指定測試項目為上面定義的server
load:
job: bombardier # 指定測試工具bombardier
variables:
serverPort: 5000 #配置http服務埠
path: / #配置http服務地址
profiles:
local:
variables:
serverAddress: localhost
jobs:
application:
endpoints:
- //localhost:5010
load:
endpoints:
- //localhost:5010
啟動Crank-Agent
啟動agent後執行crank，會有一個如下安裝sdk的動作
[09:29:05.261] Runtime: 6.0.0 (Current)
[09:29:05.262] SDK: 6.0.100 (Current)
[09:29:05.263] ASP.NET: 6.0.0 (Current)
[09:29:05.265] Creating custom global.json
[09:29:05.266] Desktop: 6.0.0 (Current)
[09:29:05.266] Installing SDK '6.0.100' ...
所以啟動agent時多指定一個dotnethome參數，避免重複安裝sdk。
crank-agent --dotnethome 'C:\Program Files\dotnet'
啟動Crank
crank --config .\demo.benchmarks.yml --scenario crank_demo --profile local
–scenario crank_demo: 指定定義的測試場景
–profile local :設置輸出結果到本地，即控制台輸出
可以通過參數指定結果輸出到本地json文件（–output results.json ）或者資料庫（–sql [connection-string] –table [table-name]）
結果輸出
這裡省略壓測期間的日誌輸出，直接列出結果。
| application | |
| --------------------- | ------------- |
| CPU Usage (%) | 56 |
| Cores usage (%) | 447 |
| Working Set (MB) | 140 |
| Private Memory (MB) | 157 |
| Build Time (ms) | 7,232 |
| Start Time (ms) | 501 |
| Published Size (KB) | 91,292 |
| .NET Core SDK Version | 6.0.100 |
| ASP.NET Core Version | 6.0.0+ae1a6cb |
| .NET Runtime Version | 6.0.0+4822e3c |
| load | |
| --------------------- | -------------- |
| CPU Usage (%) | 46 |
| Cores usage (%) | 370 |
| Working Set (MB) | 29 |
| Private Memory (MB) | 30 |
| Build Time (ms) | 11,891 |
| Start Time (ms) | 226 |
| Published Size (KB) | 68,228 |
| .NET Core SDK Version | 3.1.415 |
| ASP.NET Core Version | 3.1.21+458d974 |
| .NET Runtime Version | 3.1.21+df8abc0 |
| First Request (ms) | 185 |
| Requests | 912,005 |
| Bad responses | 0 |
| Mean latency (us) | 4,207 |
| Max latency (us) | 138,999 |
| Requests/sec | 60,305 |
| Requests/sec (max) | 128,523 |
更多內容參考:
crank readme
Benchmarks crank為Benchmarks重構版本
