Allure-pytest功能特性介紹

  • 2019 年 10 月 3 日
  • 筆記

前言

Allure框架是一個靈活的輕量級多語言測試報告工具,它不僅以web的方式展示了簡介的測試結果,而且允許參與開發過程的每個人從日常執行的測試中最大限度的提取有用信息
從dev/qa的角度來看,Allure報告簡化了常見缺陷的統計:失敗的測試可以分為bug和被中斷的測試,還可以配置日誌、步驟、fixture、附件、計時、執行歷史以及與TMS和BUG管理系統集成,所以,通過以上配置,所有負責的開發人員和測試人員可以儘可能的掌握測試信息。
從管理者的角度來看,Allure提供了一個清晰的“大圖”,其中包括已覆蓋的特性、缺陷聚集的位置、執行時間軸的外觀以及許多其他方便的事情。allure的模塊化和可擴展性保證了您總是能夠對某些東西進行微調,使Allure更適合您,那麼今天我們就來說說如何使報告更加詳細的顯示我們需要的信息,以及allure與jenkins的集成

生成報告

pytest框架編寫的項目如何生成測試報告,這裡將不再講解,具體過程可以參考:pytest進階之html測試報告

注意:python使用的allure插件為allure-pytest

測試代碼

為了大家能夠快速的認識allure的所有功能特性,附上完整的測試代碼

"""  ------------------------------------  @Time : 2019/8/28 19:50  @Auth : linux超  @File : conftest.py  @IDE  : PyCharm  @Motto: Real warriors,dare to face the bleak warning,dare to face the incisive error!  @QQ   : [email protected]  @GROUP: 878565760  ------------------------------------  """  import pytest  import allure        @pytest.mark.hookwrapper  def pytest_runtest_makereport(item):      outcome = yield      report = outcome.get_result()      report.nodeid = report.nodeid.encode("utf-8").decode("unicode_escape")  # 解決亂碼      @allure.step("打開瀏覽器")  def fixture_step():      pass      @pytest.fixture  def init_url():      fixture_step()      yield True

conftest.py

"""  ------------------------------------  @Time : 2019/9/4 21:05  @Auth : linux超  @File : test_allure_feature.py  @IDE  : PyCharm  @Motto: Real warriors,dare to face the bleak warning,dare to face the incisive error!  @QQ   : [email protected]  @GROUP: 878565760  ------------------------------------  """  import pytest  import allure  import os      def login(username=None, password=None):      """模擬登錄"""      user = "linux超"      pwd = "123456"      if user == username and pwd == password:          return {"code": 1001, "msg": "登錄成功", "data": None}      elif "" == password or password is None and username:          return {"code": 1002, "msg": "密碼不能為空", "data": None}      elif "" == username or username is None and password:          return {"code": 1003, "msg": "用戶名不能為空", "data": None}      else:          return {"code": 1004, "msg": "用戶名或密碼錯誤", "data": None}      @allure.step("輸入用戶名")  def input_username(user):      print("輸入用戶名")      return user      @allure.step("輸入密碼")  def input_password(pwd):      print("輸入密碼")      return pwd      login_success_data = [      # 測試數據      {          "case": "用戶名正確, 密碼正確",          "user": "linux超",          "pwd": "123456",          "expected": {"code": 1001, "msg": "登錄成功", "data": None}      }  ]    login_fail_data = [      {          "case": "用戶名正確, 密碼為空",          "user": "linux超",          "pwd": "",          "expected": {"code": 1002, "msg": "密碼不能為空", "data": None}      },      {          "case": "用戶名為空, 密碼正確",          "user": "",          "pwd": "linux超哥",          "expected": {"code": 1003, "msg": "用戶名不能為空", "data": None}      },      {          "case": "用戶名錯誤, 密碼錯誤",          "user": "linux",          "pwd": "linux",          "expected": {"code": 1004, "msg": "用戶名或密碼錯誤", "data": None}      }  ]    username_none = [      {          "case": "缺省用戶名參數",          "pwd": "123456",          "expected": {"code": 1003, "msg": "用戶名不能為空", "data": None}      }  ]  password_none = [      {          "case": "缺省密碼參數",          "user": "linux超",          "expected": {"code": 1002, "msg": "密碼不能為空", "data": None}      }  ]  # 改變輸出結果  ids_login_success_data = [      "測試{}用戶名:{}密碼{}期望值{}".          format(data["case"], data["user"], data["pwd"], data["expected"]) for data in login_success_data  ]  ids_login_fail_data = [      "測試{}用戶名:{}密碼{}期望值{}".          format(data["case"], data["user"], data["pwd"], data["expected"]) for data in login_fail_data  ]  ids_username_none = [      "測試{}密碼{}期望值{}".          format(data["case"], data["pwd"], data["expected"]) for data in username_none  ]  ids_password_none = [      "測試{}用戶名:{}期望值{}".          format(data["case"], data["user"], data["expected"]) for data in password_none  ]      @allure.feature("登錄模塊")  class TestLogin(object):        @allure.severity(allure.severity_level.BLOCKER)      @allure.story("測試登錄成功")      @allure.title("登錄成功場景-{data}")      @pytest.mark.parametrize("data", login_success_data, ids=ids_login_success_data)      def test_login_success(self, data):          """測試登錄成功"""          user = input_username(data["user"])          pwd = input_password(data["pwd"])          result = login(user, pwd)          assert result == data["expected"]        @allure.severity(allure.severity_level.CRITICAL)      @allure.story("測試登錄失敗")      @pytest.mark.parametrize("data", login_fail_data, ids=ids_login_fail_data)      def test_login_fail(self, data):          """測試用戶名或密碼錯誤"""          user = input_username(data["user"])          pwd = input_password(data["pwd"])          result = login(user, pwd)          assert result == data["expected"]        @allure.severity(allure.severity_level.MINOR)      @allure.story("測試用戶名參數缺失")      @pytest.mark.parametrize("data", username_none, ids=ids_username_none)      def test_username_none(self, data):          """測試缺省用戶名"""          pwd = input_password(data["pwd"])          result = login(password=pwd)          assert result == data["expected"]        @allure.severity(allure.severity_level.MINOR)      @allure.story("測試密碼參數缺失")      @pytest.mark.parametrize("data", password_none, ids=ids_password_none)      def test_password_none(self, data):          """測試缺省密碼"""          user = input_username(data["user"])          result = login(username=user)          assert result == data["expected"]        @allure.severity(allure.severity_level.MINOR)      @allure.story("測試初始化地址")      @allure.testcase("https://www.cnblogs.com/linuxchao/", "測試用例地址")      def test_init_url(self, init_url):          flag = init_url          assert flag is True        @allure.severity(allure.severity_level.NORMAL)      @allure.story("測試失敗用例與用例中添加附件")      @allure.link("https://www.cnblogs.com/linuxchao/", name="bug鏈接")      @allure.description("這是一個一直執行失敗的測試用例")      def test_failed(self):          """你也可以在這裡添加用例的描述信息,但是會被allure.description覆蓋"""          try:              assert False          except AssertionError as e:              with open("attach.png", "rb") as f:                  context = f.read()                  allure.attach(context, "錯誤圖片", attachment_type=allure.attachment_type.PNG)              raise e        @allure.severity(allure.severity_level.TRIVIAL)      @allure.story("測試broken用例")      @allure.issue("https://www.cnblogs.com/linuxchao/", "錯誤鏈接")      def test_broken(self):          """broken"""          with open("broken.json", "r", encoding='utf8') as f:              f.read()        @allure.severity(allure.severity_level.TRIVIAL)      @allure.story("測試無條件跳過測試用例")      @pytest.mark.skip(reason="無條件跳過")      def test_skip(self):          """skip"""          pass      if __name__ == '__main__':      pytest.main(["-vsq",                   "--alluredir", "./allure-results", ])      os.system(r"allure generate --clean ./allure-results -o ./allure-report")

test_allure_feature.py

"""  ------------------------------------  @Time : 2019/8/28 19:45  @Auth : linux超  @File : test_allure_fixture.py  @IDE  : PyCharm  @Motto: Real warriors,dare to face the bleak warning,dare to face the incisive error!  @QQ   : [email protected]  @GROUP: 878565760  ------------------------------------  """  import pytest  import os  import allure      def function_scope_step():      print("function_scope_step")      def class_scope_step():      print("class_scope_step")      def module_scope_step():      print("module_scope_step")      def session_scope_step():      print("session_scope_step")      def step_inside_test_body():      print("step_inside_test_body")      @pytest.fixture(params=[True, False], ids=['param_true', 'param_false'])  def function_scope_fixture_with_finalizer(request):      if request.param:          print('True')      else:          print('False')        def function_scope_finalizer():          function_scope_step()        request.addfinalizer(function_scope_finalizer)      @pytest.fixture(scope='class')  def class_scope_fixture_with_finalizer(request):      def class_finalizer_fixture():          class_scope_step()        request.addfinalizer(class_finalizer_fixture)      @pytest.fixture(scope='module')  def module_scope_fixture_with_finalizer(request):      def module_finalizer_fixture():          module_scope_step()        request.addfinalizer(module_finalizer_fixture)      @pytest.fixture(scope='session')  def session_scope_fixture_with_finalizer(request):      def session_finalizer_fixture():          session_scope_step()        request.addfinalizer(session_finalizer_fixture)      @allure.severity(allure.severity_level.BLOCKER)  @allure.feature("fixture場景")  class TestClass(object):        def test_with_scoped_finalizers(self,                                      function_scope_fixture_with_finalizer,                                      class_scope_fixture_with_finalizer,                                      module_scope_fixture_with_finalizer,                                      session_scope_fixture_with_finalizer):          step_inside_test_body()      if __name__ == '__main__':      pytest.main(["-vsq",                   "--alluredir", "./allure-results", ])      os.system(r"allure generate --clean ./allure-results -o ./allure-report")

test_allure_fixture.py

[    {      "name": "Ignored tests",      "matchedStatuses": ["skipped"]    },    {      "name": "Infrastructure problems",      "matchedStatuses": ["broken", "failed"],      "messageRegex": ".*bye-bye.*"    },    {      "name": "Outdated tests",      "matchedStatuses": ["broken"],      "traceRegex": ".*FileNotFoundException.*"    },    {      "name": "Product defects",      "matchedStatuses": ["failed"]    },    {      "name": "Test defects",      "matchedStatuses": ["broken"]    }  ]

categories.json

Browser=Chrome  Browser.Version=63.0  Stand=Production  ApiUrl=127.0.0.1/login  python.Version=3.6

environment.properties

"""  ------------------------------------  @Time : 2019/9/3 14:21  @Auth : linux超  @File : run.py  @IDE  : PyCharm  @Motto: Real warriors,dare to face the bleak warning,dare to face the incisive error!  @QQ   : [email protected]  @GROUP: 878565760  ------------------------------------  """  import pytest  import os      if __name__ == '__main__':      pytest.main(["-sq",                   "--alluredir", "./allure-results"])      os.system(r"allure generate --clean allure-results -o allure-report")

run.py

目錄結構

Allure特性

Environment

在Allure報告中添加環境信息,通過創建environment.properties或者environment.xml文件,並把文件存放到allure-results(這個目錄是生成最後的html報告之前,生成依賴文件的目錄)目錄下

environment.properties

Browser=Chrome  Browser.Version=63.0  Stand=Production  ApiUrl=127.0.0.1/login  python.Version=3.6

或者

environment.xml

<environment>      <parameter>          <key>Browser</key>          <value>Chrome</value>      </parameter>      <parameter>          <key>Browser.Version</key>          <value>63.0</value>      </parameter>      <parameter>          <key>Stand</key>          <value>Production</value>      </parameter>          <parameter>          <key>ApiUrl</key>          <value>127.0.0.1/login</value>      </parameter>          <parameter>          <key>python.Version</key>          <value>3.6</value>      </parameter>  </environment>

執行run.py查看報告

Categories

測試報告默認統計兩種類型的測試用例結果,失敗的用例和故障測試用例,我們可以自定義添加用例的統計類型,同樣需要在allure-results目錄下新建categories.json文件

[    {      "name": "Ignored tests",      "matchedStatuses": ["skipped"]    },    {      "name": "Infrastructure problems",      "matchedStatuses": ["broken", "failed"],      "messageRegex": ".*bye-bye.*"    },    {      "name": "Outdated tests",      "matchedStatuses": ["broken"],      "traceRegex": ".*FileNotFoundException.*"    },    {      "name": "Product defects",      "matchedStatuses": ["failed"]    },    {      "name": "Test defects",      "matchedStatuses": ["broken"]    }  ]

執行run.py查看報告

Fixtures and Finalizers

Fixtures和Finalizers是pytest在測試開始和測試結束調用的方法,allure會自動跟蹤每一個fixture的調用,並且詳細顯示會調用哪些fixture和參數,而且會保留正確的調用順數

測試代碼

test_allure_html.py

def function_scope_step():      print("function_scope_step")      def class_scope_step():      print("class_scope_step")      def module_scope_step():      print("module_scope_step")      def session_scope_step():      print("session_scope_step")      def step_inside_test_body():      print("step_inside_test_body")      @pytest.fixture(params=[True, False], ids=['param_true', 'param_false'])  def function_scope_fixture_with_finalizer(request):      if request.param:          print('True')      else:          print('False')        def function_scope_finalizer():          function_scope_step()        request.addfinalizer(function_scope_finalizer)      @pytest.fixture(scope='class')  def class_scope_fixture_with_finalizer(request):      def class_finalizer_fixture():          class_scope_step()        request.addfinalizer(class_finalizer_fixture)      @pytest.fixture(scope='module')  def module_scope_fixture_with_finalizer(request):      def module_finalizer_fixture():          module_scope_step()        request.addfinalizer(module_finalizer_fixture)      @pytest.fixture(scope='session')  def session_scope_fixture_with_finalizer(request):      def session_finalizer_fixture():          session_scope_step()        request.addfinalizer(session_finalizer_fixture)      class TestClass(object):        def test_with_scoped_finalizers(self,                                      function_scope_fixture_with_finalizer,                                      class_scope_fixture_with_finalizer,                                      module_scope_fixture_with_finalizer,                                      session_scope_fixture_with_finalizer):          step_inside_test_body()

執行run.py查看報告

@allure.step

pytest支持使用@allure.step修飾某些測試用例中需要的函數,使測試用例在allure報告中能夠更加詳細的顯示測試過程

測試代碼

test_allure_feature.py文件中修改如下代碼

@allure.step("輸入用戶名")  def input_username():      print("輸入用戶名")      @allure.step("輸入密碼")  def input_password():      print("輸入密碼")

執行run.py查看報告

conftest.py

@allure.step修飾的測試步驟還支持在conftest.py文件中定義,作為fixture的步驟,現在我們在項目目錄下新建conftest.py文件,寫入如下代碼

conftest.py

@allure.step("打開瀏覽器")  def fixture_step():      pass      @pytest.fixture  def init_url():      fixture_step()      yield True

test_allure_feature.py文件中添加如下用例

    def test_init_url(self, init_url):          flag = init_url          assert flag == True

執行run.py查看報告

allure.attach

使用allure.attach可以給報告中添加文件,圖片,log,html代碼等等。 我們修改test_allure_feature.py中如下用例, 並在用例所在目錄添加attach.png圖片

    def test_failed(self):          """failed"""          try:              assert False          except AssertionError as e:              with open("attach.png", "rb") as f:                  context = f.read()                  allure.attach(context, "錯誤圖片", attachment_type=allure.attachment_type.PNG)              raise e

執行run.py查看報告

@allure.description

如果你想在報告中展示測試用例的描述信息,那麼你可以使用@allure.description(string)或者@allure.description_html(html代碼)修飾你的測試用例,test_allure_feature.py文件修改如下代碼

    @allure.description("這是一個一直執行失敗的測試用例")      def test_failed(self):          """你也可以在這裡添加用例的描述信息,但是會被allure.description覆蓋"""          try:              assert False          except AssertionError as e:              with open("attach.png", "rb") as f:                  context = f.read()                  allure.attach(context, "錯誤圖片", attachment_type=allure.attachment_type.PNG)              raise e

執行run.py查看報告

@allure.title

使用allure.title(title)可以重命名測試用例在allure報告中的名稱,test_allure_feature.py文件修改如下代碼

    @allure.title("登錄成功場景-{data}")      @pytest.mark.parametrize("data", login_success_data, ids=ids_login_success_data)      def test_login_success(self, data):          """測試登錄成功"""          user = input_username(data["user"])          pwd = input_password(data["pwd"])          result = login(user, pwd)          assert result == data["expected"]

@allure.link

@allure.testcase

@allure.issue

這三種特性都可以給測試用例添加一個鏈接,test_allure_feature.py文件修改如下代碼

    @allure.testcase("https://www.cnblogs.com/linuxchao/", "測試用例地址")      def test_init_url(self, init_url):          flag = init_url          assert flag == True        @allure.link("https://www.cnblogs.com/linuxchao/", name="bug鏈接")      @allure.description("這是一個一直執行失敗的測試用例")      def test_failed(self):          """你也可以在這裡添加用例的描述信息,但是會被allure.description覆蓋"""          try:              assert False          except AssertionError as e:              with open("attach.png", "rb") as f:                  context = f.read()                  allure.attach(context, "錯誤圖片", attachment_type=allure.attachment_type.PNG)              raise e        @allure.issue("https://www.cnblogs.com/linuxchao/", "錯誤鏈接")      def test_broken(self):          """broken"""          with open("broken.json", "r", encoding='utf8') as f:              f.read()

執行run.py查看報告

 

 

 

@allure.feature

@allure.story

feature和story被稱為行為驅動標記,因為使用這個兩個標記,通過報告可以更加清楚的掌握每個測試用例的功能和每個測試用例的測試場景

在test_allure_feature.py文件中的測試類使用@allure.feature修飾, 測試方法使用@allure.story修飾

執行run.py查看報告

 以上兩種標記不僅僅能夠在測試報告中顯示,而且還可以使用命令執行指定的測試模塊或者場景

@allure.severity

此標記用來標識測試用例或者測試類的級別,分為blocker,critical,normal,minor,trivial5個級別,下面們把測試用例按級別標記,並查看一下測試報告

總結

以上就是所有的allure-pytest插件在pytest中支持的大部分功能特性,也許整理的不是很詳細,所以如果你想詳細的了解具體的特性在報告中的效果,還需自己動手嘗試一下,附上本文參考鏈接

https://docs.qameta.io/allure/