Openstack運維常見問題記錄(4)
- 2020 年 2 月 12 日
- 筆記
自動清除鏡像快取
在一台計算節點創建虛擬機，若是第一次在該節點創建次虛擬機，會先將鏡像文件複製到該計算節點目錄/var/lib/nova/instances/_base。長期下來，該目錄會佔用比較大的磁碟空間而要清理。
可以通過修改nova的配置文件來自動清理該快取目錄。
對比remove_unused_base_images=True設定前後的不同，即設定前多餘鏡像不會被自動清除，設定並重啟nova-compute服務後，多餘的鏡像文件會被自動清理，且可以正常生成雲主機。
remove_unused_base_images=True設定前：
其中鏡像文件be32a717ae7ea3aa039131689864a325fd634c92是沒有被使用的鏡像文件。 等待一段時間，文件沒有變化。
remove_unused_base_images=True設定並重啟nova-compute服務後:
image_cache_manager_interval=5，remove_unused_original_minimum_age_seconds=3是為了縮短測試時間作的設定，實際生產環境分別是用默認值40分鐘，設定值247小時，現在改成5秒，3秒。就是說不用此縮短測試時間的設定，生產環境是40分鐘檢查下image cache，超過247小時無用的base鏡像才會被刪除。
發現很快多餘的鏡像被自動刪除了。 檢查/var/log/nova/nova-compute.log，也找到了自動刪除多餘鏡像的log：
2017-02-07 16:01:51.678 14200 INFO nova.virt.libvirt.imagecache [req-ee1a7ad5-a021-4aad-b653-7db14191536e - - - - -] Active base files: /var/lib/nova/instances/_base/0522bc602608d45758d815b01a6899ff3e1e3e27 /var/lib/nova/instances/_base/dc1ed4ad70a573f2acea085b068b61f3cb99e195 /var/lib/nova/instances/_base/be32a717ae7ea3aa039131689864a325fd634c1c 2017-02-07 16:01:51.678 14200 INFO nova.virt.libvirt.imagecache [req-ee1a7ad5-a021-4aad-b653-7db14191536e - - - - -] Removable base files: /var/lib/nova/instances/_base/be32a717ae7ea3aa039131689864a325fd634c92 2017-02-07 16:01:51.679 14200 INFO nova.virt.libvirt.imagecache [req-ee1a7ad5-a021-4aad-b653-7db14191536e - - - - -] Removing base or swap file: /var/lib/nova/instances/_base/be32a717ae7ea3aa039131689864a325fd634c92
has allocations against this compute host but is not found in the database
現象： 創建vm報錯
[root@EXTENV-194-18-2-16 nova]# cat nova-compute.log | grep 620cd801-8849-481a-80e0-2980b6c8dba6 2018-08-23 15:23:36.136 3558 INFO nova.compute.resource_tracker [req-f76d5408-00f8-4a67-854e-ad3da2098811 - - - - -] Instance 620cd801-8849-481a-80e0-2980b6c8dba6 has allocations against this compute host but is not found in the database.
解決：
感覺是node的資訊資料庫不同步
nova show 出錯的vm，報cell錯誤
####每次增加一個計算節點在控制節點需要執行： # su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova
問題解決。
dashboard neutron報錯 Invalid service catalog service: network
現象： neutron服務是好的，命令行創建網路查看網路正常，就是dashboard不能查看網路相關的頁面 dashboard neutron報錯 Invalid service catalog service: network
解決： 應該是Keystone沒有正常配置。導致沒有找到相關的Catalog資訊。
[root@EXTENV-194-18-2-11 ~]# openstack catalog list +-----------+-----------+-----------------------------------------------+ | Name | Type | Endpoints | +-----------+-----------+-----------------------------------------------+ | placement | placement | RegionOne | | | | internal: http://nova-ha-vip:8778 | | | | RegionOne | | | | admin: http://nova-ha-vip:8778 | | | | RegionOne | | | | public: http://nova-ha-vip:8778 | | | | | | keystone | identity | RegionOne | | | | public: http://keystone-ha-vip:5000/v3/ | | | | RegionOne | | | | internal: http://keystone-ha-vip:35357/v3/ | | | | RegionOne | | | | admin: http://keystone-ha-vip:35357/v3/ | | | | | | glance | image | RegionOne | | | | admin: http://glance-ha-vip:9292 | | | | RegionOne | | | | internal: http://glance-ha-vip:9292 | | | | RegionOne | | | | public: http://glance-ha-vip:9292 | | | | | | nova | compute | RegionOne | | | | public: http://nova-ha-vip:8774/v2.1 | | | | RegionOne | | | | admin: http://nova-ha-vip:8774/v2.1 | | | | RegionOne | | | | internal: http://nova-ha-vip:8774/v2.1 | | | | | | neutron | network | | | neutron | network | RegionOne | | | | public: http://neutron-server-ha-vip:9696 | | | | RegionOne | | | | admin: http://neutron-server-ha-vip:9696 | | | | RegionOne | | | | internal: http://neutron-server-ha-vip:9696 | | | | | +-----------+-----------+-----------------------------------------------+
所以把第一條neutron沒有url的數據刪掉就好了，但是發現只有openstack catalog list，沒有openstack catalog delete命令，後來查看keystone的配置文件keystone.conf發現如下配置 見[catalog] 從配置文件看出catalog是從mysql裡面讀取的數據，然後從mysql的keystone庫中的service表裡找到了臟數據，然後知道了用openstack service delete去刪除『臟數據』，問題就解決了。
MariaDB [keystone]> select * from service; +----------------------------------+-----------+---------+-------------------------------------------------------------+ | id | type | enabled | extra | +----------------------------------+-----------+---------+-------------------------------------------------------------+ | 520f6bf8564240be9678c4ef25305cad | placement | 1 | {"description": "OpenStack Placement", "name": "placement"} | | 960580852a594c078e68fe3683e35db5 | identity | 1 | {"name": "keystone"} | | 98ed18fcd8104732919bb5869a5a6dc2 | image | 1 | {"description": "OpenStack Image", "name": "glance"} | | abef1b9469d94d3ab9f27c8ed72a5a48 | compute | 1 | {"description": "OpenStack Compute", "name": "nova"} | | e37085e8fb2a49c0921c2d24f5e4f9b5 | network | 1 | {"description": "OpenStack Networking", "name": "neutron"} | | f1b661407ce04f79bc24605fa59bb74c | network | 1 | {"description": "OpenStack Networking", "name": "neutron"} | +----------------------------------+-----------+---------+-------------------------------------------------------------+ 6 rows in set (0.00 sec) MariaDB [keystone]> select * from endpoint; +----------------------------------+--------------------+-----------+----------------------------------+-----------------------------------+-------+---------+-----------+ | id | legacy_endpoint_id | interface | service_id | url | extra | enabled | region_id | +----------------------------------+--------------------+-----------+----------------------------------+-----------------------------------+-------+---------+-----------+ | 142cb619cd2242828b0c9394d5baaea1 | NULL | public | f1b661407ce04f79bc24605fa59bb74c | http://neutron-server-ha-vip:9696 | {} | 1 | RegionOne | | 2252d3ef840b4c5aa1184ebe8d6094f1 | NULL | public | abef1b9469d94d3ab9f27c8ed72a5a48 | http://nova-ha-vip:8774/v2.1 | {} | 1 | RegionOne | | 476654c6e7dd4d22b290de451e3afda0 | NULL | admin | abef1b9469d94d3ab9f27c8ed72a5a48 | http://nova-ha-vip:8774/v2.1 | {} | 1 | RegionOne | | 562a5d5443af4dfab6760204d0adf3bf | NULL | internal | 520f6bf8564240be9678c4ef25305cad | http://nova-ha-vip:8778 | {} | 1 | RegionOne | | 58bd5f09811a4ebcb62a4b51fb7ae444 | NULL | admin | f1b661407ce04f79bc24605fa59bb74c | http://neutron-server-ha-vip:9696 | {} | 1 | RegionOne | | 600811f8ccaf42669d4d83b897af3933 | NULL | admin | 520f6bf8564240be9678c4ef25305cad | http://nova-ha-vip:8778 | {} | 1 | RegionOne | | 80683f619efb41dcbb6796ea04f16159 | NULL | internal | f1b661407ce04f79bc24605fa59bb74c | http://neutron-server-ha-vip:9696 | {} | 1 | RegionOne | | 8e0a684607294a729f87d7d8b1a639ca | NULL | public | 520f6bf8564240be9678c4ef25305cad | http://nova-ha-vip:8778 | {} | 1 | RegionOne | | 9ef0f18d891e45608ffc41985dc6afa6 | NULL | public | 960580852a594c078e68fe3683e35db5 | http://keystone-ha-vip:5000/v3/ | {} | 1 | RegionOne | | a0b10cb04a5b4ca3859aaf2ea4ca2a3b | NULL | admin | 98ed18fcd8104732919bb5869a5a6dc2 | http://glance-ha-vip:9292 | {} | 1 | RegionOne | | c53979becccc44f1813e9f50a619af7e | NULL | internal | 960580852a594c078e68fe3683e35db5 | http://keystone-ha-vip:35357/v3/ | {} | 1 | RegionOne | | dadbb8dc218245bbba8c9a34237413ec | NULL | internal | 98ed18fcd8104732919bb5869a5a6dc2 | http://glance-ha-vip:9292 | {} | 1 | RegionOne | | f4034b8c086a451caed52ac51a761fb0 | NULL | public | 98ed18fcd8104732919bb5869a5a6dc2 | http://glance-ha-vip:9292 | {} | 1 | RegionOne | | fc150884825544baaf4912f14e76f51a | NULL | internal | abef1b9469d94d3ab9f27c8ed72a5a48 | http://nova-ha-vip:8774/v2.1 | {} | 1 | RegionOne | | fc7132052063438895674fd7b840db68 | NULL | admin | 960580852a594c078e68fe3683e35db5 | http://keystone-ha-vip:35357/v3/ | {} | 1 | RegionOne | +----------------------------------+--------------------+-----------+----------------------------------+-----------------------------------+-------+---------+-----------+ 15 rows in set (0.00 sec) [root@EXTENV-194-18-2-11 ~]# openstack service list +----------------------------------+-----------+-----------+ | ID | Name | Type | +----------------------------------+-----------+-----------+ | 520f6bf8564240be9678c4ef25305cad | placement | placement | | 960580852a594c078e68fe3683e35db5 | keystone | identity | | 98ed18fcd8104732919bb5869a5a6dc2 | glance | image | | abef1b9469d94d3ab9f27c8ed72a5a48 | nova | compute | | e37085e8fb2a49c0921c2d24f5e4f9b5 | neutron | network | | f1b661407ce04f79bc24605fa59bb74c | neutron | network | +----------------------------------+-----------+-----------+ [root@EXTENV-194-18-2-11 ~]# openstack service delete e37085e8fb2a49c0921c2d24f5e4f9b5 [root@EXTENV-194-18-2-11 ~]# systemctl restart httpd.service memcached.service
系列文章鏈接
- Openstack運維常見問題記錄(3)
- Openstack運維常見問題記錄(2)
- Openstack運維常見問題記錄(1)
