凸函數及其性質
1. 定義
1.1 定義一
- 如果對任意\(x_1\)、\(x_2\)總有\(f[\alpha x_1 + (1 – \alpha )x_2] \ge \alpha f(x_1) + (1 – \alpha )f(x_2)\)，其中\(\displaystyle 0 \le \alpha \le 1\)，則稱\(f(x)\)為上凸函數
- 如果對任意\(x_1\)、\(x_2\)且\(x_1 \ne x_2\)，總有\(f[\alpha x_1 + (1 – \alpha )x_2] \gt \alpha f(x_1) + (1 – \alpha )f(x_2)\)，其中\(0 \lt\alpha \lt 1\)，則稱\(f(x)\)為嚴格上凸函數
1.2 定義二
- 如果對任意\(x_1\)、\(x_2\)總有\(f[\alpha x_1 + (1 – \alpha )x_2]\le \alpha f(x_1) + (1 – \alpha )f(x_2)\)，其中\(\displaystyle 0 \le \alpha \le 1\)，則稱\(f(x)\)為下凸函數
- 如果對任意\(x_1\)、\(x_2\)且\(x_1 \ne x_2\)，總有\(f[\alpha x_1 + (1 – \alpha )x_2] \lt \alpha f(x_1) + (1 – \alpha )f(x_2)\)，其中\(0 \lt\alpha \lt 1\)，則稱\(f(x)\)為嚴格下凸函數
2. 琴生（Jenson）不等式
- 對於上凸函數，\(f(E[X]) \ge E[f(x)]\)或\(\displaystyle \sum_{k=1}^q \lambda_k f(x_k)\le f(\sum_{k=1}^q \lambda_k x_k)\)，其中\(\lambda_1,\lambda_2,\cdots,\lambda_q\)為正實數（或非負實數，後者去除無影響的\(\lambda_i = 0\)的項即為前者，故二者等價）且\(\displaystyle \sum_{k=1}^q \lambda_k = 1\)；對於嚴格上凸函數，上述等號成立當且僅當\(x_1 = x_2 = \cdots = x_q\)。
- 對於下凸函數，\(f(E[X])\le E[f(x)]\)或\(\displaystyle \sum_{k=1}^q \lambda_k f(x_k) \ge f(\sum_{k=1}^q \lambda_k x_k)\)，其中\(\lambda_1,\lambda_2,\cdots,\lambda_q\)為正實數（或非負實數，後者去除無影響的\(\lambda_i = 0\)的項即為前者，故二者等價）且\(\displaystyle \sum_{k=1}^q \lambda_k = 1\)；對於嚴格下凸函數，上述等號成立當且僅當\(x_1 = x_2 = \cdots = x_q\)。
\(\downarrow\)證明過程如下\(\downarrow\)
2.1 上凸函數
證明：因為\(\lambda_i\)均為正實數，故有
\(\displaystyle f( \sum_{k=1}^q \lambda_k x_k) = f(\lambda_1 x_1 + \sum_{k=2}^q \lambda_k {\sum_{k=2}^q \lambda_k x_k \over \sum_{k=2}^q \lambda_k}) \ge \lambda_1 f(x_1) + \sum_{k=2}^q \lambda_k \cdot f({\sum_{k=2}^q \lambda_k x_k \over \sum_{k=2}^q \lambda_k})\)
\(\displaystyle = \lambda_1 f(x_1) + \sum_{k=2}^q \lambda_k \cdot f({\lambda_2 \over \sum_{k=2}^q \lambda_k} x_2 + {\sum_{k=3}^q \lambda_k \over \sum_{k=2}^q \lambda_k} \cdot {\sum_{k=3}^q \lambda_k x_k \over \sum_{k=3}^q \lambda_k})\)
\(\displaystyle \ge \lambda_1 f(x_1) + \lambda_2 f(x_2) + \sum_{k=3}^q \lambda_k \cdot f({\sum_{k=3}^q \lambda_k x_k \over \sum_{k=3}^q \lambda_k})\)
\(\displaystyle \ge \cdots \ge \sum_{k=1}^q \lambda_k f(x_k)\)
2.2 嚴格上凸函數
證明：由定義可知，對於嚴格上凸函數，\(f[\alpha x_1 + (1 – \alpha )x_2] \ge \alpha f(x_1) + (1 – \alpha )f(x_2)\)等號成立時當且僅當\(x_1 = x_2\) 。而根據上文對於上凸函數對於\(\displaystyle \sum_{k=1}^q \lambda_k f(x_k)\le f(\sum_{k=1}^q \lambda_k x_k)\)不等式推導過程可知，若上凸函數為嚴格上凸函數，則第一個\(\ge\)處等號成立當且僅當：\(x_1 = {\sum_{k=2}^q \lambda_k x_k \over \sum_{k=2}^q \lambda_k}\)；第二個\(\ge\)處等號成立當且僅當：\(x_2 = {\sum_{k=3}^q \lambda_k x_k \over \sum_{k=3}^q \lambda_k}\)；\(\cdots\)；第\(q-1\)個\(\ge\)處等號成立當且僅當：\(x_{q-1} = {\sum_{k=q}^q \lambda_k x_k \over \sum_{k=q}^q \lambda_q} = x_q\)。所有等號都成立則以上條件都需滿足，對以上條件反向推導可得：\(x_q = x_{q-1}\)；\(x_{q-2} = {\sum_{k=q-1}^q \lambda_k x_k \over \sum_{k=q-1}^q \lambda_k} = {\lambda_{q-1} x_{q-1} + \lambda_{q} x_q \over \lambda_{q-1} + \lambda_{q}} = x_{q-1}\)；\(\cdots\)；\(x_1 = x_2\)。
即\(\displaystyle \sum_{k=1}^q \lambda_k f(x_k)\le f(\sum_{k=1}^q \lambda_k x_k)\)等號成立當且僅當\(x_1 = x_2 = \cdots = x_q\)
2.3 下凸函數
證明：因為\(\lambda_i\)均為正實數，故有
\(\displaystyle f( \sum_{k=1}^q \lambda_k x_k) = f(\lambda_1 x_1 + \sum_{k=2}^q \lambda_k {\sum_{k=2}^q \lambda_k x_k \over \sum_{k=2}^q \lambda_k}) \le \lambda_1 f(x_1) + \sum_{k=2}^q \lambda_k \cdot f({\sum_{k=2}^q \lambda_k x_k \over \sum_{k=2}^q \lambda_k})\)
\(\displaystyle = \lambda_1 f(x_1) + \sum_{k=2}^q \lambda_k \cdot f({\lambda_2 \over \sum_{k=2}^q \lambda_k} x_2 + {\sum_{k=3}^q \lambda_k \over \sum_{k=2}^q \lambda_k} \cdot {\sum_{k=3}^q \lambda_k x_k \over \sum_{k=3}^q \lambda_k})\)
\(\displaystyle \le \lambda_1 f(x_1) + \lambda_2 f(x_2) + \sum_{k=3}^q \lambda_k \cdot f({\sum_{k=3}^q \lambda_k x_k \over \sum_{k=3}^q \lambda_k})\)
\(\displaystyle \le \cdots \le \sum_{k=1}^q \lambda_k f(x_k)\)
2.4 嚴格下凸函數
證明：由定義可知，對於嚴格下凸函數，\(f[\alpha x_1 + (1 – \alpha )x_2] \le \alpha f(x_1) + (1 – \alpha )f(x_2)\)等號成立時當且僅當\(x_1 = x_2\) 。而根據上文對於下凸函數對於\(\displaystyle \sum_{k=1}^q \lambda_k f(x_k)\le f(\sum_{k=1}^q \lambda_k x_k)\)不等式推導過程可知，若下凸函數為嚴格下凸函數，則第一個\(\le\)處等號成立當且僅當：\(x_1 = {\sum_{k=2}^q \lambda_k x_k \over \sum_{k=2}^q \lambda_k}\)；第二個\(\le\)處等號成立當且僅當：\(x_2 = {\sum_{k=3}^q \lambda_k x_k \over \sum_{k=3}^q \lambda_k}\)；\(\cdots\)；第\(q-1\)個\(\le\)處等號成立當且僅當：\(x_{q-1} = {\sum_{k=q}^q \lambda_k x_k \over \sum_{k=q}^q \lambda_q} = x_q\)。所有等號都成立則以上條件都需滿足，對以上條件反向推導可得：\(x_q = x_{q-1}\)；\(x_{q-2} = {\sum_{k=q-1}^q \lambda_k x_k \over \sum_{k=q-1}^q \lambda_k} = {\lambda_{q-1} x_{q-1} + \lambda_{q} x_q \over \lambda_{q-1} + \lambda_{q}} = x_{q-1}\)；\(\cdots\)；\(x_1 = x_2\)。
即\(\displaystyle \sum_{k=1}^q \lambda_k f(x_k) \ge f(\sum_{k=1}^q \lambda_k x_k)\)等號成立當且僅當\(x_1 = x_2 = \cdots = x_q\)
