使用MindSpore的線性神經網路擬合非線性函數

技術背景

在前面的幾篇部落格中，我們分別介紹了MindSpore的CPU版本在Docker下的安裝與配置方案、MindSpore的線性函數擬合以及MindSpore後來新推出的GPU版本的Docker編程環境解決方案。這裡我們在線性擬合的基礎上，再介紹一下MindSpore中使用線性神經網路來擬合多變數非線性函數的解決方案。

環境配置

在按照這篇部落格中的方法進行安裝和配置之後，可以在本地的docker鏡像倉庫中找到一個mindspore的鏡像：

[dechin-manjaro gitlab]# docker images
REPOSITORY                                                 TAG                  IMAGE ID     
mindspore                                                  latest               3a6951d9b900

由於運行Docker容器的時候最好是可以綁定本地的一個目錄，所以可以用pwd先看一下本地的執行目錄：

[dechin@dechin-manjaro mindspore]$ pwd
/home/dechin/projects/2021-python/mindspore

然後可以根據上述的名稱和路徑資訊，創建一個新的docker容器環境：

[dechin-manjaro mindspore]# docker run -it --rm -v /home/dechin/projects/2021-python/mindspore:/home mindspore

這裡運行的時候加了--rm選項，表示關閉容器後就刪除容器，是一個一次性容器的選項。如果不加這個選項的話，會在docker ps -a下看到眾多的被保存下來的容器環境，這會浪費不必要的資源。因此如果不對環境進行變更的話，建議還是加上--rm選項。

非線性函數擬合

在前面這篇部落格中我們所擬合的是一個簡單的線性函數：

\[y=ax+b \]

那麼在這裡我們先考慮一個最簡單的非線性函數的場景：

\[y=ax^2+b \]

同樣的還是兩個參數，需要注意的是，如果要用線性神經網路來擬合非線性的函數，那麼在給出參數的時候就要給出非線性的入參，以下是完整的程式碼（如果需要展示結果更好看的話可以參考上面提到的線性函數擬合的部落格，這裡我們為了提速，刪除了很多非比要的模組）：

# test_nonlinear.py

from mindspore import context
context.set_context(mode=context.GRAPH_MODE, device_target="CPU")
import numpy as np
from mindspore import dataset as ds
from mindspore import nn, Tensor, Model
import time
from mindspore.train.callback import Callback, LossMonitor

def get_data(num, a=2.0, b=3.0):
    for _ in range(num):
        x = np.random.uniform(-1.0, 1.0)
        noise = np.random.normal(0, 0.03)
        z = a * x ** 2 + b + noise
        # 返回數據的時候就返回數據的平方
        yield np.array([x**2]).astype(np.float32), np.array([z]).astype(np.float32)

def create_dataset(num_data, batch_size=16, repeat_size=1):
    input_data = ds.GeneratorDataset(list(get_data(num_data)), column_names=['x','z'])
    input_data = input_data.batch(batch_size)
    input_data = input_data.repeat(repeat_size)
    return input_data

data_number = 1600 # 一共產生1600組數據
batch_number = 16 # 分為16組分別進行優化
repeat_number = 2 # 重複2次，可以取到更低的損失函數值

ds_train = create_dataset(data_number, batch_size=batch_number, repeat_size=repeat_number)
dict_datasets = next(ds_train.create_dict_iterator())

class LinearNet(nn.Cell):
    def __init__(self):
        super(LinearNet, self).__init__()
        self.fc = nn.Dense(1, 1, 0.02, 0.02)

    def construct(self, x):
        x = self.fc(x)
        return x

net = LinearNet()
model_params = net.trainable_params()
print ('Param Shape is: {}'.format(len(model_params)))
for net_param in net.trainable_params():
    print(net_param, net_param.asnumpy())
net_loss = nn.loss.MSELoss()

# 設定優化演算法，常用的是Momentum和ADAM
optim = nn.Momentum(net.trainable_params(), learning_rate=0.005, momentum=0.9)
model = Model(net, net_loss, optim)

epoch = 1
# 設定每8個batch訓練完成後就播報一次，這裡一共播報25次
model.train(epoch, ds_train, callbacks=[LossMonitor(8)], dataset_sink_mode=False)

for net_param in net.trainable_params():
    print(net_param, net_param.asnumpy())

在容器內直接用python來運行這個程式碼：

root@9f9358769573:/home# python test_nonlinear.py 
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
[WARNING] ME(818:140122680680576,MainProcess):2021-05-15-01:59:46.275.616 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
Param Shape is: 2
Parameter (name=fc.weight) [[0.02]]
Parameter (name=fc.bias) [0.02]
epoch: 1 step: 8, loss is 7.727301
epoch: 1 step: 16, loss is 0.9204254
epoch: 1 step: 24, loss is 0.051439833
epoch: 1 step: 32, loss is 0.40419307
epoch: 1 step: 40, loss is 0.24850279
epoch: 1 step: 48, loss is 0.0988866
epoch: 1 step: 56, loss is 0.0244435
epoch: 1 step: 64, loss is 0.05162067
epoch: 1 step: 72, loss is 0.043285627
epoch: 1 step: 80, loss is 0.013224524
epoch: 1 step: 88, loss is 0.014601769
epoch: 1 step: 96, loss is 0.0086541185
epoch: 1 step: 104, loss is 0.014336493
epoch: 1 step: 112, loss is 0.007790237
epoch: 1 step: 120, loss is 0.007644728
epoch: 1 step: 128, loss is 0.0090831155
epoch: 1 step: 136, loss is 0.009039238
epoch: 1 step: 144, loss is 0.0061546825
epoch: 1 step: 152, loss is 0.006427808
epoch: 1 step: 160, loss is 0.004290967
epoch: 1 step: 168, loss is 0.003127655
epoch: 1 step: 176, loss is 0.0044714734
epoch: 1 step: 184, loss is 0.003788504
epoch: 1 step: 192, loss is 0.0025329243
epoch: 1 step: 200, loss is 0.0034840193
Parameter (name=fc.weight) [[1.8488435]]
Parameter (name=fc.bias) [3.0522664]

最後優化的結果為\(f(x)=1.8488435x+3.0522664\)，可以看到距離我們所預期的\(f(x)=2x+3\)已經是非常接近的，但是最後具體能訓練到多少的精度，其實很大程度上跟訓練集有關，這裡因為訓練集的範圍取的比較小，可能會有一定的影響。

多變數函數擬合

不論是前面提到的線性函數擬合的場景，或者是上一個章節中單變數非線性函數的擬合，其實都只有1個輸入參數，本章節介紹的場景有2個入參，這裡我們要擬合的函數模型是：

\[z(x,y)=ax^2+by^3+c \]

以下是完整的程式碼實例：

# test_nonlinear.py

from mindspore import context
context.set_context(mode=context.GRAPH_MODE, device_target="CPU")
import numpy as np
from mindspore import dataset as ds
from mindspore import nn, Tensor, Model
import time
from mindspore.train.callback import Callback, LossMonitor

def get_data(num, a=2.0, b=3.0, c=5.0):
    for _ in range(num):
        x = np.random.uniform(-1.0, 1.0)
        y = np.random.uniform(-1.0, 1.0)
        noise = np.random.normal(0, 0.03)
        z = a * x ** 2 + b * y ** 3 + c + noise
        # 返回參數的時候壓縮在一個數組內
        yield np.array([x**2,y**3]).astype(np.float32), np.array([z]).astype(np.float32)

def create_dataset(num_data, batch_size=16, repeat_size=1):
    input_data = ds.GeneratorDataset(list(get_data(num_data)), column_names=['x','z'])
    input_data = input_data.batch(batch_size)
    input_data = input_data.repeat(repeat_size)
    return input_data

data_number = 1600
batch_number = 16
repeat_number = 2

ds_train = create_dataset(data_number, batch_size=batch_number, repeat_size=repeat_number)
dict_datasets = next(ds_train.create_dict_iterator())

class LinearNet(nn.Cell):
    def __init__(self):
        super(LinearNet, self).__init__()
        # 神經網路的input和output維度設置為2,1
        self.fc = nn.Dense(2, 1, 0.02, 0.02)

    def construct(self, x):
        x = self.fc(x)
        return x

net = LinearNet()
model_params = net.trainable_params()
print ('Param Shape is: {}'.format(len(model_params)))
for net_param in net.trainable_params():
    print(net_param, net_param.asnumpy())
net_loss = nn.loss.MSELoss()

optim = nn.Momentum(net.trainable_params(), learning_rate=0.005, momentum=0.9)
model = Model(net, net_loss, optim)

epoch = 1
model.train(epoch, ds_train, callbacks=[LossMonitor(8)], dataset_sink_mode=False)

for net_param in net.trainable_params():
    print(net_param, net_param.asnumpy())

然後用同樣的方法來運行：

root@9f9358769573:/home# python test_nonlinear.py 
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
[WARNING] ME(1256:140024122265728,MainProcess):2021-05-15-02:03:11.597.713 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
Param Shape is: 2
Parameter (name=fc.weight) [[0.02 0.02]]
Parameter (name=fc.bias) [0.02]
epoch: 1 step: 8, loss is 23.713902
epoch: 1 step: 16, loss is 4.097528
epoch: 1 step: 24, loss is 0.6977275
epoch: 1 step: 32, loss is 1.6294805
epoch: 1 step: 40, loss is 1.6045574
epoch: 1 step: 48, loss is 0.33872497
epoch: 1 step: 56, loss is 0.2145875
epoch: 1 step: 64, loss is 0.15970881
epoch: 1 step: 72, loss is 0.18631926
epoch: 1 step: 80, loss is 0.093276486
epoch: 1 step: 88, loss is 0.030935105
epoch: 1 step: 96, loss is 0.065783374
epoch: 1 step: 104, loss is 0.036098357
epoch: 1 step: 112, loss is 0.021177795
epoch: 1 step: 120, loss is 0.022446468
epoch: 1 step: 128, loss is 0.024930779
epoch: 1 step: 136, loss is 0.025752623
epoch: 1 step: 144, loss is 0.019025242
epoch: 1 step: 152, loss is 0.012783089
epoch: 1 step: 160, loss is 0.008119936
epoch: 1 step: 168, loss is 0.010284519
epoch: 1 step: 176, loss is 0.0034211911
epoch: 1 step: 184, loss is 0.003315867
epoch: 1 step: 192, loss is 0.0035076286
epoch: 1 step: 200, loss is 0.0032196175
Parameter (name=fc.weight) [[1.9629341 2.9065483]]
Parameter (name=fc.bias) [5.004186]

這裡得到的結果\(z(x,y)=1.9629341x^2+2.9065483y^3+5.004186\)跟我們所預期的結果\(z(x,y)=2x^2+3y^3+5\)也是非常接近的。

其他的函數類型

使用上一章節中所介紹的方法，不僅可以擬合多參數、多冪次的函數，同樣的可以擬合一些其他的初等函數，比如：

\[z(x,y)=ax^2+b\ sin(y)+c \]

完整的程式碼如下所示：

# test_nonlinear.py

from mindspore import context
context.set_context(mode=context.GRAPH_MODE, device_target="CPU")
import numpy as np
from mindspore import dataset as ds
from mindspore import nn, Tensor, Model
import time
from mindspore.train.callback import Callback, LossMonitor

def get_data(num, a=4.0, b=2.0, c=7.0):
    for _ in range(num):
        x = np.random.uniform(-1.0, 1.0)
        y = np.random.uniform(-1.0, 1.0)
        noise = np.random.normal(0, 0.03)
        z = a * x ** 2 + b * np.sin(y) + c + noise
        yield np.array([x**2,np.sin(y)]).astype(np.float32), np.array([z]).astype(np.float32)

def create_dataset(num_data, batch_size=16, repeat_size=1):
    input_data = ds.GeneratorDataset(list(get_data(num_data)), column_names=['x','z'])
    input_data = input_data.batch(batch_size)
    input_data = input_data.repeat(repeat_size)
    return input_data

data_number = 1600
batch_number = 16
repeat_number = 2

ds_train = create_dataset(data_number, batch_size=batch_number, repeat_size=repeat_number)
dict_datasets = next(ds_train.create_dict_iterator())

class LinearNet(nn.Cell):
    def __init__(self):
        super(LinearNet, self).__init__()
        self.fc = nn.Dense(2, 1, 0.02, 0.02)

    def construct(self, x):
        x = self.fc(x)
        return x

net = LinearNet()
model_params = net.trainable_params()
print ('Param Shape is: {}'.format(len(model_params)))
for net_param in net.trainable_params():
    print(net_param, net_param.asnumpy())
net_loss = nn.loss.MSELoss()

optim = nn.Momentum(net.trainable_params(), learning_rate=0.005, momentum=0.9)
model = Model(net, net_loss, optim)

epoch = 1
model.train(epoch, ds_train, callbacks=[LossMonitor(8)], dataset_sink_mode=False)

for net_param in net.trainable_params():
    print(net_param, net_param.asnumpy())

還是同樣的執行方法：

root@9f9358769573:/home# python test_nonlinear.py 
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
[WARNING] ME(2132:139783326998656,MainProcess):2021-05-15-02:07:23.220.797 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
Param Shape is: 2
Parameter (name=fc.weight) [[0.02 0.02]]
Parameter (name=fc.bias) [0.02]
epoch: 1 step: 8, loss is 33.49754
epoch: 1 step: 16, loss is 5.38931
epoch: 1 step: 24, loss is 0.6019592
epoch: 1 step: 32, loss is 2.3307486
epoch: 1 step: 40, loss is 1.4274476
epoch: 1 step: 48, loss is 0.48646024
epoch: 1 step: 56, loss is 0.15924525
epoch: 1 step: 64, loss is 0.20410316
epoch: 1 step: 72, loss is 0.21361557
epoch: 1 step: 80, loss is 0.09532051
epoch: 1 step: 88, loss is 0.06947755
epoch: 1 step: 96, loss is 0.05294941
epoch: 1 step: 104, loss is 0.04545658
epoch: 1 step: 112, loss is 0.032882713
epoch: 1 step: 120, loss is 0.04866419
epoch: 1 step: 128, loss is 0.033258867
epoch: 1 step: 136, loss is 0.023843747
epoch: 1 step: 144, loss is 0.024475154
epoch: 1 step: 152, loss is 0.015055884
epoch: 1 step: 160, loss is 0.016890783
epoch: 1 step: 168, loss is 0.019164825
epoch: 1 step: 176, loss is 0.014314213
epoch: 1 step: 184, loss is 0.010082506
epoch: 1 step: 192, loss is 0.009690857
epoch: 1 step: 200, loss is 0.008122671
Parameter (name=fc.weight) [[3.6814337 1.9820771]]
Parameter (name=fc.bias) [7.106203]

得到的結果是\(z(x,y)=3.6814337x^2+1.9820771sin(y)+7.106203\)，而我們所預期的結果為：\(z(x,y)=4x^2+2sin(y)+7\)。

總結概要

基於前面所介紹的MindSpore框架使用線性神經網路來擬合線性函數的一個方案，這裡我們將其推廣到多參數、多冪次的函數，甚至是一些初等的函數。但是對於更加複雜的嵌套函數，可能還要思考一下更加普適的解決方案。

版權聲明

本文首發鏈接為：//www.cnblogs.com/dechinphy/p/nonlinear.html
作者ID：DechinPhy
更多原著文章請參考：//www.cnblogs.com/dechinphy/

Tags:

VirMach 便宜 VPS