使用Dropout抑制过拟合

Dropout是一种常用的神经网络正则化方法，主要用于防止过拟合。在深度学习中，由于网络层数过多，参数数量庞大，模型容易过拟合，并且在测试时产生较大的泛化误差。Dropout方法借鉴了集成学习中的Bagging思想，通过随机的方式，将一部分神经元的输出设置为0，从而减少过拟合的可能。

Dropout方法最早由Hinton等人提出，其基本思想是在训练时，以一定的概率随机地将网络中某些神经元的输出置为0。这种随机的行为可以被看作是一种对网络进行了部分剪枝，从而增加了网络的容忍性，使网络更加健壮，同时也避免了网络中某些特定的神经元对整个网络的过度依赖。

Dropout方法的具体实现如下：在每次训练过程中，以一定的概率p随机选择一部分神经元并将其置为0，被选择的神经元不参与后续的训练和反向传播。在测试时，为了保持模型的稳定性和一致性，一般不会采取随机化的方式，而是将每个神经元的权重乘以概率p，这里的p是在训练时选择的那个概率。

Dropout方法不仅可用于全连接网络，也可用于卷积神经网络和循环神经网络中，以减少过拟合现象。并且，它的实现简单，仅需要在模型训练时对每个神经元以概率p随机地进行挑选和保留，所以Dropout方法得到了广泛的应用和推广。

总之，Dropout方法可以在一定程度上提高模型的准确性和泛化能力，对于防止过拟合有着较好的效果。但是需要注意的是，Dropout方法会导致训练过程中每个mini-batch的梯度都不同，所以在使用Dropout方法时需要调整学习率，以保证模型的收敛速度和效果。

1. 环境准备

# 导入库
import keras
from keras import layers
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

Using TensorFlow backend.

2. 导入数据集

# 导入数据集
data = pd.read_csv('./dataset/credit-a.csv', header=None)
data

	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
0	0	30.83	0.000	0	0	9	0	1.250	0	0	1	1	0	202	0.0	-1
1	1	58.67	4.460	0	0	8	1	3.040	0	0	6	1	0	43	560.0	-1
2	1	24.50	0.500	0	0	8	1	1.500	0	1	0	1	0	280	824.0	-1
3	0	27.83	1.540	0	0	9	0	3.750	0	0	5	0	0	100	3.0	-1
4	0	20.17	5.625	0	0	9	0	1.710	0	1	0	1	2	120	0.0	-1
5	0	32.08	4.000	0	0	6	0	2.500	0	1	0	0	0	360	0.0	-1
6	0	33.17	1.040	0	0	7	1	6.500	0	1	0	0	0	164	31285.0	-1
7	1	22.92	11.585	0	0	2	0	0.040	0	1	0	1	0	80	1349.0	-1
8	0	54.42	0.500	1	1	5	1	3.960	0	1	0	1	0	180	314.0	-1
9	0	42.50	4.915	1	1	9	0	3.165	0	1	0	0	0	52	1442.0	-1
10	0	22.08	0.830	0	0	0	1	2.165	1	1	0	0	0	128	0.0	-1
11	0	29.92	1.835	0	0	0	1	4.335	0	1	0	1	0	260	200.0	-1
12	1	38.25	6.000	0	0	5	0	1.000	0	1	0	0	0	0	0.0	-1
13	0	48.08	6.040	0	0	5	0	0.040	1	1	0	1	0	0	2690.0	-1
14	1	45.83	10.500	0	0	8	0	5.000	0	0	7	0	0	0	0.0	-1
15	0	36.67	4.415	1	1	5	0	0.250	0	0	10	0	0	320	0.0	-1
16	0	28.25	0.875	0	0	6	0	0.960	0	0	3	0	0	396	0.0	-1
17	1	23.25	5.875	0	0	8	0	3.170	0	0	10	1	0	120	245.0	-1
18	0	21.83	0.250	0	0	1	1	0.665	0	1	0	0	0	0	0.0	-1
19	1	19.17	8.585	0	0	2	1	0.750	0	0	7	1	0	96	0.0	-1
20	0	25.00	11.250	0	0	0	0	2.500	0	0	17	1	0	200	1208.0	-1
21	0	23.25	1.000	0	0	0	0	0.835	0	1	0	1	2	300	0.0	-1
22	1	47.75	8.000	0	0	0	0	7.875	0	0	6	0	0	0	1260.0	-1
23	1	27.42	14.500	0	0	10	1	3.085	0	0	1	1	0	120	11.0	-1
24	1	41.17	6.500	0	0	8	0	0.500	0	0	3	0	0	145	0.0	-1
25	1	15.83	0.585	0	0	0	1	1.500	0	0	2	1	0	100	0.0	-1
26	1	47.00	13.000	0	0	3	2	5.165	0	0	9	0	0	0	0.0	-1
27	0	56.58	18.500	0	0	1	2	15.000	0	0	17	0	0	0	0.0	-1
28	0	57.42	8.500	0	0	11	1	7.000	0	0	3	1	0	0	0.0	-1
29	0	42.08	1.040	0	0	9	0	5.000	0	0	6	0	0	500	10000.0	-1
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
623	1	28.58	3.750	0	0	0	0	0.250	1	0	1	0	0	40	154.0	1
624	0	22.25	9.000	0	0	12	0	0.085	1	1	0	1	0	0	0.0	1
625	0	29.83	3.500	0	0	0	0	0.165	1	1	0	1	0	216	0.0	1
626	1	23.50	1.500	0	0	9	0	0.875	1	1	0	0	0	160	0.0	1
627	0	32.08	4.000	1	1	2	0	1.500	1	1	0	0	0	120	0.0	1
628	0	31.08	1.500	1	1	9	0	0.040	1	1	0	1	2	160	0.0	1
629	0	31.83	0.040	1	1	6	0	0.040	1	1	0	1	0	0	0.0	1
630	1	21.75	11.750	0	0	0	0	0.250	1	1	0	0	0	180	0.0	1
631	1	17.92	0.540	0	0	0	0	1.750	1	0	1	0	0	80	5.0	1
632	0	30.33	0.500	0	0	1	1	0.085	1	1	0	0	2	252	0.0	1
633	0	51.83	2.040	1	1	13	7	1.500	1	1	0	1	0	120	1.0	1
634	0	47.17	5.835	0	0	9	0	5.500	1	1	0	1	0	465	150.0	1
635	0	25.83	12.835	0	0	2	0	0.500	1	1	0	1	0	0	2.0	1
636	1	50.25	0.835	0	0	12	0	0.500	1	1	0	0	0	240	117.0	1
637	1	37.33	2.500	0	0	3	1	0.210	1	1	0	1	0	260	246.0	1
638	1	41.58	1.040	0	0	12	0	0.665	1	1	0	1	0	240	237.0	1
639	1	30.58	10.665	0	0	8	1	0.085	1	0	12	0	0	129	3.0	1
640	0	19.42	7.250	0	0	6	0	0.040	1	0	1	1	0	100	1.0	1
641	1	17.92	10.210	0	0	13	7	0.000	1	1	0	1	0	0	50.0	1
642	1	20.08	1.250	0	0	0	0	0.000	1	1	0	1	0	0	0.0	1
643	0	19.50	0.290	0	0	5	0	0.290	1	1	0	1	0	280	364.0	1
644	0	27.83	1.000	1	1	1	1	3.000	1	1	0	1	0	176	537.0	1
645	0	17.08	3.290	0	0	3	0	0.335	1	1	0	0	0	140	2.0	1
646	0	36.42	0.750	1	1	1	0	0.585	1	1	0	1	0	240	3.0	1
647	0	40.58	3.290	0	0	6	0	3.500	1	1	0	0	2	400	0.0	1
648	0	21.08	10.085	1	1	11	1	1.250	1	1	0	1	0	260	0.0	1
649	1	22.67	0.750	0	0	0	0	2.000	1	0	2	0	0	200	394.0	1
650	1	25.25	13.500	1	1	13	7	2.000	1	0	1	0	0	200	1.0	1
651	0	17.92	0.205	0	0	12	0	0.040	1	1	0	1	0	280	750.0	1
652	0	35.00	3.375	0	0	0	1	8.290	1	1	0	0	0	0	0.0	1

653 rows × 16 columns

data.iloc[:, -1].unique()

array([-1,  1])

3. 对所有数据的预测

3.1 数据集

x = data.iloc[:, :-1].values
y = data.iloc[:, -1].replace(-1, 0).values.reshape(-1, 1)

x.shape, y.shape

((653, 15), (653, 1))

3.2 构建神经网络

model = keras.Sequential()
model.add(layers.Dense(128, input_dim=15, activation='relu'))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 128)               2048      
_________________________________________________________________
dense_2 (Dense)              (None, 128)               16512     
_________________________________________________________________
dense_3 (Dense)              (None, 128)               16512     
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 129       
=================================================================
Total params: 35,201
Trainable params: 35,201
Non-trainable params: 0
_________________________________________________________________

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

WARNING:tensorflow:From /home/nlp/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where

3.3 训练模型

history = model.fit(x, y, epochs=1000)

WARNING:tensorflow:From /home/nlp/anaconda3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

Epoch 1/1000
653/653 [==============================] - 0s 434us/step - loss: 7.5273 - acc: 0.5988
Epoch 2/1000
653/653 [==============================] - 0s 92us/step - loss: 3.7401 - acc: 0.6187
Epoch 3/1000
653/653 [==============================] - 0s 75us/step - loss: 3.6464 - acc: 0.5712
Epoch 4/1000
653/653 [==============================] - 0s 56us/step - loss: 10.2291 - acc: 0.6631
Epoch 5/1000
653/653 [==============================] - 0s 63us/step - loss: 2.0400 - acc: 0.6233
Epoch 6/1000
653/653 [==============================] - 0s 120us/step - loss: 2.4279 - acc: 0.6217
Epoch 7/1000
653/653 [==============================] - 0s 105us/step - loss: 2.3289 - acc: 0.6325
Epoch 8/1000
653/653 [==============================] - 0s 159us/step - loss: 3.2521 - acc: 0.6294
Epoch 9/1000
653/653 [==============================] - 0s 89us/step - loss: 2.6005 - acc: 0.6294
Epoch 10/1000
653/653 [==============================] - 0s 118us/step - loss: 1.3997 - acc: 0.6738
……
Epoch 1000/1000
653/653 [==============================] - 0s 106us/step - loss: 0.2630 - acc: 0.9326

3.4 分析模型

history.history.keys()

dict_keys(['loss', 'acc'])

plt.plot(history.epoch, history.history.get('loss'), c='r')
plt.plot(history.epoch, history.history.get('acc'), c='b')

[<matplotlib.lines.Line2D at 0x7fd43c1597f0>]

4. 对未见过数据的预测

4.1 划分数据集

x_train = x[:int(len(x)*0.75)]
x_test = x[int(len(x)*0.75):]
y_train = y[:int(len(x)*0.75)]
y_test = y[int(len(x)*0.75):]

x_train.shape, x_test.shape, y_train.shape, y_test.shape

((489, 15), (164, 15), (489, 1), (164, 1))

4.2 构建神经网络

model = keras.Sequential()

model.add(layers.Dense(128, input_dim=15, activation='relu'))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

#admam:利用梯度的一阶矩估计和二阶矩估计动态调整每个参数的学习率.
model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

4.3 训练模型

history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))

Train on 489 samples, validate on 164 samples
Epoch 1/1000
489/489 [==============================] - 0s 487us/step - loss: 14.4564 - acc: 0.5951 - val_loss: 3.3778 - val_acc: 0.7256
Epoch 2/1000
489/489 [==============================] - 0s 110us/step - loss: 6.0909 - acc: 0.6012 - val_loss: 2.0924 - val_acc: 0.7195
Epoch 3/1000
489/489 [==============================] - 0s 195us/step - loss: 2.4527 - acc: 0.6074 - val_loss: 1.0763 - val_acc: 0.7378
Epoch 4/1000
489/489 [==============================] - 0s 183us/step - loss: 1.0751 - acc: 0.6585 - val_loss: 0.8990 - val_acc: 0.7134
Epoch 5/1000
489/489 [==============================] - 0s 155us/step - loss: 1.2669 - acc: 0.6503 - val_loss: 1.8094 - val_acc: 0.6585
Epoch 6/1000
489/489 [==============================] - 0s 202us/step - loss: 3.6742 - acc: 0.6892 - val_loss: 1.1836 - val_acc: 0.4573
Epoch 7/1000
489/489 [==============================] - 0s 166us/step - loss: 1.7544 - acc: 0.7301 - val_loss: 2.0060 - val_acc: 0.4573
Epoch 8/1000
489/489 [==============================] - 0s 185us/step - loss: 1.4768 - acc: 0.6605 - val_loss: 0.8917 - val_acc: 0.5427
Epoch 9/1000
489/489 [==============================] - 0s 163us/step - loss: 1.6829 - acc: 0.6667 - val_loss: 4.7695 - val_acc: 0.4573
Epoch 10/1000
489/489 [==============================] - 0s 157us/step - loss: 8.4323 - acc: 0.7239 - val_loss: 2.0879 - val_acc: 0.7439
……
Epoch 1000/1000
489/489 [==============================] - 0s 97us/step - loss: 0.0272 - acc: 0.9877 - val_loss: 2.2746 - val_acc: 0.8049

4.4 分析模型

history.history.keys()

dict_keys(['val_loss', 'val_acc', 'loss', 'acc'])

plt.plot(history.epoch, history.history.get('val_acc'), c='r', label='val_acc')
plt.plot(history.epoch, history.history.get('acc'), c='b', label='acc')
plt.legend()

<matplotlib.legend.Legend at 0x7fd417ff5978>

model.evaluate(x_train, y_train)

489/489 [==============================] - 0s 37us/step
[0.021263938083038197, 0.9897750616073608]

model.evaluate(x_test, y_test)

164/164 [==============================] - 0s 46us/step
[2.274633582976715, 0.8048780560493469]

过拟合：在训练数据正确率非常高， 在测试数据上比较低

5. 使用Dropout抑制过拟合

5.1 构建神经网络

model = keras.Sequential()
model.add(layers.Dense(128, input_dim=15, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(128, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()

Model: "sequential_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_9 (Dense)              (None, 128)               2048      
_________________________________________________________________
dropout_1 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_10 (Dense)             (None, 128)               16512     
_________________________________________________________________
dropout_2 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_11 (Dense)             (None, 128)               16512     
_________________________________________________________________
dropout_3 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_12 (Dense)             (None, 1)                 129       
=================================================================
Total params: 35,201
Trainable params: 35,201
Non-trainable params: 0
________________________________________________________________

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

5.2 训练模型

history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))

Train on 489 samples, validate on 164 samples
Epoch 1/1000
489/489 [==============================] - 1s 1ms/step - loss: 41.6885 - acc: 0.5378 - val_loss: 9.9666 - val_acc: 0.6768
Epoch 2/1000
489/489 [==============================] - 0s 298us/step - loss: 53.1358 - acc: 0.5358 - val_loss: 11.0265 - val_acc: 0.6951
Epoch 3/1000
489/489 [==============================] - 0s 173us/step - loss: 36.9899 - acc: 0.5828 - val_loss: 11.6578 - val_acc: 0.6890
Epoch 4/1000
489/489 [==============================] - 0s 177us/step - loss: 43.3404 - acc: 0.5808 - val_loss: 7.5652 - val_acc: 0.6890
Epoch 5/1000
489/489 [==============================] - 0s 197us/step - loss: 23.3085 - acc: 0.6196 - val_loss: 7.9913 - val_acc: 0.6890
Epoch 6/1000
489/489 [==============================] - 0s 254us/step - loss: 24.1833 - acc: 0.6155 - val_loss: 5.5747 - val_acc: 0.7073
Epoch 7/1000
489/489 [==============================] - 0s 229us/step - loss: 19.7051 - acc: 0.5890 - val_loss: 5.5711 - val_acc: 0.7012
Epoch 8/1000
489/489 [==============================] - 0s 180us/step - loss: 22.1131 - acc: 0.5849 - val_loss: 7.0290 - val_acc: 0.6890
Epoch 9/1000
489/489 [==============================] - 0s 172us/step - loss: 23.2305 - acc: 0.6115 - val_loss: 4.2624 - val_acc: 0.6951
Epoch 10/1000
……
Epoch 1000/1000
489/489 [==============================] - 0s 137us/step - loss: 0.3524 - acc: 0.8200 - val_loss: 0.7290 - val_acc: 0.7012

5.3 分析模型

model.evaluate(x_train, y_train)

489/489 [==============================] - 0s 41us/step
[0.3090217998422728, 0.8548057079315186]

model.evaluate(x_test, y_test)

164/164 [==============================] - 0s 64us/step
[0.7289713301309725, 0.7012194991111755]

plt.plot(history.epoch, history.history.get('val_acc'), c='r', label='val_acc')
plt.plot(history.epoch, history.history.get('acc'), c='b', label='acc')
plt.legend()

<matplotlib.legend.Legend at 0x7fd4177c87b8>

6. 正则化

l1：loss = s*abs(w1 + w2 + …) + mse

l2：loss = s*(w12 + w22 + …) + mse

from keras import regularizers

6.1 神经网络太过复杂容易过拟合

#神经网络太过复杂容易过拟合
model = keras.Sequential()
model.add(layers.Dense(128, kernel_regularizer=regularizers.l2(0.001), input_dim=15, activation='relu'))
model.add(layers.Dense(128, kernel_regularizer=regularizers.l2(0.001), activation='relu'))
model.add(layers.Dense(128, kernel_regularizer=regularizers.l2(0.001), activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))

Train on 489 samples, validate on 164 samples
Epoch 1/1000
489/489 [==============================] - 0s 752us/step - loss: 22.2560 - acc: 0.5910 - val_loss: 9.1111 - val_acc: 0.6524
Epoch 2/1000
489/489 [==============================] - 0s 137us/step - loss: 6.8963 - acc: 0.6217 - val_loss: 3.2886 - val_acc: 0.4573
Epoch 3/1000
489/489 [==============================] - 0s 161us/step - loss: 5.0407 - acc: 0.6830 - val_loss: 1.1973 - val_acc: 0.7256
Epoch 4/1000
489/489 [==============================] - 0s 218us/step - loss: 6.6088 - acc: 0.6421 - val_loss: 7.4651 - val_acc: 0.7012
Epoch 5/1000
489/489 [==============================] - 0s 233us/step - loss: 8.3945 - acc: 0.6973 - val_loss: 2.5579 - val_acc: 0.7317
Epoch 6/1000
489/489 [==============================] - 0s 192us/step - loss: 7.0204 - acc: 0.6196 - val_loss: 3.6758 - val_acc: 0.6829
Epoch 7/1000
489/489 [==============================] - 0s 152us/step - loss: 3.9961 - acc: 0.7382 - val_loss: 1.6183 - val_acc: 0.7317
Epoch 8/1000
489/489 [==============================] - 0s 94us/step - loss: 2.3441 - acc: 0.6237 - val_loss: 1.1523 - val_acc: 0.7256
Epoch 9/1000
489/489 [==============================] - 0s 114us/step - loss: 1.8178 - acc: 0.6442 - val_loss: 1.3449 - val_acc: 0.7073
Epoch 10/1000
489/489 [==============================] - 0s 157us/step - loss: 1.6122 - acc: 0.7117 - val_loss: 1.2869 - val_acc: 0.6646
……
Epoch 1000/1000
489/489 [==============================] - 0s 130us/step - loss: 0.1452 - acc: 0.9775 - val_loss: 1.0515 - val_acc: 0.7866
model.evaluate(x_train, y_train)

model.evaluate(x_train, y_train)

489/489 [==============================] - 0s 34us/step
[0.17742264538942426, 0.9611452221870422]

model.evaluate(x_test, y_test)

164/164 [==============================] - 0s 77us/step
[1.0514701096023, 0.7865853905677795]

6.2 太简单容易欠拟合

#太简单容易欠拟合
model = keras.Sequential()
model.add(layers.Dense(4, input_dim=15, activation='relu'))
model.add(layers.Dense(1,  activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))

Train on 489 samples, validate on 164 samples
Epoch 1/1000
489/489 [==============================] - 0s 502us/step - loss: 0.6932 - acc: 0.4765 - val_loss: 0.6931 - val_acc: 0.6341
Epoch 2/1000
489/489 [==============================] - 0s 91us/step - loss: 0.6931 - acc: 0.5174 - val_loss: 0.6930 - val_acc: 0.6341
Epoch 3/1000
489/489 [==============================] - 0s 107us/step - loss: 0.6931 - acc: 0.5174 - val_loss: 0.6924 - val_acc: 0.6341
Epoch 4/1000
489/489 [==============================] - 0s 91us/step - loss: 0.6930 - acc: 0.5174 - val_loss: 0.6916 - val_acc: 0.6341
Epoch 5/1000
489/489 [==============================] - 0s 101us/step - loss: 0.6930 - acc: 0.5174 - val_loss: 0.6914 - val_acc: 0.6341
Epoch 6/1000
489/489 [==============================] - 0s 113us/step - loss: 0.6930 - acc: 0.5174 - val_loss: 0.6914 - val_acc: 0.6341
Epoch 7/1000
489/489 [==============================] - 0s 147us/step - loss: 0.6929 - acc: 0.5174 - val_loss: 0.6908 - val_acc: 0.6341
Epoch 8/1000
489/489 [==============================] - 0s 166us/step - loss: 0.6929 - acc: 0.5174 - val_loss: 0.6905 - val_acc: 0.6341
Epoch 9/1000
489/489 [==============================] - 0s 162us/step - loss: 0.6929 - acc: 0.5174 - val_loss: 0.6904 - val_acc: 0.6341
Epoch 10/1000
489/489 [==============================] - 0s 129us/step - loss: 0.6928 - acc: 0.5174 - val_loss: 0.6901 - val_acc: 0.6341
……
Epoch 1000/1000
489/489 [==============================] - 0s 86us/step - loss: 0.6926 - acc: 0.5174 - val_loss: 0.6849 - val_acc: 0.6341

model.evaluate(x_train, y_train)

489/489 [==============================] - 0s 43us/step
[0.6925447341854587, 0.5173823833465576]

model.evaluate(x_test, y_test)

164/164 [==============================] - 0s 39us/step
[0.684889389247429, 0.6341463327407837]

6.3 选取适当的神经网络

# 选取适当的神经网络
model = keras.Sequential()
model.add(layers.Dense(4, input_dim=15, activation='relu'))
model.add(layers.Dense(4,  activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['acc']
)

history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))

Train on 489 samples, validate on 164 samples
Epoch 1/1000
489/489 [==============================] - 0s 575us/step - loss: 40.1825 - acc: 0.5317 - val_loss: 17.6376 - val_acc: 0.6098
Epoch 2/1000
489/489 [==============================] - 0s 104us/step - loss: 30.0785 - acc: 0.5337 - val_loss: 12.6986 - val_acc: 0.6159
Epoch 3/1000
489/489 [==============================] - 0s 148us/step - loss: 20.0469 - acc: 0.5112 - val_loss: 8.3732 - val_acc: 0.5671
Epoch 4/1000
489/489 [==============================] - 0s 151us/step - loss: 12.5171 - acc: 0.4908 - val_loss: 3.8925 - val_acc: 0.5061
Epoch 5/1000
489/489 [==============================] - 0s 113us/step - loss: 4.4324 - acc: 0.4294 - val_loss: 0.9156 - val_acc: 0.4573
Epoch 6/1000
489/489 [==============================] - 0s 79us/step - loss: 1.0313 - acc: 0.5419 - val_loss: 0.9974 - val_acc: 0.4695
Epoch 7/1000
489/489 [==============================] - 0s 88us/step - loss: 1.0071 - acc: 0.5562 - val_loss: 0.8852 - val_acc: 0.5183
Epoch 8/1000
489/489 [==============================] - 0s 88us/step - loss: 0.9085 - acc: 0.5808 - val_loss: 0.7934 - val_acc: 0.5366
Epoch 9/1000
489/489 [==============================] - 0s 107us/step - loss: 0.8235 - acc: 0.5992 - val_loss: 0.7390 - val_acc: 0.5366
Epoch 10/1000
489/489 [==============================] - 0s 114us/step - loss: 0.7711 - acc: 0.5971 - val_loss: 0.7174 - val_acc: 0.5366
……
Epoch 1000/1000
489/489 [==============================] - 0s 141us/step - loss: 0.3095 - acc: 0.8732 - val_loss: 0.3971 - val_acc: 0.8537

model.evaluate(x_train, y_train)

489/489 [==============================] - 0s 68us/step
[0.30120014958464536, 0.8813905715942383]

model.evaluate(x_test, y_test)

164/164 [==============================] - 0s 45us/step
[0.39714593858253666, 0.8536585569381714]

附：系列文章

序号	文章目录	直达链接
1	波士顿房价预测	https://want595.blog.csdn.net/article/details/132181950
2	鸢尾花数据集分析	https://want595.blog.csdn.net/article/details/132182057
3	特征处理	https://want595.blog.csdn.net/article/details/132182165
4	交叉验证	https://want595.blog.csdn.net/article/details/132182238
5	构造神经网络示例	https://want595.blog.csdn.net/article/details/132182341
6	使用TensorFlow完成线性回归	https://want595.blog.csdn.net/article/details/132182417
7	使用TensorFlow完成逻辑回归	https://want595.blog.csdn.net/article/details/132182496
8	TensorBoard案例	https://want595.blog.csdn.net/article/details/132182584
9	使用Keras完成线性回归	https://want595.blog.csdn.net/article/details/132182723
10	使用Keras完成逻辑回归	https://want595.blog.csdn.net/article/details/132182795
11	使用Keras预训练模型完成猫狗识别	https://want595.blog.csdn.net/article/details/132243928
12	使用PyTorch训练模型	https://want595.blog.csdn.net/article/details/132243989
13	使用Dropout抑制过拟合	https://want595.blog.csdn.net/article/details/132244111
14	使用CNN完成MNIST手写体识别(TensorFlow)	https://want595.blog.csdn.net/article/details/132244499
15	使用CNN完成MNIST手写体识别(Keras)	https://want595.blog.csdn.net/article/details/132244552
16	使用CNN完成MNIST手写体识别(PyTorch)	https://want595.blog.csdn.net/article/details/132244641
17	使用GAN生成手写数字样本	https://want595.blog.csdn.net/article/details/132244764
18	自然语言处理	https://want595.blog.csdn.net/article/details/132276591