ProgrammingAI

[TensorFlow 2.0] 각 Layer별 역할 및 파라미터

, ,

Load Packages

1
2
3
4
5
import tensorflow as tf
import os

import matplotlib.pyplot as plt
%matplotlib inline

Input Image

1
2
3
4
5
6
7
8
9
10
11
from tensorflow.keras import datasets

(train_x, train_y), (test_x, test_y) = datasets.mnist.load_data()
image = train_x[0]

# 차원 수 높이기
image = image[tf.newaxis, ..., tf.newaxis]
image.shape

# Out
(1, 28, 28, 1)

Feature Extraction

Convolution

  • filters: layer에서 나갈 때 몇 개의 filter를 만들 것인지
  • kernel_size: filter(Weight)의 크기
  • strides: 몇 개의 pixel을 skip 하면서 훑어지나갈 것인지 (크기에도 영향을 준다.)
  • padding: zero padding을 만들 것인지. VALID는 padding이 없고, SAME은 padding이 있습니다. (크기에도 영향을 준다.)
  • activation: Activation Function을 만들 것인지. 당장 설정 안해도 Layer 층을 따로 만들 수 있습니다. (Default: None)
1
2
3
4
tf.keras.layers.Conv2D(filters=3, kernel_size=(3, 3), strides=(1, 1), padding='VALID', activation='relu')

# (3, 3) 대신에 3으로도 대체 가능
tf.keras.layers.Conv2D(3, 3, 1, 'VALID')

Visualization

1
2
3
4
image = tf.cast(image, dtype=tf.float32)

layer = tf.keras.layers.Conv2D(3, 3, strides=(1, 1), padding='SAME')
output = layer(image)
1
2
3
4
5
plt.subplot(121)
plt.imshow(image[0, :, :, 0], 'gray')
plt.subplot(122)
plt.imshow(output[0, :, :, 0], 'gray')
plt.show()

weight 불러오기

1
2
3
4
5
6
7
8
9
10
11
12
13
14
weight = layer.get_weights()[0]

plt.figure(figsize=(15, 5))
plt.subplot(131)
plt.hist(output.numpy().ravel(), range=[-2, 2])
plt.ylim(0, 100)
plt.subplot(132)
plt.title(weight.shape)
plt.imshow(weight[:, :, 0, 0], 'gray')
plt.subplot(133)
plt.title(output.shape)
plt.imshow(output[0, :, :, 0], 'gray')
plt.colorbar()
plt.show()

Activation Function

0 미만의 값들을 0으로 바꿔준다.

1
2
3
4
5
6
7
8
import numpy as np

act_layer = tf.keras.layers.ReLU()
act_output = act_layer(output)
np.min(act_output), np.max(act_output)

# Out
(0.0, 244.73064) # 0미만 값들이 0으로 바뀜
1
2
3
4
5
6
7
8
9
10
plt.figure(figsize=(15, 5))
plt.subplot(121)
plt.hist(act_output.numpy().ravel(), range=[-2, 2])
plt.ylim(0, 100)

plt.subplot(122)
plt.title(act_output.shape)
plt.imshow(act_output[0, :, :, 0], 'gray')
plt.colorbar()
plt.show()

Pooling

강조되는 것들만 압축합니다.

1
2
3
4
5
6
pool_layer = tf.keras.layers.MaxPool2D(pool_size=(2, 2), strides=(2, 2), padding='SAME')
pool_output = pool_layer(act_output)
pool_output.shape

# Out
TensorShape([1, 14, 14, 3])
1
2
3
4
5
6
7
8
9
10
plt.figure(figsize=(15, 5))
plt.subplot(121)
plt.hist(pool_output.numpy().ravel(), range=[-2, 2])
plt.ylim(0, 100)

plt.subplot(122)
plt.title(pool_output.shape)
plt.imshow(pool_output[0, :, :, 0], 'gray')
plt.colorbar()
plt.show()

Fully Connected

Flatten

1
2
3
4
5
6
7
8
9
10
11
layer = tf.keras.layers.Flatten()
flatten = layer(output)
flatten.shape

# Out
TensorShape([1, 2352]) # 28 * 28 * 3

output.shape

# Out
TensorShape([1, 28, 28, 3])
1
2
3
4
5
6
plt.figure(figsize=(10, 5))
plt.subplot(211)
plt.hist(flatten.numpy().ravel())
plt.subplot(212)
plt.imshow(flatten[:, :100])
plt.show()

Dense

하나씩 연결하겠다는 의미다.

1
2
3
4
5
6
layer = tf.keras.layers.Dense(32, activation='relu')
output = layer(flatten)
output.shape

# Out
TensorShape([1, 32]) # 32로 줄어들었습니다.

DropOut

학습할 때만 랜덤하게 끊어준다. 학습이 끝나면 다시 복구시킨다.

1
2
3
4
5
6
7
# 0.7은 비율입니다. 얼마나 끊을 것인지 살릴 것인지
layer = tf.keras.layers.Dropout(0.7)
output = layer(output)
output.shape

# Out
TensorShape([1, 32])

Build Model

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
from tensorflow.keras import layers

input_shape = (28, 28, 1)
num_classes = 10

inputs = layers.Input(shape=input_shape)

# Feature Extraction
net = layers.Conv2D(32, 3, padding='SAME')(inputs)
net = layers.Activation('relu')(net)
net = layers.Conv2D(32, 3, padding='SAME')(net)
net = layers.Activation('relu')(net)
net = layers.MaxPool2D(2, 2)(net)
net = layers.Dropout(0.25)(net)

net = layers.Conv2D(64, 3, padding='SAME')(net)
net = layers.Activation('relu')(net)
net = layers.Conv2D(64, 3, padding='SAME')(net)
net = layers.Activation('relu')(net)
net = layers.MaxPool2D(2, 2)(net)
net = layers.Dropout(0.25)(net)

# Fully Connected
net = layers.Flatten()(net)
net = layers.Dense(512)(net)
net = layers.Activation('relu')(net)
net = layers.Dropout(0.25)(net)
net = layers.Dense(num_classes)(net)
net = layers.Activation('softmax')(net)

model = tf.keras.Model(inputs=inputs, outputs=net, name='Basic-CNN')
model.summary()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
# Out
Model: "Basic-CNN"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         [(None, 28, 28, 1)]       0
_________________________________________________________________
conv2d_25 (Conv2D)           (None, 28, 28, 32)        320
_________________________________________________________________
activation_8 (Activation)    (None, 28, 28, 32)        0
_________________________________________________________________
conv2d_26 (Conv2D)           (None, 28, 28, 32)        9248
_________________________________________________________________
activation_9 (Activation)    (None, 28, 28, 32)        0
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 14, 14, 32)        0
_________________________________________________________________
dropout_5 (Dropout)          (None, 14, 14, 32)        0
_________________________________________________________________
conv2d_27 (Conv2D)           (None, 14, 14, 64)        18496
_________________________________________________________________
activation_10 (Activation)   (None, 14, 14, 64)        0
_________________________________________________________________
conv2d_28 (Conv2D)           (None, 14, 14, 64)        36928
_________________________________________________________________
activation_11 (Activation)   (None, 14, 14, 64)        0
_________________________________________________________________
max_pooling2d_9 (MaxPooling2 (None, 7, 7, 64)          0
_________________________________________________________________
dropout_6 (Dropout)          (None, 7, 7, 64)          0
_________________________________________________________________
flatten_4 (Flatten)          (None, 3136)              0
_________________________________________________________________
dense_3 (Dense)              (None, 512)               1606144
_________________________________________________________________
activation_12 (Activation)   (None, 512)               0
_________________________________________________________________
dropout_7 (Dropout)          (None, 512)               0
_________________________________________________________________
dense_4 (Dense)              (None, 10)                5130
_________________________________________________________________
activation_13 (Activation)   (None, 10)                0
=================================================================
Total params: 1,676,266
Trainable params: 1,676,266
Non-trainable params: 0
_________________________________________________________________
Share