[CNN] architecture

CNN( = ConvNet)

- sequence of layers

- each layer of a ConvNet transforms one volue of activations to another through a differientable function

- one volume of activations = activation map = feature map

 

ReLU(nonlinear) layer : activates relevant responses

Fully-Connected Layer : each neuron in a layer will be connected to all the numbers in the previous volume

Pooling Layer : downsampling operation layer

Convolutional Layer : specially designed for ConvNet

 

* Multi-Layer perceptron : fully-connected layer(s) + ReLU(or sigmoid.. activation function)

* Multi-Layer perceptron과 CNN의 차이 : pooling layer와 convolutional layer가 추가됨

 

[(Conv-ReLU) * N - Pool ] * M - (FC -ReLU) * K - SoftMax

(N usually ~5, M > 10, 0 <= K <= 2)

 

Convolutional Layer

vs FC layer

 

CNN architecture

1. kernel

: filter

i - k + 1 ( i : input size, k: kernel size)

2. stride

: modify the amount of movement of filter

i - k/s + 1 (s : stride)

3. padding

i - k + 2p/s +1 (p : padding)

4. pooling

: generalizing featrues extracted by convolutional features

5. flatten

: 2D array to single long continuous linear array

 

Layers

1. Convolutional layer

 : extract features from an image

2. Pooling layer

 : decrease size of the convolved feature map

 : Max pooling / Average pooling

3. Fully Connected (FC) layer

 : weights and biases

 : last few layers of CNN architecture

 : connect neurons between different layers

4. Dropout

 : mask

 : nullify the contribution of some neurons towards the next layer

 

Activation function

: determine whether the neuron should be activated or not

 : Sigmoid, tanH, Softmax, ReLU