In the nodding RNN course, the teacher assigned homework to rewrite AlexNet using depthwise separable convolution.
Huh? Depthwise separable convolution? What is that? After surfing for a while, I found some good content below.
The concept is explained very well, along with background knowledge expansion, but the code demonstration is based on Keras. I will supplement with PyTorch implementation videos later (I have finished understanding the principles and skimmed through the code part).
- 【精读AI论文】谷歌轻量化网络MobileNet V1（附MobileNetV1实时图像分类代码）_哔哩哔哩_bilibili
- Extended content recommended in the previous video (watched enough videos, will not read for now)
【转载】轻量级神经网络“巡礼”（二）—— MobileNet，从V1到V3
PyTorch code implementation (don’t understand)
V1 highlights
V2 highlights (don’t understand)
Inverted residual structure
- First use PW (pointwise convolution) to increase dimensions, then DW (depthwise convolution), then PW to reduce dimensions
ReLU6
Linear bottlenecks
V3 highlights (don’t understand)
Updated Block (bneck)
- Added SE (Squeeze-and-Excitation) module
Using NAS (Neural Architecture Search) to search parameters
Redesigned time-consuming layer structure