Benchmark Analysis of Representative Deep Neural Network Architectures
Benchmark Analysis of Representative Deep Neural Network Architectures was a 2018 paper that compared dozens of neural network architectures, looking at model accuracy (on ImageNet 1k), inference speed, FLOPs, memory usage, and parameter count.
It follows a 2016 benchmark, but expands it by
- Including far more architectures
- Running experiments on two different hardware platforms: one with a top-of-the-line consumer GPU (3840 cores, $1100), and one that is more modest ($400).
The paper makes the following claims:
Finally, all models use 224x224 images, except for NASNet-A-Large (which uses 331x331) and various Inception nets which use (229x229).
The figures below shows accuracy (using center-crop only) versus FLOPs for a single forward pass. The size of the circle shows the number of model parameters.