Skip to content

Latest commit

 

History

History
54 lines (41 loc) · 2.11 KB

README.md

File metadata and controls

54 lines (41 loc) · 2.11 KB

deepmark

THE Deep Learning Benchmarks

See: soumith/convnet-benchmarks#101

Come back here on June 15th, 2016.
A bit delayed due to y'know -- a lot of co-ordination among groups.

Networks

Images

Video

Audio

Text

Platform

  • Initially multi-GPU with (1 to 4 titan-X cards)
  • However, multi-machine, custom hardware, other GPU cards such as AMD, CPUs etc. can and should be accommodated, we will work this out after the initial push.

Metrics

  • Round-trip time for 1 epoch of training (will define an epoch size separately for each network)
  • Maximum batch-size that fits (to show and focus on the extra memory consumption that the framework uses)

Frameworks

Everyone who wants to join-in, but I thought an initial set that is important to cover would be:

  • Caffe
  • Chainer
  • CNTK
  • MXNet
  • Neon
  • Theano
  • TensorFlow
  • Torch

Scripts format

Guarantees

  • I will personally to the best of my abilities make sure that the benchmarking is fair and unbiased. The hope is that the community at large will watch these and point-out / fix mistakes.

Governance

  • The benchmarks will be placed at https://github.com/DeepMark/deepmark and other key community members / organizations who want ownership will be welcome to join in proposing new benchmarks that get relevant as the field progresses.