question

[dimiy]

sorry can you give your email address or where is it convenient for you to chat? I wanted to ask something about the framework and skills.

[jnbraun]

Hi, you may post your questions here if you don't mind.

[dimiy]

Hello. OK!
1)What do I need to know to create a deep learning framework for working with computer vision? 2)What you need to know and where to study better? Please advise the resources for learning (tensors, autograd, NN abstractions, optimizers, training loop abstraction, cnn, distributed training, etc )
3)where did you get the knowledge after which you could write a framework? (courses, books, etc.)
4)xpress your opinion on the given resources. you are welcome.
http://blog.ezyang.com/2019/05/pytorch-internals/
https://medium.com/@whitesell.ben/how-to-build-your-own-deep-learning-framework-its-easy-i-ll-prove-it-c859cb790386
https://github.com/pjreddie/uwimg
https://nbviewer.jupyter.org/github/domluna/labs/blob/master/Build%20Your%20Own%20TensorFlow.ipynb
https://www.xinjianl.com/blog/2019/01/01/pytensor-operation/
http://dlsys.cs.washington.edu/
https://course.fast.ai/videos/?lesson=8

Thanks!
Regards Erron

[jnbraun]

Hi, sorry for the delay, it has been a busy week :)

To answer your questions in a straightforward way, I would say that it really depends on your end goal here: is it for learning the basics of deep learning applied to computer vision, for understanding how the theoretical concepts of NN are actually implemented, or for building something useful in a particular context ?

In a general way, I would not recommend to create one from scratch but instead, pick one existing framework and expand (or reduce) it according to your focus and needs.
For that, I would list the different steps as follows:

1. Have a grasp on the theoretical concepts behind ConvNets, nowadays, learning resources are abundant but if there is one to cite, it is probably the online course CS231N from Stanford Uni.

2. Study code from early days frameworks. Mainstream frameworks such as Tensorflow or Pytorch try to cover all possible use-cases, therefore the risk is to get overwhelmed by their large codebase.
   My personal favorites here are :

• cxxnet (https://github.com/dmlc/cxxnet), one of MXNet ancestor,
• Darknet from Joseph Redmon (https://github.com/pjreddie/darknet), probably the easiest to read and understand. (Also, I like his way to come back to the basics as he does in the link you mentioned: https://github.com/pjreddie/uwimg)

Those two libraries are very CV oriented and offer a very good trade-off between performance, features covered and codebase complexity. Other good candidates could be caffe and TinyDNN.
The resources you give in your message are good to start with, but at one point, if you want to build something competitive, I guess you will have to go at the C/C++ level anyway.

3. Start from one of the framework you studied, and decide on what you want to focus: for example, inference speed, cross-platform capabilities, distributed training, multi-gpu, efficient data loading / augmentation pipeline, reducing the number of dependencies ...
   Then you could start to add your own bricks according to your needs. For that, you will probably need to know some stuff about specific SW engineering (for example, if you want to optimize the CPU speed, go for multi-threading and SIMD).

Good luck anyway !

[dimiy]

THANKS!

[dimiy]

sorry. Will there be enough skills after CS231N? and can you advise what else is worth seeing?

[dimiy]

Excuse for troubling
where is it more convenient for you to talk?
I would like to ask something
please give feedback