Scalable AI and Edge Computing Challenges

Deep learning, especially deep neural networks have shown considerable promise through tremendous results in recent years, significantly improving the accuracy of a variety of challenging problems when compared to other machine learning methods. However, deep neural networks require high performance computing systems due to the tremendous quantity of computational layers they possess, leading to a massive quantity of parameters to learn and compute.
Although the deep neural networks outperform conventional machine learning algorithms, due to their computational burden they have to be performed on cloud side which imposes a huge cost to companies. As an example, Facebook has 1.86 billion users worldwide. Let us assume there is an application which categorizes images for users. Assume that each user uploads only one images per day and we know every 16 classifications takes about 147 ms on Titan x. Now if the cost of using a Titan X GPU for 1 h is 5.625 cent, this application costs about $27K1. It is always possible to shrink the cost and expenses!
Furthermore, transferring data to and process them on the cloud accompanied with privacy issues and it causes a lot of headache for industries. The privacy issue is a big concern especially in medical applications and healthcare when the patient information needs to be transferred to the cloud servers. Due to this fact, processing the information in local machines is highly interested in several applications such as decision supports in healthcare.
Scalable AI and more specifically, scalable deep learning is one of the interesting topics which attracts researchers and industries nowadays. The concept of Edge computing has been introduced recently to focus on this problem. At this talk, I will introduce different approaches which have been proposed to address this problem and enables edge computing for deep learning algorithms. Then, I will introduce a new framework that we are pursuing in our research group to tackle this problem. Taking inspiration from biological evolution, we explore the idea of “Can deep neural networks evolve naturally over successive generations into highly efficient deep neural networks?” by introducing the notion of synthesizing new highly efficient, yet powerful deep neural networks over successive generations via an evolutionary process from ancestor deep neural networks.