Abstract
By a News Reporter-Staff News Editor at Network Daily News - Research findingson networks are discussed in a new report. According to news originating from the Chinese Academy ofSciences by NewsRx editors, the research stated, “As neural network inference demands are ever-increasingin intelligent applications, the performance optimization of model serving becomes a challenging problem.”Funders for this research include National Natural Science Foundation of China; China PostdoctoralScience Foundation; Institute of Computing Technology, Chinese Academy of Sciences.The news editors obtained a quote from the research from Chinese Academy of Sciences: “Dynamicbatching is an important feature of contemporary deep learning serving systems, which combines multiplerequests of model inference and executes them together to improve the system’s throughput. However, thebehavior characteristics of each part in deep neural network batching systems as well as their performanceimpact on different model structures are still unknown. In this paper, we characterize the batching systemby leveraging three representative deep neural networks on GPUs, performing a systematic analysis of theperformance effects from the request batching module, model slicing module, and stage reorchestratingmodule.”
NETL
NSTL