首页|Researchers Submit Patent Application, 'Masked Autoencoders for Computer Vision' , for Approval (USPTO 20240096072)
Researchers Submit Patent Application, 'Masked Autoencoders for Computer Vision' , for Approval (USPTO 20240096072)
扫码查看
点击上方二维码区域，可以放大扫码查看
原文链接
NETL
NSTL
No assignee for this patent application has been made.News editors obtained the following quote from the background information suppli ed by the inventors:“Deep learning has witnessed an explosion of architectures of continuously growing capability and capacity.Aided by the rapid gains in har dware, models today may easily overfit many images (e.g., million images)and be gin to demand hundreds of millions of, often publicly inaccessible, labeled imag es. This appetite fordata has been successfully addressed in natural language p rocessing (NLP) by self-supervised pre-training.Solutions, based on autoregress ive language modelling and masked autoencoding in prior approaches(e.g., genera tive pre-trained transformer (GPT), bidirectional encoder representations from t ransformers(BERT)), are conceptually simple. This may include removing a portio n of the data and learn to predictthe removed content. These methods now enable training of generalizable NLP models containing overone hundred billion parame ters.
NETL
NSTL
