Abstract
Reporters obtained the following quote from the background information supplied by the inventors:“The present invention relates generally to the field of computing, and more particularly to data labeling.“Even with the rise of unsupervised learning and weak supervision techniques, human-labeled data isoften still a necessary part of machine learning pipelines for different contexts and/or applications. Thisbranch of artificial intelligence (AI) may be referred to as a human-in-the-loop (HITL) system. A HITLsystem may leverage both human and machine intelligence in the creation of a machine learning model.This may often involve using humans for the task of labeling large amounts of data which may be anasynchronous process and may lead to conflict amongst labelers, where individual labelers potentiallysubmit labels in disagreement from each other. When noisy data is fed to a machine learning model, theaccuracy and/or performance of the overall system may be jeopardized.
NETL
NSTL