Partial multi-label learning (PML) deals with the problem where each training example is associated with an overcomplete set of candidate labels, among which only some candidate labels are valid. The task of PML naturally arises in learning scenarios with inaccurate supervision, and the goal is to induce a multi-label predictor which can assign a set of proper labels for unseen instance. The PML training procedure is prone to be misled by false positive labels concealed in the candidate label set, which serves as the major modeling difficulty for partial multi-label learning. In this paper, a novel two-stage PML approach is proposed which works by eliciting credible labels from the candidate label set for model induction. In this first stage, the labeling confidence of candidate label for each PML training example is estimated via iterative label propagation. In the second stage, by utilizing credible labels with high labeling confidence, multi-label predictor is induced via pairwise label ranking coupled with virtual label splitting or maximum a posteriori (MAP) reasoning. Experimental studies show that the proposed approach can achieve highly competitive generalization performance by excluding most false positive labels from the training procedure via credible label elicitation.

Multi-label learning deals with training examples each represented by a single instance while associated with multiple class labels, and the task is to train a predictive model which can assign a set of proper labels for the unseen instance. Existing approaches employ the common assumption of equal labeling-importance, i.e. all associated labels are regarded to be relevant to the training instance while their relative importance in characterizing its semantics are not differentiated. Nonetheless, this common assumption does not reflect the fact that the importance degree of each relevant label is generally different, though the importance information is not directly accessible from the training examples. In this paper, we show that it is beneficial to leverage the implicit relative labeling-importance (RLI) information to help induce multi-label predictive model with strong generalization performance. Specifically, RLI degrees are formalized as multinomial distribution over the label space, which can be estimated by either global label propagation procedure or local k-nearest neighbor reconstruction. Correspondingly, the multi-label predictive model is induced by fitting modeling outputs with estimated RLI degrees along with multi-label empirical loss regularization. Extensive experiments clearly validate that leveraging implicit RLI information serves as a favorable strategy to achieve effective multi-label learning.

Multi-label learning deals with training examples each represented by a single instance while associated with multiple class labels. Due to the exponential number of possible label sets to be considered by the predictive model, it is commonly assumed that label correlations should be well exploited to design an effective multi-label learning approach. On the other hand, class-imbalance stands as an intrinsic property of multi-label data which significantly affects the generalization performance of the multi-label predictive model. For each class label, the number of training examples with positive labeling assignment is generally much less than those with negative labeling assignment. To deal with the class-imbalance issue for multi-label learning, a simple yet effective class-imbalance aware learning strategy called cross-coupling aggregation (Cocoa) is proposed in this article. Specifically, Cocoa works by leveraging the exploitation of label correlations as well as the exploration of class-imbalance simultaneously. For each class label, a number of multiclass imbalance learners are induced by randomly coupling with other labels, whose predictions on the unseen instance are aggregated to determine the corresponding labeling relevancy. Extensive experiments on 18 benchmark datasets clearly validate the effectiveness of Cocoa against state-of-the-art multi-label learning approaches especially in terms of imbalance-specific evaluation metrics.

Multi-label classification aims to assign a set ofproper labels for each instance, where distance metric learningcan help improve the generalization ability of instance-basedmulti-label classification models. Existing multi-label metriclearning techniques work by utilizing pairwise constraints toenforce that examples with similar label assignments shouldhave close distance in the embedded feature space. In thispaper, a novel distance metric learning approach for multi-label classification is proposed by modeling structural interac-tions between instance space and label space. On one hand,compositional distance metric is employed which adopts therepresentation of a weighted sum of rank-1 PSD matricesbased on component bases. On the other hand, compositionalweights are optimized by exploiting triplet similarity con-straints derived from both instance and label spaces. Due tothe compositional nature of employed distance metric, theresulting problem admits quadratic programming formulationwith linear optimization complexity w.r.t. the number of train-ing examples. We also derive the generalization bound forthe proposed approach based on algorithmic robustness anal-ysis of the compositional metric. Extensive experiments onsixteen benchmark data sets clearly validate the usefulness ofcompositional metric in yielding effective distance metric formulti-label classification.

Multi-dimensional classification (MDC) aims to build classification models for multiple heteroge-nous class spaces simultaneously, where each class space characterizes the semantics of an object w.r.t. onespecific dimension. Modeling dependencies among class spaces plays a key role in solving MDC tasks, wheremost approaches work by assuming directed acyclic graph (DAG) structure or random chaining structureover class spaces. Different from existing probabilistic strategies, a deterministic strategy namedSeemfordependency modeling is proposed in this paper via stacked dependency exploitation. In the first-level, pair-wise dependencies are considered which can be modeled more reliably than modeling full dependencies amongall class spaces by DAG or chaining structure. In the second-level, the class label of unseen instance w.r.t.each class space is determined by adaptively stacking predictive outputs from first-level pairwise classifiers.Experimental results show that stacked dependency exploitation leads to superior performance against state-of-the-art MDC approaches.

In multi-dimensional classification (MDC), each training example is represented by a single instance (feature vector) while associated with multiple class variables, each of which specifies its class membership w.r.t. one specific class space. Most existing MDC approaches try to model dependencies among class variables in output space when inducing predictive functions, while the potential usefulness of manipulating feature space hasn’t been investigated. As a first attempt towards feature manipulation in input space for MDC, a simple yet effective approach named Kram is proposed which enriches the original feature space with augmented features based on kNN techniques. Specifically, simple counting statistics on the class membership of neighboring MDC examples as well as distance information between MDC examples and their k nearest neighbors are used to generate augmented feature vector. In this way, discriminative information from class space is expected to be brought into the feature space which would be helpful to the following MDC predictive model induction. To validate the effectiveness of the proposed feature augmentation techniques, comprehensive comparative studies are conducted over fifteen benchmark data sets. Compared to the original feature space, it is clearly shown that the kNN-augmented features generated by the proposed Kram approach can significantly improve generalization abilities of existing MDC approaches.

In this paper, we investigate the large-scale multi-label image classification problem when images with unknown novel classes come in stream during the training stage. It coincides with the practical requirement that usually novel classes are detected and used to update an existing image recognition system. Most existing multi-label image classification methods cannot be directly applied in this scenario, where the training and testing stages must have the same label set. In this paper, we proposed to learn a multi-label classifier and a novel-class detector alternately to solve this problem. The multi-label classifier is learned using a convolutional neural network (CNN) from the images in the known classes. We proposed a recurrent novel-class detector which is learned in the supervised manner to detect the novel class by encoding image features with the multi-label information. In the experiment, our method is evaluated on several large-scale multi-label benchmarks including MS COCO. The results show the proposed method is comparable to most existing multi-label image classification methods, which validate its efficacy when encountering streaming images with unknown classes.

Representation learning is one of the most important aspects of multi-label learning because of the intricate nature of multi-label data. Current research on representation learning either fails to consider label knowledge or is affected by the lack of labeled data. Moreover, most of them learn the representations and incorporate the label information in a two-step manner. In this paper, due to the success of representation learning by deep learning we propose a novel framework based on neural networks named SERL to learn global feature representation by jointly considering all labels in an effective supervised manner. At its core, a two-encoding-layer autoencoder, which can utilize labeled and unlabeled data, is adopted to learn feature representation in the supervision of softmax regression. Specifically, the softmax regression incorporates label knowledge to improve the performance of both representation learning and multi-label learning by being jointly optimized with the autoencoder. Moreover, the autoencoder is expanded into two encoding layers to share knowledge with the softmax regression by sharing the second encoding weight matrix. We conduct extensive experiments on five real-world datasets to demonstrate the superiority of SERL over other state-of-the-art multi-label learning approaches.

Weakly supervised part-of-speech (POS) tagging is to learn to predict the POS tag for a given word in context by making use of partial annotated data instead of the fully tagged corpora. Weakly supervised POS tagging would benefit various natural language processing applications in such languages where tagged corpora are mostly unavailable. In this article, we propose a novel framework for weakly supervised POS tagging based on a dictionary of words with their possible POS tags. In the constrained error-correcting output codes (ECOC)-based approach, a unique L-bit vector is assigned to each POS tag. The set of bitvectors is referred to as a coding matrix with value { 1, -1}. Each column of the coding matrix specifies a dichotomy over the tag space to learn a binary classifier. For each binary classifier, its training data is generated in the following way: each pair of words and its possible POS tags are considered as a positive training example only if the whole set of its possible tags falls into the positive dichotomy specified by the column coding and similarly for negative training examples. Given a word in context, its POS tag is predicted by concatenating the predictive outputs of the L binary classifiers and choosing the tag with the closest distance according to some measure. By incorporating the ECOC strategy, the set of all possible tags for each word is treated as an entirety without the need of performing disambiguation. Moreover, instead of manual feature engineering employed in most previous POS tagging approaches, features for training and testing in the proposed framework are automatically generated using neural language modeling. The proposed framework has been evaluated on three corpora for English, Italian, and Malagasy POS tagging, achieving accuracies of 93.21%, 90.9%, and 84.5% individually, which shows a significant improvement compared to the state-of-the-art approaches.

Multi-label learning deals with problems where each example is represented by a single instance while being associated with multiple class labels simultaneously. Binary relevance is arguably the most intuitive solution for learning from multi-label examples. It works by decomposing the multi-label learning task into a number of independent binary learning tasks (one per class label). In view of its potential weakness in ignoring correlations between labels, many correlation-enabling extensions to binary relevance have been proposed in the past decade. In this paper, we aim to review the state of the art of binary relevance from three perspectives. First, basic settings for multi-label learning and binary relevance solutions are briefly summarized. Second, representative strategies to provide binary relevancewith label correlation exploitation abilities are discussed. Third, some of our recent studies on binary relevance aimed at issues other than label correlation exploitation are introduced. As a conclusion, we provide suggestions on future research directions.