Population metaheuristics manipulate a set of solutions at each iteration from which the population of the next iteration is produced. Examples are evolutionary algorithms and scatter search, and construction-oriented techniques such as ant colony optimization and the greedy randomized adaptive search procedure. The metaheuristics that deal with only one solution at any given time are called trajectory metaheuristics where the search process describes a trajectory in the search space [ 1 , 2 , 4 ] as shown in Figure 1 [ 12 ]. When they first appeared, pure metaheuristics quickly became state-of-the-art algorithms for many optimization problems as they found high-quality solutions for these optimization problems.
This was reported in many specific conferences and workshops. This success had motivated researches toward finding answers to questions such as:. Which metaheuristic is best for a given optimization problem?
Despite this success, it became recently evident that the focus on pure metaheuristics is restrictive when tackling particular optimization problems such as real-world and large-scale optimization problems [ 2 ]. A skilled combination of a metaheuristic with components from other metaheuristics or with other optimization algorithms such as operations research techniques mathematical programming , artificial intelligence techniques constraint programming , or complete algorithms branch and bound can lead to getting much better solutions for these optimization problems.
This interdisciplinary field is called hybrid metaheuristics which goes beyond the scope of a pure metaheuristic [ 1 ]. Over the years, many algorithms that do not purely follow the paradigm of a pure metaheuristic were developed. They combine various algorithmic components originating from different optimization algorithms [ 2 ].
This is explained in Section 3. The rest of this chapter is organized as follows. The following section introduced classification problems. Section 3 explains the main forms of hybridizing metaheuristics. Section 4 demonstrates designing a hybrid metaheuristic. The fifth section demonstrates hybrid metaheuristics for classification problems. The discussion is given in Section 6.
The last section concludes this chapter and highlights future work in this area. Classification involves training and testing data which consist of data instances objects. Each instance in the training set contains one class label called target, dependent, response, or features and other features called attributes, inputs, predictors, or independent features [ 13 — 15 ].
Classification consists of examining the features of a new object and then assigning it to one of the predefined set of classes. The objects to be classified are generally represented by records in a dataset. The classification task is to build a model that will be applied to unclassified data to classify it, that is, predicting the target values of instances that are given only the input features in the testing set [ 15 , 16 ].
The classification task determining which of the fixed set of classes an example belongs to is illustrated in Figure 2. The accuracy of a classifier refers to how well it can predict the value of the predictor feature for a previous unseen data and how well it captured the dependencies among the input features. Classifier accuracy is the main measure for classification and is widely used. The classifier accuracy goes up when comparing between different classifiers [ 18 — 20 ].
The classifier is considered the basic component of any classification system, and its task is to partition the feature space into class-labeled decision regions one for each category. This choice affects the accuracy of these classifiers, the time needed for learning, and the number of examples needed for learning. Feature selection FS can be seen as an optimization problem that involves searching the space of possible solutions feature subsets to identify the optimal one.
Many metaheuristics such as ant colony optimization algorithms, particle swarm optimization, genetic algorithms, simulated annealing, and tabu search have been used for solving the feature selection problem [ 20 , 21 ]. Feature selection deleting a column from a dataset is one of the main aspects of dimension reduction besides instance reduction deleting a row from a dataset. This is illustrated in Figure 3 [ 18 ]. Both of these should keep the characteristics of the original input data after excluding some of it. Figure 4 [ 22 ] illustrates the revised classification with the use of dimension reduction phase as an intermediate step.
In Figure 4 , dimension reduction is performed first to the given data, and then, the prediction methods are applied to the reduced data.
Although combining different algorithms together dates back to s, in recent years only hybrid metaheuristics have been commonly used. Then, the advantage of combining different algorithms together has been widely recognized [ 1 , 4 ]. Forms of hybridization can be classified into two categories as in Figure 5 : 1 combining components from a metaheuristic into another metaheuristic examples are: using trajectory methods into population algorithms or using a specific local search method into a more general trajectory algorithm such as iterated local search and 2 combining metaheuristics with other techniques such as artificial intelligence and operations research examples are: combining metaheuristics with constraint programming CP , integer programming IP , tree-based search methods, data mining techniques, etc.
The following two subsections explain these types. This category represents the beginning of hybridizing metaheuristics. Later, it got widely used especially integrating nature-inspired metaheuristics with local search methods. This is well illustrated in the most common type of this category which is in ant colony optimization algorithms and evolutionary algorithms that often use local search methods in order to refine the generated solutions during the search process.
The reason for that is these nature-inspired metaheuristics explore well the search space and identify the regions having high-quality solutions since they first capture a global picture of the search space and then they successively focus the search toward the promising regions. However, these nature-inspired metaheuristics are not effective in exploiting the accumulated search experiences that can be achieved by adding local search methods into them. Therefore, the resulting hybrid metaheuristic will work as follows: the nature-inspired metaheuristic will identify the promising search areas from which the local search method can then determine quickly the best solutions.
Based on the above—mentioned fact, the resulting hybrid metaheuristic combining the strengths of both metaheuristics is often very successful. Apart from this hybridization, there are other hybrids. We mentioned it only here as it is considered the standard way of hybridization [ 1 , 2 ]. There are many possible ways of integration between metaheuristics and other algorithms.
For example, metaheuristics and tree search methods can be interleaved or sequentially applied. This can be achieved by using a tree search method for generating a partial solution that a metaheuristic can then complete. Alternatively, a metaheuristic improves a solution generated by a tree search method.
Another example is that constraint programming techniques can be used to reduce the search space or the neighborhoods that will be explored by a local search method [ 1 , 4 ]. It should be noted that all of the hybrid metaheuristics mentioned above are integrative combinations in which there is some kind of master algorithm including one or more subordinate components either embedded or called.
Another way of combinations is called either collaborative or cooperative combinations in which the search is performed by different algorithms that exchange information about states, models, entire subproblems, solutions, or search space characteristics. The cooperative search algorithms consist of parallel execution of search algorithms that can be different or instances of the same algorithm working on different models or running with different parameter settings.
Therefore, the control strategy in hybrid metaheuristics can be integrative or collaborative, and the order of executing the combined parts can be sequential, parallel, or interleaved [ 1 , 4 , 12 ]. These are shown in Figures 6 and 7. The main motivation behind combining various algorithmic ideas from different metaheuristics is to get better performing system that exploits and includes advantages of the combined algorithms [ 3 , 4 ].
These advantages should be complementary to each other so that the resulting hybrid metaheuristic can benefit from them [ 2 , 3 , 23 ]. The key to achieving high performance in the resulting hybrid metaheuristic especially when tackling hard optimization problem is to choose suitable combinations of complementary algorithmic concepts. Therefore, this task of developing a highly effective hybrid metaheuristic is complicated and not easy [ 3 ].
The reasons for that are as follows:. Designing and implementing a hybrid metaheuristic involves wide knowledge about algorithms, data structure, programming, and statistics [ 3 ]. It requires expertise from different optimization areas [ 2 ]. According to Blum et al. This can be achieved by answering the following questions:. What is the optimization objective?
Do we need a reasonable good solution? And whether this solution is needed very quickly or not? Petros Dellaportas. Bayesian methods; calibration; classification; multivariate analysis; near infrared spectroscopy.
In some cases, the strongest signal is a confounding factor, and the variation of interest is captured by higher-order PCs. In: Multivariate Analysis. Sample normalization and variance stabilization together are effective and sufficient preprocessing steps for high-throughput data. Comments By submitting a comment you agree to abide by our Terms and Community Guidelines. Even when your primary goal is data visualization, in which only two or three axes can be displayed at a time, you still need to select a sufficient number of new features to generate. Heartbeat classification using morphological and dynamic features of ECG signals. Sandra L.
Simon Harden. Christian Henning. Volume 1 Issue 1 May Previous Article. Next Article. Part I. Part II. About the article Published Online : Citing Articles Here you can find all Crossref-listed publications in which this article is cited. Combining location-and-scale batch effect adjustment with data cleaning by latent factor adjustment. DOI: The diversity in the applications of partial least squares: an overview. Magwaza, Timothy Dube, Shirly T. Magwaza, Alfred O. Odindo, Asanda Mditshwa, and Paramu L. A road to classification in high dimensional space: the regularized optimal affine discriminant.
Classifying cultivars of rice Oryza sativa L. Review of sparse methods in regression and classification with application to chemometrics. Rasch-based high-dimensionality data reduction and class prediction with applications to microarray gene expression data. Incomplete label distribution learning based on supervised neighborhood information. Canonical correlation analysis for multivariate regression and its application to metabolic fingerprinting.
Covariate adjustment in partial least squares PLS regression for the extraction of the spatial—temporal pattern from positron emission tomography data. Borrowing information from relevant microarray studies for sample classification using weighted partial least squares.
Linking physiology and transcriptional profiles by quantitative predictive models. Importance of data structure in comparing two dimension reduction methods for classification of microarray gene expression data. Dimension reduction with gene expression data using targeted variable importance measurement. Sparse PLS discriminant analysis: biologically relevant feature selection and graphical displays for multiclass problems. Structure-activity models of oral clearance, cytotoxicity, and LD a screen for promising anticancer compounds.
A Partial Least Squares based algorithm for parsimonious variable selection.
Sequential rank agreement methods for comparison of ranked lists. Non-invasive staging of chronic kidney allograft damage using urine metabolomic profiling.
Read the latest chapters of Handbook of Statistics at cydyqywyty.cf, Elsevier's leading Search in this handbook 1 - Statistical Data Mining 7 - Canonical Variate Analysis and Related Methods for Reduction of 8 - Pattern Recognition 12 - Fast Algorithms for Classification Using Class Cover Catch Digraphs. This book provides an introduction to statistical pattern recognition theory and decision-tree approach to discrimination, describing the classification and regression tree to use the Gaussian classifier in the reduced dimension space.
Gibson, David Rush, David S. Wishart, and Tom D. Deciphering the multicomponent synergy mechanism from a systems pharmacology perspective: Application to Gualou Xiebai Decoction for coronary heart disease. Sand Hills Streams. Predicting athlete ground reaction forces and moments from motion capture. A metabolomics approach using juvenile cystic mice to identify urinary biomarkers and altered pathways in polycystic kidney disease.