Hybrid Filter-Wrapper Feature Selection using Modified Flower Pollination Algorithm

Computer and Knowledge Engineering

دوره 8، شماره 2 - شماره پیاپی 16، مرداد 2025، صفحه 55-74   

نوع مقاله: Original Article

شناسه دیجیتال (DOI): 10.22067/cke.2025.89191.1124

نویسندگان

Mohammad Ansari Shiri¹؛ Najme Mansouri^{* 2}

¹Department of Computer Science, Shahid Bahonar University of Kerman

²shahid ba honar university of kerman

چکیده

A major challenge in machine learning and data science is feature selection. Feature selection involves selecting the optimal (or suboptimal) subset of features to derive useful conclusions from a dataset based on the relevant information contained in those features. Flower Pollination Algorithm (FPA) is a metaheuristic algorithm developed recently based on flower pollination. In this paper, we propose a new type of binary FPA, called the Filter-Wrapper Modified Binary FPA (FWMBFPA), which aims to improve convergence rate and solution quality by combining filter and wrapper advantages. Using FWMBFPA, the exploration process is directed towards specific search areas by extracting the features of existing solutions. 18 UCI datasets are used to evaluate the performance of the method. FWMBFPA generally performs better than the other algorithms in terms of average classification accuracy. FWMBFPA achieves highest classification accuracy with the smallest number of selected features when compared to other algorithms when dealing with datasets with a large number of features.

کلیدواژه‌ها

Feature selection؛ Flower Pollination Algorithm؛ Filter؛ Wrapper

مراجع

[1]    P. Dhal and C. Azad. (2023, Sep.). A Lightweight Filter Based Feature Selection Approach for Multi-Label Text Classification. Journal of Ambient Intelligence and Humanized Computing. [Online]. 14(9), pp. 12345–12357. Available: https://doi.org/10.1007/s12652-022-04335-5

[2]    X. Cui, Y. Li, J. Fan, and T. Wang. (2022, May). A Novel Filter Feature Selection Algorithm Based on Relief. Applied Intelligence. [Online]. 52(5), pp. 5063–5081. Available: https://doi.org/10.1007/s10489-021-02659-x

[3]    S. S. Shekhawat, H. Sharma, S. Kumar, A. Nayyar, and B. Qureshi. (2021, Jan.). bSSA: Binary Salp Swarm Algorithm with Hybrid Data Transformation for Feature Selection. IEEE Access. [Online]. 9, pp. 14867–14882. Available: https://doi.org/10.1109/ACCESS.2021.3049547

[4]    F. Kılıç, Y. Kaya, and S. Yildirim. (2021, Mar.). A Novel Multi-Population Based Particle Swarm Optimization for Feature Selection. Knowledge-Based Systems. [Online]. 219, p. 106894. Available: https://doi.org/10.1016/j.knosys.2021.106894

[5]    M. Kelidari and J. Hamidzadeh. (2021, Apr.). Feature Selection by Using Chaotic Cuckoo Optimization Algorithm with Levy Flight, Opposition-Based Learning and Disruption Operator. Soft Computing. [Online]. 25(4), pp. 2911–2933. Available: https://doi.org/10.1007/s00500-020-05349-x

[6]    D. Wang, H. Chen, T. Li, J. Wan, and Y. Huang. (2020, Dec.). A Novel Quantum Grasshopper Optimization Algorithm for Feature Selection. International Journal of Approximate Reasoning. [Online]. 127, pp. 33–53. Available: https://doi.org/10.1016/j.ijar.2020.08.010

[7]    Y. Xue, X. Cai, and W. Jia. (2023, Jun.). Particle Swarm Optimization Based on Filter-Based Population Initialization Method for Feature Selection in Classification. Journal of Ambient Intelligence and Humanized Computing. [Online]. 14(6), pp. 7355–7366. Available: https://doi.org/10.1007/s12652-022-04444-1

[8]    N. S. M. Nafis and S. Awang. (2021, Apr.). An Enhanced Hybrid Feature Selection Technique Using Term Frequency-Inverse Document Frequency and Support Vector Machine-Recursive Feature Elimination for Sentiment Classification. IEEE Access. [Online]. 9, pp. 52177–52192. Available: https://doi.org/10.1109/ACCESS.2021.3069001

[9]    L. Peng, Z. Cai, A. A. Heidari, L. Zhang, and H. Chen. (2023, Nov.). Hierarchical Harris Hawks Optimizer for Feature Selection. Journal of Advanced Research. [Online]. 53, pp. 261–278. Available: https://doi.org/10.1016/j.jare.2023.01.014

[10]  K. K. Ghosh, S. Ahmed, P. K. Singh, Z. W. Geem, and R. Sarkar. (2020, Apr.). Improved Binary Sailfish Optimizer Based on Adaptive β-Hill Climbing for Feature Selection. IEEE Access. [Online]. 8, pp. 83548–83560. Available: https://doi.org/10.1109/ACCESS.2020.2991543

[11]  W. Ma, X. Zhou, H. Zhu, L. Li, and L. Jiao. (2021, Aug.). A Two-Stage Hybrid Ant Colony Optimization for High-Dimensional Feature Selection. Pattern Recognition. [Online]. 116, p. 107933. Available: https://doi.org/10.1016/J.PATCOG.2021.107933

[12]  M. Mafarja, I. Aljarah, A. A. Heidari, A. I. Hammouri, H. Faris, A. M. Al-Zoubi, and S. Mirjalili. (2018, Apr.). Evolutionary Population Dynamics and Grasshopper Optimization Approaches for Feature Selection Problems. Knowledge-Based Systems. [Online]. 145, pp. 25–45. Available: https://doi.org/10.1016/j.knosys.2017.12.037

[13]  M. M. Mafarja and S. Mirjalili. (2017, Oct.). Hybrid Whale Optimization Algorithm with Simulated Annealing for Feature Selection. Neurocomputing. [Online]. 260, pp. 302–312. Available: https://doi.org/10.1016/j.neucom.2017.04.053

[14]  H. Faris, A. A. Heidari, A. M. Al-Zoubi, M. Mafarja, I. Aljarah, M. Eshtay, and S. Mirjalili. (2020, Mar.). Time-Varying Hierarchical Chains of Salps with Random Weight Networks for Feature Selection. Expert Systems with Applications. [Online]. 140, p. 112898. Available: https://doi.org/10.1016/J.PATCOG.2021.107933

[15]  A. Faramarzi, M. Heidarinejad, B. Stephens, and S. Mirjalili. (2020, Mar.). Equilibrium Optimizer: A Novel Optimization Algorithm. Knowledge-Based Systems. [Online]. 191, p. 105190. Available: https://doi.org/10.1016/j.knosys.2019.105190

[16]  P. Agrawal, H. F. Abutarboush, T. Ganesh, and A. W. Mohamed. (2021, Mar.). Metaheuristic Algorithms on Feature Selection: A Survey of One Decade of Research (2009–2019). IEEE Access. [Online]. 9, pp. 26766–26791. Available: https://doi.org/10.1109/TMTT.2021.3056433

[17]  A. M. Anter and M. Ali. (2020, Mar.). Feature Selection Strategy Based on Hybrid Crow Search Optimization Algorithm Integrated with Chaos Theory and Fuzzy C-Means Algorithm for Medical Diagnosis Problems. Soft Computing. [Online]. 24(3), pp. 1565–1584. Available: https://doi.org/10.1007/s00500-019-03988-3

[18]  L. Abualigah, B. Alsalibi, M. Shehab, M. Alshinwan, A. M. Khasawneh, and H. Alabool. (2021, Sep.). A Parallel Hybrid Krill Herd Algorithm for Feature Selection. International Journal of Machine Learning and Cybernetics. [Online]. 12, pp. 783–806. Available: https://doi.org/10.1007/s13042-020-01202-7

[19]  M. Alweshah, S. Alkhalaileh, D. Albashish, M. Mafarja, Q. Bsoul, and O. Dorgham. (2021, Jul.). A Hybrid Mine Blast Algorithm for Feature Selection Problems. Soft Computing. [Online]. 25, pp. 517–534. Available: https://doi.org/10.1007/s00500-020-05164-4

[20]  E. Nabil. (2016, Feb.). A Modified Flower Pollination Algorithm for Global Optimization. Expert Systems with Applications. [Online]. 57, pp. 192–203. Available: https://doi.org/10.1016/j.eswa.2016.03.005

[21]  Z. A. A. Alyasseri, A. T. Khader, M. A. Al-Betar, M. A. Awadallah, and X.-S. Yang. (2018). Variants of the Flower Pollination Algorithm: A Review. In Nature-Inspired Algorithms and Applied Optimization, pp. 91–118. [Online]. Available: https://doi.org/10.1007/978-3-319-67669-2_5

[22]  X.-S. Yang. (2012). Flower Pollination Algorithm for Global Optimization. In International Conference on Unconventional Computing and Natural Computation, Lecture Notes in Computer Science, vol. 7445, pp. 240–249. Springer. Available: https://doi.org/10.1007/978-3-642-32894-7_27

[23]  M. Abdel-Basset and L. A. Shawky. (2019, Dec.). Flower Pollination Algorithm: A Comprehensive Review. Artificial Intelligence Review. [Online]. 52, pp. 2533–2557. Available: https://doi.org/10.1007/s10462-018-9624-4

[24]  M. Abdel-Basset, R. Mohamed, S. Saber, S. S. Askar, and M. Abouhawwash. (2021, Jul.). Modified Flower Pollination Algorithm for Global Optimization. Mathematics. [Online]. 9(14), p. 1661. Available: https://doi.org/10.3390/math9141661

[25]  S. Mirjalili, A. H. Gandomi, S. Z. Mirjalili, S. Saremi, H. Faris, and S. M. Mirjalili. (2017, Dec.). Salp Swarm Algorithm: A Bio-Inspired Optimizer for Engineering Design Problems. Advances in Engineering Software. [Online]. 114, pp. 163–191. Available: https://doi.org/10.1016/j.advengsoft.2017.07.002

[26]  A. E. Hegazy, M. A. Makhlouf, and G. S. El-Tawel. (2019, Apr.). Feature Selection Using Chaotic Salp Swarm Algorithm for Data Classification. Arabian Journal for Science and Engineering. [Online]. 44, pp. 3801–3816. Available: https://doi.org/10.1007/s13369-018-3680-6

[27]  A. K. Naik, V. Kuppili, and D. R. Edla. (2020, Jun.). Efficient Feature Selection Using One-Pass Generalized Classifier Neural Network and Binary Bat Algorithm with a Novel Fitness Function. Soft Computing. [Online]. 24(6), pp. 4575–4587. Available: https://doi.org/10.1007/s00500-019-04218-6

[28]  Y. Gao, Y. Zhou, and Q. Luo. (2020, Aug.). An Efficient Binary Equilibrium Optimizer Algorithm for Feature Selection. IEEE Access. [Online]. 8, pp. 140936–140963. Available: https://doi.org/10.1109/ACCESS.2020.3013617

[29]  S. Saremi, S. Mirjalili, and A. Lewis. (2017, Mar.). Grasshopper Optimisation Algorithm: Theory and Application. Advances in Engineering Software. [Online]. 105, pp. 30–47. Available: https://doi.org/10.1016/j.advengsoft.2017.01.004

[30]  S. Zhao, P. Wang, A. A. Heidari, X. Zhao, C. Ma, and H. Chen. (2022). An Enhanced Cauchy Mutation Grasshopper Optimization with Trigonometric Substitution: Engineering Design and Feature Selection. Engineering with Computers. [Online]. 38(Suppl 5), pp. 4583–4616. Available: https://doi.org/10.1007/s00366-021-01448-x

[31]  B. Doğan and T. Ölmez. (2015, Jan.). A New Metaheuristic for Numerical Function Optimization: Vortex Search Algorithm. Information Sciences. [Online]. 293, pp. 125–145. Available: https://doi.org/10.1016/j.ins.2014.08.053

[32]  F. S. Gharehchopogh, I. Maleki, and Z. A. Dizaji. (2022, Mar.). Chaotic Vortex Search Algorithm: Metaheuristic Algorithm for Feature Selection. Evolutionary Intelligence. [Online]. 15(3), pp. 1777–1808. Available: https://doi.org/10.1007/s12065-021-00590-1

[33]  N. Bacanin, K. Venkatachalam, T. Bezdan, M. Zivkovic, and M. Abouhawwash. (2023, Feb.). A Novel Firefly Algorithm Approach for Efficient Feature Selection with COVID-19 Dataset. Microprocessors and Microsystems. [Online]. 98, p. 104778. Available: https://doi.org/10.1016/j.micpro.2023.104778

[34]  Y. Liu, X. Zou, S. Ma, M. Avdeev, and S. Shi. (2022, Oct.). Feature Selection Method Reducing Correlations Among Features by Embedding Domain Knowledge. Acta Materialia. [Online]. 238, p. 118195. Available: https://doi.org/10.1016/j.actamat.2022.118195

[35]  J. R. Vergara and P. A. Estévez. (2014). A Review of Feature Selection Methods Based on Mutual Information. Neural Computing and Applications. [Online]. 24, pp. 175–186. Available: https://doi.org/10.1007/s00521-013-1368-0

[36]  L. Wang, S. Jiang, and S. Jiang. (2021, Nov.). A Feature Selection Method via Analysis of Relevance, Redundancy, and Interaction. Expert Systems with Applications. [Online]. 183, p. 115365. Available: https://doi.org/10.1016/j.eswa.2021.115365

[37]  M. Mafarja, A. Qasem, A. A. Heidari, I. Aljarah, H. Faris, and S. Mirjalili. (2020). Efficient Hybrid Nature-Inspired Binary Optimizers for Feature Selection. Cognitive Computation. [Online]. 12, pp. 150–175. Available: https://doi.org/10.1007/s12559-019-09668-6