[2505.11601] Continuous Optimization for Feature Selection with Permutation-Invariant Embedding and Policy-Guided Search

Computer Science > Machine Learning arXiv:2505.11601 (cs) [Submitted on 16 May 2025 (v1), last revised 27 Feb 2026 (this version, v3)] Title:Continuous Optimization for Feature Selection with Permutation-Invariant Embedding and Policy-Guided Search Authors:Rui Liu, Rui Xie, Zijun Yao, Yanjie Fu, Dongjie Wang View a PDF of the paper titled Continuous Optimization for Feature Selection with Permutation-Invariant Embedding and Policy-Guided Search, by Rui Liu and 4 other authors View PDF HTML (experimental) Abstract:Feature selection removes redundant features to enhanc performance and computational efficiency in downstream tasks. Existing works often struggle to capture complex feature interactions and adapt to diverse scenarios. Recent advances in this domain have incorporated generative intelligence to address these drawbacks by uncovering intricate relationships between features. However, two key limitations remain: 1) embedding feature subsets in a continuous space is challenging due to permutation sensitivity, as changes in feature order can introduce biases and weaken the embedding learning process; 2) gradient-based search in the embedding space assumes convexity, which is rarely guaranteed, leading to reduced search effectiveness and suboptimal subsets. To address these limitations, we propose a new framework that can: 1) preserve feature subset knowledge in a continuous embedding space while ensuring permutation invariance; 2) effectively explore the embedding space...