Scaling Closed-Loop Feature Channel Configuration with LLMs
Scaling Closed-Loop Feature Channel Configuration with LLMs
利用大语言模型扩展闭环特征通道配置
Abstract: Promising initial results in closed-loop large-language-model-based channel-configuration search demonstrated that neural-network widths can be optimized directly through executable code generation and accuracy feedback. However, those results were obtained from a relatively sparse set of valid evaluations, leaving open whether the observed optimization behavior transfers to a denser sampling regime and whether additional architectural regularities emerge when more generated networks are evaluated.
摘要： 在基于大语言模型的闭环通道配置搜索中，初步研究结果显示，通过可执行代码生成和准确率反馈，可以直接优化神经网络的宽度。然而，这些结果源于相对稀疏的有效评估集，因此尚不清楚所观察到的优化行为是否适用于更密集的采样方案，以及在评估更多生成的网络时是否会出现额外的架构规律。
To test this, the same search setting is scaled to 250 candidate networks per fine-tuning cycle. The analysis covers 2000 generated candidates from 8 complete cycles, yielding 462 verified CIFAR-100 evaluations after task and metadata filtering. Per-cycle mean accuracy exhibits a positive linear trend with slope 9.87e-4 (p=0.043), while the high-performing frontier improves more strongly: the best observed accuracy increases from 0.3144 to 0.3676, and both the top-5 and top-10 cycle-level means exhibit positive trends.
为了验证这一点，研究将相同的搜索设置扩展到每个微调周期 250 个候选网络。分析涵盖了 8 个完整周期中生成的 2000 个候选网络，在经过任务和元数据过滤后，获得了 462 个经过验证的 CIFAR-100 评估结果。每个周期的平均准确率呈现出斜率为 9.87e-4 (p=0.043) 的正向线性趋势，而高性能前沿的提升更为显著：观测到的最佳准确率从 0.3144 提高到 0.3676，且周期级的前 5 名和前 10 名平均值均呈现出正向趋势。
The scaled run also reveals improved parameter efficiency. The best model reaches 0.3676 with 11.8M parameters, compared with an early high-performing model at 0.3144 with 166.5M parameters. Beyond accuracy, the larger sample exposes architectural regularities that were difficult to assess from sparse observations. Non-power-of-two channel widths occur in 41.8% of verified candidates, and the strongest models share structured channel-allocation patterns characterized by moderate early widths and expanded middle or later blocks. These findings indicate that the channel-search signal observed in the initial study transfers.
扩展后的运行还揭示了参数效率的提升。最佳模型在 11.8M 参数下达到了 0.3676 的准确率，而早期的高性能模型在 166.5M 参数下仅达到 0.3144。除了准确率之外，更大的样本量还揭示了在稀疏观测中难以评估的架构规律。非 2 的幂次方通道宽度出现在 41.8% 的验证候选网络中，且最强的模型共享结构化的通道分配模式，其特征是早期宽度适中，而中间或后期模块则有所扩展。这些发现表明，在初步研究中观察到的通道搜索信号具有可迁移性。