SubmittedACML 2026·Conference · double-blind
When Is an LLM Worth It for Hyperparameter Optimization? A Budget-Matched Study
Carson RodriguesiD (Celabe), Oysturn VasiD (University of Waterloo), Isaiah Abner DCosta (University of Queensland), Nithish Kumar Prabhakaran (University of Queensland)
A budget-matched, multi-seed study of whether an LLM advisor actually helps hyperparameter optimization on tabular data. The deflationary finding: the advisor's strong first guess is not an LLM output but a fixed default configuration evaluated before any model call. Once classical search (random, Optuna-TPE, Bayesian optimization, successive halving) is seeded with that same default, the apparent lead collapses within a handful of evaluations and the LLM adds no measurable generalization benefit.
LLM warm-start = a default config, not the modelAutoMLHyperparameter optimizationLLMs
Submitted to ACML 2026 on 26 June 2026 (Conference Track, double-blind; notification 15 September 2026). Preprint on arXiv (cs.LG · cs.AI); the harness and a script that reproduces every statistic are released.