Framework for Organized Reproducible Generalized Experiments: A reproducible experimental framework for systematic prompt design and evaluation

Prompting has become the dominant mechanism for adapting large language models (LLMs) to downstream tasks, yet experimental practice around prompt experiment design remains ad-hoc and difficult to reproduce. Existing work has produced technique surveys, taxonomies, and domain-specific reporting guidelines, but none provides a prompt-experiment-specific specification framework that structures how experimental setups are declared, controlled, and varied. This paper proposes the Framework for Organized Reproducible Generalized Experiments (FORGE), a structured reporting and design scaffold for benchmark-style prompt studies, not a universal methodology for all LLM evaluation contexts. FORGE represents a prompt experiment as a tuple of seven explicitly declared components: model configuration (M), control condition (C), task specification (T), prompt architecture (A), prompting techniques under study (P), evaluation and analysis specification (E), and deployment and sampling configuration (D). Its contributions are: (1) shared terminology for prompt experiment components, (2) the (M, C, T, A, P, E, D) component model, (3) prescriptive design guidance for within-model and cross-model studies, and (4) a minimum-standard reporting checklist. FORGE is most applicable to quantitative, benchmark-style prompt comparison studies and is intended to complement, not replace, qualitative or human-computer interaction research methods. Expected advantages include improved cross-study comparability, support for meta-analytic synthesis, and more complete disclosure of experimental choices; these advantages are anticipated rather than empirically demonstrated, and formal validation of the framework remains a priority for future work.