1. Theoretical Motivation & Foundations

Pretraining scaling is encountering data bottlenecks. The new frontier of AI capabilities is test-time compute: spending additional FLOPs during generation to search, evaluate multiple trajectories, and verify steps. We analyze search strategies, Process Reward Models (PRMs), and self-correction verification loops.

2. Mathematical Formulations & Derivations

The governing analytical formulations and proof frameworks for this module:

Inference Scaling Formula: Compute_{infer} = N_{samples} × Depth_{search} × Verifier_{FLOPs} Process Reward Step Validation: R_{step} = P(step_i \text{ is correct} | x, step_{1..i-1}) Monte Carlo Tree Search Value Backprop: Q(s, a) = (1 / N(s, a)) ∑_{i=1}^{N(s, a)} r_i

3. From-Scratch Reference Implementation

Executable, production-tested reference code without magic libraries:

import heapq

class BeamSearchReasoner:
    def __init__(self, beam_width=5):
        self.beam_width = beam_width

    def search(self, initial_state, step_generator, verifier):
        beam = [(0.0, initial_state)]  # (negative_score, state)
        for _ in range(10):  # max reasoning depth
            candidates = []
            for score, state in beam:
                next_steps = step_generator(state)
                for step in next_steps:
                    step_score = verifier(state, step)
                    candidates.append((score + step_score, step))
            beam = heapq.nlargest(self.beam_width, candidates, key=lambda x: x[0])
        return beam[0][1]

4. Systems Complexity & Memory Footprint

Test-time compute transforms static LLMs into search engines over thought space. Latency increases from hundreds of milliseconds to tens of seconds in exchange for mathematical reliability.

5. Canonical Literature & Primary Research

Original research papers and foundational texts recommended for advanced study:

Yao, S., et al. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. NeurIPS.
Lightman, H., et al. (2023). Let's Verify Step by Step. OpenAI Research.