1. Theoretical Motivation & Foundations

As the marginal returns of scaling pretraining compute and web-scraped token volume encounter physical and thermodynamic bounds, frontier AI research has unlocked a new scaling axis: test-time compute. By giving models the ability to deliberate, generate intermediate reasoning chains ('thinking tokens'), explore branching search trees, and self-correct prior errors, performance scales with inference computation instead of training parameter count. This module formalizes the mathematics of test-time scaling: comparing Outcome-supervised Reward Models (ORMs, which score only the final answer) against Process-supervised Reward Models (PRMs, which score every intermediate reasoning step); implementing Monte Carlo Tree Search (MCTS) and Best-of-N beam sampling over generation graphs; and analyzing the compute-accuracy trade-offs across OpenAI o1/o3-mini, DeepSeek-R1, and Claude 3.7 Sonnet.

2. Mathematical Formulations & Derivations

The governing analytical formulations and proof frameworks for this module:

Test-Time Scaling Law Formulation (Accuracy A as function of inference FLOPs C_test): A(C_test) = A_base + β · log(C_test / C_0)^α Process Reward Model (PRM) Path Evaluation (Step t ∈ 1..T): Score(Path) = ∏_{t=1}^T P(Step_t is Correct | Context, Step_{1..t-1}) Upper Confidence Bound for Trees (UCT) Selection: UCT(v) = Q(v) + c_puct · P(v) · \sqrt{N(Parent)} / (1 + N(v))

3. From-Scratch Reference Implementation

Executable, production-tested reference code without magic libraries:

# Process Reward Model Step-by-Step Verifier Simulation
class StepVerifier:
    def __init__(self):
        # Mock PRM token-level step confidence scores
        self.step_scores = [0.98, 0.94, 0.42, 0.96]  # Step 3 introduces an error!
        
    def verify_chain(self, steps: list) -> tuple:
        cumulative_confidence = 1.0
        for i, (step, conf) in enumerate(zip(steps, self.step_scores)):
            cumulative_confidence *= conf
            if conf < 0.70:
                return False, i + 1, cumulative_confidence
        return True, len(steps), cumulative_confidence

steps = [
    'Let x be the speed of the train in km/h.',
    'The distance equation is D = x * t.',
    'Assume time t is 2 hours (incorrectly misread from 3 hours).',
    'Calculate final speed.'
]

verifier = StepVerifier()
valid, err_step, conf = verifier.verify_chain(steps)
print(f'Chain Valid: {valid} | Error detected at step: {err_step} | Path Confidence: {conf:.4f}')

4. Systems Complexity & Memory Footprint

Test-time compute transforms models from intuitive immediate answer engines into deliberate systems of reasoning. By pruning incorrect reasoning steps before they pollute the context window, PRM-guided search unlocks dramatic accuracy gains on competitive mathematics and software synthesis.

5. Canonical Literature & Primary Research

Original research papers and foundational texts recommended for advanced study:

Lightman, H., et al. (2023). Let's Verify Step by Step. arXiv:2305.20050.
Brown, N., et al. (2024). Large Language Monkeys: Scaling Inference Compute with Verifiers. arXiv:2407.21787.
DeepSeek-AI. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.