Domain-Specialized Company Memory for Grounded Marketing Recommendations

Abstract

Can a marketing-specialized company memory improve next-action recommendations, or is strong generic context sufficient? We separate the policy value recoverable from retrieved evidence from the grounded feasible value actually returned by a fixed reader. A frozen benchmark contains 120 fictional company worlds, 360 multi-account cases, three compound stress levels, six executed context treatments and 4,320 actual GPT-6.1 Sol/Claude Opus 5.5 responses. On positive cases, general company context yields 47.9% of the full-information optimum for both readers; GTM context yields 88.4% for GPT and 88.2% for Opus. Paired gains are 40.6 percentage points (95% company-bootstrap interval 36.7–44.4) and 40.3 points (36.3–44.3). No company evidence yields zero grounded positive value, as expected under the evidence-required contract. The advantage is principally information selection: a generic query-word selector produces exactly the same GTM packets, and wider generic contexts close the retrieval gap. Full current context performs better, at greater API cost. On a separate one-company raw-document/wiki development set, the general wiki answers 22/22 questions correctly versus 21/22 for the marketing wiki. The results support task-aware evidence selection under a budget, not universal vertical-memory superiority, a proprietary architecture requirement or measured revenue improvement. Data, code, raw attempts, uncertainty analysis and publication figures are packaged for reproduction.

What the comparison shows

With the same reader, marketing-specialized context recovers more grounded decision value than the general company-context baseline. The paired gain is +40.6 percentage points for GPT-6.1 Sol (95% interval: 36.7–44.4) and +40.3 for Claude Opus 5.5 (36.3–44.3).

Percentage of each positive case’s full-information optimum, averaged equally by company. These are synthetic decision scores. The table shows selected comparisons; the paper reports all six executed treatments.

What was measured

The benchmark asks a fixed reader to select up to three marketing actions across several accounts under a shared resource limit. Recommendations must cite current evidence, satisfy prerequisites and follow the approved policy. Retrieval quality is scored separately from the decisions the reader returns.

• 120 fictional companies and 360 multi-account cases: 288 positive and 72 negative cases.
• Six executed context treatments and 4,320 actual reader responses across two models.
• Three compound stress levels, with 4/6/8 accounts and 16/12/8 KiB evidence allowances.
• Paired comparisons and company-clustered bootstrap uncertainty. All treatments abstain on all 72 negative cases.

Dataset card · Machine-readable results

Where the evidence stops

This is an AI-assisted industrial technical report using synthetic, schema-controlled histories. It has not been peer reviewed. It does not measure campaign outcomes, acquisition costs, revenue, human preference or the performance of a deployed Pancake recommender.

On a separate, one-company raw-document/wiki development set, the general wiki answers 22/22 questions correctly and the marketing wiki answers 21/22. That small development set is not an independent multi-company validation.

The supported conclusion is task-aware evidence selection under a budget. A proprietary architecture or universal advantage for vertical memory is not established.

Inspect and reproduce

The public repository contains the generator, frozen inputs, saved requests and responses, scoring code, uncertainty analysis, figures and LaTeX sources. Historical studies and failed attempts remain separate from the current experiment.

Saved-output reproduction makes no provider requests. It verifies the artifact manifest, rebuilds contexts, rescores responses and regenerates the report. Fresh model experiments require separate API access and incur costs; follow the release instructions for the appropriate path.

Cite this work

@techreport{marquis2026specializedmemory,
  author = {Marquis, Guillaume},
  title = {Domain-Specialized Company Memory for Grounded Marketing Recommendations},
  year = {2026},
  institution = {Pancake},
  note = {AI-assisted industrial technical report},
  url = {https://github.com/get-pancake/marketing-memory-benchmark}
}

Download BibTeX · CITATION.cff

Original code: MIT. Specified original data, outputs and documentation: CC BY 4.0. Third-party material retains its upstream license; see the repository’s attribution notes.