A recent paper by Google DeepMind, Google Cloud AI, and University of California researchers proposes a new method for using LLMs that improves accuracy. It uses a two-part system that involves a smaller model drafting responses and a larger model that checks and selects the best one.

How Speculative RAG Works

• Specialist Drafter: Quickly creates multiple drafts using retrieved documents.
• Generalist Verifier: Validates drafts and picks the most accurate one, speeding up the process.

Result: Speculative RAG outperforms existing systems on standard benchmarks.

Let's compare this to LegalDuel

• Specialist Drafter: LegalDuel uses a variety of models and third-parties who generate responses.
• Generalist Verifier: We use our Arbiter agents to pick the best results.

Result: We might be onto something at LegalDuel. And, by the way, LegalDuel was launched in April 2023 before the paper was published.

The paper is available at: https://arxiv.org/pdf/2407.08223