Back to Report
APPENDIX D

Secondary Research Synthesis

Palace Research Report · HCDE Capstone · Winter 2026

Research Context

This secondary research builds on the Winter 2026 Palace research findings and conference tutorial notes on Agentic Information Retrieval (CHIIR 2026, Dammu & Roosta). The goal is to understand current developments in AI-enhanced information retrieval that may inform the Palace intervention design.

1. The Evolution from Traditional IR to Agentic IR

Traditional IR Paradigm

The conference tutorial outlined the classical IR flow: - Query understanding → Document ranking → Relevance estimation - User inspects results → Clicks documents → Reformulates query until satisfied

Agentic IR Paradigm Shift

Current research shows a fundamental shift from reactive/passive systems to goal-seeking, dynamic systems that: - Break down objectives and execute actions across multiple steps - Employ multi-step reasoning and task decomposition - Manage context, memory, and tool orchestration autonomously

Key insight: Agentic RAG transcends traditional limitations by embedding autonomous AI agents into retrieval pipelines. These agents leverage reflection, planning, tool use, and multiagent collaboration to dynamically manage retrieval strategies.

Relevance to Palace

The Palace problem of “retrieval breakdown under vague recall” aligns with the agentic IR critique of traditional systems: users don’t formulate queries in fleshed-out outcomes. The conference notes reference Belkin’s ASK (Anomalous State of Knowledge)—the request is an “incomplete, distorted expression of the underlying need.” This validates Palace’s focus on recognition-based retrieval over query-driven search.

Agentic vs Traditional RAG
Fig. 1 — Traditional RAG vs. agentic RAG with iterative planning and reflection

2. Memory Systems for AI Agents: A Core Primitive

The Memory Problem

The survey “Memory in the Age of AI Agents” (2025) establishes that memory has emerged as a core capability of foundation model-based agents. Key insight: “Long-lived, safe, and useful agents require a principled memory substrate supporting: - Single-shot learning of instances - Context-aware retrieval - Consolidation into generalizable knowledge”

Memory Taxonomies

Modern frameworks analyze memory through three dimensions: 1. Forms: Token-level, parametric, and latent memory 2. Functions: Factual, experiential, and working memory 3. Dynamics: How memory is formed, evolves, and retrieved

A-MEM: Zettelkasten-Inspired Memory

The A-MEM paper (NeurIPS 2025) proposes a Zettelkasten-inspired approach to agent memory: - Creates interconnected knowledge networks through dynamic indexing and linking - Generates comprehensive notes with contextual descriptions, keywords, and tags - Analyzes historical memories to identify relevant connections - New memories trigger updates to existing representations

Relevance to Palace

Palace’s design principle of “idea-centric with source traceability” parallels the A-MEM approach. Both treat knowledge units (ideas/notes) as the primary object and emphasize: - Interconnections between knowledge units - Dynamic linking rather than static hierarchies - Contextual metadata that evolves over time

The memory frameworks suggest Palace could benefit from: - Intelligent decay and consolidation (scoring memories based on relevance/usage) - Memory lifecycle management (avoiding memory inflation) - Hybrid episodic + semantic memory (from cognitive science)

Memory Taxonomy
Fig. 2 — Agent memory taxonomy: forms, functions, and dynamics
Zettelkasten-Inspired Memory
Fig. 3 — Zettelkasten-inspired agentic memory system (A-MEM)

3. Recognition vs. Recall: The Semantic Search Advantage

The Vocabulary Gap Problem

Research confirms that traditional keyword search creates significant vocabulary gaps between how users remember information and how it was originally stored. As noted in Azure AI Search documentation: “Keyword search prioritizes matching specific, important words that might be diluted in an embedding” but fails when users cannot recall exact terminology.

Semantic Search Capabilities

AI semantic search understands: - Query meaning using semantic embeddings and NLP - User intent, not just keywords - Contextual relationships between concepts

Key finding: “Semantic search delivers relevant answers to even vague or unconventional queries.”

Hybrid Search: The Pragmatic Solution

Current best practice combines both approaches: - Vector retrieval: Semantically matches queries to content with similar meanings (handles synonyms, misspellings, phrasing differences) - Keyword retrieval: Prioritizes matching specific important words (handles “out of domain” data like product codes, proper nouns, technical terms)

As one source noted: “We are not moving from keywords to AI, but from keywords plus AI.”

Hybrid Search Architecture
Fig. 4 — Hybrid search architecture combining semantic and keyword retrieval

Relevance to Palace

This directly validates Palace’s design principle of “support recognition over recall.” The research suggests: - Semantic search handles the vague recall problem - Visual and relational cues complement semantic matching - Hybrid approaches preserve precision for technical/exact retrieval needs

4. Personal Knowledge Management AI Tools

Emerging Solutions

Several tools now address the vague recall problem:

Recall AI - Builds personal knowledge graphs from consumed content - Uses semantic search to find information even with vague queries - Automatically creates connections between information - “Safely forget everything and trust that Recall will resurface it when needed”

Constella App - “Smart Retrieval” uses AI to recall relevant notes as you type - Automatic organization and connection of notes

Heptabase / Scrintal - Focus on idea connectivity and visual thinking surfaces

Knowledge Graph Approaches

“Knowledge graphs map relationships between concepts, people, projects, and ideas across your repository, enabling multi-hop reasoning that mirrors human thought processes.”

Relevance to Palace

The market validates Palace’s problem space—multiple startups now address retrieval breakdown. Palace’s differentiation may lie in: - Project scoping (bounded context vs. global knowledge base) - Source traceability (connecting synthesis back to originals) - In-flow retrieval (ambient presence alongside existing tools)

5. Project-Scoped AI Assistants

Enterprise Adoption Patterns

OpenAI reports Custom GPTs and Projects usage increased 19x year-to-date, with 20% of Enterprise messages processed via a Custom GPT or Project. This suggests strong demand for bounded, context-aware AI assistants.

The “Frontier Firm” Model

Microsoft’s 2025 Work Trend Index predicts: - Traditional org charts replaced by “Work Charts”—dynamic, outcome-driven models - Teams form around goals, not functions, powered by agents - This mirrors movie production: tailored teams assemble for a project and disband

Context Understanding as Key Differentiator

When evaluating AI tools, a critical factor is: “How effectively each tool maintains understanding of project structure, coding standards, and ongoing work across multiple sessions.”

Relevance to Palace

The project-scoped approach aligns with enterprise trends. Key implications: - Project as the primary container is validated by market behavior - Context persistence across sessions is a recognized need - Bounded context may prevent the “new silo” problem identified in competitive analysis

6. Critical Thinking and AI Assistance

Microsoft Research Findings (CHI 2025)

The “Tools for Thought” research reveals a critical tension: - Confidence paradox: “Higher confidence in AI was associated with less critical thinking” - AI shifts cognitive work toward verification, integration, and task oversight - Workers expend more effort on high-stakes tasks, less on routine work

Implications for Design

Microsoft advocates positioning AI as a “thought partner” and “provocateur” rather than an answer-delivery system.

Relevance to Palace

Palace should consider: - Supporting human judgment rather than replacing it - Making sources and reasoning visible (not black-box retrieval) (Palace design implication, not a direct Microsoft claim) - Preserving agency in the retrieval and synthesis process - Design for verification, not just convenience

7. Evaluation Challenges for Agentic Systems

New Metrics Required

The conference tutorial and subsequent research highlight that traditional IR metrics are insufficient. New dimensions include:

Agentic Metrics - Initiative index - Delegation score - Task success rate - Multi-step process efficiency

Faithfulness Metrics - Evidence support - Source authority score - Source freshness - Viewpoint diversity

Temporal Metrics - Staleness error rate - Temporal consistency

Key Insight

Evaluation must shift: - From model weights → entire decision process - From predictions → actions with consequences - From static evaluation → dynamic trajectories

Relevance to Palace

Palace’s proposed measures (time-to-retrieve, tool switches, failed attempts, abandonment rate) align with agentic evaluation thinking. Additional measures to consider: - Source traceability preservation - Recognition accuracy (did user find what they vaguely remembered?) - Context maintenance (how much flow disruption occurred?)

8. Open Research Questions

From the Literature

  1. Memory design for long-horizon IR agents: How to balance explicit vs. compressed (embedding-based) memory?
  2. Governance of agentic systems: Who is responsible when agents make retrieval decisions?
  3. Temporal drift: How to handle stored memory becoming stale?
  4. Information contamination: How to prevent pretraining/retrieval overlap problems?

For Palace Specifically

  1. Can a project-scoped surface avoid becoming “another dumping ground” while enabling effortless capture?
  2. How to balance semantic (recognition-based) retrieval with precision (keyword) search for technical terms?
  3. How to measure retrieval improvement without longitudinal access?
  4. How to maintain source traceability when synthesis artifacts evolve?

9. Synthesis: Implications for Palace Design

Validated Design Principles

The secondary research validates all four Palace design principles:

Principle Supporting Evidence
Support Recognition Over Recall Semantic search handles vague queries; vocabulary gap in keyword search
Preserve Persistent Project Context Enterprise 19x growth in project-scoped tools; memory systems research on context persistence
Maintain Visible Source Traceability A-MEM Zettelkasten approach; faithfulness metrics in agentic evaluation
Reduce Context Switching “Tools for Thought” research on cognitive load; in-flow retrieval demand

Design Opportunities from Research

  1. Adopt hybrid search architecture
  2. Semantic embeddings for vague recall
  3. Keyword matching for technical terms, proper nouns

  4. Let the system decide which to weight based on query characteristics

  5. Implement intelligent memory management

  6. Relevance scoring based on recency, usage, project salience
  7. Dynamic linking between related materials (Zettelkasten-inspired)

  8. Consolidation mechanisms to prevent memory inflation

  9. Support multi-hop reasoning

  10. Knowledge graphs that connect concepts across sources

  11. Enable “find the paper that connects to the note that relates to this slide”

  12. Design for verification, not just retrieval

  13. Make reasoning visible (why was this surfaced?)

  14. Preserve provenance chains from raw material → idea → synthesis

  15. Consider temporal awareness

  16. Track when materials were added, last accessed, last relevant
  17. Surface staleness warnings for time-sensitive information

Potential Technical Approaches

Based on the research, Palace implementation might explore: - Vector databases (Pinecone, Weaviate) for semantic search - Knowledge graph layers (Cognee, Neo4j) for relationship mapping - LLM orchestration (LangChain, LlamaIndex) for agentic retrieval - Memory frameworks (Letta, LangMem) for context persistence

10. References

Academic Papers

Conference Materials

Industry Reports & Analysis

Technical Documentation