项目简介
MemoryRush 是一个围绕“阅读记忆”设计的本地 AI/ML 项目。它不是普通的文档聊天工具，也不是只生成摘要的工具。它更关注一件事：读完一篇文章之后，哪些内容真正值得被长期记住？
项目希望把长文本阅读材料转成一组可以复习、可以追溯、可以检查的记忆结构：
文章
-> 稳定原文段落
-> 核心观点
-> 记忆单元
-> 原文证据
-> 复习问题
-> 可评估的处理结果
这些记忆点不是脱离原文的灵感摘抄，而是要能回到文章中的具体段落。这样之后再回顾时，读者既能看到压缩后的观点，也能知道它来自哪里。
为什么做这个项目
普通阅读很容易留下一个问题：当时觉得有启发，但过一段时间后只剩下模糊印象。摘要可以帮助快速回顾，但摘要通常不区分哪些观点值得长期保存，也不一定保留清楚的证据来源。
MemoryRush 想探索的是另一种方式：让 AI 帮助读者从文章中提取少量高价值记忆点，并为每个记忆点绑定原文依据。它更像一个“阅读后的记忆整理器”，而不是一个临时问答系统。
这个方向也更适合学习型项目：输出不是一段看起来流畅的回答，而是一份可以被检查的结构化 artifact。每个核心观点、记忆单元和复习问题，都应该能被追问：它依据哪段原文？它是否重复？它是否真的适合之后回忆？
输入与输出
MemoryRush 当前面向的是 .txt、.md 和 .markdown 阅读材料。输入可以是一篇文章、课程笔记、论文摘录或其它长文本。系统会先把文档解析成稳定段落，再把段落交给后续生成与校验流程。
理想输出包含几类信息：
summary：文章的整体压缩说明。core_ideas：少量核心观点，每个观点带有重要性说明和证据引用。memory_units：适合长期记忆的复习单元，带类型、标签、置信度和证据段落。recall_questions：围绕记忆单元生成的问题，用于之后主动回忆。validation_report：检查输出是否引用了真实存在的段落，是否出现空内容、重复内容或错误索引。
这套输出设计的重点不是“内容多”，而是“结构清楚”。如果一个记忆点无法回到原文，或者复习问题找不到对应记忆单元，它就不应该被当成可靠结果。
Source-Grounded Memory Unit
MemoryRush 的核心概念是 source-grounded memory unit，也就是“有来源证据的记忆点”。它把普通摘要进一步拆成可复习、可验证的最小单元。
一个理想的记忆单元应该包含：
- 简洁的观点内容。
memory_type，例如概念、论点、机制或例子。- 支持它的
evidence_paragraph_ids。 - 用于组织的
tags。 - 表示生成可信度的
confidence。 - 可以绑定到它的 recall question。
这样设计之后，系统不是只说“这篇文章讲了阅读记忆”，而是生成一个更适合复习的记忆声明，并指出它对应文章里的哪几个段落。它把“读懂了什么”和“凭什么这样说”放在同一个对象里。
文档解析机制
项目先把阅读材料解析成 SourceDocument 和 SourceParagraph。每个段落都有稳定的 paragraph_id，格式类似 p_001、p_002。文档本身也有 document_id，由文件路径和文本内容生成短哈希。
这个步骤看起来朴素，但它决定了后面所有 evidence link 是否可靠。MemoryRush 会：
- 统一 Windows、Unix 和旧式换行。
- 从 Markdown 一级标题或文件名中提取标题。
- 删除 Markdown 开头的标题，避免标题被当成正文段落重复处理。
- 用空行切分段落，再把段落内部换行整理成连续文本。
- 按稳定顺序生成段落编号。
也就是说，算法不是先急着问模型“文章讲了什么”，而是先建立一套稳定的 source reference system。没有稳定段落 ID，后面的引用、评估和复习问题都会变得不可追踪。
Pipeline 机制
MemoryRush 的处理流程可以理解为一个小型 article-memory pipeline：
parse_document()
-> SourceDocument
-> build_prompt_context()
-> MemoryExtractionProvider.generate()
-> ArticleMemoryOutput
-> validate_article_memory_output()
-> ProcessingArtifact
build_prompt_context() 会把文档 ID、标题和每个段落的 paragraph_id: text 拼成模型上下文。这样模型在生成核心观点和记忆单元时，可以直接引用段落 ID，而不是依赖模糊的位置描述。
生成层通过 MemoryExtractionProvider 协议抽象出来。只要 provider 能接收 prompt context 并返回 ArticleMemoryOutput，pipeline 就不需要关心底层是确定性测试 provider，还是本地模型 provider。
最终产物是 ProcessingArtifact。它同时保存文档 ID、prompt 版本、模型名、结构化输出、原始输出以及校验报告。这个对象让一次处理不只是“生成了文本”，而是变成一份可以复现、可以检查、可以比较的实验记录。
算法与校验
MemoryRush 目前最关键的算法机制不是复杂模型结构，而是围绕 evidence 的结构约束和一致性检查。
在输出合约里，CoreIdea 包含 idea、why_it_matters、EvidenceSpan 和 salience_score。EvidenceSpan 必须包含 paragraph_id 和原文 quote。MemoryUnit 则必须包含内容、类型、证据段落列表、标签和置信度。
校验器会主动检查几类问题：
- 核心观点引用的段落 ID 是否真实存在。
- 核心观点里的 quote 是否确实出现在对应原文段落中。
- 记忆单元内容是否为空。
- 记忆单元是否与前面的内容重复。
- 记忆单元是否至少引用一个证据段落。
- 复习问题引用的
memory_unit_index 是否越界。
这些规则让项目的重点从“模型是否听起来聪明”转到“模型输出能不能被证据约束”。这也是 MemoryRush 和普通摘要工具最重要的区别之一。
用到的技术
项目整体保持轻量，主要使用 Python 标准库和少量清晰的工程约束，而不是先堆复杂框架。
- Python
dataclasses 定义核心领域对象和输出合约。 pathlib 处理跨平台文件路径。hashlib.sha1 生成稳定文档 ID。re 负责 Markdown 标题识别、段落切分和换行整理。json 负责 artifact 序列化和 provider 输出解析。urllib.request 接入本地 Ollama /api/generate。argparse 提供命令行处理入口。- Streamlit 用作本地交互界面的方向。
- Ollama /
qwen2.5:7b-instruct 可作为本地模型实验入口。
这种选择和项目定位一致：MemoryRush 是本地研究原型，不是 SaaS、浏览器插件或商业阅读平台。代码更重视可读性、可复现性和结构约束。
和 RAG 的区别
传统 RAG 更像是：
用户提问
-> 检索相关 chunk
-> 生成回答
MemoryRush 更像是：
用户读文章
-> 提取值得记住的观点
-> 绑定原文证据
-> 生成复习问题
-> 以后用于回忆和检索
它关注的不是“提问时临时找答案”，而是“阅读后留下什么记忆结构”。RAG 的核心交互通常发生在问题提出之后；MemoryRush 的核心交互发生在阅读完成之后。它先把文章变成可复习的知识痕迹，再让之后的回忆和检索有更稳定的基础。
Local First
MemoryRush 选择 local-first 的方向，因为阅读材料可能包含私人笔记、课程内容、论文摘录或未公开想法。让处理流程尽量留在本地，可以减少隐私风险，也方便之后做可复现实验。
本地模型可以通过 Ollama 接入。项目的目标不是追求最大模型，而是先让 pipeline 清楚、输出结构稳定、证据关系可检查。对这个项目来说，一个小而稳定的本地流程，往往比一个强但不可控的远程黑箱更适合早期实验。
我觉得它有意思的地方
MemoryRush 有意思的地方在于，它把“AI 帮我总结”改成了一个更具体的问题：AI 能不能帮我决定哪些东西值得进入记忆？
这个问题要求系统同时处理三层关系：
- 文章原文是什么。
- 值得记住的观点是什么。
- 这些观点如何在之后被重新唤起。
所以它不是单纯的 NLP demo，也不是传统意义上的知识库工具。它更像一个把阅读、证据、记忆和复习连起来的小型实验系统。
Project Overview
MemoryRush is a local AI/ML project built around reading memory. It is not a normal document chatbot, and it is not only a summarizer. The central question is: after reading an article, which ideas are actually worth remembering?
The project turns long-form reading material into a structure that is reviewable, source-grounded, and inspectable:
article
-> stable source paragraphs
-> core ideas
-> memory units
-> source evidence
-> recall questions
-> evaluable processing artifact
The memory units are not detached notes. Each one should point back to concrete source paragraphs, so later review keeps both the compressed idea and the evidence behind it.
Why This Project Exists
Reading often leaves behind a vague impression: something felt useful at the time, but the exact idea fades later. Summaries help with quick review, but they do not always identify what should become long-term memory, and they do not always preserve clear source evidence.
MemoryRush explores a different approach: using AI to extract a small set of high-value memory units from an article, while keeping each unit linked to the original text. It is closer to a post-reading memory organizer than a temporary question-answering tool.
That makes the project useful as a learning experiment as well. The output is not just a fluent paragraph. It is a structured artifact that can be questioned: which source paragraph supports this idea, is this unit duplicated, and can this recall question actually map back to a memory unit?
Input and Output
MemoryRush currently focuses on .txt, .md, and .markdown reading materials. The input can be an article, course note, paper excerpt, or another long-form text. The system first parses the document into stable paragraphs, then passes those paragraphs into the generation and validation flow.
The intended output contains several parts:
summary: a compact description of the whole article.core_ideas: a small set of key ideas, each with rationale and evidence.memory_units: reviewable units with type, tags, confidence, and evidence paragraphs.recall_questions: questions for later active recall.validation_report: checks for source references, empty content, duplicated units, and invalid indexes.
The goal is not to produce as much content as possible. The goal is to keep the structure clear. If a memory unit cannot point back to the source, or a recall question references no valid memory unit, it should not be treated as a reliable result.
Source-Grounded Memory Unit
The core concept is a source-grounded memory unit: a compact memory object that remains attached to evidence.
An ideal memory unit includes:
- concise content,
memory_type, such as concept, claim, mechanism, or example,- supporting
evidence_paragraph_ids, - organizing
tags, - a
confidence score, - recall questions that can refer back to it.
Instead of only saying "this article is about reading memory," the system should produce a reviewable memory statement and show which paragraphs support it. It keeps "what did I learn" and "why can I trust that extraction" inside the same object.
Document Parsing
The project first parses reading material into SourceDocument and SourceParagraph. Each paragraph has a stable paragraph_id, such as p_001 or p_002. The document itself receives a document_id generated from the file path and text content.
This step looks simple, but it is the foundation for reliable evidence links. MemoryRush:
- normalizes Windows, Unix, and older newline styles,
- extracts the title from a Markdown heading or file name,
- removes the leading Markdown title so it is not duplicated as a body paragraph,
- splits paragraphs on blank lines,
- folds internal line breaks into continuous paragraph text,
- assigns stable paragraph IDs in document order.
In other words, the system does not start by asking the model what the article says. It first builds a stable source-reference system. Without stable paragraph IDs, citation, evaluation, and recall-question mapping all become fragile.
Pipeline Mechanism
MemoryRush can be understood as a small article-memory pipeline:
parse_document()
-> SourceDocument
-> build_prompt_context()
-> MemoryExtractionProvider.generate()
-> ArticleMemoryOutput
-> validate_article_memory_output()
-> ProcessingArtifact
build_prompt_context() formats the document ID, title, and every paragraph_id: text pair into model context. That gives the model explicit IDs to cite instead of vague location descriptions.
Generation is abstracted through the MemoryExtractionProvider protocol. As long as a provider accepts prompt context and returns an ArticleMemoryOutput, the pipeline does not need to care whether the provider is deterministic for local checks or backed by a local model.
The final result is a ProcessingArtifact. It stores the document ID, prompt version, model name, structured output, raw output, and validation report. A run is therefore not just "some generated text"; it becomes an experiment record that can be reproduced, inspected, and compared.
Algorithms and Validation
The most important algorithmic part of MemoryRush is not a complex model architecture. It is the evidence-centered structure and the consistency checks around it.
In the output contract, CoreIdea contains idea, why_it_matters, EvidenceSpan, and salience_score. EvidenceSpan must include a paragraph_id and an exact source quote. MemoryUnit must include content, type, evidence paragraph IDs, tags, and confidence.
The validator checks several failure modes:
- whether a core idea references a paragraph ID that exists,
- whether the quoted evidence appears in the referenced paragraph,
- whether a memory unit is empty,
- whether a memory unit duplicates an earlier unit,
- whether every memory unit references at least one evidence paragraph,
- whether each recall question points to an existing memory unit index.
These checks shift the project from "does the model sound smart" to "can the model output be constrained by evidence." That is one of the most important differences between MemoryRush and a generic summarizer.
Technologies Used
The project intentionally stays lightweight. It relies on Python and a small set of clear engineering contracts instead of starting with a heavy framework.
- Python
dataclasses define the domain objects and structured output contracts. pathlib handles cross-platform paths.hashlib.sha1 creates stable document IDs.re handles Markdown title detection, paragraph splitting, and newline normalization.json serializes processing artifacts and parses provider output.urllib.request connects to the local Ollama /api/generate endpoint.argparse provides a command-line processing entry.- Streamlit is used for the local interactive-app direction.
- Ollama /
qwen2.5:7b-instruct can be used for local model experiments.
This stack matches the project identity: MemoryRush is a local research prototype, not a SaaS product, browser extension, or commercial reading app. The code favors readability, reproducibility, and source-grounded structure.
Difference From RAG
Traditional RAG is usually:
user question
-> retrieve related chunks
-> generate answer
MemoryRush is closer to:
user reads an article
-> extract memorable ideas
-> attach source evidence
-> generate recall questions
-> support later recall and retrieval
It is less about finding an answer at question time, and more about deciding what memory structure should remain after reading. RAG usually begins when a user asks a question; MemoryRush begins after a reading session and creates a stable memory surface for future recall.
Local First
MemoryRush is designed as a local-first project because reading material may include private notes, course content, paper excerpts, or unfinished ideas. Keeping the workflow local reduces privacy risk and makes research experiments easier to reproduce.
Local models can be connected through Ollama. The point is not to chase the largest model first, but to keep the pipeline clear, the output structured, and the evidence links inspectable. For this project, a smaller but controllable local workflow is often more useful than a stronger remote black box during early experimentation.
What Makes It Interesting
The interesting part of MemoryRush is that it turns "AI summarizes this for me" into a more precise question: can AI help decide what deserves to become memory?
That question forces the system to connect three layers:
- what the source article says,
- which ideas are worth keeping,
- how those ideas can be recalled later.
So it is not only an NLP demo and not quite a traditional knowledge-base tool. It is a small experimental system connecting reading, evidence, memory, and review.