feat: initial release — AI document intelligence for Paperless-ngx
PaperCortex adds semantic search, auto-classification, receipt extraction, bank statement matching, and DATEV export to Paperless-ngx — powered entirely by local AI through Ollama. Exposes everything as an MCP Server for Claude Code and AI agent integration. - MCP Server with 5 tools (search, classify, receipt, query, export) - Local Ollama embeddings for semantic document search - Receipt data extraction (vendor, amount, date, tax, line items) - DATEV Buchungsstapel CSV export for German accounting - Bank CSV transaction matching - Paperless-ngx REST API client - Docker deployment - Zero cloud dependencies — 100% self-hosted
This commit is contained in:
Rene Fichtmueller
commit
2052d87ba1
20
.env.example
Normal file
20
.env.example
Normal file
@ -0,0 +1,20 @@
|
||||
# PaperCortex Configuration
|
||||
# Copy this file to .env and fill in your values
|
||||
|
||||
# Paperless-ngx connection
|
||||
PAPERLESS_URL=http://localhost:8000
|
||||
PAPERLESS_TOKEN=your-paperless-api-token-here
|
||||
|
||||
# Ollama connection
|
||||
OLLAMA_URL=http://localhost:11434
|
||||
OLLAMA_MODEL=qwen2.5:14b
|
||||
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
|
||||
|
||||
# Vector store
|
||||
VECTOR_DB_PATH=./data/vectors.db
|
||||
|
||||
# MCP Server
|
||||
MCP_SERVER_PORT=3100
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=info
|
||||
35
.gitignore
vendored
Normal file
35
.gitignore
vendored
Normal file
@ -0,0 +1,35 @@
|
||||
# Dependencies
|
||||
node_modules/
|
||||
|
||||
# Build output
|
||||
dist/
|
||||
|
||||
# Environment files
|
||||
.env
|
||||
.env.local
|
||||
.env.*.local
|
||||
|
||||
# Data directory (vectors, cache)
|
||||
data/
|
||||
|
||||
# OS files
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# Logs
|
||||
logs/
|
||||
*.log
|
||||
npm-debug.log*
|
||||
|
||||
# Test coverage
|
||||
coverage/
|
||||
|
||||
# Temporary files
|
||||
tmp/
|
||||
temp/
|
||||
34
Dockerfile
Normal file
34
Dockerfile
Normal file
@ -0,0 +1,34 @@
|
||||
FROM node:22-alpine AS builder
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY package.json package-lock.json* ./
|
||||
RUN npm ci
|
||||
|
||||
COPY tsconfig.json ./
|
||||
COPY src/ ./src/
|
||||
RUN npm run build
|
||||
|
||||
# --- Production image ---
|
||||
FROM node:22-alpine
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN addgroup -g 1001 -S papercortex && \
|
||||
adduser -S papercortex -u 1001
|
||||
|
||||
COPY package.json package-lock.json* ./
|
||||
RUN npm ci --omit=dev && npm cache clean --force
|
||||
|
||||
COPY --from=builder /app/dist ./dist
|
||||
|
||||
RUN mkdir -p /app/data && chown papercortex:papercortex /app/data
|
||||
|
||||
USER papercortex
|
||||
|
||||
ENV NODE_ENV=production
|
||||
ENV VECTOR_DB_PATH=/app/data/vectors.db
|
||||
|
||||
EXPOSE 3100
|
||||
|
||||
CMD ["node", "dist/mcp-server/index.js"]
|
||||
21
LICENSE
Normal file
21
LICENSE
Normal file
@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 PaperCortex Contributors
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
737
README.md
Normal file
737
README.md
Normal file
@ -0,0 +1,737 @@
|
||||
<p align="center">
|
||||
<img src="docs/assets/papercortex-logo.svg" alt="PaperCortex Logo" width="120" />
|
||||
<h1 align="center">PaperCortex</h1>
|
||||
<p align="center">
|
||||
<strong>AI-Powered Document Intelligence for Paperless-ngx</strong><br/>
|
||||
<em>Semantic search, auto-classification, receipt extraction, and accounting export — 100% local, 100% private.</em>
|
||||
</p>
|
||||
<p align="center">
|
||||
<a href="#-quick-start"><img src="https://img.shields.io/badge/Docker-one--command-2496ED?logo=docker&logoColor=white" alt="Docker"></a>
|
||||
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-22c55e.svg" alt="MIT License"></a>
|
||||
<img src="https://img.shields.io/badge/TypeScript-5.x-3178C6?logo=typescript&logoColor=white" alt="TypeScript">
|
||||
<img src="https://img.shields.io/badge/Ollama-Local_AI-7C3AED?logo=ollama&logoColor=white" alt="Ollama">
|
||||
<img src="https://img.shields.io/badge/MCP-Server-F97316" alt="MCP Server">
|
||||
<img src="https://img.shields.io/badge/Paperless--ngx-Compatible-EF4444?logo=data:image/svg+xml;base64,..." alt="Paperless-ngx">
|
||||
<img src="https://img.shields.io/badge/DATEV-Export-EAB308" alt="DATEV Export">
|
||||
<img src="https://img.shields.io/badge/Privacy-First-10B981" alt="Privacy First">
|
||||
</p>
|
||||
<p align="center">
|
||||
<a href="#-quick-start">Quick Start</a> · <a href="#-features">Features</a> · <a href="#-mcp-server-tools">MCP Tools</a> · <a href="#-receipt-intelligence">Receipts</a> · <a href="#-documentation">Docs</a>
|
||||
</p>
|
||||
</p>
|
||||
|
||||
---
|
||||
|
||||
## What is PaperCortex?
|
||||
|
||||
**PaperCortex** turns your [Paperless-ngx](https://github.com/paperless-ngx/paperless-ngx) document archive into an intelligent, queryable knowledge base — powered entirely by local AI running on your own hardware.
|
||||
|
||||
If you use Paperless-ngx to store invoices, receipts, contracts, tax documents, letters, or any other scanned paperwork, PaperCortex adds the intelligence layer that Paperless-ngx is missing:
|
||||
|
||||
- **Ask questions in plain English** — "Show me all invoices from Amazon over 100 EUR in 2025"
|
||||
- **Find documents by meaning**, not just keywords — searching for "office rent" finds "Bueromiete" and "monthly lease payment"
|
||||
- **Auto-tag and classify** every new document the moment it arrives
|
||||
- **Extract structured data from receipts** — vendor, date, amount, tax rate, line items
|
||||
- **Match receipts to bank transactions** automatically
|
||||
- **Export to DATEV** for your German tax advisor — or plain CSV for any accounting software
|
||||
|
||||
Everything runs locally through [Ollama](https://ollama.com). No document content ever leaves your network. No cloud APIs. No subscriptions. No data harvesting.
|
||||
|
||||
PaperCortex exposes all capabilities as an **[MCP (Model Context Protocol)](https://modelcontextprotocol.io) Server**, making it a first-class tool for [Claude Code](https://docs.anthropic.com/en/docs/claude-code), AI coding agents, and automated workflows.
|
||||
|
||||
---
|
||||
|
||||
## The Problem
|
||||
|
||||
Paperless-ngx is an outstanding document management system with 37,000+ GitHub stars. It handles scanning, OCR, storage, and basic tagging beautifully. But once your documents are in Paperless-ngx, finding and working with them has real limitations:
|
||||
|
||||
| What you want to do | Paperless-ngx alone | With PaperCortex |
|
||||
|---|---|---|
|
||||
| Find a document by what it's about | Keyword search only — misses synonyms, translations, related concepts | **Semantic search** understands meaning across languages |
|
||||
| Classify incoming documents | Manual rules or basic auto-matching | **LLM-powered classification** understands document content |
|
||||
| Extract data from a receipt | Read it yourself and type it in | **Automatic extraction** of vendor, amount, date, tax, line items |
|
||||
| Answer "How much did I spend on X?" | Export everything, open spreadsheet, filter manually | **Natural language query** returns the answer instantly |
|
||||
| Send receipt data to accounting | Manual data entry or copy-paste | **One-click DATEV/CSV export** ready for your tax advisor |
|
||||
| Use documents in AI workflows | No API integration for AI agents | **Full MCP Server** for Claude Code and any MCP-compatible agent |
|
||||
| Keep data private | Self-hosted (good!) | Self-hosted AI too — **zero cloud dependency** |
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
### Semantic Document Search
|
||||
|
||||
Traditional keyword search fails when you don't remember the exact words. PaperCortex generates vector embeddings for every document using local Ollama models and stores them in a lightweight SQLite vector database.
|
||||
|
||||
**Search by meaning, not by memory:**
|
||||
- Search for `"electricity bill"` → finds documents containing "Stromrechnung", "utility payment", "power invoice"
|
||||
- Search for `"office supplies"` → finds "Bueroausstattung", "paper and toner", "desk accessories order"
|
||||
- Search for `"tax deductible travel"` → finds flight bookings, hotel receipts, train tickets, taxi invoices
|
||||
|
||||
**Supported embedding models:**
|
||||
- `nomic-embed-text` (recommended — fast, accurate, 768 dimensions)
|
||||
- `mxbai-embed-large` (higher accuracy, slower)
|
||||
- Any Ollama-compatible embedding model
|
||||
|
||||
### Automatic Document Classification
|
||||
|
||||
Every new document arriving in Paperless-ngx gets analyzed by a local LLM that reads the OCR content and assigns:
|
||||
|
||||
- **Document type** — Invoice, Receipt, Contract, Letter, Statement, Tax Document, Certificate
|
||||
- **Tags** — Contextual tags based on content (e.g., "office", "travel", "insurance", "subscription")
|
||||
- **Correspondent** — Identifies the sender/vendor from document content
|
||||
- **Date extraction** — Finds the document date (not just the scan date)
|
||||
- **Language detection** — Identifies the document language
|
||||
|
||||
Classification runs asynchronously in the background. New documents are processed within minutes of arriving in Paperless-ngx.
|
||||
|
||||
### Receipt Intelligence
|
||||
|
||||
PaperCortex includes a dedicated receipt processing pipeline optimized for expense management:
|
||||
|
||||
**Data extraction from receipts and invoices:**
|
||||
- Vendor / merchant name and address
|
||||
- Date of purchase
|
||||
- Total amount (gross and net)
|
||||
- Tax rate and tax amount (supports multiple VAT rates)
|
||||
- Currency
|
||||
- Individual line items with quantities and prices
|
||||
- Payment method
|
||||
- Invoice/receipt number
|
||||
|
||||
**Works with:**
|
||||
- Scanned paper receipts (via Paperless-ngx OCR)
|
||||
- Digital PDF invoices
|
||||
- Photographed receipts (mobile upload to Paperless-ngx)
|
||||
- Multi-page invoices
|
||||
- Receipts in German, English, French, Spanish, and other languages
|
||||
|
||||
### Bank Statement Matching
|
||||
|
||||
Import your bank statement as CSV and let PaperCortex automatically match transactions to receipts:
|
||||
|
||||
- **Fuzzy matching** on amount, date, and vendor name
|
||||
- **Confidence scoring** — high/medium/low match indicators
|
||||
- **Unmatched detection** — highlights receipts without matching transactions and vice versa
|
||||
- **Multi-currency support** — handles EUR, USD, GBP, CHF, and 20+ currencies
|
||||
|
||||
### DATEV Export
|
||||
|
||||
For German businesses and freelancers, PaperCortex generates DATEV-compatible export files that your Steuerberater can import directly:
|
||||
|
||||
- **DATEV CSV format** (Buchungsstapel) — the standard German accounting import format
|
||||
- **SKR03 / SKR04** account mapping
|
||||
- **Automatic account assignment** based on document classification
|
||||
- **Beleglink** — links each DATEV entry back to the original document in Paperless-ngx
|
||||
- **Period exports** — monthly, quarterly, or annual
|
||||
|
||||
Also supports plain CSV export for use with any accounting software worldwide.
|
||||
|
||||
### Natural Language Queries
|
||||
|
||||
Ask questions about your document archive in plain language:
|
||||
|
||||
```
|
||||
"How much did I spend on hotels in Q1 2025?"
|
||||
"Show me all contracts expiring this year"
|
||||
"What was my highest single expense last month?"
|
||||
"Find all invoices from Deutsche Telekom"
|
||||
"Which receipts don't have a matching bank transaction?"
|
||||
"Summarize my office supply spending trend over the last 12 months"
|
||||
```
|
||||
|
||||
PaperCortex translates natural language into document queries, retrieves relevant documents via semantic search, and uses the local LLM to synthesize answers with source references.
|
||||
|
||||
### MCP Server Integration
|
||||
|
||||
PaperCortex implements the [Model Context Protocol (MCP)](https://modelcontextprotocol.io) — the open standard for connecting AI agents to external tools. This means any MCP-compatible AI agent can use your document archive as a knowledge source.
|
||||
|
||||
**Compatible with:**
|
||||
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code) (Anthropic)
|
||||
- [Claude Desktop](https://claude.ai)
|
||||
- Any MCP-compatible AI agent or IDE plugin
|
||||
- Custom AI workflows via the MCP SDK
|
||||
|
||||
---
|
||||
|
||||
## Feature Comparison
|
||||
|
||||
| Feature | PaperCortex | paperless-ai | Veryfi | Taggun | Rossum |
|
||||
|---|:---:|:---:|:---:|:---:|:---:|
|
||||
| Fully self-hosted | :white_check_mark: | :white_check_mark: | :x: | :x: | :x: |
|
||||
| Local AI (no cloud API) | :white_check_mark: | :x: OpenAI | :x: | :x: | :x: |
|
||||
| Semantic search | :white_check_mark: | :x: | :x: | :x: | :x: |
|
||||
| Auto-classification | :white_check_mark: | :white_check_mark: | :x: | :x: | :white_check_mark: |
|
||||
| Receipt data extraction | :white_check_mark: | :x: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| Bank statement matching | :white_check_mark: | :x: | :x: | :x: | :x: |
|
||||
| DATEV export | :white_check_mark: | :x: | :x: | :x: | :x: |
|
||||
| CSV accounting export | :white_check_mark: | :x: | :white_check_mark: | :x: | :white_check_mark: |
|
||||
| MCP Server | :white_check_mark: | :x: | :x: | :x: | :x: |
|
||||
| Natural language queries | :white_check_mark: | :x: | :x: | :x: | :x: |
|
||||
| Multi-language documents | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| Free and open source | :white_check_mark: | :white_check_mark: | :x: $$$ | :x: $$$ | :x: $$$$ |
|
||||
| Privacy — data stays local | :white_check_mark: | :warning: API calls | :x: | :x: | :x: |
|
||||
| Works with Paperless-ngx | :white_check_mark: | :white_check_mark: | :x: | :x: | :x: |
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────┐ ┌──────────────────────────┐ ┌────────────────────┐
|
||||
│ │ │ │ │ │
|
||||
│ Claude Code / │ MCP │ PaperCortex │ REST │ Paperless-ngx │
|
||||
│ AI Agents / ├────────►│ ├────────►│ │
|
||||
│ Automation │ │ ┌──────────────────┐ │ API │ OCR + Storage + │
|
||||
│ │ │ │ MCP Server │ │ │ Tagging │
|
||||
└─────────────────────┘ │ │ (stdio / HTTP) │ │ │ │
|
||||
│ └──────────────────┘ │ └────────────────────┘
|
||||
│ │
|
||||
│ ┌──────────────────┐ │ ┌────────────────────┐
|
||||
│ │ Intelligence │ │ │ │
|
||||
│ │ Layer │ │ LLM │ Ollama │
|
||||
│ │ ├────────────►│ │
|
||||
│ │ - Classifier │ │ API │ qwen2.5 / llama3 │
|
||||
│ │ - Extractor │ │ │ nomic-embed-text │
|
||||
│ │ - Query Engine │ │ │ │
|
||||
│ └──────────────────┘ │ └────────────────────┘
|
||||
│ │
|
||||
│ ┌──────────────────┐ │
|
||||
│ │ Vector Store │ │
|
||||
│ │ (SQLite + HNSW) │ │
|
||||
│ └──────────────────┘ │
|
||||
│ │
|
||||
└──────────────────────────┘
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Documents arrive** in Paperless-ngx through scanning, email, or manual upload
|
||||
2. **PaperCortex polls** the Paperless-ngx API for new and updated documents
|
||||
3. **Embedding generation** — Ollama creates vector embeddings from OCR text
|
||||
4. **Classification** — the local LLM analyzes content and assigns types, tags, and metadata
|
||||
5. **Storage** — embeddings and extracted data are stored in a local SQLite vector database
|
||||
6. **Query interface** — the MCP Server exposes search, classify, extract, query, and export tools
|
||||
7. **AI agents connect** via MCP and interact with your documents using natural language
|
||||
|
||||
All processing happens on your hardware. The only network traffic is between PaperCortex and your local Paperless-ngx and Ollama instances.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- **[Docker](https://docs.docker.com/get-docker/)** and Docker Compose
|
||||
- **[Paperless-ngx](https://github.com/paperless-ngx/paperless-ngx)** — running instance with API access
|
||||
- **[Ollama](https://ollama.com)** — running locally or on your network
|
||||
|
||||
**Pull the required Ollama models:**
|
||||
|
||||
```bash
|
||||
ollama pull qwen2.5:14b # LLM for classification, extraction, queries
|
||||
ollama pull nomic-embed-text # Embedding model for semantic search
|
||||
```
|
||||
|
||||
### Option 1: Docker Compose (Recommended)
|
||||
|
||||
```bash
|
||||
git clone https://github.com/renefichtmueller/PaperCortex.git
|
||||
cd PaperCortex
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Edit `.env` with your configuration:
|
||||
|
||||
```env
|
||||
PAPERLESS_URL=http://your-paperless-instance:8000
|
||||
PAPERLESS_TOKEN=your-paperless-api-token
|
||||
OLLAMA_URL=http://your-ollama-host:11434
|
||||
OLLAMA_MODEL=qwen2.5:14b
|
||||
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
|
||||
```
|
||||
|
||||
Start PaperCortex:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
PaperCortex will begin indexing your existing documents automatically.
|
||||
|
||||
### Option 2: Manual Installation
|
||||
|
||||
```bash
|
||||
git clone https://github.com/renefichtmueller/PaperCortex.git
|
||||
cd PaperCortex
|
||||
npm install
|
||||
cp .env.example .env
|
||||
# Edit .env with your settings
|
||||
npm run build
|
||||
npm start
|
||||
```
|
||||
|
||||
### Option 3: npx (MCP Server only)
|
||||
|
||||
```bash
|
||||
npx papercortex --paperless-url http://localhost:8000 --paperless-token YOUR_TOKEN
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## MCP Server Tools
|
||||
|
||||
PaperCortex exposes five MCP tools that AI agents can call:
|
||||
|
||||
### `papercortex_search` — Semantic Document Search
|
||||
|
||||
Find documents by meaning, not just keywords.
|
||||
|
||||
```json
|
||||
{
|
||||
"tool": "papercortex_search",
|
||||
"arguments": {
|
||||
"query": "electricity bills from last winter",
|
||||
"limit": 10,
|
||||
"date_from": "2024-12-01",
|
||||
"date_to": "2025-02-28"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Returns:** Ranked list of documents with relevance scores, titles, dates, and Paperless-ngx document IDs.
|
||||
|
||||
### `papercortex_classify` — Auto-Classification
|
||||
|
||||
Analyze a document and assign type, tags, and metadata.
|
||||
|
||||
```json
|
||||
{
|
||||
"tool": "papercortex_classify",
|
||||
"arguments": {
|
||||
"document_id": 1234,
|
||||
"apply": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Returns:** Suggested document type, tags, correspondent, and confidence scores. Set `apply: true` to write classifications back to Paperless-ngx.
|
||||
|
||||
### `papercortex_receipt` — Receipt Data Extraction
|
||||
|
||||
Extract structured financial data from receipts and invoices.
|
||||
|
||||
```json
|
||||
{
|
||||
"tool": "papercortex_receipt",
|
||||
"arguments": {
|
||||
"document_id": 5678
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Returns:**
|
||||
```json
|
||||
{
|
||||
"vendor": "Amazon EU S.a.r.l.",
|
||||
"date": "2025-03-15",
|
||||
"total_gross": 119.99,
|
||||
"total_net": 100.83,
|
||||
"tax_rate": 19,
|
||||
"tax_amount": 19.16,
|
||||
"currency": "EUR",
|
||||
"items": [
|
||||
{ "description": "USB-C Hub", "quantity": 1, "price": 49.99 },
|
||||
{ "description": "Monitor Arm", "quantity": 1, "price": 70.00 }
|
||||
],
|
||||
"invoice_number": "INV-DE-2025-1234567"
|
||||
}
|
||||
```
|
||||
|
||||
### `papercortex_query` — Natural Language Questions
|
||||
|
||||
Ask questions about your entire document archive.
|
||||
|
||||
```json
|
||||
{
|
||||
"tool": "papercortex_query",
|
||||
"arguments": {
|
||||
"question": "How much did I spend on business travel in Q1 2025?"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Returns:** A natural language answer with source document references and a breakdown of the calculation.
|
||||
|
||||
### `papercortex_export` — Accounting Export
|
||||
|
||||
Export extracted receipt data in accounting-ready formats.
|
||||
|
||||
```json
|
||||
{
|
||||
"tool": "papercortex_export",
|
||||
"arguments": {
|
||||
"format": "datev",
|
||||
"date_from": "2025-01-01",
|
||||
"date_to": "2025-03-31",
|
||||
"account_plan": "SKR03"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Supported formats:** `datev` (German standard), `csv` (universal), `json` (programmatic).
|
||||
|
||||
---
|
||||
|
||||
## Claude Code Integration
|
||||
|
||||
### Register as MCP Server
|
||||
|
||||
Add to your `~/.claude.json` or project `.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"papercortex": {
|
||||
"command": "node",
|
||||
"args": ["./dist/mcp-server/index.js"],
|
||||
"cwd": "/path/to/PaperCortex",
|
||||
"env": {
|
||||
"PAPERLESS_URL": "http://localhost:8000",
|
||||
"PAPERLESS_TOKEN": "your-token",
|
||||
"OLLAMA_URL": "http://localhost:11434"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Example Conversations
|
||||
|
||||
Once connected, you can ask Claude Code about your documents naturally:
|
||||
|
||||
```
|
||||
You: Search my documents for anything related to the office lease renewal
|
||||
|
||||
Claude: I found 4 relevant documents:
|
||||
1. "Mietvertrag Verlängerung 2025" (Score: 0.94) — Document #1234
|
||||
2. "Office Lease Agreement Amendment" (Score: 0.91) — Document #1235
|
||||
3. "Nebenkostenabrechnung 2024" (Score: 0.78) — Document #1240
|
||||
4. "Facilities Management Invoice" (Score: 0.72) — Document #1251
|
||||
```
|
||||
|
||||
```
|
||||
You: Extract the receipt data from document #5678 and export it for DATEV
|
||||
|
||||
Claude: Extracted receipt data:
|
||||
Vendor: Deutsche Bahn AG
|
||||
Date: 2025-03-20
|
||||
Amount: 89.90 EUR (net: 75.55 EUR, 19% VAT: 14.35 EUR)
|
||||
Description: ICE Frankfurt-Berlin, 1st class
|
||||
|
||||
DATEV export saved to: exports/datev_2025_03.csv
|
||||
```
|
||||
|
||||
```
|
||||
You: How much did I spend on cloud services this year?
|
||||
|
||||
Claude: Based on 23 matching documents, your cloud service spending in 2025:
|
||||
- AWS: 2,340.00 EUR (12 invoices)
|
||||
- Hetzner: 456.00 EUR (3 invoices)
|
||||
- Cloudflare: 240.00 EUR (3 invoices)
|
||||
- Vercel: 180.00 EUR (3 invoices)
|
||||
- GitHub: 132.00 EUR (2 invoices)
|
||||
Total: 3,348.00 EUR
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Receipt Workflow
|
||||
|
||||
### End-to-End Receipt Processing
|
||||
|
||||
```
|
||||
┌──────────┐ ┌─────────────┐ ┌──────────────┐ ┌──────────┐ ┌──────────┐
|
||||
│ Scan / │ │ Paperless- │ │ PaperCortex │ │ Match │ │ Export │
|
||||
│ Photo / ├───►│ ngx ├───►│ Receipt ├───►│ Bank ├───►│ DATEV / │
|
||||
│ Email │ │ OCR+Store │ │ Extraction │ │ CSV │ │ CSV │
|
||||
└──────────┘ └─────────────┘ └──────────────┘ └──────────┘ └──────────┘
|
||||
```
|
||||
|
||||
### CLI Commands
|
||||
|
||||
```bash
|
||||
# Process all unprocessed receipts
|
||||
npm run receipt:process
|
||||
|
||||
# Extract data from a specific document
|
||||
npm run receipt:extract -- --document-id 1234
|
||||
|
||||
# Import bank statement and match transactions
|
||||
npm run receipt:match -- --bank-csv ./bank_export_2025_q1.csv
|
||||
|
||||
# Export matched data as DATEV
|
||||
npm run receipt:export -- --format datev --period 2025-Q1
|
||||
|
||||
# Export as plain CSV
|
||||
npm run receipt:export -- --format csv --period 2025-03
|
||||
```
|
||||
|
||||
### DATEV Integration Details
|
||||
|
||||
The DATEV export generates a `Buchungsstapel` CSV file following the official DATEV format specification:
|
||||
|
||||
- **Header row** with advisor number, client number, fiscal year start, and export period
|
||||
- **Transaction rows** with amount, debit/credit account, tax code, date, and booking text
|
||||
- **Beleglink** — each row includes a reference to the source document in Paperless-ngx
|
||||
- **Account mapping** — automatic assignment based on vendor and document type (configurable)
|
||||
- **SKR03 and SKR04** chart of accounts supported
|
||||
|
||||
---
|
||||
|
||||
## Privacy and Security
|
||||
|
||||
### Why Local AI Matters
|
||||
|
||||
Your documents contain some of the most sensitive data in your life:
|
||||
|
||||
- **Tax returns** with income, deductions, and financial details
|
||||
- **Contracts** with confidential terms and personal information
|
||||
- **Medical bills** with health information
|
||||
- **Bank statements** with account numbers and transaction history
|
||||
- **Personal correspondence** with private content
|
||||
|
||||
Cloud-based document AI services require uploading this data to external servers for processing. Even with encryption and privacy policies, you are trusting a third party with your most sensitive information.
|
||||
|
||||
**PaperCortex takes a fundamentally different approach:**
|
||||
|
||||
- All AI processing runs on **your hardware** via Ollama
|
||||
- Document content is sent only to **your local Ollama instance**
|
||||
- Embeddings and extracted data are stored in a **local SQLite database**
|
||||
- The only network traffic is between PaperCortex, your Paperless-ngx instance, and your Ollama server
|
||||
- **No telemetry, no analytics, no external API calls**
|
||||
|
||||
**Your documents stay in your network. Period.**
|
||||
|
||||
### Security Best Practices
|
||||
|
||||
- Store the Paperless-ngx API token in environment variables, never in source code
|
||||
- Run PaperCortex on the same network as Paperless-ngx and Ollama
|
||||
- Use Docker networks to isolate services
|
||||
- Regularly update Ollama and PaperCortex for security patches
|
||||
|
||||
---
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
All configuration is done through environment variables. See `.env.example` for a complete template.
|
||||
|
||||
### Core Settings
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `PAPERLESS_URL` | `http://localhost:8000` | Paperless-ngx instance URL |
|
||||
| `PAPERLESS_TOKEN` | *(required)* | Paperless-ngx API authentication token |
|
||||
| `OLLAMA_URL` | `http://localhost:11434` | Ollama API endpoint |
|
||||
| `OLLAMA_MODEL` | `qwen2.5:14b` | LLM model for classification and extraction |
|
||||
| `OLLAMA_EMBEDDING_MODEL` | `nomic-embed-text` | Embedding model for semantic search |
|
||||
| `VECTOR_DB_PATH` | `./data/vectors.db` | Path to the SQLite vector database |
|
||||
|
||||
### Processing Settings
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `POLL_INTERVAL` | `300` | Seconds between polling Paperless-ngx for new documents |
|
||||
| `BATCH_SIZE` | `10` | Number of documents to process per batch |
|
||||
| `EMBEDDING_DIMENSIONS` | `768` | Vector dimensions (must match embedding model) |
|
||||
| `CLASSIFICATION_CONFIDENCE` | `0.7` | Minimum confidence to auto-apply classifications |
|
||||
|
||||
### Export Settings
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `DATEV_ADVISOR_NUMBER` | *(optional)* | Steuerberater number for DATEV export header |
|
||||
| `DATEV_CLIENT_NUMBER` | *(optional)* | Mandantennummer for DATEV export header |
|
||||
| `DATEV_FISCAL_YEAR_START` | `01-01` | Fiscal year start (MM-DD) |
|
||||
| `DEFAULT_ACCOUNT_PLAN` | `SKR03` | Default chart of accounts (`SKR03` or `SKR04`) |
|
||||
| `EXPORT_DIR` | `./exports` | Directory for generated export files |
|
||||
|
||||
### MCP Server Settings
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `MCP_TRANSPORT` | `stdio` | MCP transport mode (`stdio` or `http`) |
|
||||
| `MCP_PORT` | `3100` | Port for HTTP transport mode |
|
||||
| `MCP_AUTH_TOKEN` | *(optional)* | Bearer token for HTTP transport authentication |
|
||||
|
||||
---
|
||||
|
||||
## Supported Models
|
||||
|
||||
PaperCortex works with any Ollama-compatible model. Recommended configurations:
|
||||
|
||||
### For Classification and Extraction
|
||||
|
||||
| Model | VRAM | Speed | Quality | Recommended For |
|
||||
|---|---|---|---|---|
|
||||
| `qwen2.5:7b` | 5 GB | Fast | Good | Raspberry Pi, low-end servers |
|
||||
| `qwen2.5:14b` | 10 GB | Medium | Very Good | Most homelab setups |
|
||||
| `qwen2.5:32b` | 20 GB | Slow | Excellent | High-accuracy requirements |
|
||||
| `llama3.1:8b` | 5 GB | Fast | Good | Alternative to Qwen |
|
||||
| `mistral:7b` | 5 GB | Fast | Good | European language focus |
|
||||
|
||||
### For Embeddings
|
||||
|
||||
| Model | Dimensions | Speed | Quality |
|
||||
|---|---|---|---|
|
||||
| `nomic-embed-text` | 768 | Very Fast | Very Good |
|
||||
| `mxbai-embed-large` | 1024 | Fast | Excellent |
|
||||
| `all-minilm` | 384 | Fastest | Good |
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
PaperCortex/
|
||||
├── src/
|
||||
│ ├── mcp-server/ # MCP Server for AI agent integration
|
||||
│ │ ├── index.ts # Server entry point and tool registration
|
||||
│ │ └── tools/
|
||||
│ │ ├── search.ts # Semantic document search tool
|
||||
│ │ ├── classify.ts # Auto-classification tool
|
||||
│ │ ├── receipt.ts # Receipt data extraction tool
|
||||
│ │ ├── query.ts # Natural language query tool
|
||||
│ │ └── export.ts # DATEV/CSV export tool
|
||||
│ ├── embeddings/
|
||||
│ │ ├── ollama.ts # Ollama embedding API client
|
||||
│ │ └── store.ts # SQLite vector store with HNSW index
|
||||
│ ├── paperless/
|
||||
│ │ ├── client.ts # Paperless-ngx REST API client
|
||||
│ │ └── types.ts # TypeScript type definitions
|
||||
│ └── receipt/
|
||||
│ ├── extractor.ts # Receipt OCR content parsing and extraction
|
||||
│ ├── matcher.ts # Bank CSV transaction matching engine
|
||||
│ └── datev.ts # DATEV Buchungsstapel CSV formatter
|
||||
├── docs/
|
||||
│ ├── architecture.md # Detailed architecture documentation
|
||||
│ ├── setup.md # Step-by-step installation guide
|
||||
│ └── receipts.md # Receipt workflow documentation
|
||||
├── docker-compose.yml # Production deployment
|
||||
├── Dockerfile # Container build
|
||||
├── .env.example # Configuration template (no secrets!)
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
└── LICENSE # MIT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Roadmap
|
||||
|
||||
- [x] Core MCP Server with 5 tools
|
||||
- [x] Paperless-ngx API client
|
||||
- [x] Ollama embedding generation
|
||||
- [x] SQLite vector store
|
||||
- [x] Receipt data extraction
|
||||
- [x] DATEV export
|
||||
- [x] Docker deployment
|
||||
- [ ] Bank CSV matching engine
|
||||
- [ ] Web dashboard UI
|
||||
- [ ] Webhook support (instant processing on document arrival)
|
||||
- [ ] Multi-user support with separate vector stores
|
||||
- [ ] Additional export formats (SKR04 mapping, FiBu, CSV+)
|
||||
- [ ] Ollama vision model support for direct image analysis
|
||||
- [ ] Automated document workflow triggers
|
||||
- [ ] Plugin system for custom extractors
|
||||
- [ ] Prometheus metrics endpoint
|
||||
|
||||
---
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome! PaperCortex is early-stage and there are many ways to help:
|
||||
|
||||
### Getting Started
|
||||
|
||||
```bash
|
||||
git clone https://github.com/renefichtmueller/PaperCortex.git
|
||||
cd PaperCortex
|
||||
npm install
|
||||
cp .env.example .env
|
||||
# Edit .env with your local Paperless-ngx and Ollama settings
|
||||
npm run dev
|
||||
```
|
||||
|
||||
### How to Contribute
|
||||
|
||||
1. **Fork** the repository
|
||||
2. **Create** a feature branch (`git checkout -b feat/amazing-feature`)
|
||||
3. **Write tests** for your changes
|
||||
4. **Commit** using conventional commits (`feat:`, `fix:`, `docs:`, `refactor:`)
|
||||
5. **Push** and open a Pull Request
|
||||
|
||||
### Areas Where Help is Needed
|
||||
|
||||
| Area | Description | Difficulty |
|
||||
|---|---|---|
|
||||
| **Bank CSV Parsers** | Add parsers for different bank export formats (Sparkasse, ING, N26, Revolut, etc.) | Easy |
|
||||
| **Export Formats** | Additional accounting export formats beyond DATEV | Medium |
|
||||
| **Web Dashboard** | Build a simple web UI for browsing indexed documents and extracted data | Medium |
|
||||
| **Multi-language** | Improve extraction accuracy for non-English/German receipts | Medium |
|
||||
| **Vision Models** | Use Ollama vision models to extract data directly from receipt images | Hard |
|
||||
| **Webhooks** | React to Paperless-ngx document events in real-time | Medium |
|
||||
|
||||
---
|
||||
|
||||
## Frequently Asked Questions
|
||||
|
||||
**Q: Does PaperCortex modify my documents in Paperless-ngx?**
|
||||
A: By default, PaperCortex only reads documents. When you use the `classify` tool with `apply: true`, it can write tags, document types, and correspondents back to Paperless-ngx. Extraction results and embeddings are stored in PaperCortex's own database.
|
||||
|
||||
**Q: How much disk space does the vector database need?**
|
||||
A: Roughly 1-2 KB per document for embeddings. A collection of 10,000 documents needs about 10-20 MB of vector storage.
|
||||
|
||||
**Q: Can I use OpenAI instead of Ollama?**
|
||||
A: PaperCortex is designed for local-first operation with Ollama. Support for OpenAI-compatible APIs (including local alternatives like LM Studio, vLLM, or LocalAI) is on the roadmap.
|
||||
|
||||
**Q: What Paperless-ngx version is required?**
|
||||
A: PaperCortex works with Paperless-ngx 2.0 and later (REST API v3+).
|
||||
|
||||
**Q: Can I run PaperCortex on a Raspberry Pi?**
|
||||
A: PaperCortex itself is lightweight. The bottleneck is Ollama — you'll need a model that fits in your available RAM. `qwen2.5:7b` works on 8GB devices.
|
||||
|
||||
**Q: Is DATEV export only for Germany?**
|
||||
A: The DATEV format is the German standard, but PaperCortex also exports plain CSV that works with any accounting software worldwide.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT License — see [LICENSE](LICENSE) for details.
|
||||
|
||||
Free to use, modify, and distribute. Commercial use welcome.
|
||||
|
||||
---
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
Built on the shoulders of giants:
|
||||
|
||||
- **[Paperless-ngx](https://github.com/paperless-ngx/paperless-ngx)** — The incredible open-source document management system (37k+ stars)
|
||||
- **[Ollama](https://ollama.com)** — Making local AI accessible to everyone
|
||||
- **[Model Context Protocol](https://modelcontextprotocol.io)** — The open standard for AI tool integration by Anthropic
|
||||
- **[better-sqlite3](https://github.com/WiseLibs/better-sqlite3)** — Fast, reliable SQLite bindings for Node.js
|
||||
|
||||
---
|
||||
|
||||
## Star History
|
||||
|
||||
If PaperCortex is useful to you, please consider giving it a star — it helps others discover the project!
|
||||
|
||||
---
|
||||
|
||||
<p align="center">
|
||||
<strong>Your documents. Your AI. Your hardware.</strong><br/>
|
||||
<em>No cloud required.</em>
|
||||
</p>
|
||||
36
docker-compose.yml
Normal file
36
docker-compose.yml
Normal file
@ -0,0 +1,36 @@
|
||||
services:
|
||||
papercortex:
|
||||
build: .
|
||||
container_name: papercortex
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "3100:3100"
|
||||
volumes:
|
||||
- papercortex-data:/app/data
|
||||
env_file:
|
||||
- .env
|
||||
environment:
|
||||
- NODE_ENV=production
|
||||
depends_on:
|
||||
- ollama
|
||||
|
||||
ollama:
|
||||
image: ollama/ollama:latest
|
||||
container_name: papercortex-ollama
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "11434:11434"
|
||||
volumes:
|
||||
- ollama-models:/root/.ollama
|
||||
# Uncomment for NVIDIA GPU support:
|
||||
# deploy:
|
||||
# resources:
|
||||
# reservations:
|
||||
# devices:
|
||||
# - driver: nvidia
|
||||
# count: all
|
||||
# capabilities: [gpu]
|
||||
|
||||
volumes:
|
||||
papercortex-data:
|
||||
ollama-models:
|
||||
64
docs/architecture.md
Normal file
64
docs/architecture.md
Normal file
@ -0,0 +1,64 @@
|
||||
# Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
PaperCortex is structured as three layers:
|
||||
|
||||
1. **MCP Server Layer** -- Exposes tools via the Model Context Protocol for AI agent integration.
|
||||
2. **Intelligence Layer** -- Embedding generation, classification, receipt extraction, and query answering.
|
||||
3. **Data Layer** -- Paperless-ngx API client and local SQLite vector store.
|
||||
|
||||
## Components
|
||||
|
||||
### MCP Server (`src/mcp-server/`)
|
||||
|
||||
The entry point for all AI agent interactions. Implements the MCP standard using `@modelcontextprotocol/sdk` and communicates via stdio transport.
|
||||
|
||||
Each tool is implemented as a separate handler module under `src/mcp-server/tools/`.
|
||||
|
||||
### Embeddings (`src/embeddings/`)
|
||||
|
||||
- **ollama.ts** -- Client for the Ollama API. Handles embedding generation and LLM completions.
|
||||
- **store.ts** -- SQLite-backed vector store using `better-sqlite3`. Stores document embeddings and supports cosine similarity search.
|
||||
|
||||
Current implementation uses brute-force search, which is performant up to ~100k documents. For larger archives, consider migrating to `sqlite-vss` or a dedicated vector database.
|
||||
|
||||
### Paperless Integration (`src/paperless/`)
|
||||
|
||||
- **client.ts** -- REST API client for Paperless-ngx. Supports document CRUD, search, tags, correspondents, and document types.
|
||||
- **types.ts** -- TypeScript type definitions matching the Paperless-ngx API v3+ schema.
|
||||
|
||||
### Receipt Processing (`src/receipt/`)
|
||||
|
||||
- **extractor.ts** -- Uses LLM to extract structured data from receipt OCR text.
|
||||
- **matcher.ts** -- Matches extracted receipts against bank CSV transaction exports.
|
||||
- **datev.ts** -- Generates DATEV Buchungsstapel format CSV for German accounting software.
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
Paperless-ngx --(REST API)--> PaperCortex --(Ollama API)--> Ollama
|
||||
|
|
||||
v
|
||||
SQLite Vector DB
|
||||
|
|
||||
v
|
||||
MCP Server (stdio)
|
||||
|
|
||||
v
|
||||
Claude Code / AI Agents
|
||||
```
|
||||
|
||||
## Security Model
|
||||
|
||||
- All data stays local -- no external API calls except to Paperless-ngx and Ollama (both self-hosted).
|
||||
- API tokens are read from environment variables, never hardcoded.
|
||||
- The SQLite database is stored on the local filesystem with configurable path.
|
||||
- MCP Server communicates via stdio (no network port required for MCP).
|
||||
|
||||
## Future Considerations
|
||||
|
||||
- **Webhook support** -- Listen for Paperless-ngx webhooks to auto-process new documents.
|
||||
- **Plugin system** -- Allow custom extractors and exporters.
|
||||
- **Web dashboard** -- Optional UI for monitoring and manual review.
|
||||
- **Multi-user** -- Support multiple Paperless-ngx instances and user isolation.
|
||||
101
docs/receipts.md
Normal file
101
docs/receipts.md
Normal file
@ -0,0 +1,101 @@
|
||||
# Receipt Workflow
|
||||
|
||||
## Overview
|
||||
|
||||
PaperCortex provides a complete receipt-to-accounting pipeline:
|
||||
|
||||
1. **Scan** -- Upload receipts to Paperless-ngx (scan, email, photo)
|
||||
2. **Extract** -- AI extracts structured data (vendor, date, amounts, line items)
|
||||
3. **Match** -- Reconcile against bank CSV exports
|
||||
4. **Export** -- Generate DATEV-compatible CSV for accounting software
|
||||
|
||||
## Receipt Extraction
|
||||
|
||||
### Via MCP Server (Claude Code)
|
||||
|
||||
```
|
||||
Extract receipt data from document #1234
|
||||
```
|
||||
|
||||
### Via CLI
|
||||
|
||||
```bash
|
||||
npm run receipt:extract -- --document-id 1234
|
||||
```
|
||||
|
||||
### Extracted Fields
|
||||
|
||||
| Field | Description | Example |
|
||||
|---|---|---|
|
||||
| vendor | Company name | "IKEA Deutschland GmbH" |
|
||||
| vendorAddress | Full address | "Am Wanderweg 1, 65719 Hofheim" |
|
||||
| vendorTaxId | Tax ID / VAT number | "DE 129 341 800" |
|
||||
| date | Receipt date | "2024-03-15" |
|
||||
| currency | ISO 4217 code | "EUR" |
|
||||
| subtotal | Before tax | 84.03 |
|
||||
| taxRate | Tax percentage | 19 |
|
||||
| taxAmount | Tax amount | 15.97 |
|
||||
| totalAmount | Total with tax | 100.00 |
|
||||
| paymentMethod | How it was paid | "card" |
|
||||
| lineItems | Individual items | Array of items |
|
||||
| category | Expense category | "office_supplies" |
|
||||
|
||||
## Bank Statement Matching
|
||||
|
||||
Match receipts against bank CSV exports to verify which receipts correspond to which bank transactions.
|
||||
|
||||
### Supported Bank Formats
|
||||
|
||||
- Sparkasse (semicolon-separated, German format)
|
||||
- ING (semicolon-separated)
|
||||
- DKB (semicolon-separated)
|
||||
- Volksbank (semicolon-separated)
|
||||
- Generic CSV
|
||||
|
||||
### Matching Algorithm
|
||||
|
||||
1. **Amount match** -- Exact or close amount (within 1.00 tolerance)
|
||||
2. **Date proximity** -- Same day, within 3 days, or within 7 days
|
||||
3. **Vendor name** -- Partial match in transaction description
|
||||
|
||||
Results include a confidence score (0.0 - 1.0) and match reasons.
|
||||
|
||||
## DATEV Export
|
||||
|
||||
### Format
|
||||
|
||||
PaperCortex generates DATEV Buchungsstapel (posting batch) format CSV, compatible with:
|
||||
|
||||
- DATEV Unternehmen Online
|
||||
- lexoffice
|
||||
- sevDesk
|
||||
- FastBill
|
||||
- Any DATEV-import-capable software
|
||||
|
||||
### Account Mapping (SKR03)
|
||||
|
||||
| Category | Account | Description |
|
||||
|---|---|---|
|
||||
| office_supplies | 4930 | Buerokosten |
|
||||
| travel | 4660 | Reisekosten |
|
||||
| food | 4650 | Bewirtungskosten |
|
||||
| telephone | 4920 | Telefon |
|
||||
| postage | 4910 | Porto |
|
||||
| rent | 4210 | Miete |
|
||||
| advertising | 4600 | Werbekosten |
|
||||
| software | 4964 | Software |
|
||||
| consulting | 4950 | Rechts- und Beratungskosten |
|
||||
| default | 4900 | Sonstige Aufwendungen |
|
||||
|
||||
### Export via CLI
|
||||
|
||||
```bash
|
||||
# Export all receipts from March 2024 as DATEV CSV
|
||||
npm run receipt:export -- --format datev --year 2024 --month 03
|
||||
```
|
||||
|
||||
### Export via MCP Server
|
||||
|
||||
```
|
||||
Export documents #100, #101, #102 as DATEV CSV
|
||||
```
|
||||
107
docs/setup.md
Normal file
107
docs/setup.md
Normal file
@ -0,0 +1,107 @@
|
||||
# Setup Guide
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Node.js** 20+ (or Docker)
|
||||
- **Paperless-ngx** instance with API access
|
||||
- **Ollama** with required models
|
||||
|
||||
## Step 1: Install Ollama Models
|
||||
|
||||
```bash
|
||||
# Required: LLM for classification and extraction
|
||||
ollama pull qwen2.5:14b
|
||||
|
||||
# Required: Embedding model for semantic search
|
||||
ollama pull nomic-embed-text
|
||||
```
|
||||
|
||||
Verify Ollama is running:
|
||||
```bash
|
||||
curl http://localhost:11434/api/tags
|
||||
```
|
||||
|
||||
## Step 2: Get Paperless-ngx API Token
|
||||
|
||||
1. Open your Paperless-ngx web UI
|
||||
2. Go to Settings > API
|
||||
3. Generate a new API token
|
||||
4. Copy the token for the next step
|
||||
|
||||
## Step 3: Configure PaperCortex
|
||||
|
||||
```bash
|
||||
git clone https://github.com/YOUR_USERNAME/PaperCortex.git
|
||||
cd PaperCortex
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Edit `.env` with your values:
|
||||
```env
|
||||
PAPERLESS_URL=http://localhost:8000
|
||||
PAPERLESS_TOKEN=<your-api-token>
|
||||
OLLAMA_URL=http://localhost:11434
|
||||
```
|
||||
|
||||
## Step 4: Run
|
||||
|
||||
### Option A: Docker (Recommended)
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### Option B: Manual
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run build
|
||||
npm start
|
||||
```
|
||||
|
||||
### Option C: Development
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
## Step 5: Register MCP Server
|
||||
|
||||
Add to your Claude Code configuration (`~/.claude.json`):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"papercortex": {
|
||||
"command": "node",
|
||||
"args": ["/absolute/path/to/PaperCortex/dist/mcp-server/index.js"],
|
||||
"env": {
|
||||
"PAPERLESS_URL": "http://localhost:8000",
|
||||
"PAPERLESS_TOKEN": "your-token",
|
||||
"OLLAMA_URL": "http://localhost:11434"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Step 6: Populate Vector Store
|
||||
|
||||
On first run, you need to embed your existing documents. This will be automated in a future release. For now, the vector store is populated as documents are queried or classified.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Connection refused" to Paperless-ngx
|
||||
- Verify the URL in `.env` is reachable
|
||||
- Check that the API token is valid
|
||||
- Ensure Paperless-ngx is running
|
||||
|
||||
### "Connection refused" to Ollama
|
||||
- Run `ollama serve` if not already running
|
||||
- Check the port (default: 11434)
|
||||
- Verify models are pulled: `ollama list`
|
||||
|
||||
### Slow first query
|
||||
- The first embedding generation may take longer as Ollama loads the model into memory
|
||||
- Subsequent queries will be faster once the model is loaded
|
||||
57
package.json
Normal file
57
package.json
Normal file
@ -0,0 +1,57 @@
|
||||
{
|
||||
"name": "papercortex",
|
||||
"version": "0.1.0",
|
||||
"description": "Self-hosted AI intelligence layer for Paperless-ngx with semantic search, receipt extraction, and MCP Server integration",
|
||||
"main": "dist/mcp-server/index.js",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"start": "node dist/mcp-server/index.js",
|
||||
"dev": "tsx watch src/mcp-server/index.ts",
|
||||
"lint": "eslint src/",
|
||||
"test": "vitest",
|
||||
"test:coverage": "vitest --coverage",
|
||||
"receipt:extract": "tsx src/receipt/extractor.ts",
|
||||
"receipt:match": "tsx src/receipt/matcher.ts",
|
||||
"receipt:export": "tsx src/receipt/datev.ts"
|
||||
},
|
||||
"keywords": [
|
||||
"paperless-ngx",
|
||||
"ollama",
|
||||
"mcp",
|
||||
"mcp-server",
|
||||
"semantic-search",
|
||||
"document-ai",
|
||||
"receipt-extraction",
|
||||
"datev",
|
||||
"self-hosted",
|
||||
"local-ai",
|
||||
"embeddings",
|
||||
"vector-search"
|
||||
],
|
||||
"author": "",
|
||||
"license": "MIT",
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": ""
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
},
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "^1.12.0",
|
||||
"better-sqlite3": "^11.8.0",
|
||||
"csv-parse": "^5.6.0",
|
||||
"csv-stringify": "^6.5.0",
|
||||
"dotenv": "^16.4.0",
|
||||
"zod": "^3.24.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/better-sqlite3": "^7.6.12",
|
||||
"@types/node": "^22.10.0",
|
||||
"eslint": "^9.17.0",
|
||||
"tsx": "^4.19.0",
|
||||
"typescript": "^5.7.0",
|
||||
"vitest": "^3.0.0"
|
||||
}
|
||||
}
|
||||
148
src/embeddings/ollama.ts
Normal file
148
src/embeddings/ollama.ts
Normal file
@ -0,0 +1,148 @@
|
||||
/**
|
||||
* Ollama embedding and LLM integration.
|
||||
*
|
||||
* Generates vector embeddings and LLM completions using a local Ollama instance.
|
||||
* All functions are pure and return new objects -- no mutation.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const ollama = createOllamaClient({ baseUrl: "http://localhost:11434" });
|
||||
* const embedding = await ollama.embed("Office rent invoice March 2024");
|
||||
* const answer = await ollama.complete("Classify this document: ...");
|
||||
* ```
|
||||
*/
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface OllamaConfig {
|
||||
readonly baseUrl: string;
|
||||
readonly model: string;
|
||||
readonly embeddingModel: string;
|
||||
readonly timeout?: number;
|
||||
}
|
||||
|
||||
export interface EmbeddingResult {
|
||||
readonly vector: readonly number[];
|
||||
readonly model: string;
|
||||
readonly dimensions: number;
|
||||
}
|
||||
|
||||
export interface CompletionResult {
|
||||
readonly text: string;
|
||||
readonly model: string;
|
||||
readonly totalDuration: number;
|
||||
}
|
||||
|
||||
export interface OllamaClient {
|
||||
/** Generate an embedding vector for the given text. */
|
||||
embed(text: string): Promise<EmbeddingResult>;
|
||||
|
||||
/** Generate a chat/instruct completion. */
|
||||
complete(prompt: string, systemPrompt?: string): Promise<CompletionResult>;
|
||||
|
||||
/** Check if the Ollama server is reachable and models are available. */
|
||||
healthCheck(): Promise<{ ok: boolean; models: readonly string[] }>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Implementation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Create an Ollama client for embeddings and completions.
|
||||
*/
|
||||
export function createOllamaClient(config: OllamaConfig): OllamaClient {
|
||||
const { baseUrl, model, embeddingModel, timeout = 120_000 } = config;
|
||||
|
||||
async function post<T>(path: string, body: unknown): Promise<T> {
|
||||
const url = `${baseUrl.replace(/\/+$/, "")}${path}`;
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), timeout);
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const text = await response.text().catch(() => "");
|
||||
throw new Error(`Ollama API error: ${response.status} -- ${text}`);
|
||||
}
|
||||
|
||||
return (await response.json()) as T;
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
async embed(text) {
|
||||
// TODO: implement chunking for texts exceeding model context window
|
||||
// TODO: add retry logic with exponential backoff
|
||||
|
||||
interface OllamaEmbedResponse {
|
||||
embedding: number[];
|
||||
}
|
||||
|
||||
const result = await post<OllamaEmbedResponse>("/api/embeddings", {
|
||||
model: embeddingModel,
|
||||
prompt: text,
|
||||
});
|
||||
|
||||
return {
|
||||
vector: result.embedding,
|
||||
model: embeddingModel,
|
||||
dimensions: result.embedding.length,
|
||||
};
|
||||
},
|
||||
|
||||
async complete(prompt, systemPrompt) {
|
||||
// TODO: implement streaming support for long completions
|
||||
// TODO: add structured output parsing (JSON mode)
|
||||
|
||||
interface OllamaGenerateResponse {
|
||||
response: string;
|
||||
model: string;
|
||||
total_duration: number;
|
||||
}
|
||||
|
||||
const result = await post<OllamaGenerateResponse>("/api/generate", {
|
||||
model,
|
||||
prompt,
|
||||
system: systemPrompt ?? "",
|
||||
stream: false,
|
||||
});
|
||||
|
||||
return {
|
||||
text: result.response,
|
||||
model: result.model,
|
||||
totalDuration: result.total_duration,
|
||||
};
|
||||
},
|
||||
|
||||
async healthCheck() {
|
||||
try {
|
||||
const url = `${baseUrl.replace(/\/+$/, "")}/api/tags`;
|
||||
const response = await fetch(url);
|
||||
if (!response.ok) return { ok: false, models: [] };
|
||||
|
||||
interface OllamaTagsResponse {
|
||||
models: Array<{ name: string }>;
|
||||
}
|
||||
|
||||
const data = (await response.json()) as OllamaTagsResponse;
|
||||
return {
|
||||
ok: true,
|
||||
models: data.models.map((m) => m.name),
|
||||
};
|
||||
} catch {
|
||||
return { ok: false, models: [] };
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
231
src/embeddings/store.ts
Normal file
231
src/embeddings/store.ts
Normal file
@ -0,0 +1,231 @@
|
||||
/**
|
||||
* Local SQLite-backed vector store for document embeddings.
|
||||
*
|
||||
* Stores embedding vectors alongside document metadata in a SQLite database
|
||||
* using better-sqlite3. Supports cosine similarity search for semantic
|
||||
* document retrieval.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const store = createVectorStore({ dbPath: "./data/vectors.db" });
|
||||
* await store.upsert({ documentId: 42, vector: [...], content: "..." });
|
||||
* const results = await store.search(queryVector, { limit: 10 });
|
||||
* ```
|
||||
*/
|
||||
|
||||
import Database from "better-sqlite3";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface VectorStoreConfig {
|
||||
readonly dbPath: string;
|
||||
}
|
||||
|
||||
export interface DocumentEmbedding {
|
||||
readonly documentId: number;
|
||||
readonly vector: readonly number[];
|
||||
readonly content: string;
|
||||
readonly title: string;
|
||||
readonly tags: readonly string[];
|
||||
readonly createdAt: string;
|
||||
}
|
||||
|
||||
export interface SearchResult {
|
||||
readonly documentId: number;
|
||||
readonly title: string;
|
||||
readonly content: string;
|
||||
readonly score: number;
|
||||
readonly tags: readonly string[];
|
||||
}
|
||||
|
||||
export interface SearchOptions {
|
||||
readonly limit?: number;
|
||||
readonly minScore?: number;
|
||||
readonly tagFilter?: readonly string[];
|
||||
}
|
||||
|
||||
export interface VectorStore {
|
||||
/** Insert or update a document embedding. */
|
||||
upsert(embedding: DocumentEmbedding): void;
|
||||
|
||||
/** Search for similar documents using cosine similarity. */
|
||||
search(queryVector: readonly number[], options?: SearchOptions): readonly SearchResult[];
|
||||
|
||||
/** Remove an embedding by document ID. */
|
||||
remove(documentId: number): void;
|
||||
|
||||
/** Get the total count of stored embeddings. */
|
||||
count(): number;
|
||||
|
||||
/** Check if a document has been embedded. */
|
||||
has(documentId: number): boolean;
|
||||
|
||||
/** Close the database connection. */
|
||||
close(): void;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Compute cosine similarity between two vectors.
|
||||
* Returns a value between -1 and 1 (1 = identical direction).
|
||||
*/
|
||||
function cosineSimilarity(a: readonly number[], b: readonly number[]): number {
|
||||
if (a.length !== b.length) {
|
||||
throw new Error(
|
||||
`Vector dimension mismatch: ${a.length} vs ${b.length}`,
|
||||
);
|
||||
}
|
||||
|
||||
let dotProduct = 0;
|
||||
let normA = 0;
|
||||
let normB = 0;
|
||||
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
dotProduct += a[i] * b[i];
|
||||
normA += a[i] * a[i];
|
||||
normB += b[i] * b[i];
|
||||
}
|
||||
|
||||
const denominator = Math.sqrt(normA) * Math.sqrt(normB);
|
||||
if (denominator === 0) return 0;
|
||||
|
||||
return dotProduct / denominator;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Implementation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Create a local vector store backed by SQLite.
|
||||
*
|
||||
* TODO: Consider migrating to sqlite-vss or DuckDB for ANN search at scale.
|
||||
* The current brute-force approach works well for <100k documents.
|
||||
*/
|
||||
export function createVectorStore(config: VectorStoreConfig): VectorStore {
|
||||
const db = new Database(config.dbPath);
|
||||
|
||||
// Enable WAL mode for better concurrent read performance
|
||||
db.pragma("journal_mode = WAL");
|
||||
|
||||
// Create tables if they don't exist
|
||||
db.exec(`
|
||||
CREATE TABLE IF NOT EXISTS embeddings (
|
||||
document_id INTEGER PRIMARY KEY,
|
||||
vector BLOB NOT NULL,
|
||||
content TEXT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
tags TEXT NOT NULL DEFAULT '[]',
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_embeddings_created
|
||||
ON embeddings (created_at);
|
||||
`);
|
||||
|
||||
// Prepared statements for performance
|
||||
const upsertStmt = db.prepare(`
|
||||
INSERT INTO embeddings (document_id, vector, content, title, tags, created_at, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, datetime('now'))
|
||||
ON CONFLICT(document_id) DO UPDATE SET
|
||||
vector = excluded.vector,
|
||||
content = excluded.content,
|
||||
title = excluded.title,
|
||||
tags = excluded.tags,
|
||||
updated_at = datetime('now')
|
||||
`);
|
||||
|
||||
const getAllStmt = db.prepare(`
|
||||
SELECT document_id, vector, content, title, tags FROM embeddings
|
||||
`);
|
||||
|
||||
const removeStmt = db.prepare(`
|
||||
DELETE FROM embeddings WHERE document_id = ?
|
||||
`);
|
||||
|
||||
const countStmt = db.prepare(`
|
||||
SELECT COUNT(*) as count FROM embeddings
|
||||
`);
|
||||
|
||||
const hasStmt = db.prepare(`
|
||||
SELECT 1 FROM embeddings WHERE document_id = ? LIMIT 1
|
||||
`);
|
||||
|
||||
return {
|
||||
upsert(embedding) {
|
||||
const vectorBlob = Buffer.from(new Float32Array(embedding.vector).buffer);
|
||||
upsertStmt.run(
|
||||
embedding.documentId,
|
||||
vectorBlob,
|
||||
embedding.content,
|
||||
embedding.title,
|
||||
JSON.stringify(embedding.tags),
|
||||
embedding.createdAt,
|
||||
);
|
||||
},
|
||||
|
||||
search(queryVector, options = {}) {
|
||||
const { limit = 10, minScore = 0.5, tagFilter } = options;
|
||||
|
||||
// TODO: Implement ANN (approximate nearest neighbor) for large datasets
|
||||
// Current approach: brute-force scan -- fine for <100k documents
|
||||
|
||||
interface EmbeddingRow {
|
||||
document_id: number;
|
||||
vector: Buffer;
|
||||
content: string;
|
||||
title: string;
|
||||
tags: string;
|
||||
}
|
||||
|
||||
const rows = getAllStmt.all() as EmbeddingRow[];
|
||||
|
||||
const scored = rows
|
||||
.map((row) => {
|
||||
const storedVector = Array.from(new Float32Array(row.vector.buffer));
|
||||
const tags: string[] = JSON.parse(row.tags);
|
||||
const score = cosineSimilarity(queryVector, storedVector);
|
||||
|
||||
return {
|
||||
documentId: row.document_id,
|
||||
title: row.title,
|
||||
content: row.content,
|
||||
score,
|
||||
tags,
|
||||
};
|
||||
})
|
||||
.filter((result) => result.score >= minScore)
|
||||
.filter((result) => {
|
||||
if (!tagFilter || tagFilter.length === 0) return true;
|
||||
return tagFilter.some((tag) => result.tags.includes(tag));
|
||||
})
|
||||
.sort((a, b) => b.score - a.score)
|
||||
.slice(0, limit);
|
||||
|
||||
return scored;
|
||||
},
|
||||
|
||||
remove(documentId) {
|
||||
removeStmt.run(documentId);
|
||||
},
|
||||
|
||||
count() {
|
||||
const row = countStmt.get() as { count: number };
|
||||
return row.count;
|
||||
},
|
||||
|
||||
has(documentId) {
|
||||
return hasStmt.get(documentId) !== undefined;
|
||||
},
|
||||
|
||||
close() {
|
||||
db.close();
|
||||
},
|
||||
};
|
||||
}
|
||||
249
src/mcp-server/index.ts
Normal file
249
src/mcp-server/index.ts
Normal file
@ -0,0 +1,249 @@
|
||||
/**
|
||||
* PaperCortex MCP Server entry point.
|
||||
*
|
||||
* Exposes document intelligence tools via the Model Context Protocol (MCP)
|
||||
* for integration with Claude Code and other AI agents.
|
||||
*
|
||||
* @see https://modelcontextprotocol.io
|
||||
*/
|
||||
|
||||
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
|
||||
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
|
||||
import {
|
||||
CallToolRequestSchema,
|
||||
ListToolsRequestSchema,
|
||||
} from "@modelcontextprotocol/sdk/types.js";
|
||||
import { config } from "dotenv";
|
||||
|
||||
import { createOllamaClient } from "../embeddings/ollama.js";
|
||||
import { createVectorStore } from "../embeddings/store.js";
|
||||
import { createPaperlessClient } from "../paperless/client.js";
|
||||
import { handleClassify } from "./tools/classify.js";
|
||||
import { handleExport } from "./tools/export.js";
|
||||
import { handleQuery } from "./tools/query.js";
|
||||
import { handleReceipt } from "./tools/receipt.js";
|
||||
import { handleSearch } from "./tools/search.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Configuration
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
config(); // Load .env
|
||||
|
||||
function requireEnv(key: string): string {
|
||||
const value = process.env[key];
|
||||
if (!value) {
|
||||
throw new Error(`Missing required environment variable: ${key}`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Service initialization
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const paperless = createPaperlessClient({
|
||||
baseUrl: requireEnv("PAPERLESS_URL"),
|
||||
token: requireEnv("PAPERLESS_TOKEN"),
|
||||
});
|
||||
|
||||
const ollama = createOllamaClient({
|
||||
baseUrl: process.env["OLLAMA_URL"] ?? "http://localhost:11434",
|
||||
model: process.env["OLLAMA_MODEL"] ?? "qwen2.5:14b",
|
||||
embeddingModel: process.env["OLLAMA_EMBEDDING_MODEL"] ?? "nomic-embed-text",
|
||||
});
|
||||
|
||||
const vectorStore = createVectorStore({
|
||||
dbPath: process.env["VECTOR_DB_PATH"] ?? "./data/vectors.db",
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Shared context for tool handlers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ToolContext {
|
||||
readonly paperless: typeof paperless;
|
||||
readonly ollama: typeof ollama;
|
||||
readonly vectorStore: typeof vectorStore;
|
||||
}
|
||||
|
||||
const ctx: ToolContext = { paperless, ollama, vectorStore };
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MCP Server setup
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const server = new Server(
|
||||
{
|
||||
name: "papercortex",
|
||||
version: "0.1.0",
|
||||
},
|
||||
{
|
||||
capabilities: {
|
||||
tools: {},
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
/**
|
||||
* List all available PaperCortex tools.
|
||||
*/
|
||||
server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
||||
tools: [
|
||||
{
|
||||
name: "papercortex_search",
|
||||
description:
|
||||
"Semantic search across all documents in Paperless-ngx. " +
|
||||
"Finds documents by meaning, not just keywords.",
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
properties: {
|
||||
query: {
|
||||
type: "string",
|
||||
description: "Natural language search query",
|
||||
},
|
||||
limit: {
|
||||
type: "number",
|
||||
description: "Maximum number of results (default: 10)",
|
||||
},
|
||||
tags: {
|
||||
type: "array",
|
||||
items: { type: "string" },
|
||||
description: "Filter by tag names",
|
||||
},
|
||||
},
|
||||
required: ["query"],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "papercortex_classify",
|
||||
description:
|
||||
"Auto-classify a document using local AI. " +
|
||||
"Suggests tags, document type, and correspondent.",
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
properties: {
|
||||
documentId: {
|
||||
type: "number",
|
||||
description: "Paperless-ngx document ID",
|
||||
},
|
||||
applyTags: {
|
||||
type: "boolean",
|
||||
description: "Automatically apply suggested tags (default: false)",
|
||||
},
|
||||
},
|
||||
required: ["documentId"],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "papercortex_receipt",
|
||||
description:
|
||||
"Extract structured data from a receipt document: " +
|
||||
"vendor, date, amounts, tax, line items.",
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
properties: {
|
||||
documentId: {
|
||||
type: "number",
|
||||
description: "Paperless-ngx document ID of the receipt",
|
||||
},
|
||||
},
|
||||
required: ["documentId"],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "papercortex_query",
|
||||
description:
|
||||
"Ask natural language questions about your documents. " +
|
||||
'Example: "How much did I spend on office supplies in Q1 2024?"',
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
properties: {
|
||||
question: {
|
||||
type: "string",
|
||||
description: "Natural language question about your documents",
|
||||
},
|
||||
maxDocuments: {
|
||||
type: "number",
|
||||
description:
|
||||
"Maximum documents to include in context (default: 5)",
|
||||
},
|
||||
},
|
||||
required: ["question"],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "papercortex_export",
|
||||
description:
|
||||
"Export receipt data as DATEV-compatible CSV for German accounting, " +
|
||||
"or as generic CSV.",
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
properties: {
|
||||
documentIds: {
|
||||
type: "array",
|
||||
items: { type: "number" },
|
||||
description: "Document IDs to export",
|
||||
},
|
||||
format: {
|
||||
type: "string",
|
||||
enum: ["datev", "csv"],
|
||||
description: "Export format (default: datev)",
|
||||
},
|
||||
},
|
||||
required: ["documentIds"],
|
||||
},
|
||||
},
|
||||
],
|
||||
}));
|
||||
|
||||
/**
|
||||
* Route tool calls to their respective handlers.
|
||||
*/
|
||||
server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
||||
const { name, arguments: args } = request.params;
|
||||
|
||||
try {
|
||||
switch (name) {
|
||||
case "papercortex_search":
|
||||
return await handleSearch(ctx, args as Record<string, unknown>);
|
||||
case "papercortex_classify":
|
||||
return await handleClassify(ctx, args as Record<string, unknown>);
|
||||
case "papercortex_receipt":
|
||||
return await handleReceipt(ctx, args as Record<string, unknown>);
|
||||
case "papercortex_query":
|
||||
return await handleQuery(ctx, args as Record<string, unknown>);
|
||||
case "papercortex_export":
|
||||
return await handleExport(ctx, args as Record<string, unknown>);
|
||||
default:
|
||||
return {
|
||||
content: [
|
||||
{ type: "text" as const, text: `Unknown tool: ${name}` },
|
||||
],
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
} catch (error) {
|
||||
const message =
|
||||
error instanceof Error ? error.message : "Unknown error occurred";
|
||||
return {
|
||||
content: [{ type: "text" as const, text: `Error: ${message}` }],
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Start server
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const transport = new StdioServerTransport();
|
||||
await server.connect(transport);
|
||||
console.error("PaperCortex MCP Server running on stdio");
|
||||
}
|
||||
|
||||
main().catch((error) => {
|
||||
console.error("Fatal error starting PaperCortex:", error);
|
||||
process.exit(1);
|
||||
});
|
||||
117
src/mcp-server/tools/classify.ts
Normal file
117
src/mcp-server/tools/classify.ts
Normal file
@ -0,0 +1,117 @@
|
||||
/**
|
||||
* Auto-classification tool for the PaperCortex MCP Server.
|
||||
*
|
||||
* Uses local LLM to analyze document content and suggest appropriate
|
||||
* tags, document types, and correspondents.
|
||||
*/
|
||||
|
||||
import type { ToolContext } from "../index.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface ClassifyArgs {
|
||||
readonly documentId: number;
|
||||
readonly applyTags?: boolean;
|
||||
}
|
||||
|
||||
interface ClassificationResult {
|
||||
readonly suggestedTags: readonly string[];
|
||||
readonly suggestedType: string | null;
|
||||
readonly suggestedCorrespondent: string | null;
|
||||
readonly summary: string;
|
||||
readonly language: string;
|
||||
readonly confidence: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Prompts
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const CLASSIFY_SYSTEM_PROMPT = `You are a document classification assistant. Analyze the document content and provide structured classification.
|
||||
|
||||
Respond with valid JSON only:
|
||||
{
|
||||
"suggestedTags": ["tag1", "tag2"],
|
||||
"suggestedType": "invoice|contract|receipt|letter|report|tax_document|bank_statement|insurance|warranty|manual|other",
|
||||
"suggestedCorrespondent": "Company or person name",
|
||||
"summary": "One sentence summary",
|
||||
"language": "ISO 639-1 code",
|
||||
"confidence": 0.0 to 1.0
|
||||
}`;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Handler
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Handle a `papercortex_classify` tool call.
|
||||
*
|
||||
* 1. Fetch document content from Paperless-ngx.
|
||||
* 2. Send content to Ollama for classification.
|
||||
* 3. Optionally apply suggested tags back to Paperless-ngx.
|
||||
*
|
||||
* TODO: Match suggested tags against existing Paperless-ngx tags
|
||||
* TODO: Create new tags automatically when confidence is high
|
||||
* TODO: Learn from user corrections to improve classification
|
||||
*/
|
||||
export async function handleClassify(
|
||||
ctx: ToolContext,
|
||||
args: Record<string, unknown>,
|
||||
): Promise<{ content: Array<{ type: "text"; text: string }> }> {
|
||||
const { documentId, applyTags = false } = args as unknown as ClassifyArgs;
|
||||
|
||||
// Fetch document from Paperless-ngx
|
||||
const document = await ctx.paperless.getDocument(documentId);
|
||||
|
||||
if (!document.content || document.content.trim().length === 0) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: `Document #${documentId} has no text content. OCR may not have completed.`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
// Classify using Ollama
|
||||
const prompt = `Classify this document:\n\nTitle: ${document.title}\n\nContent:\n${document.content.slice(0, 4000)}`;
|
||||
const completion = await ctx.ollama.complete(prompt, CLASSIFY_SYSTEM_PROMPT);
|
||||
|
||||
let classification: ClassificationResult;
|
||||
try {
|
||||
classification = JSON.parse(completion.text) as ClassificationResult;
|
||||
} catch {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: `Classification failed: LLM did not return valid JSON.\nRaw response: ${completion.text.slice(0, 500)}`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
// Optionally apply tags
|
||||
let appliedNote = "";
|
||||
if (applyTags && classification.suggestedTags.length > 0) {
|
||||
// TODO: Look up tag IDs from Paperless-ngx, create missing tags
|
||||
appliedNote =
|
||||
"\n\nNote: Tag application is not yet implemented. " +
|
||||
"Tags need to be matched against existing Paperless-ngx tags.";
|
||||
}
|
||||
|
||||
const output =
|
||||
`Classification for Document #${documentId} "${document.title}":\n\n` +
|
||||
`Type: ${classification.suggestedType ?? "unknown"}\n` +
|
||||
`Correspondent: ${classification.suggestedCorrespondent ?? "unknown"}\n` +
|
||||
`Tags: ${classification.suggestedTags.join(", ") || "none"}\n` +
|
||||
`Language: ${classification.language}\n` +
|
||||
`Summary: ${classification.summary}\n` +
|
||||
`Confidence: ${(classification.confidence * 100).toFixed(0)}%` +
|
||||
appliedNote;
|
||||
|
||||
return { content: [{ type: "text", text: output }] };
|
||||
}
|
||||
116
src/mcp-server/tools/export.ts
Normal file
116
src/mcp-server/tools/export.ts
Normal file
@ -0,0 +1,116 @@
|
||||
/**
|
||||
* DATEV/CSV export tool for the PaperCortex MCP Server.
|
||||
*
|
||||
* Exports receipt data in accounting-compatible formats.
|
||||
*/
|
||||
|
||||
import { createReceiptExtractor } from "../../receipt/extractor.js";
|
||||
import { createDatevExporter } from "../../receipt/datev.js";
|
||||
import type { ToolContext } from "../index.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface ExportArgs {
|
||||
readonly documentIds: readonly number[];
|
||||
readonly format?: "datev" | "csv";
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Handler
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Handle a `papercortex_export` tool call.
|
||||
*
|
||||
* 1. Extract receipt data from all specified documents.
|
||||
* 2. Format as DATEV or generic CSV.
|
||||
* 3. Return the CSV content.
|
||||
*
|
||||
* TODO: Add file output option (save to disk)
|
||||
* TODO: Add date range filtering
|
||||
* TODO: Add DATEV header metadata (consultant/client numbers from config)
|
||||
*/
|
||||
export async function handleExport(
|
||||
ctx: ToolContext,
|
||||
args: Record<string, unknown>,
|
||||
): Promise<{ content: Array<{ type: "text"; text: string }> }> {
|
||||
const { documentIds, format = "datev" } = args as unknown as ExportArgs;
|
||||
|
||||
if (!documentIds || documentIds.length === 0) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: "Error: at least one document ID is required for export.",
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
// Extract receipt data from all documents
|
||||
const extractor = createReceiptExtractor({
|
||||
ollama: ctx.ollama,
|
||||
paperless: ctx.paperless,
|
||||
});
|
||||
|
||||
const receipts = await extractor.extractBatch(documentIds);
|
||||
|
||||
if (format === "datev") {
|
||||
// TODO: Read consultant/client numbers from configuration
|
||||
const exporter = createDatevExporter({
|
||||
consultantNumber: 0,
|
||||
clientNumber: 0,
|
||||
});
|
||||
|
||||
const receiptsForExport = receipts.map((r) => ({
|
||||
documentId: r.documentId,
|
||||
vendor: r.vendor,
|
||||
date: r.date,
|
||||
totalAmount: r.totalAmount,
|
||||
taxRate: r.taxRate,
|
||||
category: r.category,
|
||||
}));
|
||||
|
||||
const csv = exporter.generateCsv(receiptsForExport);
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`DATEV export for ${receipts.length} receipt(s):\n\n` +
|
||||
"```csv\n" +
|
||||
csv +
|
||||
"\n```\n\n" +
|
||||
"Copy this CSV content into a file and import into your " +
|
||||
"DATEV-compatible accounting software.",
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
// Generic CSV format
|
||||
const header = "Document ID;Vendor;Date;Amount;Tax Rate;Tax Amount;Currency;Category";
|
||||
const rows = receipts.map(
|
||||
(r) =>
|
||||
`${r.documentId};${r.vendor};${r.date};${r.totalAmount.toFixed(2)};` +
|
||||
`${r.taxRate ?? ""};${r.taxAmount?.toFixed(2) ?? ""};${r.currency};${r.category ?? ""}`,
|
||||
);
|
||||
|
||||
const csv = [header, ...rows].join("\n");
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`CSV export for ${receipts.length} receipt(s):\n\n` +
|
||||
"```csv\n" +
|
||||
csv +
|
||||
"\n```",
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
110
src/mcp-server/tools/query.ts
Normal file
110
src/mcp-server/tools/query.ts
Normal file
@ -0,0 +1,110 @@
|
||||
/**
|
||||
* Natural language query tool for the PaperCortex MCP Server.
|
||||
*
|
||||
* Answers questions about documents using RAG (Retrieval-Augmented Generation):
|
||||
* retrieves relevant documents via semantic search, then generates an answer
|
||||
* using the local LLM with document context.
|
||||
*/
|
||||
|
||||
import type { ToolContext } from "../index.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface QueryArgs {
|
||||
readonly question: string;
|
||||
readonly maxDocuments?: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Prompts
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const QUERY_SYSTEM_PROMPT = `You are a document analysis assistant. Answer the user's question based ONLY on the provided document excerpts. If the documents don't contain enough information to answer, say so clearly.
|
||||
|
||||
Be precise with numbers, dates, and amounts. Cite document IDs when referencing specific documents.`;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Handler
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Handle a `papercortex_query` tool call.
|
||||
*
|
||||
* Uses RAG (Retrieval-Augmented Generation):
|
||||
* 1. Embed the question and retrieve relevant documents.
|
||||
* 2. Build a context from retrieved documents.
|
||||
* 3. Generate an answer using the local LLM.
|
||||
*
|
||||
* TODO: Add conversation history for follow-up questions
|
||||
* TODO: Add source citation with page numbers
|
||||
* TODO: Implement query decomposition for complex questions
|
||||
*/
|
||||
export async function handleQuery(
|
||||
ctx: ToolContext,
|
||||
args: Record<string, unknown>,
|
||||
): Promise<{ content: Array<{ type: "text"; text: string }> }> {
|
||||
const { question, maxDocuments = 5 } = args as unknown as QueryArgs;
|
||||
|
||||
if (!question || question.trim().length === 0) {
|
||||
return {
|
||||
content: [{ type: "text", text: "Error: question cannot be empty." }],
|
||||
};
|
||||
}
|
||||
|
||||
// Step 1: Retrieve relevant documents
|
||||
const queryEmbedding = await ctx.ollama.embed(question);
|
||||
const relevantDocs = ctx.vectorStore.search(queryEmbedding.vector, {
|
||||
limit: maxDocuments,
|
||||
minScore: 0.3,
|
||||
});
|
||||
|
||||
if (relevantDocs.length === 0) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`I couldn't find any relevant documents to answer: "${question}"\n\n` +
|
||||
"The vector store may need to be populated first, or your documents " +
|
||||
"may not contain information related to this question.",
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
// Step 2: Build context from retrieved documents
|
||||
const context = relevantDocs
|
||||
.map(
|
||||
(doc) =>
|
||||
`--- Document #${doc.documentId}: ${doc.title} (relevance: ${doc.score.toFixed(2)}) ---\n` +
|
||||
doc.content.slice(0, 2000),
|
||||
)
|
||||
.join("\n\n");
|
||||
|
||||
// Step 3: Generate answer with context
|
||||
const prompt =
|
||||
`Based on the following documents, answer this question: "${question}"\n\n` +
|
||||
`Documents:\n${context}`;
|
||||
|
||||
const completion = await ctx.ollama.complete(prompt, QUERY_SYSTEM_PROMPT);
|
||||
|
||||
const sourcesNote = relevantDocs
|
||||
.map(
|
||||
(doc) =>
|
||||
` - Document #${doc.documentId}: ${doc.title} (score: ${doc.score.toFixed(2)})`,
|
||||
)
|
||||
.join("\n");
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text:
|
||||
`${completion.text}\n\n` +
|
||||
`---\nSources (${relevantDocs.length} documents):\n${sourcesNote}`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
76
src/mcp-server/tools/receipt.ts
Normal file
76
src/mcp-server/tools/receipt.ts
Normal file
@ -0,0 +1,76 @@
|
||||
/**
|
||||
* Receipt extraction tool for the PaperCortex MCP Server.
|
||||
*
|
||||
* Extracts structured receipt data from Paperless-ngx documents
|
||||
* using local LLM analysis.
|
||||
*/
|
||||
|
||||
import { createReceiptExtractor } from "../../receipt/extractor.js";
|
||||
import type { ToolContext } from "../index.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface ReceiptArgs {
|
||||
readonly documentId: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Handler
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Handle a `papercortex_receipt` tool call.
|
||||
*
|
||||
* 1. Fetch document from Paperless-ngx.
|
||||
* 2. Extract receipt data using LLM.
|
||||
* 3. Return structured receipt information.
|
||||
*
|
||||
* TODO: Cache extraction results to avoid re-processing
|
||||
* TODO: Add confidence thresholds and human review flags
|
||||
* TODO: Store extracted data back as Paperless-ngx custom fields
|
||||
*/
|
||||
export async function handleReceipt(
|
||||
ctx: ToolContext,
|
||||
args: Record<string, unknown>,
|
||||
): Promise<{ content: Array<{ type: "text"; text: string }> }> {
|
||||
const { documentId } = args as unknown as ReceiptArgs;
|
||||
|
||||
const extractor = createReceiptExtractor({
|
||||
ollama: ctx.ollama,
|
||||
paperless: ctx.paperless,
|
||||
});
|
||||
|
||||
const receipt = await extractor.extract(documentId);
|
||||
|
||||
// Format line items table
|
||||
const lineItemsTable =
|
||||
receipt.lineItems.length > 0
|
||||
? receipt.lineItems
|
||||
.map(
|
||||
(item, i) =>
|
||||
` ${i + 1}. ${item.description} | ` +
|
||||
`${item.quantity}x ${item.unitPrice.toFixed(2)} = ${item.totalPrice.toFixed(2)}`,
|
||||
)
|
||||
.join("\n")
|
||||
: " No line items extracted";
|
||||
|
||||
const output =
|
||||
`Receipt Data for Document #${documentId}:\n\n` +
|
||||
`Vendor: ${receipt.vendor}\n` +
|
||||
`Address: ${receipt.vendorAddress ?? "N/A"}\n` +
|
||||
`Tax ID: ${receipt.vendorTaxId ?? "N/A"}\n` +
|
||||
`Date: ${receipt.date}\n` +
|
||||
`Currency: ${receipt.currency}\n` +
|
||||
`\nAmounts:\n` +
|
||||
` Subtotal: ${receipt.subtotal?.toFixed(2) ?? "N/A"}\n` +
|
||||
` Tax (${receipt.taxRate ?? "?"}%): ${receipt.taxAmount?.toFixed(2) ?? "N/A"}\n` +
|
||||
` Total: ${receipt.totalAmount.toFixed(2)}\n` +
|
||||
`\nPayment: ${receipt.paymentMethod ?? "N/A"}\n` +
|
||||
`Category: ${receipt.category ?? "uncategorized"}\n` +
|
||||
`Confidence: ${(receipt.confidence * 100).toFixed(0)}%\n` +
|
||||
`\nLine Items:\n${lineItemsTable}`;
|
||||
|
||||
return { content: [{ type: "text", text: output }] };
|
||||
}
|
||||
87
src/mcp-server/tools/search.ts
Normal file
87
src/mcp-server/tools/search.ts
Normal file
@ -0,0 +1,87 @@
|
||||
/**
|
||||
* Semantic search tool for the PaperCortex MCP Server.
|
||||
*
|
||||
* Performs vector similarity search across all embedded documents,
|
||||
* returning the most semantically relevant results.
|
||||
*/
|
||||
|
||||
import type { ToolContext } from "../index.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface SearchArgs {
|
||||
readonly query: string;
|
||||
readonly limit?: number;
|
||||
readonly tags?: readonly string[];
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Handler
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Handle a `papercortex_search` tool call.
|
||||
*
|
||||
* 1. Generate an embedding for the search query via Ollama.
|
||||
* 2. Search the local vector store for similar documents.
|
||||
* 3. Return ranked results with scores and metadata.
|
||||
*
|
||||
* TODO: Add hybrid search (combine vector + keyword for better recall)
|
||||
* TODO: Add date range filtering
|
||||
* TODO: Add result caching for repeated queries
|
||||
*/
|
||||
export async function handleSearch(
|
||||
ctx: ToolContext,
|
||||
args: Record<string, unknown>,
|
||||
): Promise<{ content: Array<{ type: "text"; text: string }> }> {
|
||||
const { query, limit = 10, tags } = args as unknown as SearchArgs;
|
||||
|
||||
if (!query || query.trim().length === 0) {
|
||||
return {
|
||||
content: [{ type: "text", text: "Error: search query cannot be empty." }],
|
||||
};
|
||||
}
|
||||
|
||||
// Generate embedding for the query
|
||||
const queryEmbedding = await ctx.ollama.embed(query);
|
||||
|
||||
// Search vector store
|
||||
const results = ctx.vectorStore.search(queryEmbedding.vector, {
|
||||
limit,
|
||||
minScore: 0.4,
|
||||
tagFilter: tags ? [...tags] : undefined,
|
||||
});
|
||||
|
||||
if (results.length === 0) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: `No documents found matching "${query}". The vector store may need to be populated first.`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
// Format results
|
||||
const formatted = results
|
||||
.map(
|
||||
(r, i) =>
|
||||
`${i + 1}. [Document #${r.documentId}] (score: ${r.score.toFixed(3)})\n` +
|
||||
` Title: ${r.title}\n` +
|
||||
` Tags: ${r.tags.length > 0 ? r.tags.join(", ") : "none"}\n` +
|
||||
` Preview: ${r.content.slice(0, 200).replace(/\n/g, " ")}...`,
|
||||
)
|
||||
.join("\n\n");
|
||||
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text",
|
||||
text: `Found ${results.length} documents matching "${query}":\n\n${formatted}`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
182
src/paperless/client.ts
Normal file
182
src/paperless/client.ts
Normal file
@ -0,0 +1,182 @@
|
||||
/**
|
||||
* Paperless-ngx REST API client.
|
||||
*
|
||||
* Provides typed access to documents, correspondents, tags, and document types.
|
||||
* All methods return immutable result objects.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const client = createPaperlessClient({
|
||||
* baseUrl: "http://localhost:8000",
|
||||
* token: "your-api-token",
|
||||
* });
|
||||
* const docs = await client.getDocuments({ query: "invoice" });
|
||||
* ```
|
||||
*/
|
||||
|
||||
import type {
|
||||
Correspondent,
|
||||
DocumentSearchParams,
|
||||
DocumentType,
|
||||
PaginatedResponse,
|
||||
PaperlessConfig,
|
||||
PaperlessDocument,
|
||||
Tag,
|
||||
} from "./types.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Client interface
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface PaperlessClient {
|
||||
/** Fetch a single document by ID. */
|
||||
getDocument(id: number): Promise<PaperlessDocument>;
|
||||
|
||||
/** Search / list documents with optional filters. */
|
||||
getDocuments(
|
||||
params?: DocumentSearchParams,
|
||||
): Promise<PaginatedResponse<PaperlessDocument>>;
|
||||
|
||||
/** Fetch all correspondents. */
|
||||
getCorrespondents(): Promise<PaginatedResponse<Correspondent>>;
|
||||
|
||||
/** Fetch all tags. */
|
||||
getTags(): Promise<PaginatedResponse<Tag>>;
|
||||
|
||||
/** Fetch all document types. */
|
||||
getDocumentTypes(): Promise<PaginatedResponse<DocumentType>>;
|
||||
|
||||
/** Download the original file content of a document. */
|
||||
downloadDocument(id: number): Promise<ArrayBuffer>;
|
||||
|
||||
/** Update tags on a document (immutable -- returns the updated doc). */
|
||||
updateDocumentTags(
|
||||
id: number,
|
||||
tagIds: readonly number[],
|
||||
): Promise<PaperlessDocument>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Implementation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Create a new Paperless-ngx API client.
|
||||
*
|
||||
* @param config - Connection configuration (URL + token).
|
||||
* @returns A {@link PaperlessClient} instance.
|
||||
*/
|
||||
export function createPaperlessClient(config: PaperlessConfig): PaperlessClient {
|
||||
const { baseUrl, token, timeout = 30_000 } = config;
|
||||
|
||||
const headers: Record<string, string> = {
|
||||
Authorization: `Token ${token}`,
|
||||
"Content-Type": "application/json",
|
||||
Accept: "application/json; version=3",
|
||||
};
|
||||
|
||||
/**
|
||||
* Internal fetch wrapper with timeout and error handling.
|
||||
*/
|
||||
async function request<T>(
|
||||
path: string,
|
||||
options: RequestInit = {},
|
||||
): Promise<T> {
|
||||
const url = `${baseUrl.replace(/\/+$/, "")}/api${path}`;
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), timeout);
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
...options,
|
||||
headers: { ...headers, ...((options.headers as Record<string, string>) ?? {}) },
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const body = await response.text().catch(() => "");
|
||||
throw new Error(
|
||||
`Paperless API error: ${response.status} ${response.statusText} -- ${body}`,
|
||||
);
|
||||
}
|
||||
|
||||
return (await response.json()) as T;
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build query string from search params.
|
||||
*/
|
||||
function buildQuery(params?: DocumentSearchParams): string {
|
||||
if (!params) return "";
|
||||
const entries = Object.entries(params).filter(
|
||||
([, v]) => v !== undefined && v !== null,
|
||||
);
|
||||
if (entries.length === 0) return "";
|
||||
const searchParams = new URLSearchParams();
|
||||
for (const [key, value] of entries) {
|
||||
if (Array.isArray(value)) {
|
||||
searchParams.set(key, value.join(","));
|
||||
} else {
|
||||
searchParams.set(key, String(value));
|
||||
}
|
||||
}
|
||||
return `?${searchParams.toString()}`;
|
||||
}
|
||||
|
||||
return {
|
||||
async getDocument(id) {
|
||||
return request<PaperlessDocument>(`/documents/${id}/`);
|
||||
},
|
||||
|
||||
async getDocuments(params) {
|
||||
return request<PaginatedResponse<PaperlessDocument>>(
|
||||
`/documents/${buildQuery(params)}`,
|
||||
);
|
||||
},
|
||||
|
||||
async getCorrespondents() {
|
||||
return request<PaginatedResponse<Correspondent>>("/correspondents/");
|
||||
},
|
||||
|
||||
async getTags() {
|
||||
return request<PaginatedResponse<Tag>>("/tags/");
|
||||
},
|
||||
|
||||
async getDocumentTypes() {
|
||||
return request<PaginatedResponse<DocumentType>>("/document_types/");
|
||||
},
|
||||
|
||||
async downloadDocument(id) {
|
||||
const url = `${baseUrl.replace(/\/+$/, "")}/api/documents/${id}/download/`;
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), timeout);
|
||||
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
headers: { Authorization: `Token ${token}` },
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(
|
||||
`Paperless download error: ${response.status} ${response.statusText}`,
|
||||
);
|
||||
}
|
||||
|
||||
return await response.arrayBuffer();
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
},
|
||||
|
||||
async updateDocumentTags(id, tagIds) {
|
||||
return request<PaperlessDocument>(`/documents/${id}/`, {
|
||||
method: "PATCH",
|
||||
body: JSON.stringify({ tags: [...tagIds] }),
|
||||
});
|
||||
},
|
||||
};
|
||||
}
|
||||
126
src/paperless/types.ts
Normal file
126
src/paperless/types.ts
Normal file
@ -0,0 +1,126 @@
|
||||
/**
|
||||
* TypeScript type definitions for the Paperless-ngx REST API.
|
||||
*
|
||||
* Based on Paperless-ngx API v3+.
|
||||
* @see https://docs.paperless-ngx.com/api/
|
||||
*/
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Pagination
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Generic paginated response envelope from Paperless-ngx. */
|
||||
export interface PaginatedResponse<T> {
|
||||
readonly count: number;
|
||||
readonly next: string | null;
|
||||
readonly previous: string | null;
|
||||
readonly results: readonly T[];
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Core entities
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface PaperlessDocument {
|
||||
readonly id: number;
|
||||
readonly correspondent: number | null;
|
||||
readonly document_type: number | null;
|
||||
readonly storage_path: number | null;
|
||||
readonly title: string;
|
||||
readonly content: string;
|
||||
readonly tags: readonly number[];
|
||||
readonly created: string;
|
||||
readonly created_date: string;
|
||||
readonly modified: string;
|
||||
readonly added: string;
|
||||
readonly archive_serial_number: number | null;
|
||||
readonly original_file_name: string;
|
||||
readonly archived_file_name: string | null;
|
||||
readonly owner: number | null;
|
||||
readonly notes: readonly DocumentNote[];
|
||||
readonly custom_fields: readonly CustomFieldValue[];
|
||||
}
|
||||
|
||||
export interface DocumentNote {
|
||||
readonly id: number;
|
||||
readonly note: string;
|
||||
readonly created: string;
|
||||
readonly user: number;
|
||||
}
|
||||
|
||||
export interface CustomFieldValue {
|
||||
readonly field: number;
|
||||
readonly value: string | number | boolean | null;
|
||||
}
|
||||
|
||||
export interface Correspondent {
|
||||
readonly id: number;
|
||||
readonly slug: string;
|
||||
readonly name: string;
|
||||
readonly match: string;
|
||||
readonly matching_algorithm: number;
|
||||
readonly is_insensitive: boolean;
|
||||
readonly document_count: number;
|
||||
readonly last_correspondence: string | null;
|
||||
}
|
||||
|
||||
export interface DocumentType {
|
||||
readonly id: number;
|
||||
readonly slug: string;
|
||||
readonly name: string;
|
||||
readonly match: string;
|
||||
readonly matching_algorithm: number;
|
||||
readonly is_insensitive: boolean;
|
||||
readonly document_count: number;
|
||||
}
|
||||
|
||||
export interface Tag {
|
||||
readonly id: number;
|
||||
readonly slug: string;
|
||||
readonly name: string;
|
||||
readonly color: string;
|
||||
readonly text_color: string;
|
||||
readonly match: string;
|
||||
readonly matching_algorithm: number;
|
||||
readonly is_insensitive: boolean;
|
||||
readonly is_inbox_tag: boolean;
|
||||
readonly document_count: number;
|
||||
}
|
||||
|
||||
export interface StoragePath {
|
||||
readonly id: number;
|
||||
readonly slug: string;
|
||||
readonly name: string;
|
||||
readonly path: string;
|
||||
readonly match: string;
|
||||
readonly matching_algorithm: number;
|
||||
readonly is_insensitive: boolean;
|
||||
readonly document_count: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Search & filter
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface DocumentSearchParams {
|
||||
readonly query?: string;
|
||||
readonly correspondent__id?: number;
|
||||
readonly document_type__id?: number;
|
||||
readonly tags__id__all?: readonly number[];
|
||||
readonly tags__id__none?: readonly number[];
|
||||
readonly created__date__gt?: string;
|
||||
readonly created__date__lt?: string;
|
||||
readonly ordering?: string;
|
||||
readonly page?: number;
|
||||
readonly page_size?: number;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// API client configuration
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface PaperlessConfig {
|
||||
readonly baseUrl: string;
|
||||
readonly token: string;
|
||||
readonly timeout?: number;
|
||||
}
|
||||
171
src/receipt/datev.ts
Normal file
171
src/receipt/datev.ts
Normal file
@ -0,0 +1,171 @@
|
||||
/**
|
||||
* DATEV export formatter.
|
||||
*
|
||||
* Generates DATEV-compatible CSV files for import into German accounting
|
||||
* software (DATEV Unternehmen Online, lexoffice, sevDesk, etc.).
|
||||
*
|
||||
* Implements the DATEV "Buchungsstapel" (posting batch) format v7.0+.
|
||||
*
|
||||
* @see https://developer.datev.de/datev/platform/en/dtvf/formate
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const exporter = createDatevExporter({ consultantNumber: 12345, clientNumber: 67890 });
|
||||
* const csv = exporter.generateCsv(receiptData);
|
||||
* writeFileSync("./export.csv", csv);
|
||||
* ```
|
||||
*/
|
||||
|
||||
import { stringify } from "csv-stringify/sync";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface DatevConfig {
|
||||
/** DATEV consultant number (Beraternummer). */
|
||||
readonly consultantNumber: number;
|
||||
/** DATEV client number (Mandantennummer). */
|
||||
readonly clientNumber: number;
|
||||
/** Fiscal year start (1-12, default: 1 for January). */
|
||||
readonly fiscalYearStart?: number;
|
||||
/** Default debit account length (SKR03/SKR04). */
|
||||
readonly accountLength?: 4 | 5;
|
||||
}
|
||||
|
||||
export interface DatevBookingEntry {
|
||||
readonly amount: number;
|
||||
readonly debitAccount: string;
|
||||
readonly creditAccount: string;
|
||||
readonly taxCode: string;
|
||||
readonly date: string;
|
||||
readonly description: string;
|
||||
readonly documentNumber: string;
|
||||
readonly costCenter?: string;
|
||||
}
|
||||
|
||||
export interface ReceiptForExport {
|
||||
readonly documentId: number;
|
||||
readonly vendor: string;
|
||||
readonly date: string;
|
||||
readonly totalAmount: number;
|
||||
readonly taxRate: number | null;
|
||||
readonly category: string | null;
|
||||
}
|
||||
|
||||
export interface DatevExporter {
|
||||
/** Generate DATEV CSV from receipt data. */
|
||||
generateCsv(receipts: readonly ReceiptForExport[]): string;
|
||||
|
||||
/** Map a receipt to a DATEV booking entry. */
|
||||
mapToBooking(receipt: ReceiptForExport): DatevBookingEntry;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Constants
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Map expense categories to SKR03 accounts.
|
||||
* TODO: Add SKR04 mapping support
|
||||
* TODO: Make configurable via user settings
|
||||
*/
|
||||
const SKR03_ACCOUNT_MAP: Record<string, string> = {
|
||||
office_supplies: "4930",
|
||||
travel: "4660",
|
||||
food: "4650",
|
||||
telephone: "4920",
|
||||
postage: "4910",
|
||||
insurance: "4360",
|
||||
rent: "4210",
|
||||
advertising: "4600",
|
||||
software: "4964",
|
||||
hardware: "4980",
|
||||
consulting: "4950",
|
||||
training: "4945",
|
||||
vehicle: "4500",
|
||||
default: "4900",
|
||||
};
|
||||
|
||||
/**
|
||||
* Map tax rates to DATEV tax codes (Steuerschluessel).
|
||||
*/
|
||||
const TAX_CODE_MAP: Record<number, string> = {
|
||||
19: "9", // 19% USt (standard)
|
||||
7: "8", // 7% USt (reduced)
|
||||
0: "0", // Tax-free
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Implementation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Create a DATEV-format exporter for receipt data.
|
||||
*
|
||||
* TODO: Implement DATEV header line with metadata (consultant, client, date range)
|
||||
* TODO: Add validation for account numbers against SKR03/SKR04
|
||||
* TODO: Support DATEV XML format (Buchungsdaten v5.0)
|
||||
*/
|
||||
export function createDatevExporter(config: DatevConfig): DatevExporter {
|
||||
const {
|
||||
consultantNumber: _consultantNumber,
|
||||
clientNumber: _clientNumber,
|
||||
fiscalYearStart: _fiscalYearStart = 1,
|
||||
accountLength: _accountLength = 4,
|
||||
} = config;
|
||||
|
||||
function mapToBooking(receipt: ReceiptForExport): DatevBookingEntry {
|
||||
const category = receipt.category ?? "default";
|
||||
const debitAccount =
|
||||
SKR03_ACCOUNT_MAP[category] ?? SKR03_ACCOUNT_MAP["default"];
|
||||
|
||||
const taxRate = receipt.taxRate ?? 19;
|
||||
const taxCode = TAX_CODE_MAP[taxRate] ?? TAX_CODE_MAP[19];
|
||||
|
||||
// Parse date to DD.MM format for DATEV
|
||||
const dateParts = receipt.date.split("-");
|
||||
const datevDate =
|
||||
dateParts.length === 3
|
||||
? `${dateParts[2]}${dateParts[1]}`
|
||||
: receipt.date;
|
||||
|
||||
return {
|
||||
amount: receipt.totalAmount,
|
||||
debitAccount,
|
||||
creditAccount: "1200", // Bank account (SKR03 default)
|
||||
taxCode,
|
||||
date: datevDate,
|
||||
description: receipt.vendor.slice(0, 60), // DATEV max 60 chars
|
||||
documentNumber: `PC-${receipt.documentId}`,
|
||||
costCenter: undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function generateCsv(receipts: readonly ReceiptForExport[]): string {
|
||||
const bookings = receipts.map(mapToBooking);
|
||||
|
||||
// DATEV Buchungsstapel columns
|
||||
const rows = bookings.map((b) => [
|
||||
b.amount.toFixed(2).replace(".", ","), // Umsatz (amount with comma)
|
||||
"S", // Soll/Haben (S = Soll/Debit)
|
||||
b.taxCode, // BU-Schluessel (tax code)
|
||||
b.debitAccount, // Gegenkonto (offset account)
|
||||
b.date, // Belegdatum (document date)
|
||||
b.documentNumber, // Belegfeld 1 (document number)
|
||||
"", // Belegfeld 2
|
||||
b.description, // Buchungstext (description)
|
||||
"", // Umsatzsteuer-ID
|
||||
b.creditAccount, // Konto (account)
|
||||
b.costCenter ?? "", // Kostenstelle (cost center)
|
||||
]);
|
||||
|
||||
return stringify(rows, {
|
||||
delimiter: ";",
|
||||
quoted: true,
|
||||
record_delimiter: "\r\n",
|
||||
});
|
||||
}
|
||||
|
||||
return { generateCsv, mapToBooking };
|
||||
}
|
||||
170
src/receipt/extractor.ts
Normal file
170
src/receipt/extractor.ts
Normal file
@ -0,0 +1,170 @@
|
||||
/**
|
||||
* Receipt data extraction using local LLM via Ollama.
|
||||
*
|
||||
* Extracts structured data from receipt documents: vendor, date, amounts,
|
||||
* tax breakdown, line items, and payment method. Uses the Paperless-ngx
|
||||
* OCR content and enriches it with LLM analysis.
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const extractor = createReceiptExtractor({ ollama, paperless });
|
||||
* const receipt = await extractor.extract(documentId);
|
||||
* console.log(receipt.vendor, receipt.totalAmount, receipt.taxAmount);
|
||||
* ```
|
||||
*/
|
||||
|
||||
import type { OllamaClient } from "../embeddings/ollama.js";
|
||||
import type { PaperlessClient } from "../paperless/client.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ReceiptData {
|
||||
readonly documentId: number;
|
||||
readonly vendor: string;
|
||||
readonly vendorAddress: string | null;
|
||||
readonly vendorTaxId: string | null;
|
||||
readonly date: string;
|
||||
readonly currency: string;
|
||||
readonly subtotal: number | null;
|
||||
readonly taxRate: number | null;
|
||||
readonly taxAmount: number | null;
|
||||
readonly totalAmount: number;
|
||||
readonly paymentMethod: string | null;
|
||||
readonly lineItems: readonly LineItem[];
|
||||
readonly category: string | null;
|
||||
readonly confidence: number;
|
||||
readonly rawText: string;
|
||||
}
|
||||
|
||||
export interface LineItem {
|
||||
readonly description: string;
|
||||
readonly quantity: number;
|
||||
readonly unitPrice: number;
|
||||
readonly totalPrice: number;
|
||||
readonly taxRate: number | null;
|
||||
}
|
||||
|
||||
export interface ReceiptExtractorConfig {
|
||||
readonly ollama: OllamaClient;
|
||||
readonly paperless: PaperlessClient;
|
||||
}
|
||||
|
||||
export interface ReceiptExtractor {
|
||||
/** Extract structured receipt data from a Paperless-ngx document. */
|
||||
extract(documentId: number): Promise<ReceiptData>;
|
||||
|
||||
/** Batch-extract receipts from multiple documents. */
|
||||
extractBatch(documentIds: readonly number[]): Promise<readonly ReceiptData[]>;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Prompts
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const EXTRACTION_SYSTEM_PROMPT = `You are a receipt data extraction assistant. Given the OCR text of a receipt, extract structured data in JSON format.
|
||||
|
||||
Extract the following fields:
|
||||
- vendor: Company/store name
|
||||
- vendorAddress: Full address if visible
|
||||
- vendorTaxId: Tax ID / VAT number if visible (e.g., USt-IdNr, Steuernummer)
|
||||
- date: Date in ISO 8601 format (YYYY-MM-DD)
|
||||
- currency: ISO 4217 currency code (e.g., EUR, USD)
|
||||
- subtotal: Amount before tax (null if not distinguishable)
|
||||
- taxRate: Tax percentage as decimal (e.g., 19 for 19%)
|
||||
- taxAmount: Tax amount
|
||||
- totalAmount: Total amount including tax
|
||||
- paymentMethod: Payment method if visible (cash, card, etc.)
|
||||
- lineItems: Array of { description, quantity, unitPrice, totalPrice, taxRate }
|
||||
- category: Suggested expense category (office_supplies, travel, food, etc.)
|
||||
- confidence: Your confidence in the extraction (0.0 to 1.0)
|
||||
|
||||
Respond ONLY with valid JSON. No explanation, no markdown.`;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Implementation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Create a receipt data extractor.
|
||||
*
|
||||
* TODO: Add support for image-based receipts (pass images to multimodal LLM)
|
||||
* TODO: Add receipt template matching for common vendors
|
||||
* TODO: Add currency conversion support
|
||||
*/
|
||||
export function createReceiptExtractor(
|
||||
config: ReceiptExtractorConfig,
|
||||
): ReceiptExtractor {
|
||||
const { ollama, paperless } = config;
|
||||
|
||||
async function extractSingle(documentId: number): Promise<ReceiptData> {
|
||||
// Fetch the document content from Paperless-ngx
|
||||
const document = await paperless.getDocument(documentId);
|
||||
const ocrText = document.content;
|
||||
|
||||
if (!ocrText || ocrText.trim().length === 0) {
|
||||
throw new Error(
|
||||
`Document ${documentId} has no OCR content. Ensure Paperless-ngx has processed the document.`,
|
||||
);
|
||||
}
|
||||
|
||||
// Send to Ollama for structured extraction
|
||||
const prompt = `Extract receipt data from the following OCR text:\n\n---\n${ocrText}\n---`;
|
||||
const completion = await ollama.complete(prompt, EXTRACTION_SYSTEM_PROMPT);
|
||||
|
||||
// Parse LLM response
|
||||
// TODO: Add robust JSON extraction (handle markdown code blocks, partial JSON)
|
||||
// TODO: Validate against Zod schema for type safety
|
||||
let parsed: Record<string, unknown>;
|
||||
try {
|
||||
parsed = JSON.parse(completion.text);
|
||||
} catch {
|
||||
throw new Error(
|
||||
`Failed to parse receipt extraction result for document ${documentId}. ` +
|
||||
`LLM response was not valid JSON.`,
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
documentId,
|
||||
vendor: String(parsed.vendor ?? "Unknown"),
|
||||
vendorAddress: parsed.vendorAddress ? String(parsed.vendorAddress) : null,
|
||||
vendorTaxId: parsed.vendorTaxId ? String(parsed.vendorTaxId) : null,
|
||||
date: String(parsed.date ?? new Date().toISOString().split("T")[0]),
|
||||
currency: String(parsed.currency ?? "EUR"),
|
||||
subtotal: typeof parsed.subtotal === "number" ? parsed.subtotal : null,
|
||||
taxRate: typeof parsed.taxRate === "number" ? parsed.taxRate : null,
|
||||
taxAmount: typeof parsed.taxAmount === "number" ? parsed.taxAmount : null,
|
||||
totalAmount: typeof parsed.totalAmount === "number" ? parsed.totalAmount : 0,
|
||||
paymentMethod: parsed.paymentMethod ? String(parsed.paymentMethod) : null,
|
||||
lineItems: Array.isArray(parsed.lineItems)
|
||||
? parsed.lineItems.map((item: Record<string, unknown>) => ({
|
||||
description: String(item.description ?? ""),
|
||||
quantity: Number(item.quantity ?? 1),
|
||||
unitPrice: Number(item.unitPrice ?? 0),
|
||||
totalPrice: Number(item.totalPrice ?? 0),
|
||||
taxRate: typeof item.taxRate === "number" ? item.taxRate : null,
|
||||
}))
|
||||
: [],
|
||||
category: parsed.category ? String(parsed.category) : null,
|
||||
confidence: typeof parsed.confidence === "number" ? parsed.confidence : 0.5,
|
||||
rawText: ocrText,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
extract: extractSingle,
|
||||
|
||||
async extractBatch(documentIds) {
|
||||
// TODO: Add concurrency control (process N at a time)
|
||||
// TODO: Add progress reporting callback
|
||||
const results: ReceiptData[] = [];
|
||||
for (const id of documentIds) {
|
||||
const result = await extractSingle(id);
|
||||
results.push(result);
|
||||
}
|
||||
return results;
|
||||
},
|
||||
};
|
||||
}
|
||||
231
src/receipt/matcher.ts
Normal file
231
src/receipt/matcher.ts
Normal file
@ -0,0 +1,231 @@
|
||||
/**
|
||||
* Bank CSV transaction matching for receipts.
|
||||
*
|
||||
* Matches extracted receipt data against bank CSV exports to reconcile
|
||||
* transactions. Supports common German bank export formats (Sparkasse,
|
||||
* Volksbank, ING, DKB).
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* const matcher = createTransactionMatcher();
|
||||
* const bankTxns = await matcher.parseBankCsv("./bank_export.csv");
|
||||
* const matches = matcher.matchReceipts(receipts, bankTxns);
|
||||
* ```
|
||||
*/
|
||||
|
||||
import { parse } from "csv-parse/sync";
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface BankTransaction {
|
||||
readonly date: string;
|
||||
readonly description: string;
|
||||
readonly amount: number;
|
||||
readonly currency: string;
|
||||
readonly iban: string | null;
|
||||
readonly bic: string | null;
|
||||
readonly reference: string | null;
|
||||
readonly rawLine: string;
|
||||
}
|
||||
|
||||
export interface ReceiptMatchCandidate {
|
||||
readonly documentId: number;
|
||||
readonly vendor: string;
|
||||
readonly date: string;
|
||||
readonly totalAmount: number;
|
||||
readonly currency: string;
|
||||
}
|
||||
|
||||
export interface MatchResult {
|
||||
readonly receipt: ReceiptMatchCandidate;
|
||||
readonly transaction: BankTransaction;
|
||||
readonly confidence: number;
|
||||
readonly matchReasons: readonly string[];
|
||||
}
|
||||
|
||||
export interface UnmatchedItem {
|
||||
readonly type: "receipt" | "transaction";
|
||||
readonly item: ReceiptMatchCandidate | BankTransaction;
|
||||
}
|
||||
|
||||
export interface MatchSummary {
|
||||
readonly matched: readonly MatchResult[];
|
||||
readonly unmatchedReceipts: readonly ReceiptMatchCandidate[];
|
||||
readonly unmatchedTransactions: readonly BankTransaction[];
|
||||
readonly matchRate: number;
|
||||
}
|
||||
|
||||
export interface TransactionMatcher {
|
||||
/** Parse a bank CSV export file into structured transactions. */
|
||||
parseBankCsv(filePath: string, format?: BankCsvFormat): readonly BankTransaction[];
|
||||
|
||||
/** Match receipts against bank transactions. */
|
||||
matchReceipts(
|
||||
receipts: readonly ReceiptMatchCandidate[],
|
||||
transactions: readonly BankTransaction[],
|
||||
): MatchSummary;
|
||||
}
|
||||
|
||||
export type BankCsvFormat = "auto" | "sparkasse" | "ing" | "dkb" | "volksbank" | "generic";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Implementation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Create a transaction matcher for bank CSV reconciliation.
|
||||
*
|
||||
* TODO: Add ML-based fuzzy matching for vendor names
|
||||
* TODO: Add support for MT940/CAMT.053 bank statement formats
|
||||
* TODO: Add date tolerance configuration (match within N days)
|
||||
*/
|
||||
export function createTransactionMatcher(): TransactionMatcher {
|
||||
/**
|
||||
* Parse bank CSV with auto-detected or specified format.
|
||||
*/
|
||||
function parseBankCsv(
|
||||
filePath: string,
|
||||
format: BankCsvFormat = "auto",
|
||||
): readonly BankTransaction[] {
|
||||
const raw = readFileSync(filePath, "utf-8");
|
||||
|
||||
// TODO: Implement format auto-detection based on header patterns
|
||||
// TODO: Add support for different CSV delimiters (semicolon for German exports)
|
||||
// TODO: Handle different date formats (DD.MM.YYYY, YYYY-MM-DD, MM/DD/YYYY)
|
||||
|
||||
const _format = format; // Acknowledge format parameter for future use
|
||||
|
||||
const records = parse(raw, {
|
||||
columns: true,
|
||||
skip_empty_lines: true,
|
||||
delimiter: ";",
|
||||
relaxColumnCount: true,
|
||||
}) as Record<string, string>[];
|
||||
|
||||
return records.map((record): BankTransaction => {
|
||||
// Generic column mapping -- override per format
|
||||
// TODO: Implement format-specific column mappings
|
||||
return {
|
||||
date: record["Buchungstag"] ?? record["Date"] ?? record["Datum"] ?? "",
|
||||
description:
|
||||
record["Verwendungszweck"] ??
|
||||
record["Description"] ??
|
||||
record["Buchungstext"] ??
|
||||
"",
|
||||
amount: parseFloat(
|
||||
(record["Betrag"] ?? record["Amount"] ?? "0")
|
||||
.replace(/\./g, "")
|
||||
.replace(",", "."),
|
||||
),
|
||||
currency: record["Waehrung"] ?? record["Currency"] ?? "EUR",
|
||||
iban: record["IBAN"] ?? null,
|
||||
bic: record["BIC"] ?? null,
|
||||
reference: record["Kundenreferenz"] ?? record["Reference"] ?? null,
|
||||
rawLine: JSON.stringify(record),
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Match receipts against bank transactions by amount and date proximity.
|
||||
*/
|
||||
function matchReceipts(
|
||||
receipts: readonly ReceiptMatchCandidate[],
|
||||
transactions: readonly BankTransaction[],
|
||||
): MatchSummary {
|
||||
const matched: MatchResult[] = [];
|
||||
const matchedReceiptIds = new Set<number>();
|
||||
const matchedTxnIndices = new Set<number>();
|
||||
|
||||
// TODO: Implement smarter matching with vendor name fuzzy matching
|
||||
// TODO: Add configurable date tolerance window
|
||||
// TODO: Handle split transactions (one receipt, multiple bank entries)
|
||||
|
||||
for (const receipt of receipts) {
|
||||
let bestMatch: { index: number; confidence: number; reasons: string[] } | null =
|
||||
null;
|
||||
|
||||
for (let i = 0; i < transactions.length; i++) {
|
||||
if (matchedTxnIndices.has(i)) continue;
|
||||
|
||||
const txn = transactions[i];
|
||||
const reasons: string[] = [];
|
||||
let confidence = 0;
|
||||
|
||||
// Amount matching (exact or close)
|
||||
const amountDiff = Math.abs(Math.abs(txn.amount) - receipt.totalAmount);
|
||||
if (amountDiff < 0.01) {
|
||||
confidence += 0.5;
|
||||
reasons.push("exact_amount_match");
|
||||
} else if (amountDiff < 1.0) {
|
||||
confidence += 0.3;
|
||||
reasons.push("close_amount_match");
|
||||
}
|
||||
|
||||
// Date matching
|
||||
const receiptDate = new Date(receipt.date).getTime();
|
||||
const txnDate = new Date(txn.date).getTime();
|
||||
const daysDiff = Math.abs(receiptDate - txnDate) / (1000 * 60 * 60 * 24);
|
||||
|
||||
if (daysDiff < 1) {
|
||||
confidence += 0.3;
|
||||
reasons.push("same_day");
|
||||
} else if (daysDiff < 3) {
|
||||
confidence += 0.15;
|
||||
reasons.push("within_3_days");
|
||||
} else if (daysDiff < 7) {
|
||||
confidence += 0.05;
|
||||
reasons.push("within_7_days");
|
||||
}
|
||||
|
||||
// Vendor name in description
|
||||
if (
|
||||
txn.description
|
||||
.toLowerCase()
|
||||
.includes(receipt.vendor.toLowerCase().slice(0, 8))
|
||||
) {
|
||||
confidence += 0.2;
|
||||
reasons.push("vendor_in_description");
|
||||
}
|
||||
|
||||
if (
|
||||
confidence > 0.5 &&
|
||||
(!bestMatch || confidence > bestMatch.confidence)
|
||||
) {
|
||||
bestMatch = { index: i, confidence, reasons };
|
||||
}
|
||||
}
|
||||
|
||||
if (bestMatch) {
|
||||
matched.push({
|
||||
receipt,
|
||||
transaction: transactions[bestMatch.index],
|
||||
confidence: bestMatch.confidence,
|
||||
matchReasons: bestMatch.reasons,
|
||||
});
|
||||
matchedReceiptIds.add(receipt.documentId);
|
||||
matchedTxnIndices.add(bestMatch.index);
|
||||
}
|
||||
}
|
||||
|
||||
const unmatchedReceipts = receipts.filter(
|
||||
(r) => !matchedReceiptIds.has(r.documentId),
|
||||
);
|
||||
const unmatchedTransactions = transactions.filter(
|
||||
(_, i) => !matchedTxnIndices.has(i),
|
||||
);
|
||||
|
||||
return {
|
||||
matched,
|
||||
unmatchedReceipts,
|
||||
unmatchedTransactions,
|
||||
matchRate:
|
||||
receipts.length > 0 ? matched.length / receipts.length : 0,
|
||||
};
|
||||
}
|
||||
|
||||
return { parseBankCsv, matchReceipts };
|
||||
}
|
||||
72
src/skill/SKILL.md
Normal file
72
src/skill/SKILL.md
Normal file
@ -0,0 +1,72 @@
|
||||
# PaperCortex -- Document Intelligence Skill
|
||||
|
||||
> A Claude Code skill for interacting with your Paperless-ngx document archive through AI-powered semantic search, classification, receipt extraction, and accounting export.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- PaperCortex MCP Server running (see project README)
|
||||
- Paperless-ngx instance with API access
|
||||
- Ollama with `qwen2.5:14b` and `nomic-embed-text` models
|
||||
|
||||
## Available Tools
|
||||
|
||||
### papercortex_search
|
||||
Search documents by meaning, not just keywords.
|
||||
|
||||
```
|
||||
Search for: "office lease agreements from last year"
|
||||
Search for: "tax-relevant receipts over 500 EUR"
|
||||
Search for: "correspondence with insurance companies"
|
||||
```
|
||||
|
||||
### papercortex_classify
|
||||
Auto-classify a document with AI-suggested tags, type, and correspondent.
|
||||
|
||||
```
|
||||
Classify document #1234
|
||||
Classify document #1234 and apply suggested tags
|
||||
```
|
||||
|
||||
### papercortex_receipt
|
||||
Extract structured data from receipt documents.
|
||||
|
||||
```
|
||||
Extract receipt from document #5678
|
||||
```
|
||||
|
||||
Returns: vendor, date, amounts, tax breakdown, line items, category.
|
||||
|
||||
### papercortex_query
|
||||
Ask natural language questions about your document archive.
|
||||
|
||||
```
|
||||
"How much did I spend on office supplies in Q1 2024?"
|
||||
"Which invoices are still unpaid?"
|
||||
"Summarize all contracts expiring this year"
|
||||
```
|
||||
|
||||
### papercortex_export
|
||||
Export receipt data for accounting software.
|
||||
|
||||
```
|
||||
Export documents #100, #101, #102 as DATEV CSV
|
||||
Export documents #200, #201 as generic CSV
|
||||
```
|
||||
|
||||
## Workflow Examples
|
||||
|
||||
### Monthly Bookkeeping
|
||||
1. Search for all receipts from the current month
|
||||
2. Extract data from each receipt
|
||||
3. Export as DATEV CSV
|
||||
4. Import into accounting software
|
||||
|
||||
### Document Organization
|
||||
1. Find unclassified documents (no tags)
|
||||
2. Auto-classify each document
|
||||
3. Review and approve suggested tags
|
||||
|
||||
### Expense Analysis
|
||||
1. Query: "What were my top 5 expense categories last quarter?"
|
||||
2. Drill into specific categories with follow-up queries
|
||||
3. Export relevant receipts for documentation
|
||||
24
tsconfig.json
Normal file
24
tsconfig.json
Normal file
@ -0,0 +1,24 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"lib": ["ES2022"],
|
||||
"outDir": "./dist",
|
||||
"rootDir": "./src",
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"resolveJsonModule": true,
|
||||
"declaration": true,
|
||||
"declarationMap": true,
|
||||
"sourceMap": true,
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true,
|
||||
"noImplicitReturns": true,
|
||||
"noFallthroughCasesInSwitch": true
|
||||
},
|
||||
"include": ["src/**/*"],
|
||||
"exclude": ["node_modules", "dist", "**/*.test.ts"]
|
||||
}
|
||||
Loading…
x
Reference in New Issue
Block a user