132 lines
3.6 KiB
Markdown
132 lines
3.6 KiB
Markdown
# IdentityDB
|
|
|
|
IdentityDB is a TypeScript package for building structured AI memory on top of relational databases.
|
|
|
|
## What it is
|
|
|
|
IdentityDB stores memory as a graph made of:
|
|
|
|
- **Topics** — named nodes such as `TypeScript`, `programming language`, `2025`, or `I`
|
|
- **Facts** — statements that connect multiple topics
|
|
- **Fact-topic links** — the relationships that turn one fact into a bridge between many topics
|
|
|
|
A single fact like `I have worked with TypeScript since 2025.` can connect the topics `I`, `TypeScript`, and `2025` at the same time.
|
|
|
|
## Current capabilities
|
|
|
|
- SQLite, PostgreSQL, MySQL, and MariaDB connection adapters
|
|
- Automatic schema initialization for `topics`, `facts`, `fact_topics`, `topic_relations`, `topic_aliases`, and `fact_embeddings`
|
|
- High-level APIs for adding topics and facts
|
|
- Topic hierarchy APIs for parent/child traversal and lineage lookup
|
|
- Topic alias and canonical resolution APIs so facts and queries can resolve alternate names
|
|
- Semantic fact indexing and search APIs built around provider-agnostic embeddings
|
|
- Dedup-aware ingestion hooks that can reuse an existing fact when a semantic near-duplicate is detected
|
|
- Pluggable fact extraction so callers can use a small LLM or a deterministic extractor
|
|
|
|
## Install
|
|
|
|
```bash
|
|
bun install
|
|
```
|
|
|
|
## Quick start
|
|
|
|
```ts
|
|
import { IdentityDB, NaiveExtractor, type EmbeddingProvider } from 'identitydb';
|
|
|
|
const db = await IdentityDB.connect({
|
|
client: 'sqlite',
|
|
filename: ':memory:',
|
|
});
|
|
|
|
await db.initialize();
|
|
|
|
await db.ingestStatement('I have worked with TypeScript since 2025.', {
|
|
extractor: new NaiveExtractor(),
|
|
});
|
|
|
|
await db.addFact({
|
|
statement: 'TypeScript is a programming language.',
|
|
topics: [
|
|
{
|
|
name: 'TypeScript',
|
|
category: 'entity',
|
|
granularity: 'concrete',
|
|
},
|
|
{
|
|
name: 'programming language',
|
|
category: 'concept',
|
|
granularity: 'abstract',
|
|
},
|
|
],
|
|
});
|
|
|
|
await db.linkTopics({
|
|
parentName: 'programming language',
|
|
childName: 'TypeScript',
|
|
});
|
|
|
|
await db.addTopicAlias('TypeScript', 'TS');
|
|
|
|
const provider: EmbeddingProvider = {
|
|
model: 'example-embedding-v1',
|
|
dimensions: 3,
|
|
async embed(input) {
|
|
if (input.toLowerCase().includes('typescript')) {
|
|
return [1, 0, 0];
|
|
}
|
|
|
|
return [0, 1, 0];
|
|
},
|
|
};
|
|
|
|
await db.indexFactEmbeddings({ provider });
|
|
|
|
const topic = await db.getTopicByName('TS', { includeFacts: true });
|
|
const children = await db.getTopicChildren('programming language');
|
|
const lineage = await db.getTopicLineage('TS');
|
|
const connected = await db.findConnectedTopics('TypeScript');
|
|
const matches = await db.searchFacts({
|
|
query: 'TypeScript experience',
|
|
provider,
|
|
limit: 5,
|
|
});
|
|
|
|
console.log(topic?.name);
|
|
console.log(children.map((entry) => entry.name));
|
|
console.log(lineage.map((entry) => entry.name));
|
|
console.log(connected.map((entry) => [entry.name, entry.sharedFactCount]));
|
|
console.log(matches.map((entry) => [entry.statement, entry.score]));
|
|
|
|
await db.close();
|
|
```
|
|
|
|
## Semantic ingestion and duplicate detection
|
|
|
|
If you provide an embedding provider during ingestion, IdentityDB can index the new fact automatically and reuse an existing fact when a semantic near-duplicate is already present.
|
|
|
|
```ts
|
|
await db.ingestStatement('Bun makes TypeScript tooling fast.', {
|
|
extractor: new NaiveExtractor(),
|
|
embeddingProvider: provider,
|
|
duplicateThreshold: 0.95,
|
|
});
|
|
```
|
|
|
|
## Development
|
|
|
|
```bash
|
|
bun run test
|
|
bun run check
|
|
bun run build
|
|
```
|
|
|
|
## Current status
|
|
|
|
This repository is in active MVP expansion development.
|
|
|
|
See these implementation plans for the current roadmap:
|
|
|
|
- `docs/plans/2026-05-11-identitydb-foundation.md`
|
|
- `docs/plans/2026-05-11-identitydb-memory-expansion.md`
|