Vector Search

CongraphDB includes built-in support for vector similarity search using the HNSW (Hierarchical Navigable Small World) algorithm. This makes it ideal for AI/ML applications that work with embeddings.

VectorStore API (v0.2.1+)

For high-performance vector operations, CongraphDB provides a specialized VectorStore API inspired by LanceDB. It features native parallel ingestion and batch search optimizations.

const { createVectorStore } = require('congraphdb');

const store = createVectorStore('./ai-data.cgraph');
await store.init();

// Create table with optimized HNSW defaults
await store.createTable('Fragments', {
  vectorProperty: 'embedding',
  vectorDim: 1536,
  properties: { text: 'string', source: 'string' }
});

// High-speed parallel ingestion (297x faster than individual inserts)
await store.addBulk('Fragments', largeBatchOfEmbeddings);

// Batch similarity search - process multiple queries in parallel
const results = await store.searchBatch('Fragments', [vec1, vec2, vec3]);

Batch Vector Performance

The HNSW insert_batch_at method uses a two-phase parallel approach:

Phase A - Parallel Greedy Search (Read Phase) - Uses Rayon's into_par_iter() for parallel top-to-bottom search - All vectors in the batch find their entry points simultaneously - Only requires read access to the existing graph - Random level assignment happens in parallel

Phase B - Batched Insertion (Write Phase) - Single write lock acquisition for the entire batch - Bulk update of next_id and max_level - Sequential insertion within the batch (acceptable since search is already done)

Performance Results: - 500 vectors × 128 dimensions: ~4.6 seconds total - Individual inserts (baseline): ~7.73ms per insertion - Batch inserts (optimized): ~0.026ms per insertion (297x faster)

Search quality is maintained - batch insert produces comparable nearest neighbor results to individual inserts.

What are Embeddings?

Embeddings are numerical representations of data (text, images, audio) that capture semantic meaning. Similar items have similar embeddings.

Creating a Vector Column

Define a table with a FLOAT_VECTOR column:

await conn.query(`
  CREATE NODE TABLE Document(
    id STRING,
    content STRING,
    embedding FLOAT_VECTOR[128],
    PRIMARY KEY (id)
  )
`);

Inserting Vectors

// Assuming you have a function to generate embeddings
async function getEmbedding(text) {
  // Call your embedding model (OpenAI, local model, etc.)
  // Returns an array of 128 floats
  return [/* ...128 floats... */];
}

const embedding = await getEmbedding("Hello, world!");

await conn.query(`
  CREATE (d:Document {
    id: 'doc1',
    content: 'Hello, world!',
    embedding: $vec
  })
`, { vec: embedding });

Vector Similarity Search

Use the <-> operator for cosine similarity (recommended):

const queryEmbedding = await getEmbedding("greetings");

const result = await conn.query(`
  MATCH (d:Document)
  RETURN d.id, d.content, d.embedding <-> $query AS distance
  ORDER BY distance
  LIMIT 5
`, { query: queryEmbedding });

for (const row of result.getAll()) {
  console.log(`${row.content} (distance: ${row.distance})`);
}

Distance Operators

Operator	Description	Use Case
<->	Cosine distance	Recommended for embeddings
<=>	Euclidean distance (L2)	General purpose
<=	Negative inner product	Some embedding models

Indexing Vectors

For large datasets, create an HNSW index for faster search:

await conn.query(`
  CREATE HNSW INDEX ON Document(embedding, dim=128, M=16)
`);

HNSW Parameters

Parameter	Description	Default	Recommendation
dim	Vector dimension	-	Must match your column
M	Max connections per node	16	Higher = more accurate, slower
ef_construction	Build-time candidates	100	Higher = better quality, slower build

Complete Example

const { Database } = require('congraphdb');

async function semanticSearchExample() {
  const db = new Database('./semantic-search.cgraph');
  db.init();
  const conn = db.createConnection();

  // Create schema with vector column
  await conn.query(`
    CREATE NODE TABLE Document(
      id STRING,
      title STRING,
      content STRING,
      embedding FLOAT_VECTOR[384],
      PRIMARY KEY (id)
    )
  `);

  // Create HNSW index for fast search
  await conn.query(`
    CREATE HNSW INDEX ON Document(embedding, dim=384, M=16)
  `);

  // Insert documents with embeddings
  const docs = [
    { id: '1', title: 'Machine Learning', content: 'Introduction to ML algorithms' },
    { id: '2', title: 'Deep Learning', content: 'Neural networks and backpropagation' },
    { id: '3', title: 'Natural Language', content: 'Text processing and transformers' },
  ];

  for (const doc of docs) {
    const embedding = await getEmbedding(doc.content); // Your embedding function
    await conn.query(`
      CREATE (d:Document {
        id: $id,
        title: $title,
        content: $content,
        embedding: $embedding
      })
    `, { id: doc.id, title: doc.title, content: doc.content, embedding });
  }

  // Semantic search
  const query = "how do neural networks learn";
  const queryEmbedding = await getEmbedding(query);

  const result = await conn.query(`
    MATCH (d:Document)
    RETURN d.title, d.content, d.embedding <-> $query AS distance
    ORDER BY distance
    LIMIT 3
  `, { query: queryEmbedding });

  console.log('Search results for:', query);
  for (const row of result.getAll()) {
    console.log(`- ${row.title}: ${row.content}`);
  }

  db.close();
}

Use Cases

• Semantic Search — Find documents by meaning, not keywords
• Recommendation Systems — Similar items, collaborative filtering
• Image Search — Find visually similar images
• Anomaly Detection — Find outliers based on distance
• Deduplication — Find near-duplicate records

Tips

1. Normalize embeddings — Use L2 normalization for best results with cosine distance
2. Batch insertions — Insert all documents before creating the index for faster builds
3. Dimension choice — Lower dimensions (128-384) are faster; higher (768-1536) are more accurate
4. Index tuning — Start with default HNSW parameters, tune based on your data

Next Steps

• Performance — Optimization tips
• Internals — How HNSW works