Building a Sub-Millisecond Offline OSV Vulnerability Database Cache in Rust & Node.js

The Latency Paradox in Developer Security

In modern software engineering, developer tools are judged by a single ruthless metric: latency. A security scanner that adds 3 seconds to every npm install command will inevitably be disabled by developers. Conversely, a tool that relies exclusively on remote HTTP API queries for every dependency check introduces network flakiness, latency spikes, and severe privacy risks (exposing private internal package names to external cloud servers).

To achieve zero-overhead security for local CLI terminals and AI coding agents, security checks must execute in under 1 millisecond (< 1ms). At this speed, security becomes completely imperceptible to the engineer.

Achieving sub-millisecond execution requires an offline-first local database cache combined with a high-performance systems language like Rust. In this article, we break down the architecture of Agentinel's local vulnerability caching engine using Rust, SQLite/LMDB, and Node.js Native Binding (NAPI-RS).

Why Cloud API Queries Are Too Slow

Consider the network lifecycle of a traditional cloud-based dependency scanner:

Developer issues npm install lodash.
Tool intercepts command and sends HTTP POST request to https://api.security-vendor.com/v1/scan.
DNS resolution + TLS 1.3 handshake: ~35ms.
Remote server database query & response serialization: ~80ms.
Client HTTP payload parsing: ~10ms.
Total Overhead: ~125ms to 300ms per invocation.

In contrast, a local zero-copy disk query against an indexed, memory-mapped database executes in 0.15ms (150 microseconds)—over 800x faster than a cloud API lookup.

Architectural Design: Memory-Mapped Files (LMDB/SQLite)

To achieve microsecond read latency, Agentinel utilizes a hybrid storage model:

Lightning Memory-Mapped Database (LMDB / libmdbx): Used for ultra-fast Key-Value lookups where Key = hash(pkg_name + version) and Value = compact_binary_security_flags.
SQLite (WAL Mode): Used for complex range queries and CVE vulnerability details.

Memory-Mapped Virtual Memory (VFS)

LMDB maps the database file directly into the process's virtual address space using mmap(). When Agentinel queries a package hash, the Operating System kernel handles page caching. Subsequent reads fetch data directly from RAM without context switching or system call overhead.

// Agentinel Rust Local Cache Lookup Engine
use lmdb::{Environment, Geometry, Transaction};
use std::path::Path;
 
pub struct LocalVulnerabilityCache {
    env: Environment,
}
 
impl LocalVulnerabilityCache {
    pub fn new(db_path: &Path) -> Self {
        let env = Environment::new()
            .set_max_dbs(2)
            .set_map_size(104857600) // 100MB Memory Map
            .open(db_path)
            .expect("Failed to open LMDB environment");
 
        LocalVulnerabilityCache { env }
    }
 
    pub fn is_package_vulnerable(&self, pkg_hash: &[u8]) -> Result<bool, lmdb::Error> {
        let db = self.env.open_db(None)?;
        let txn = self.env.begin_ro_txn()?;
        
        match txn.get(db, &pkg_hash) {
            Ok(bytes) => {
                // Byte 0: 0x01 = Vulnerable, 0x00 = Clean
                Ok(bytes[0] == 1)
            }
            Err(lmdb::Error::NotFound) => Ok(false),
            Err(e) => Err(e),
        }
    }
}

Incremental Data Sync & OIDC Verification

An offline database is only as good as its freshness. To maintain zero-latency local reads while ensuring up-to-the-minute threat coverage:

Delta Updates: Agentinel downloads small, compressed binary diffs (containing newly reported OSV advisories) in the background once every 24 hours.
Atomic Swap: DB updates are written to a secondary file and atomically swapped using renameat2(RENAME_EXCHANGE) to prevent database locks or corruption during active terminal sessions.
Cryptographic Verification: Every dataset delta is signed via OIDC identity tokens to guarantee data authenticity before merging into the local cache.

Benchmarking Performance: Rust vs Node.js vs Cloud Query

| :--- | :--- | :--- | :--- |

| Cloud HTTP REST API | 140.0 ms | 450.0 ms | Required |

| Node.js JSON Disk Read | 4.2 ms | 12.5 ms | Offline |

| SQLite WAL Indexed Query | 0.8 ms | 1.9 ms | Offline |

| Agentinel Rust + LMDB (mmap) | 0.12 ms | 0.35 ms | Offline |

Conclusion & Integration into Developer Tooling

Designing security tools with an offline-first, low-latency mindset transforms developer adoption. Security ceases to be an annoying bottleneck and becomes an invisible, automatic guardrail.

For more technical breakdowns on building sub-millisecond developer infrastructure, follow the updates on blog.habitwala.in.