AI

Looking for AI assistance while developing your app? Niral DI runs in your terminal with your chosen provider and approval-gated workspace tools. This guide covers the separate server-side AI features used inside your application.

Niral speaks the OpenAI-compatible wire format — point it at OpenAI, Azure, Ollama, vLLM, llama.cpp or your own model:

NIRAL_AI_URL=http://localhost:11434/v1   # e.g. Ollama
NIRAL_AI_MODEL=llama3.2
NIRAL_AI_KEY=…                           # cloud providers

The ambient ai

const text = await ai.chat("explain closures", { system: "be brief" })
const data = await ai.chat(prompt, { json: true })        // parsed JSON output
const [vec] = await ai.embed(["some text"])               // embeddings

export async function* ask(messages) {                     // token streaming
  for await (const chunk of ai.stream(messages)) yield chunk
}

Streaming server functions pair with streaming RPC — the client consumes tokens with for await. niral add chat scaffolds a complete streaming chat page.

RAG — retrieval built in

await rag.ingest(handbookText, { source: "handbook.md" })
const hits = await rag.search("refund policy", { k: 5 })
const answer = await ai.chat(
  "Answer from the context:\n" + hits.map((h) => h.content).join("\n---\n") +
  "\n\nQuestion: " + q
)

Chunks and embeddings live in data/rag.db — they survive deploys.

Fully local

niral add llm          # official llama.cpp server binary, one download

Run a GGUF model on localhost, point NIRAL_AI_URL at it — every AI feature works offline, no keys, no cloud.