Platform

Private AI

New Dialogue AI is sovereign AI built for confidential computing, edge-to-cloud inference, multi-agent orchestration, and inference economics that actually deliver ROI.

New Dialogue allows enterprises to deploy leading open-source and open-weight LLMs, including our own agentic framework directly into your own cloud tenancy (AWS or Azure) or air-gapped on-prem Kubernetes clusters. Your data, your models, your infrastructure. Nothing touches a shared SaaS tenancy.

New Dialogue also exists in a public AWS environment utilizing Amazon Bedrock. Start a free 7-day trial which enables you to evaluate the full platform within this environment.

Why does a Private AI Matter?

It's unclear what public AI services are training their models on your knowledge. These public LLM's leak your IP. Shared infrastructure creates compliance nightmares and legacy on-prem solutions can’t keep up with frontier model velocity.

New Dialogue changes these scenarios.

We deploy production-grade, continuously batched inference stacks that run Mistral or OSS-20B by default entirely within your security boundary.

New Dialogue AI architecture

Layer	What’s Implemented Today	Why It Matters
Models	Claude Opus, Sonnet, Haiku, GPT-OSS-20B & 120B, Llama 3.x, Mixtral AWQ.	Saves cost, reduces latency, improves quality without blind retries.
Inference Engine	AWS Bedrock + self-hosted OSS-20b endpoint, vLLM	vLLM boosts GPU throughput 3-5×, but only useful if self-hosting grows.
RAG Pipeline	Hybrid vector + keyword search. OpenNLP chunking. Tenant + role filtering. Multi-tenant Weaviate isolation.	Reranking = immediate answer quality gain. GraphRAG only when structurally needed.
Agents	Agentic framework. Workflow orchestrator. Parallel + sequential steps. LLM routing with confidence scoring.	Prevents irreversible mistakes. Improves trust, UX, and personalization.
Scheduling	Immediate + scheduled execution. LLM scheduling intent extraction. Task state persistence.	Cron = power-user flexibility. Retry + DLQ prevents silent failures when APIs fail temporarily.
Integrations / Tools	Native integrations include Google Workspace, Slack, Microsoft Teams, SharePoint, Xero, MCP (HTTP + stdio).	MCP is major leverage: new tools can be added instantly. Platform value scales with integrations.
Web Search	Custom Search + Spider Cloud scraping.	Better summarization, fewer calls, lower cost, higher reliability. Fallback prevents outages. Caching avoids duplicate paid queries.
Orchestration	Spring Boot single-service, Docker Compose.	Avoid premature complexity; scale infrastructure when SLA/user count grows.
Security	AES-256 encryption (CBC new / ECB legacy). JWT (ES256). OAuth2 (Google). RBAC for docs. Multi-tenant DB + vector DB isolation.	Reduces breach risk, improves compliance, protects LLM tools from abuse. Vault enables secret rotation without redeploy and avoids storing secrets in git.
Multi-tenancy	Hibernate discriminator tenant isolation. Tenant-per-user. Tenant ID propagated via SecurityContext.	Isolation exists - but no visibility. Metering enables billing, quota control, and noisy-tenant protection.
Observability	SLF4J, Agent + tool error logging. OpenTelemetry tracing.	Distributed tracing shows exact latency sources.

Key Capabilities

Agentic Sovereignty

Build and deploy custom agents that securely call internal APIs, connect privately to your internal knowledge, query structured databases, trigger workflows, and maintain long-term private memory, all without ever phoning home.

Confidential RAG 2.0

Your documents, policies and contracts live in encrypted vector + knowledge graphs. Agents retrieve, reason, and act with source-grounded confidence scores.

Inference Economics That Scale

Run 14B & 20B models on a fraction of the GPUs others need, thanks to quantization and intelligent routing between Small Language Models (SLMs) and large models.

True Air-Gap Ready

One-click export to disconnected environments. Models, weights, vector stores, and agent definitions ship as signed OCI artifacts.

Enterprise Guardrails Built-In

Zero data retention by default
IAM/SCIM + SSO integration
Full audit logs with immutable provenance
Redaction & content moderation layers that run locally

Open Source Core. Enterprise Grade

New Dialogue is built on the best open-source foundations, then hardened with enterprise security, observability, and one-click deployment templates. We deliver in moments what would take most teams 6–9 months to build, test, and deploy with confidence.

Blog