Agent Multi-Tenancy Design: Tenant Isolation, Resource Quotas, and Cost Allocation

Background & Challenges

When an enterprise SaaS platform evolves from single-tenant Agents to multi-tenancy, the core challenge is no longer "how to run an Agent," but "how to securely, fairly, and metrically run Agents for hundreds of tenants." Traditional web multi-tenancy approaches (like shared databases with row-level security) fall short in the Agent context—because Agents involve not just data reads/writes, but also entirely new resource dimensions such as LLM calls, tool executions, context windows, and memory storage.

A typical Agent request triggers multiple LLM calls, several tool executions, potentially reads/writes to tenant-specific vector databases, and maintains state in short-term memory. The complexity of this call chain means that if tenant isolation only exists at the data layer, one tenant's Agent could easily impact others through tool calls or shared model quotas. More critically, cost is no longer a fixed server bill but highly dynamic token consumption—without fine-grained quotas and allocation, it's financially impossible to bill tenants accurately.

Therefore, Agent multi-tenancy design must answer three questions: How to isolate? (Data, execution, network, identity) How to quota? (Resources, calls, costs) How to allocate? (Cost attribution, billing, chargeback). This article will elaborate on these three aspects, providing a set of actionable architectures and practices.

Core Architecture Design

Drawing from practices in the Google Cloud Architecture Center and Zylos Research, a multi-tenant Agent platform should adopt a Three-Zone Architecture:

• Control Plane (Shared): Tenant registration, authentication, quota policy enforcement, audit log collection. This plane is fully shared, but all operations carry a tenant_id.
• Orchestration Layer (Tenant-Level Isolation): Agent instances, workflow engines, tool registries. Each tenant has an independent namespace or Kubernetes Namespace, running their own Agent orchestrators.
• Data Plane (Partitioned by Tenant): Vector databases, object storage, caches, session history. Data is strictly partitioned by tenant_id, using schema-per-tenant or bucket-per-tenant.

The tenant identifier (tenant_id) must permeate every layer of the architecture—from authentication tokens to LLM call metadata, to storage partition keys. Shared infrastructure (like the LLM gateway, monitoring systems) achieves multi-tenant reuse via tenant_id, while isolation boundaries ensure one tenant's failure or malicious activity doesn't affect others.

Balancing sharing and isolation is key: complete isolation (dedicated cluster per tenant) is too costly, while complete sharing loses security boundaries. The recommended approach is a shared control plane + isolated orchestration and data planes, which is also the choice of most enterprise platforms.

Tenant Isolation Strategies

Isolation is the cornerstone of multi-tenancy and must be implemented across four dimensions:

Sandbox selection involves a trade-off between isolation strength and startup time: Firecracker MicroVMs offer hardware-level isolation but take ~125ms to start; gVisor starts faster (~50ms) but offers slightly weaker isolation; containers are fastest but share the kernel. For multi-tenant Agents, it's recommended to use at least gVisor, and MicroVMs for financial-grade scenarios.

Resource Quota Management

Quota management must be implemented at two levels: the infrastructure layer and the Agent orchestration layer.

Infrastructure Layer (Kubernetes): Use ResourceQuota and LimitRange to set hard limits for each tenant namespace. Here's an example quota for a tenant namespace:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: tenant-quota
  namespace: tenant-acme
spec:
  hard:
    requests.cpu: "10"
    requests.memory: 16Gi
    limits.cpu: "20"
    limits.memory: 32Gi
    persistentvolumeclaims: "5"
---
apiVersion: v1
kind: LimitRange
metadata:
  name: tenant-limit-range
  namespace: tenant-acme
spec:
  limits:
  - default:
      cpu: "500m"
      memory: 512Mi
    defaultRequest:
      cpu: "100m"
      memory: 128Mi
    max:
      cpu: "2"
      memory: 4Gi
    type: Container
        

Agent Orchestration Layer: Limit the number of concurrent Agent instances, LLM calls per minute, and maximum tokens per session at the application layer. This is typically implemented via an API gateway or Agent orchestrator. For example, using a Redis counter for sliding window rate limiting:

# Python pseudocode: Tenant-level LLM call rate limiting
import redis
import time

r = redis.Redis(host='redis', port=6379)

def check_quota(tenant_id: str, limit_per_min: int = 100) -> bool:
    key = f"quota:{tenant_id}:{int(time.time() // 60)}"
    current = r.incr(key)
    if current == 1:
        r.expire(key, 60)
    return current <= limit_per_min
        

Overage Enforcement: When a tenant's pre-allocated quota is exhausted, new Agent deployments should be blocked. Running Agents may be allowed to finish, but new tasks should be rejected. This requires integration between the quota system and CI/CD gates.

Cost Allocation & Billing

Cost allocation is crucial for financial sustainability. The core idea is to inject tenant context at the LLM gateway layer to achieve per-call cost attribution.

Using Portkey or a self-built gateway as an example, each LLM request carries tenant_id in its metadata:

# LLM Gateway call example (pseudocode)
import requests

def call_llm(tenant_id: str, prompt: str, model: str):
    response = requests.post(
        "https://llm-gateway.internal/v1/chat/completions",
        headers={"Authorization": "Bearer sk-..."},
        json={
            "model": model,
            "messages": [{"role": "user", "content": prompt}],
            "metadata": {
                "tenant_id": tenant_id,
                "agent_id": "customer-support-v2"
            }
        }
    )
    return response.json()
        

The gateway records the token count and model unit price for each call, aggregating by tenant_id. Suitable cost tracking toolchains include:

• Langfuse: Open-source LLM observability, supports tracking costs by tenant tags.
• Portkey: Gateway with built-in cost analysis and budget alerts.
• Amberflo: Real-time metering and billing platform, suitable for complex chargeback models.

For the Chargeback model, a combined billing approach is recommended: based on Token consumption (base) + Agent instance count (fixed) + tool call count (additional). Each tenant should have self-service access to a cost dashboard for daily/weekly/monthly analysis.

Performance & Security

Performance Isolation: Avoiding the "noisy neighbor" problem is a direct goal of quota management. Kubernetes ResourceQuota ensures CPU/Memory limits, but also pay attention to LLM API rate limits—if multiple tenants share the same OpenAI deployment, one tenant's traffic spike could trigger 429 errors affecting others. Solutions include using Azure OpenAI's Provisioned Throughput or configuring dedicated deployments for high-priority tenants.

Security Hardening: Sandbox escape is the biggest risk. Layered mitigation measures include:

• Disable host IPC and PID namespace sharing within the sandbox.
• Use a read-only root filesystem, with /tmp as tmpfs.
• Network egress allowlisting, permitting access only to necessary APIs.
• Regularly scan sandbox images for vulnerabilities, use distroless base images.

Observability: All monitoring metrics and logs must carry tenant_id. Partition Prometheus metrics by tenant labels, index logs with tenant fields, and record every cross-tenant access attempt in audit logs. This is not only a security requirement but also the basis for arbitrating cost disputes.

Enterprise Implementation

The path from single-tenant to multi-tenant: Step 1, introduce a tenant_id field into existing systems, tagging all data tables and call chains with tenant identifiers. Step 2, containerize the Agent execution environment and migrate to a Kubernetes cluster with per-tenant namespaces. Step 3, implement cost attribution at the LLM gateway layer. Step 4, integrate with existing IAM and billing systems.

For cost chargeback across internal business units, a "fixed fee + variable fee" model is recommended. Fixed fees cover shared infrastructure costs (like the control plane), while variable fees are calculated based on actual token consumption and Agent instance count. Finance departments need to export per-tenant bills monthly and reconcile them with cloud provider invoices.

When integrating with existing systems, focus on: SSO Integration (mapping enterprise IdP to tenants), Audit Log Export (to SIEM systems), Cost Anomaly Alerts (e.g., automatically freezing a tenant if daily costs spike by 500%).

Common Pitfalls

Frequently Asked Questions (FAQ)

Further Reading

Multi-tenancy design is part of the Agent governance system. We recommend continuing with the following topics:

• Agent Audit Log Design: Compliance Tracking in Multi-Tenancy — Learn how to build tamper-proof audit trails for each tenant.
• Agent Tool Permission Control: Tenant-Level Least Privilege Practices — On top of isolation, finely control which tools each tenant can invoke.
• Agent Security Evaluation: Multi-Tenant Risk Checklist — Systematically assess the effectiveness of isolation strategies.
• Agent Release Gate Design: Quota and Security Admission — Embed quota checks into CI/CD to prevent over-quota deployments.
• Agent Resilience Patterns: Fault Isolation in Multi-Tenancy — Learn circuit breakers, retries, and degradation strategies.