On-Premise AI: Why Banks and Government Agencies Demand Air-Gapped Deployment

The enterprise AI adoption story has a footnote that most vendors skip: a significant segment of the market cannot use cloud-hosted AI infrastructure. Full stop. No exceptions.

For Tier-1 banks, central banks, defense contractors, government agencies, and regulated healthcare providers, the question isn't "which cloud AI gateway should we use?" It's "can this run inside our network perimeter with zero external dependencies?"

The Regulatory Landscape

Financial Services: A Web of Data Residency Rules

SEC Rule 17a-4 requires broker-dealers to preserve electronic records in a non-rewriteable, non-erasable format (WORM). Any AI system that processes trade-related communications must produce audit trails that meet this standard. Cloud-hosted audit logs with vendor-controlled retention policies don't qualify.

FFIEC Guidelines require financial institutions to maintain control over data processed by third-party technology providers. When a bank sends a customer query through a cloud AI gateway, that query — potentially containing account numbers, transaction history, or financial advice — transits infrastructure the bank doesn't control. For many institutions, this is a clear policy violation.

MiFID II and DORA (Digital Operational Resilience Act) in the EU impose strict requirements on outsourcing arrangements, including AI processing. Firms must demonstrate they can audit and examine any third-party AI service, and must maintain operational continuity if the vendor goes down. Cloud dependency on an external AI gateway fails both requirements.

Basel III operational risk frameworks require banks to maintain comprehensive control over their technology stack. Embedding a cloud AI gateway in critical workflows creates a vendor concentration risk that risk committees increasingly reject.

Government: Classified and Sensitive Compartmented Information

For agencies handling classified or controlled unclassified information (CUI), cloud AI is almost never an option. FedRAMP authorization addresses some concerns but not all — many systems operate at classification levels where data simply cannot leave approved government networks.

Beyond classification, government procurement policies often mandate:

• Data must remain within national boundaries
• No vendor access to raw query data
• Auditable logs that can be subpoenaed without vendor involvement
• Ability to operate during internet outages or deliberate network isolation

Healthcare: HIPAA and Beyond

Healthcare organizations subject to HIPAA must ensure any AI processing of PHI (Protected Health Information) occurs under a valid Business Associate Agreement. While BAAs are available from some cloud providers, the risk calculus changes when AI models are involved — PHI may be retained in model context, logging pipelines, or fine-tuning datasets in ways that BAAs don't fully address.

SharkRouter's Air-Gapped Deployment Model

SharkRouter is architected from the ground up to operate without internet connectivity. Every component can run on-premises:

Containerized, Self-Contained Deployment

The full SharkRouter stack runs as a Docker Compose or Kubernetes deployment:

```yaml
services:
shark-api:
image: sharkrouter/api:6.0
environment:
SHARK_ENV: production
SHARK_REDIS_URL: redis://redis:6379
networks:
- internal

redis:
image: redis:7-alpine
command: redis-server --requirepass ${SHARK_REDIS_PASSWORD}
networks:
- internal

postgres:
image: pgvector/pgvector:pg16
networks:
- internal

networks:
internal:
driver: bridge
internal: true # No external connectivity
```

The `internal: true` flag on the Docker network means containers cannot reach the internet even if compromised. All traffic stays within the customer's infrastructure.

Customer-Managed Keys

SharkRouter's KMS module supports three key management backends:

Local KMS — Keys stored encrypted on the customer's hardware. No key material ever leaves the premises. Suitable for most on-prem deployments.

AWS KMS (Private Endpoint) — For organizations using AWS GovCloud or PrivateLink, SharkRouter can call AWS KMS via a private VPC endpoint with no public internet routing.

HashiCorp Vault — The preferred choice for enterprises with existing Vault infrastructure. SharkRouter authenticates via AppRole or Kubernetes service accounts. Key operations never leave the Vault cluster.

```python

Example KMS configuration for Vault backend

KMS_PROVIDER=vault
VAULT_ADDR=https://vault.internal.bank.com:8200
VAULT_ROLE_ID=sharkrouter-prod
VAULT_SECRET_ID_FILE=/run/secrets/vault-secret-id
VAULT_MOUNT_PATH=transit
VAULT_KEY_NAME=sharkrouter-master
```

FIPS 140-2 Compliance

SharkRouter's cryptographic operations use FIPS 140-2 validated algorithms throughout:

• AES-256-GCM for data at rest
• Ed25519 for audit log signatures
• SHA-256 for hash chains
• PBKDF2 for key derivation

When deployed on FIPS-enabled operating systems (RHEL with FIPS mode, Ubuntu with FIPS kernel), SharkRouter inherits the system's FIPS validation for all cryptographic operations.

WORM Audit Trails

SharkRouter's audit system produces tamper-evident logs that satisfy SEC 17a-4 WORM requirements:

Every audit event is:

1. Ed25519-signed with a key stored in the customer's KMS
2. Hash-chained — each event includes the SHA-256 hash of the previous event
3. Sequence-numbered — gaps in sequence numbers indicate tampering
4. Exportable to S3 (with object lock), Azure Blob (with immutability policies), or local WORM storage

Tamper detection is automatic: any modification to a historical record breaks the hash chain, and verification tools will flag the exact event where the chain was violated.

No Telemetry, No Callbacks

SharkRouter has no phone-home functionality. There are no analytics endpoints, no license validation calls, no update checks that require internet access. The software is delivered as container images; once pulled, it operates indefinitely with no external dependencies.

License validation for enterprise deployments uses offline license files that are verified cryptographically against a customer-specific key, with no call to a license server required.

Deployment Patterns

Single-Server Deployment — For smaller agencies or proof-of-concept deployments, the full stack runs on a single server with Docker Compose. Suitable for up to ~1,000 requests/minute.

Kubernetes HA Deployment — Production deployments use Kubernetes with multiple API replicas behind an internal load balancer. Redis Sentinel or Redis Cluster for high availability. PostgreSQL with streaming replication. All within the internal network.

Air-Gapped Image Transfer — For truly air-gapped networks, container images are transferred via approved removable media following the customer's change management process. SharkRouter provides signed manifests and checksums for every release.

What This Means for Your AI Program

Regulated institutions don't have to choose between AI capabilities and compliance. On-premise deployment gives you:

• Full data sovereignty — Every byte stays in your infrastructure
• Audit independence — Your logs, your keys, your evidence
• Zero vendor lock-in — The software runs without our involvement
• Compliance-ready architecture — WORM logs, FIPS crypto, customer-managed keys

The AI governance problem isn't going away. The question is whether you solve it with architecture or with policy exceptions that accumulate risk.

---

Interested in an air-gapped deployment? Contact our enterprise team for a technical briefing.