WORK / ACTIVE SYSTEM

Narwhal

A reproducible and verifiable Kubernetes Internal Developer Platform

Kubernetes · Vagrant · GitOps · IDP · Istio Ambient · Argo CD · Cilium · Air-Gap · Keycloak · Observability
01 / Problem02 / Architecture03 / Development04 / Proof05 / Knowledge06 / Record
01 / PROBLEM

Integrating dozens of cloud-native components creates recurring DNS, TLS, identity, networking, startup-order, and version-compatibility failures that must be re-proven after upgrades

RESPONSE

Integrate 35 GitOps-managed applications into one reproducible IDP and turn 263 documented incidents into 51 CI regression checks plus live cluster verification

02 / ARCHITECTURE

Read the boundaries before the features.

Identity, delivery, network, storage and workload integration are shown as operational seams rather than a feature inventory.

Narwhal architecture: developer portal through APISIX and Keycloak to Argo CD/Gitea, HA Kubernetes and platform services
Narwhal platform map — identity, GitOps, Kubernetes and integrated platform services.
ARCHITECTURE / HOW IT WORKS

Narwhal Internal Developer Platform

01
Developer / Operator
Portal · service access
02
APISIX + Keycloak
Gateway · OIDC · SSO
03
Argo CD + Gitea
GitOps app-of-apps
04
Kubernetes HA
Cilium · Istio ambient · kube-vip
05
Platform Services
Observability · storage · backup · policy
Developer access flows through gateway and identity into GitOps-managed Kubernetes, with platform services operated as one integrated system.
PLATFORM MAP / VISUAL EVIDENCE

One login, one GitOps path, many platform capabilities

01
Portal + Keycloak SSO
02
Argo CD + Gitea GitOps
03
Cilium + Istio + APISIX
04
Prometheus · Loki · Tempo · Hubble
A compact visual summary; the architecture diagram below explains the execution flow in more detail.
03 / DEVELOPMENT OVER TIME

Evolution is part of the evidence.

First commit, cumulative commits, releases and recent activity show whether the system is still being engineered.

DEVELOPMENT OBSERVATORY
dasomel/narwhal

Observe a system evolving over time, not a repository frozen at one release.

Contributors: 3
First commit
Feb 8, 2026
Commits
1,182
Releases
4
Latest release
v1.2.0
Last push
Oct 17, 2026
Active for
8 months
Recent development activity
1,091 commits / 20 weeks
PAST → NOW
Language: TypeScriptLicense: Apache-2.0Stars: 0Forks: 1Open issues: 160Latest release
04 / PROOF, NOT BADGES
35
GitOps-managed applications

Desired platform state under GitOps

51
CI regression checks

Integration behavior protected in CI

263
integration / incident lessons

Failures retained as engineering knowledge

120+
cluster checks

Repeatable platform verification

06 / ENGINEERING RECORD

Implementation details, operating notes and project-specific context.

Project Overview

Narwhal is an open-source Internal Developer Platform (IDP) built around Kubernetes. It combines GitOps, IAM/SSO, service mesh, observability, registry, storage, backup, policy, API gateway, and management portal capabilities into one reproducible platform unit.

Narwhal is not primarily a Kubernetes installer. Its product boundary is the integration seam between independently developed projects.

Current Scale

MetricCurrent state
Activity612 commits since 2026-02-08 (as of 2026-09-23), 4 releases, latest v1.2.0
Integration35 GitOps-managed applications
CI regression192 checks
Live verification120+ cluster checks, 49 SSO checks
Integration knowledge318 documented incidents
DeploymentVagrant ARM64, Kakao Cloud AMD64, air-gapped
Offline bundle104 images, 27 Helm charts, binaries, manifests, and OS packages per architecture

These numbers are engineering evidence of repeated integration and verification, not simply activity metrics.

Current Status

The latest tagged release is v1.2.0 (2026-08-09), which turned air-gapped installation from "images only" into an install that genuinely completes with no route to the internet. It promoted Kakao Cloud to a first-class OpenTofu-provisioned provider and added a Chaos Mesh 2.8.3 + k6 load-test suite plus a 36-check clean-install regression suite in CI.

Since v1.2.0, main has accumulated further security hardening that is not yet in a tagged release (verified against the commit history as of 2026-09-23):

  • Enforced Keycloak OIDC TLS verification and internal CA trust; removed ArgoCD/APISIX OIDC TLS bypasses
  • Restricted the APISIX Admin API allowlist to the cluster pod CIDR
  • Enforced SSH host-key verification on Kakao bastion/node paths and split the bastion security group from the node security group
  • Switched the default NFS export to root_squash with a non-world-writable root
  • Added a default-deny NetworkPolicy for the database namespace and a Kyverno admission gate for the node-tuning job service account
  • Narrowed the portal's OpenBao access to least privilege, removing unnecessary secret/data/* grants

This work is release-candidate material still under verification on main, not yet part of a published tag.

Platform Components

Kubernetes and networking

  • Kubernetes v1.35
  • Cilium v1.19.x
  • Hubble v1.19.x
  • kube-vip v1.1.x
  • MetalLB v0.16.x
  • Apache APISIX 3.15.x

GitOps and identity

  • Argo CD v3.4.x
  • Gitea v1.26.x
  • Keycloak 26.5.x

Observability

  • Prometheus Stack v0.91.x
  • Loki 3.7.x
  • Grafana Alloy v1.17.x
  • Tempo 2.9.x
  • Hubble

Platform services

  • Harbor v2.15.x
  • OpenBao v2.5.x
  • Kyverno v1.18.x
  • Headlamp v0.42.x
  • SeaweedFS v4.34.x
  • Velero v1.18.x
  • CloudNative-PG v1.29.x
  • Istio v1.30.x in ambient mode

Integration Seams as Product

A typical identity-to-application path crosses several independent systems:

Keycloak OIDC
      ↓
APISIX authentication
      ↓
service routing
      ↓
Istio ambient / ztunnel
      ↓
Kubernetes workload

Narwhal records the assumptions at these boundaries as documentation, scripts, health checks, and regression checks so they can be revalidated during upgrades.

Knowledge as Tests

The core maintenance loop is:

Incident
   ↓
Root Cause
   ↓
Discriminator
   ↓
Regression Check
   ↓
Upgrade Gate

The incident log records not only what fixed an incident, but also how to distinguish it from similar failures and which tempting fixes do not work. This knowledge base is connected to the regression suite, producing 318 documented incidents and 192 CI checks.

Verification Model

Narwhal validates different layers for different questions.

LayerScopeQuestion
Cluster verification120+Is the cluster and platform actually healthy?
SSO verification49Does identity work end to end across integrated apps?
CI regression192Have known integration failures returned?

This separation keeps fast CI regression checks independent from deeper live-cluster validation.

Air-Gapped Installation

Disconnected installation is a first-class deployment target.

Online build
   ↓
images / charts / binaries / manifests / packages
   ↓
architecture-specific offline bundle
   ↓
verification
   ↓
install without live Internet

The bundle contains the artifacts required to reproduce the platform without reaching upstream registries during installation.

Management Portal

Narwhal Portal provides the day-2 user experience for the platform: dashboard, applications, security, cost, governance, catalog, and architecture views.

The product boundary is therefore:

Narwhal = platform integration + operation
Portal  = developer / operator experience

Getting Started

git clone https://github.com/dasomel/narwhal.git
cd narwhal
vagrant up --provider=vmware_desktop
vagrant ssh master-1 -c "kubectl get nodes"
vagrant ssh master-1 -c "kubectl get applications -A"
vagrant ssh master-1 -c "bash /home/vagrant/scripts/test/verify-cluster.sh"

Documentation Index

TopicDocumentPurpose
OverviewPlatform OverviewIDP scope and integration philosophy
ArchitectureCluster ArchitectureHA control plane, networking, service layout
GitOpsGitOps WorkflowArgo CD + Gitea App-of-Apps
NetworkingNetworking & IngressCilium, MetalLB, APISIX, DNS
SecuritySecurity & SSOKeycloak, OpenBao, Kyverno, TLS
ObservabilityMonitoring & LogsPrometheus, Grafana, Loki, Tempo, Hubble
StorageStorage & DatabasesNFS CSI, SeaweedFS, quota, PostgreSQL
OperationsDay-2 Operations & DRBackup, restore, upgrades, air-gap
TestingRegression Testing & ChaosLive verification and regression practice

Project Relationship

Architecture diagram
Responsive vector rendering · source preserved
kube-ready-box ↓ Narwhal IDP nfs-quota-agent ldapium Narwhal Portal OpenForge shared engineering / supply-chain practices