> ## Documentation Index
> Fetch the complete documentation index at: https://jacobpevans-docs-reusable-workflow-main-pin.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Observability

> OTEL telemetry from every AI coding tool, routed through Cribl, stored in Splunk, visualized by purpose-built dashboards.

> If an AI agent touched code, there's a trace.

Every AI coding interaction emits OpenTelemetry — IDE events, model calls, token counts, latency, cost. The Cribl Edge tier collects and reshapes it. Cribl Stream routes it. Splunk indexes it. Purpose-built Splunk apps make it readable.

## AI telemetry pipeline

{/* Boundary crossings: 0. Total nodes: 11 (≤12). Aspect: ~3:1 LR. Pass. */}

```mermaid theme={null}
%%{init: {'theme':'base','look':'handDrawn','themeVariables':{'fontFamily':'Geist','fontSize':'14px','primaryColor':'#102937','primaryTextColor':'#F4EFE6','primaryBorderColor':'#4FB3A9','lineColor':'#4FB3A9','secondaryColor':'#0B1D2A','tertiaryColor':'#1A2A38','clusterBkg':'rgba(79,179,169,0.08)','clusterBorder':'#4FB3A9'}}}%%
flowchart LR
  Claude([Claude Code])
  Copilot([Copilot])
  Gemini([Gemini CLI])
  VSC([VS Code])

  CCO([cc-edge-claude-code-otel])
  CCP([cc-edge-copilot-otel])
  CAG([cc-edge-gemini-antigravity-io])
  CCV([cc-edge-vscode-io])

  Stream([Cribl Stream])
  SP[(Splunk indexer)]
  App([Splunk apps])

  Claude --> CCO --> Stream
  Copilot --> CCP --> Stream
  Gemini --> CAG --> Stream
  VSC --> CCV --> Stream
  Stream --> SP --> App

  classDef tool  fill:#102937,stroke:#4FB3A9,stroke-width:2px,color:#F4EFE6;
  classDef pack  fill:#102937,stroke:#E06B4A,stroke-width:2px,color:#F4EFE6;
  classDef route fill:#102937,stroke:#F4EFE6,stroke-width:1.5px,color:#F4EFE6;

  class Claude,Copilot,Gemini,VSC tool
  class CCO,CCP,CAG,CCV pack
  class Stream,SP,App route

  click SP "/observability/repos/ansible-splunk" "Splunk install + indexes"
  %% Cribl Edge pack repos (CCO/CCP/CAG/CCV) intentionally NOT clickable —
  %% Mermaid drops the _blank target attribute on flowchart clicks
  %% (https://github.com/mermaid-js/mermaid/issues/3077,
  %% https://github.com/mermaid-js/mermaid/issues/5550). The pack repo links
  %% are listed in the table below this diagram, where Mintlify
  %% auto-applies target="_blank" to external markdown links.

  linkStyle 0,1,2,3,4,5,6,7 stroke:#E06B4A,stroke-width:2px,stroke-dasharray:4 3;
  linkStyle 8,9 stroke:#4FB3A9,stroke-width:2px;
```

Coral dashed edges are telemetry; solid green are the routing hops. Per-tool packs keep parsing isolated — a Copilot schema change doesn't break the Claude pipeline.

## Splunk apps

The AI-observability apps and TAs live under a separate organization at
[github.com/visicore](https://github.com/visicore). Two pieces matter on this site:

| Component              | What it does                                                                           |
| ---------------------- | -------------------------------------------------------------------------------------- |
| Dashboards (app)       | Dashboard Studio v2. Cost, usage, performance per model, per project, per developer.   |
| Technology add-on (TA) | Field aliases, CIM mappings, lookups, search-time transforms. The normalization layer. |

## Cribl Edge packs (collectors)

| Repo                                                                                          | Collects from                          |
| --------------------------------------------------------------------------------------------- | -------------------------------------- |
| [cc-edge-claude-code-otel](https://github.com/JacobPEvans/cc-edge-claude-code-otel)           | Claude Code (OTEL hooks)               |
| [cc-edge-copilot-otel](https://github.com/JacobPEvans/cc-edge-copilot-otel)                   | GitHub Copilot Chat (OTLP gRPC)        |
| [cc-edge-vscode-io](https://github.com/JacobPEvans/cc-edge-vscode-io)                         | VS Code (logs + telemetry)             |
| [cc-edge-gemini-antigravity-io](https://github.com/JacobPEvans/cc-edge-gemini-antigravity-io) | Gemini Antigravity                     |
| [cc-edge-the-mac-pack-io](https://github.com/JacobPEvans/cc-edge-the-mac-pack-io)             | macOS system health (auxiliary signal) |
| [cc-edge-macos-system](https://github.com/JacobPEvans/cc-edge-macos-system)                   | macOS-native system events             |

## Cribl Stream collectors

| Repo                                                                                                | What it does                                                              |
| --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| [cc-stream-github-copilot-rest-io](https://github.com/JacobPEvans/cc-stream-github-copilot-rest-io) | GitHub Copilot usage metrics via REST API. Per-org / per-seat usage data. |

## Why per-tool packs

Each AI coding tool emits slightly different telemetry shapes. Per-tool packs keep the parsing and enrichment isolated; a Copilot schema change shouldn't break the Claude pipeline. The shared CIM mapping in the TA is where the normalization happens.

## Where Splunk runs

<CardGroup cols={2}>
  <Card title="tf-splunk-aws" icon="aws" href="/observability/repos/tf-splunk-aws">
    OpenTofu for the AWS-side Splunk footprint. VPC, KMS, EC2, IAM — DR-ready.
  </Card>

  <Card title="ansible-splunk" icon="screwdriver-wrench" href="/observability/repos/ansible-splunk">
    The configuration tier. Splunk install, indexes, HEC tokens, storage tiering.
  </Card>
</CardGroup>

## Where to go next

<CardGroup cols={2}>
  <Card title="Mac Pack" icon="apple" href="/observability/repos/cc-edge-the-mac-pack">
    The macOS host telemetry pack — unified logs, system metrics, power.
  </Card>

  <Card title="Monitoring agents" icon="chart-line" href="/observability/monitoring-agents">
    Cross-stack map of every collector and where it runs.
  </Card>

  <Card title="Data pipelines" icon="diagram-project" href="/architecture/data-pipelines">
    Log and NetFlow ingest — the non-AI side of the pipeline.
  </Card>

  <Card title="Configuration" icon="screwdriver-wrench" href="/configuration/overview">
    Ansible playbooks that deploy the Cribl tier this pipeline runs on.
  </Card>
</CardGroup>
