# LogSource

A way logs reach Guardian, backed by a Guardian ingest token of the same type. Fields, YAML, HCL and CLI samples, and how apply treats it.

A way logs reach Guardian, backed by a Guardian ingest token of the same type. The token is minted on create and shown once, by the apply that creates it; it is never exported or shown in a plan. aws: the token for the vulnetix-siem CloudFormation stack you deploy (IngestToken), which sends VPC flow logs, Route 53 Resolver logs and CloudTrail events by Firehose. cloudflare: Guardian creates Logpush jobs with your Cloudflare API token, which is used for those calls only and never stored. events: the token for POST /v1/ingest/events, used by your own scripts and the Vulnesis runtime sensor. Guardian has no update for a token or a Logpush job, so a changed spec is applied as delete and create: the old token is revoked and a new one minted. Managing sources needs the Guardian manager role.

| | |
|---|---|
| `apiVersion` | `vulnetix.com/v1` |
| `kind` | `LogSource` |
| Group | [SIEM](/platform-iac/kinds/catalogue.md#siem) |
| Status | **Served.** The [config engine](/platform-iac/reference/engine.md) plans and applies it. |
| Objects per tenant | many, keyed by `metadata.name` |
| Token and role | An owner, or a service account with the developer role, or an Authentik access token for either. The kind is written in Guardian, which the Vulnesis API vouches to for the caller; a member can read it but not change it. See [Credentials for automation](/platform-iac/reference/config-api.md#credentials-for-automation). |
| Secret fields | `spec.cloudflare.apiToken`: read on the client from `valueFrom`, sent once, never exported or shown in a plan |
| Terraform resource | [`vulnetix_log_source`](/platform-iac/terraform/resources.md#vulnetix_log_source), imported by name (see [Terraform and OpenTofu](#terraform-and-opentofu)) |
| CLI | [`vulnetix config apply`](/platform-iac/cli/apply.md), CLI v3.165.1 and later. |
| Pinned schema | [`v1.19.1/LogSource.schema.json`](https://github.com/Vulnesis/config-schemas/releases/download/v1.19.1/LogSource.schema.json) |
| Newest schema | [`latest/LogSource.schema.json`](https://github.com/Vulnesis/config-schemas/releases/latest/download/LogSource.schema.json) |
| First released in | [`v1.3.0`](https://github.com/Vulnesis/config-schemas/releases/tag/v1.3.0) |
| Print the built-in copy | `vulnetix config schema LogSource` |

## Minimal

Only the fields the kind requires. From `examples/LogSource/minimal.yaml`.

**YAML**

```yaml
# yaml-language-server: $schema=https://github.com/Vulnesis/config-schemas/releases/latest/download/LogSource.schema.json
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: sensor-events
spec:
  type: events
```

**HCL** (resource `vulnetix_log_source`)

```hcl
# Only the arguments the kind requires. From config-schemas examples/LogSource/minimal.yaml.

resource "vulnetix_log_source" "sensor_events" {
  name = "sensor-events"
  type = "events"
}
```

Every field that has a schema default, set to that default: [`terraform/log_source.tf`](https://github.com/Vulnesis/platform-iac-examples/blob/main/terraform/log_source.tf) and [`yaml/LogSource.yaml`](https://github.com/Vulnesis/platform-iac-examples/blob/main/yaml/LogSource.yaml) in [Vulnesis/platform-iac-examples](https://github.com/Vulnesis/platform-iac-examples).

**CLI** (applies it)

```bash
export VULNESIS_ORG_ID=...    # the organization UUID
export VULNESIS_API_KEY=...   # the ApiKey of a developer or owner service account
vulnetix config apply -f - --manager gitops:acme/platform --auto-approve <<'EOF'
# yaml-language-server: $schema=https://github.com/Vulnesis/config-schemas/releases/latest/download/LogSource.schema.json
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: sensor-events
spec:
  type: events
EOF
```

## Advanced

Every field the kind supports, with a `when` expression where it takes one. From `examples/LogSource/advanced.yaml`.

**YAML**

```yaml
# yaml-language-server: $schema=https://github.com/Vulnesis/config-schemas/releases/latest/download/LogSource.schema.json
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: cloudflare-edge
  labels:
    team: netsec
  annotations:
    note: The Cloudflare token is used for this apply only; deleting the source needs it again.
spec:
  type: cloudflare
  label: Cloudflare edge and Gateway
  cloudflare:
    apiToken: { env: CLOUDFLARE_LOGPUSH_TOKEN }
    datasets: [firewall_events, http_requests, gateway_dns]
    accountId: 0123456789abcdef0123456789abcdef
    zoneIds: [fedcba9876543210fedcba9876543210]
---
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: aws-prod
spec:
  type: aws
  label: Production VPCs
---
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: sensor-events
spec:
  type: events
  label: Runtime sensors
```

**HCL** (resource `vulnetix_log_source`)

```hcl
# Every argument the kind supports. From config-schemas examples/LogSource/advanced.yaml.

variable "cloudflare_edge_cloudflare_api_token" {
  type      = string
  sensitive = true
  ephemeral = true
}

resource "vulnetix_log_source" "cloudflare_edge" {
  name = "cloudflare-edge"
  cloudflare = {
    account_id           = "0123456789abcdef0123456789abcdef"
    api_token_wo         = var.cloudflare_edge_cloudflare_api_token
    api_token_wo_version = 1
    datasets             = ["firewall_events", "http_requests", "gateway_dns"]
    zone_ids             = ["fedcba9876543210fedcba9876543210"]
  }
  label = "Cloudflare edge and Gateway"
  type  = "cloudflare"
  labels = {
    team = "netsec"
  }
  annotations = {
    note = "The Cloudflare token is used for this apply only; deleting the source needs it again."
  }
}

resource "vulnetix_log_source" "aws_prod" {
  name  = "aws-prod"
  label = "Production VPCs"
  type  = "aws"
}

resource "vulnetix_log_source" "sensor_events" {
  name  = "sensor-events"
  label = "Runtime sensors"
  type  = "events"
}
```

Every field that has a schema default, set to that default: [`terraform/log_source.tf`](https://github.com/Vulnesis/platform-iac-examples/blob/main/terraform/log_source.tf) and [`yaml/LogSource.yaml`](https://github.com/Vulnesis/platform-iac-examples/blob/main/yaml/LogSource.yaml) in [Vulnesis/platform-iac-examples](https://github.com/Vulnesis/platform-iac-examples).

**CLI** (applies it)

```bash
export VULNESIS_ORG_ID=...    # the organization UUID
export VULNESIS_API_KEY=...   # the ApiKey of a developer or owner service account
export CLOUDFLARE_LOGPUSH_TOKEN=...   # read on this machine by valueFrom, never written to the document
vulnetix config apply -f - --manager gitops:acme/platform --auto-approve <<'EOF'
# yaml-language-server: $schema=https://github.com/Vulnesis/config-schemas/releases/latest/download/LogSource.schema.json
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: cloudflare-edge
  labels:
    team: netsec
  annotations:
    note: The Cloudflare token is used for this apply only; deleting the source needs it again.
spec:
  type: cloudflare
  label: Cloudflare edge and Gateway
  cloudflare:
    apiToken: { env: CLOUDFLARE_LOGPUSH_TOKEN }
    datasets: [firewall_events, http_requests, gateway_dns]
    accountId: 0123456789abcdef0123456789abcdef
    zoneIds: [fedcba9876543210fedcba9876543210]
---
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: aws-prod
spec:
  type: aws
  label: Production VPCs
---
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: sensor-events
spec:
  type: events
  label: Runtime sensors
EOF
```

## Fields

Paths use dots for nesting and `[]` for the items of a list. **Required** means required whenever the parent object is present. A default is what the server uses when the field is left out.

| Field | Type | Required | Default | Allowed values | Secret | Description |
|---|---|---|---|---|---|---|
| <span id="apiVersion"></span>`apiVersion` | string | yes |  | exactly `vulnetix.com/v1` |  | The document format version. Every kind in this release is vulnetix.com/v1. |
| <span id="kind"></span>`kind` | string | yes |  | exactly `LogSource` |  | Always LogSource. |
| <span id="metadata"></span>`metadata` | object | yes |  | no other keys |  |  |
| <span id="metadata-name"></span>`metadata.name` | string | yes |  | pattern `^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$` |  | The object's name, unique per kind within a tenant. It is the natural key: plan, apply, export and the Terraform provider all address an object by kind and name. |
| <span id="metadata-org"></span>`metadata.org` | string | no |  | format `uuid` |  | The tenant (organization UUID) the document belongs to. When set, plan and apply refuse to run against any other tenant. |
| <span id="metadata-labels"></span>`metadata.labels` | object | no |  | map of string to string (length 0 to 256); keys match `^[a-z0-9]([a-z0-9._/-]{0,61}[a-z0-9])?$` |  | Free key/value labels. Stored with the object, never interpreted by the server. |
| <span id="metadata-annotations"></span>`metadata.annotations` | object | no |  | map of string to string (length 0 to 4096) |  | Free key/value notes for people and tools, such as a ticket link. |
| <span id="spec"></span>`spec` | object | yes |  | no other keys |  |  |
| <span id="spec-type"></span>`spec.type` | string | yes |  | `aws`, `cloudflare`, `events` |  |  |
| <span id="spec-label"></span>`spec.label` | string | no |  | length 1 to 120 |  | The token's label on the Sources page. Defaults to the type and the date. |
| <span id="spec-cloudflare"></span>`spec.cloudflare` | object | no |  | no other keys |  | The Logpush jobs to create. Running it again does not make a second job for a dataset that already sends to Guardian. Cloudflare allows four Logpush jobs per zone. |
| <span id="spec-cloudflare-apiToken"></span>`spec.cloudflare.apiToken` | object | yes |  | 1 to 1 keys; no other keys | yes | Where the client reads the Cloudflare API token. Sent for this apply only; Guardian never stores it, so deleting the source needs it again. |
| <span id="spec-cloudflare-apiToken-env"></span>`spec.cloudflare.apiToken.env` | string | no |  | pattern `^[A-Za-z_][A-Za-z0-9_]*$` |  | Name of an environment variable. |
| <span id="spec-cloudflare-apiToken-file"></span>`spec.cloudflare.apiToken.file` | string | no |  | length 1 to any |  | Path to a file holding the value. Trailing newlines are trimmed. |
| <span id="spec-cloudflare-datasets"></span>`spec.cloudflare.datasets` | array of string | yes |  | 1 to any items; items unique; each item: `firewall_events`, `http_requests`, `dns_logs`, `workers_trace_events`, `gateway_dns`, `dns_firewall_logs`, `gateway_network`, `network_analytics_logs`, `audit_logs_v2` |  | firewall_events, http_requests and dns_logs are zone datasets, created on every zone the token can see (or those in zoneIds). The others are account datasets and need accountId. |
| <span id="spec-cloudflare-accountId"></span>`spec.cloudflare.accountId` | string | no |  | pattern `^[0-9a-f]{32}$` |  | The Cloudflare account id, for account datasets. |
| <span id="spec-cloudflare-zoneIds"></span>`spec.cloudflare.zoneIds` | array of string | no |  | 1 to 50 items; items unique; each item: pattern `^[0-9a-f]{32}$` |  | Only these zones. Left out, every zone the token can see (at most 1,000). |

## Rules

Conditions the schema enforces beyond each field's own type. `vulnetix config validate` and the engine's plan report a document that breaks one.

| On | Rule |
|---|---|
| `spec` | When `type` is `cloudflare`: `cloudflare` is required. Otherwise: `cloudflare` is not allowed. |
| `spec.cloudflare` | When `datasets` is : `accountId` is required. |

## Applying this kind

- **Command.** `vulnetix config apply -f <file>`; the CLI line above says from which release. Plan first with [`vulnetix config plan`](/platform-iac/cli/plan.md): it shows each object as create, update, delete or no change, with a `planHash` that apply then holds to. Apply writes in one transaction, except for the second phase below.
- **Manager.** Each object records the manager that applied it (`terraform`, `console` or `gitops:<repo>`). `--prune` deletes only objects its own manager owns. See [The manager](/platform-iac/cli/plan.md#the-manager) and the [config engine](/platform-iac/reference/engine.md).
- **Second phase.** This kind is not written inside the apply transaction. The other kinds apply and commit first, then each LogSource is written on its own, in Guardian. One that fails reports `status.phase: Failed` and a `message`, and nothing is recorded for it. The others stay applied and the apply answers 207. Plan again to see what is left. A single `PUT` from the Terraform provider answers 502 `external_failed` instead.
- **Status.** `status.ingestToken` for an aws or events source, in the apply that creates it and never again. A failed second-phase write reports phase `Failed`. The phases the engine reports are explained under [status phases](/platform-iac/reference/engine.md#status-phases).

## How apply treats it

Written in Guardian, in the second phase; see [SIEM kinds](/platform-iac/reference/engine.md#siem-kinds-written-in-guardian).

- Only `label` changes in place. Any other change blocks, because Guardian edits Logpush jobs only with the Cloudflare token, which it does not keep.
- Create returns the aws or events ingest token once, as `status.ingestToken` in the apply result, and never again.
- Cloudflare's `apiToken` (`valueFrom`) is used for the create call only and is never compared or exported.
- Delete revokes the ingest token. A cloudflare source's Logpush jobs stay in Cloudflare until removed there, and the plan warns.
- `label` and `zoneIds` are compared only when the document sets them.

## Terraform and OpenTofu

The resource is `vulnetix_log_source` in the `vulnetix/resolve` provider. Its arguments are the `spec` in snake case, beside `name`, `labels` and `annotations` from `metadata`. The Minimal and Advanced HCL are above. The provider's own reference is [vulnetix_log_source](https://registry.terraform.io/providers/vulnetix/resolve/latest/docs/resources/log_source).

Import an object that already exists by its name:

```shell
# Import an existing LogSource by its name (metadata.name).
terraform import vulnetix_log_source.sensor_events sensor-events
```

A secret is two arguments, never stored in the plan or the state: `api_token_wo` (write-only) and `api_token_wo_version`. Change the version number to send a new value. See [Secrets](/platform-iac/terraform.md#secrets).

Install, authentication and how state is read: [Terraform and OpenTofu](/platform-iac/terraform.md).

## More examples

`examples/LogSource/sources.yaml` from `Vulnesis/config-schemas`:

```yaml
# yaml-language-server: $schema=https://github.com/Vulnesis/config-schemas/releases/latest/download/LogSource.schema.json
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: aws-prod
spec:
  type: aws
  label: Production VPCs
---
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: cloudflare-edge
spec:
  type: cloudflare
  cloudflare:
    apiToken: { env: CLOUDFLARE_LOGPUSH_TOKEN }
    datasets: [firewall_events, http_requests, gateway_dns]
    accountId: 0123456789abcdef0123456789abcdef
    zoneIds: [fedcba9876543210fedcba9876543210]
---
apiVersion: vulnetix.com/v1
kind: LogSource
metadata:
  name: sensor-events
spec:
  type: events
  label: Runtime sensors
```

Check a file with `vulnetix config validate -f <file>`. See [Validate documents](/platform-iac/cli/validate.md). In an editor, start the file with the [schema comment](/platform-iac/schemas/editors.md).
