Manage Cribl Deployments Across Environments with Cribl as Code

Forum|Forum|7 days ago
September 1, 2026
0 replies
71 views

nnagar
Employee

This guide shows you how to manage a Cribl deployment as code and promote changes safely from a development environment to a production environment. You define your Cribl configuration in Terraform and your CI/CD pipeline applies the changes to the right environment. The result is a repeatable dev-to-prod workflow where every change is reviewed, version-controlled, and applied automatically.

What This Guide Covers

You'll set up two isolated environments, wire them to two Git branches, and automate deployments so that:

Merging to the dev branch deploys your configuration to the development environment.
Merging to the prod branch deploys the same configuration to the production environment.

The guide walks through the Terraform configuration for a basic Cribl Stream setup, a remote Terraform backend, and a sample GitHub Actions pipeline. It closes with the day-to-day workflow you follow to move a change from development to production.

This guide is written for someone who is comfortable with Git and basic Terraform. You don't need deep Terraform expertise to follow along, but you should understand the basics of defining infrastructure as code. It also helps if you are comfortable with Git branching and configuring environment variables in CI/CD pipelines. The examples in this guide use GitHub, though you can adapt them to other Git providers.

Why Isolate Environments

Making changes directly in Production is risky. A misconfiguration can result in data loss or an unpredictable system.

A separate development environment gives you a place to make and validate changes without touching live traffic. When you isolate environments, you get:

A contained blast radius: A mistake in development affects only test data, keeping your production environment stable.
A place to validate: You can send sample data through a new Pipeline or Route and confirm the output before anything reaches production.
A clear promotion path: Changes move in one direction: from development to production, so the production environment only receives configuration that has already been validated.
Reproducibility: Because the environments are defined in code, you can rebuild or spin up additional test environments exactly the same way - every time.

Benefits of Terraform

Terraform lets you define your Cribl configuration as declarative code and apply it the same way you manage the rest of your infrastructure. Managing your environments this way gives you:

A version-controlled, auditable change history: Every change to your configuration is a commit. You can see who changed what, when, and why.
Consistency with your other infrastructure: You define Cribl Sources, Destinations, Pipelines, and Routes with the same tool and workflow you already use for cloud resources.
Peer review through pull requests: Changes go through a pull request before they merge, so a teammate reviews them and terraform plan shows exactly what will change.
Simpler rollbacks: To undo a change, revert the commit and let the pipeline apply the previous state.
Automated deployments: Terraform runs in CI/CD, so merging a change is what deploys it. No one applies configuration by hand.

Assumptions

This guide uses a deliberately small setup so the moving parts stay clear and assumes you have the following:

One Git repository that holds all of the Terraform code.
Two environments: Development and Production. However, note that the steps in this guide can be extended to support any number of additional environments.
Two branches that map to the two environments: A dev branch that maps to the Development environment and a prod branch that maps to the Production environment.
GitHub as the Git provider and GitHub Actions as the CI/CD system. However, the steps can be modified to fit other Git providers such as GitLab and Bitbucket.
Cribl.Cloud as the deployment type with two Cribl.Cloud Workspaces, one for each environment. However, these steps can also be applied to an on-prem deployment.

Note: If using Cribl.Cloud, consider segmenting your environments with separate Workspaces for better environment isolation.

When you make a change to the Cribl config, you will develop against the dev branch, validate the changes in the Development environment, and then promote the changes to the Production environment once you are satisfied with the changes.

Configure Cribl with Terraform

Start by declaring the Cribl Terraform Provider. Pin the provider version so every environment and every pipeline run uses the same one.

terraform {
  required_providers {
    criblio = {
      source  = "criblio/criblio"
      version = "1.25.55"
    }
  }
}

provider "criblio" {
  # For Cribl.Cloud, the provider reads these environment variables:
  # CRIBL_CLIENT_ID
  # CRIBL_CLIENT_SECRET
  # CRIBL_ORGANIZATION_ID
  # CRIBL_WORKSPACE_ID
}

The example below defines a basic Stream flow: a syslog Source, an S3 Destination, a Pipeline that keeps a selected set of fields, and a Route that connects the Source to the Destination. Each resource attaches to a Worker Group through its group_id, which references the wg-syslog Worker Group defined at the top of the example.

The example below attaches each resource directly to the Worker Group. However, Cribl as Code works equally well with Packs so you can also define your configuration as Packs in Terraform.

Note: If you have existing configuration in Cribl, you don't have to write all of this by hand. The Cribl Terraform Config Exporter CLI exports your Cribl configuration into equivalent Terraform code - accelerating your move to infrastructure as code.

# Worker Group that the resources below attach to
resource "criblio_group" "wg-syslog" {
  id      = "wg-syslog"
  name    = "wg-syslog"
  product = "stream"
  on_prem = false
  estimated_ingest_rate = 1024
  cloud = {
    provider = "aws"
    region = var.aws_region
  }
}

# Syslog Source
resource "criblio_source" "in-syslog-9021" {
  id       = "in-syslog-9021"
  group_id = criblio_group.wg-syslog.id
  input_syslog = {
    id             = "in-syslog-9021"
    type           = "syslog"
    host           = "0.0.0.0"
    tcp_port       = 9021
    disabled       = false
    send_to_routes = true
  }
  depends_on = [criblio_group.wg-syslog]
}

# S3 Destination
resource "criblio_destination" "out-s3" {
  id       = "out-s3"
  group_id = criblio_group.wg-syslog.id
  output_s3 = {
    id             = "out-s3"
    type           = "s3"
    bucket         = var.cribl_out_s3_bucket # Defined in variables.tf
    region         = var.aws_region      # Defined in variables.tf
    aws_api_key    = var.aws_api_key     # Defined in variables.tf
    aws_secret_key = var.aws_secret_key  # Defined in variables.tf
    stage_path     = "/tmp/cribl_stage"
    compress       = "gzip"
  }
  depends_on = [criblio_group.wg-syslog]
}

# Pipeline that keeps only selected fields
resource "criblio_pipeline" "syslog_field_filter" {
  id       = "syslog_field_filter"
  group_id = criblio_group.wg-syslog.id
  conf = {
    functions = [
      {
        id       = "eval"
        filter   = "true"
        disabled = false
        final    = true
        conf = jsonencode({
          remove = ["*"]
          keep   = ["_time", "host", "message"]
        })
      }
    ]
  }
  depends_on = [criblio_group.wg-syslog]
}

# Route that connects the Source to the Destination through the Pipeline
resource "criblio_routes" "default" {
  id       = "default"
  group_id = criblio_group.wg-syslog.id
  routes = [
    {
      name     = "syslog_to_s3"
      filter   = "__inputId=='in-syslog-9021'"
      pipeline = criblio_pipeline.syslog_field_filter.id
      output   = criblio_destination.out-s3.id
      final    = true
      disabled = false
    }
  ]
  depends_on = [
    criblio_source.in-syslog-9021,
    criblio_destination.out-s3,
    criblio_pipeline.syslog_field_filter,
  ]
}

The Route ties everything together: its filter matches events from the syslog Source by __inputId, sends them through the syslog_field_filter Pipeline, and writes the result to the S3 Destination.

Configure a Terraform Backend

By default, Terraform stores its state on the local disk. That doesn't work for a CI/CD pipeline, where each run may happen on a fresh machine that needs the latest state. Configure a remote backend so every run reads and writes the same shared state, and so concurrent runs can't corrupt it.

There are two common options:

Self-managed object store. Store state in AWS S3, Google Cloud Storage, or Azure Blob Storage. Enable state locking so two applies can't run against the same state at once. On AWS, for example, you can use an S3 bucket for the state and a DynamoDB table for the lock.
Managed SaaS. Use HCP Terraform, which hosts remote state and handles locking as part of the service.

This guide uses HCP Terraform as the backend.

Before you configure the backend, set up HCP Terraform:

Create an HCP Terraform account and organization at app.terraform.io.
Create one Terraform Workspace per environment, Development and Production, and add the same tag, such as cribl, to both. The tag lets your configuration bind to both Workspaces as a group.
Set each Terraform Workspace's execution mode to Local. Terraform will run in your pipeline and will read your Cribl credentials from the pipeline's environment variables.
Create an API token that Terraform uses to authenticate with HCP. You'll store this token in your CI/CD system in the next section.

Next, add a cloud block in your Terraform code to set HCP Terraform as your backend. Terraform does not allow variables inside the cloud block, so bind it to your Terraform Workspaces by the tag property and let each run select its own Terraform Workspace with the TF_WORKSPACE environment variable (we’ll set this in the next section). Each run will set the TF_WORKSPACE variable to choose its target Terraform Workspace, Development or Production.

terraform {
  cloud {
    organization = "your-hcp-org"
    workspaces {
      # Bind to every Terraform Workspace tagged 
      # "cribl": Development and Production.
      tags = ["cribl"]

      # Each run selects its Terraform Workspace 
      # with the TF_WORKSPACE variable that's set 
      # in the GitHub Actions yml file (next section).
    }
  }
}

Configure Your CI/CD Pipeline

Before you set up the CI/CD pipeline, create two GitHub Environments in the GitHub repository's settings. Create one for your Development environment and one for your Production environment. You will set your CI/CD environment variables and secrets in each of these GitHub Environments.

Once you have the environments created, set the necessary environment variables and secrets that the CI/CD pipeline will use.

For a Cribl Cloud deployment, you'll want to set the following values:

CRIBL_CLIENT_ID
CRIBL_CLIENT_SECRET
CRIBL_ORGANIZATION_ID
CRIBL_WORKSPACE_ID

If you manage an on-prem deployment instead, the provider authenticates with the following:

CRIBL_ONPREM_SERVER_URL
CRIBL_ONPREM_USERNAME
CRIBL_ONPREM_PASSWORD

Additionally, store the HCP API token as a secret named TF_TOKEN_app_terraform_io. Terraform reads this variable automatically, so the pipeline authenticates to HCP without a separate login step.

Once you have the environment variables and secrets set, the next step is to create the CI/CD pipeline with GitHub Actions.

The CI/CD pipeline will be triggered when a pull request is merged into either the dev or prod branch. If merged into the dev branch, the pipeline will deploy the Cribl configuration to the Development Cribl environment. If instead the pull request is merged into the prod branch, the pipeline will deploy the Cribl configuration to the Production Cribl environment.

The GitHub Actions workflow defined below selects the proper GitHub Environment and Terraform Workspace based on the branch the pull request is merged into. It then sets the required environment variables before executing a terraform apply.

To set up the GitHub Actions workflow, create a deploy.yml file within the .github/workflows folder and add the following code within it:

name: Deploy Cribl Configuration

on:
  pull_request:
    types:
      - closed
    branches:
      - dev
      - prod

jobs:
  deploy:
    # Run only when the pull request was merged, not simply closed.
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest

    # Map the target branch to its GitHub Environment: 
    # dev to Development, prod to Production.
    environment: ${{ github.base_ref == 'prod' && 'Production' || 'Development' }}

    env:
      CRIBL_CLIENT_ID: ${{ secrets.CRIBL_CLIENT_ID }}
      CRIBL_CLIENT_SECRET: ${{ secrets.CRIBL_CLIENT_SECRET }}
      CRIBL_ORGANIZATION_ID: ${{ vars.CRIBL_ORGANIZATION_ID }}
      CRIBL_WORKSPACE_ID: ${{ vars.CRIBL_WORKSPACE_ID }}

      # Authenticate to HCP Terraform and 
      # select the Workspace that matches the branch.
      TF_TOKEN_app_terraform_io: ${{ secrets.TF_TOKEN_app_terraform_io }}
      TF_WORKSPACE: ${{ github.base_ref == 'prod' && 'Production' || 'Development' }}

      # Other environment variables defined here

    steps:
      - name: Check out the repository
        uses: actions/checkout@v4
      - name: Set up Terraform
        uses: hashicorp/setup-terraform@v3
      - name: Initialize Terraform
        run: terraform init
      - name: Apply the configuration
        run: terraform apply -auto-approve

Recap of Setup Steps

At this point you've built the foundation for a dev-to-prod workflow. You have:

A Git repository that contains your Terraform code with a dev branch and a prod branch.
Terraform code that defines your Cribl configuration.
Two separate GitHub Environments: Development and Production - each configured with its environment variables and secrets for the CI/CD pipeline to use.
A remote Terraform backend on HCP Terraform to manage each environment's state within the CI/CD pipeline.
A GitHub Actions workflow that deploys to the Development environment on a merge to the dev branch and to the Production environment on a merge to the prod branch.

The Dev-to-Prod Workflow in Practice

Once the setup is in place, every change follows the same path from Development to Production.

Create a feature branch off dev: Make your Cribl configuration changes in the feature branch, such as adding a Route or adjusting a Cribl Pipeline.
Open a pull request into dev: A teammate reviews the change, and terraform plan shows what it will change.
Merge into dev: The CI/CD pipeline deploys the change to the Development environment.
Validate in Development: Send test data through the updated configuration and confirm it behaves as you expect.
Open a pull request to merge dev into prod: Review the changes validated in the Development environment once more before merging into the prod branch.
Merge into prod: The pipeline then deploys the changes to the Production environment.

Because a change reaches Production only after it runs and passes validation in Development, Production always receives configuration that you've already tested.

Conclusion

This guide walked through how to isolate your Cribl environment into Development and Production environments. It then showed you how to manage deployment through each environment in a safe, auditable way. The steps in this guide give you a controlled promotion process: changes are reviewed, validated in the Development environment, and applied to the Production environment only after they are tested.

Isolating environments protects your production environment from untested changes, and a code-driven promotion process makes every deployment reviewable, repeatable, and easy to roll back. The examples here are provided as a starting point. Cribl as Code is flexible, so you can adapt the integration and suggested flow to fit how your team works.