Skip to main content

This guide provides comprehensive considerations for migrating your data collection infrastructure from Splunk Universal Forwarders (UFs) to Cribl Edge. Universal Forwarder migrations present unique challenges due to fundamental architectural differences between Splunk's agent model and Cribl Edge's Fleet-based approach.

The UF to Edge migration generally follows these phases:

  • Decide on your migration strategy
  • Complete pre-migration planning specific to UF environments
  • Implement your Edge configuration with UF-specific considerations
  • Execute a controlled rollout

This document focuses on Splunk UF-specific considerations. For general migration principles, strategy overview, and general considerations, refer to the Migrating from Third-Party Agents to Cribl Edge document. This guide assumes familiarity with the general migration framework outlined in that document.

Important Note: This is a considerations guide, not step-by-step instructions. Complex UF migrations require careful planning and often benefit from professional services engagement.
 

Pre-Migration Planning for UF Environments

This phase involves understanding your current UF deployment, identifying potential challenges, and ensuring you have the necessary resources in place.
 

UF Configuration Analysis

The goal of configuration analysis is to determine the data flow and topology of your current UF system, so that you can use this information to inform your Fleet design. 

The way your current UF deployment and configuration management is organized will significantly impact your Edge Fleet design decisions.

Whether you use Splunk Deployment Server with serverclasses, or enterprise configuration management tools like CFEngine, Puppet, Chef, Ansible, or SaltStack to manage UF configurations and deploy apps or inputs to hosts, your existing approach will influence how you architect your Edge Fleet implementation.
 

Key UF Components to Document:

  • Deployment Server configuration using serverclass.conf
  • Splunk Apps stored in $SPLUNK_HOME/deployment-apps
  • Client-to-serverclass mappings

Key Endpoints to Review:

  • /services/deployment/server/serverclasses (lists serverclasses)
  • /services/deployment/server/applications (lists deployment apps)
  • /services/deployment/server/clients (shows client mappings)

You can use Splunk's btool to examine configurations:

./bin/splunk btool inputs list --app=system
 

Understanding UF to Edge Architectural Differences

The most significant challenge in UF to Edge migration stems from how the two systems handle configuration management and Node organization. This architectural shift affects every aspect of your migration planning.
 

Configuration Management Models

The migration from UF to Cribl Edge involves fundamental architectural changes:
 

Composition vs. Inheritance

  • UFs: Use composition (one UF can belong to multiple serverclasses)
  • Cribl Edge: Uses inheritance (one Edge Node belongs to a single Fleet/Subfleet)

This difference is crucial for understanding why direct translation of serverclasses to Fleets often fails.

 

Fleet Design Strategy

Fleet design is perhaps the most critical decision in your UF migration. Poor Fleet architecture can lead to management complexity, performance issues, and operational challenges. The goal is to balance functionality with maintainability while leveraging Cribl Edge's strengths.

 

Fleet Rationalization Approach

  • Review your serverclass.conf file to understand what needs to be collected
  • Use tags, Pipelines, and Cribl Stream to reduce configuration variability
  • Consolidate similar configurations to minimize Fleet count
  • Group by function or organization/region/control span rather than direct serverclass translation

Fleet Count Optimization

  • Keep Fleet counts to a minimum for optimal performance
  • Avoid direct translation of Deployment Server Classes to Edge Fleets (leads to Fleet proliferation)
  • Consider creating Fleets for unique combinations of serverclasses only when necessary

⚠️ Implementation Warning: Direct translation of Deployment Server Classes to Edge Fleets often leads to management complexity and potential performance issues.

 

UF Configuration Translation Challenges

Not all Universal Forwarder configurations can be directly translated to Cribl Edge. Understanding these limitations upfront allows you to plan alternative approaches and avoid migration roadblocks.

 

Single-Source Support Limitations

Some Cribl Edge input sources allow only one Source per Fleet:

  • System State
  • Windows Metrics

UF deployment apps with inputs that convert to these equivalents must be merged, potentially requiring conflict resolution.

 

Automatic Metadata Population

One advantage of Cribl Edge is its automatic handling of common metadata fields, reducing configuration complexity compared to Universal Forwarders.

Cribl Edge automatically populates these metadata fields without additional configuration:

  • sourcetype
  • index
  • source
  • host
  • connection_host

Configurations Requiring Alternative Approaches

The following UF configuration settings cannot be directly translated to Cribl Edge and require alternative implementations:

  • host_regex: Extract host value from file path using regex
  • host_segment: Extract path segment as host value (requires pipeline creation)
  • crcSalt: Add salt value to file fingerprinting
  • followSymlink: Control symbolic link following behavior
  • alwaysOpenFile: Control file handling behavior

Alternative Implementation Methods

  • Create corresponding Pipelines in Cribl Edge
  • Use Edge Node metadata mapping for DNS information
  • Implement data transformation through Routes (all inputs forward to Routes)

Migration Implementation Considerations

Once you've completed your planning phase, the implementation phase requires attention to configuration details and platform-specific considerations specific to UF environments.

 

Configuration Review Areas

Each configuration area requires careful review to ensure optimal performance and functionality:

 

File Monitor Settings

  • Determine whether to start from the beginning or end of files
  • Consider the impact of large file sizes or many old events
  • Assess network bandwidth and Destination capacity implications

System State Configuration

  • System State Source is enabled by default on Cribl Edge Fleets
  • Disable the Source or specific collectors if System State data is not needed

Pipeline Configuration

  • Review existing UF configurations and enable similar functionality in Cribl
  • For example, review your UF data dropping configurations and create a Stream Pipeline that replaces this functionality
  • Optimize regexes for performance
  • Consider data transformation requirements previously handled in UF, and enable similar functionality in Cribl by using Functions in Pipelines

Persistent Queue Management

Review Persistent Queue settings in your UFs, Cribl Edge, and Stream, to ensure proper queue sizing for expected data volumes.

 

Windows-Specific Considerations

Windows environments present unique challenges during UF migration, particularly around policy management and PowerShell usage.

 

Group Policy Objects

Check for GPOs that might automatically reinstall or enable removed third-party agents. And update policies before migration to prevent conflicts.

 

PowerShell Usage

For optimal performance on Windows Edge Nodes, ensure the "Use Windows Tools" setting is disabled for all data Sources. When disabled, the system uses native WMI API methods for faster and more reliable data collection. When enabled, the system falls back to legacy PowerShell cmdlets.

The PowerShell-based collection methods will be removed in future releases as native methods provide superior speed and reliability.

 

Policy Management

Some sites might have policies to re-enable/re-install UFs  if missing/removed. Update or exclude policies before migration.

 

Addressing Inheritance Limitations

When working with Edge's inheritance model, be aware of these challenges:

  • Configuration integrity: Changes to a Source/Destination in a Subfleet can break inheritance from the parent.
  • Limited flexibility with Single-Source Support: Sources that allow only one configuration at a time, like Windows Event Logs, create inheritance challenges when customization is needed.

 

Appendix: Input Configuration Mapping Tables

This appendix provides detailed mapping tables for translating Universal Forwarder input configurations to their Cribl Edge equivalents. These tables serve as reference materials for understanding how specific UF settings correspond to Edge configurations. Use these mappings in conjunction with the migration considerations outlined in the main document.

 

File Monitor and Batch Input Mapping

For details on configuring the settings, see Configuring the File Monitor Source documentation.

UF Configuration

Edge Configuration

amonitor://<path>]

path and filenames

blacklist

filenames (negated)

whitelist

filenames

recursive

depth

time_before_close

idleTimeout

initCrcLength

hashLen

followTail

tailOnly

ignoreOlderThan

maxAgeDur

multiline_event_extra_waittime

staleChannelFlushMs

disabled

disabled

abatch://<path>]

path, filenames, deleteFiles (For batch)

 

Windows Event Log Mapping

For details on configuring the settings, see the Windows Event Logs Source documentation.

UF Configuration

Edge Configuration

bWinEventLog://<name>]

logNames

current_only

readMode

batch_size

batchSize

renderXml

eventFormat

 

TCP Input Mapping

For details on configuring the settings, see the TCP Source documentation.

UF Configuration

Edge Configuration

"tcp://<port>]

port

connection_timeout

connectionTimeout

acceptFrom

ipWhitelistRegex

ssl

tls

ssl_cert_path

certPath

ssl_root_ca_path

caPath

ssl_password

passphrase

ssl_verify_server_cert

requestCert

 

UDP Input Mapping

For details on configuring the settings, see the UDP Source documentation.

UF Configuration

Edge Configuration

=udp://<remote_server>:<port>]

port, ipWhitelistRegex

acceptFrom

ipWhitelistRegex

_rcvbuf

udpSocketRxBufSize

no_appending_timestamp

singleMsgUdpPackets

 

Windows Host Monitor Mapping

For details on configuring the settings, see the System State Source documentation.

UF Configuration

Edge Configuration

type values

collectors

interval

interval

 

Performance Monitor Mapping

For details on configuring the settings, see the System State Source documentation.

UF Configuration

Edge Configuration

object values

collectors

interval

interval

counters

mode

 
Be the first to reply!