Using Available Fields for Initial Datatype Rule Filters in Cribl Search Lakehouse

Objective

Write an initial Getting Data In (GDI) datatype rule filter in Cribl Search Lakehouse using the correct available fields.

Environment

• Cribl Search 4.17.0 (Lakehouse engines)
• Search > Data > Getting Data In
• Initial Datatype / Dataset rule KQL expression field

Procedure

1. Identify which fields are available at the initial GDI rule stage. Only a small set of metadata exists at this point, before any parsing or time extraction has occurred:

   • _raw -- full raw event string as received from the source

   • __inputId -- ID or name of the GDI source that ingested the event

   • port -- destination TCP/UDP port of the source

   • host -- source host (IP or FQDN) that sent the data

2. Write the filter expression using only the fields listed above. Do not reference fields that are created after parsing or later in the pipeline, such as type, action, source_ip, dest_ip, datatype, _time, or schema. Conditions that reference those fields will not match because they do not yet exist when this rule runs.

3. Select the correct KQL syntax for the filter. Use a bare expression for simple equality checks:

   __inputId == "wef_search_lakehouse_source"

   Use a | where clause for operators like matches regex or has:

   | where _raw matches regex "TRAFFIC"

4. Keep the expression focused and inexpensive. This filter runs against every ingested event, so avoid overly complex regular expressions.

Additional Information

For more information, see Getting Data In - Cribl Search Documentation.