Skip to main content
Question

Can we merge two APIs response in Cribl?

  • Mar 11, 2025 3:15 AM UTC
  • 13 replies
  • 191 views

Hey,Can we merge two APIs response in Cribl? I have implemented discover and used the data from discover in collect but in the response, I am getting the data of collect only. So can we get the data of discover and collect in the response?

13 replies

Raanan Dagan

Normally the Discover data is used by the Collection phase (i.e. Cribl use one list of files (from the Discover phase) as an Input for the Collection)


Raanan Dagan

If yes, you can combine these using Redis .. or do the combination on the destination (Elasticsearch .. Splunk .. etc)


Raanan Dagan

Can I assume you need both set of events / files?


  • Author
  • Employee
  • Mar 11, 2025 4:29 AM UTC

But my data depends on discovery phase as well


  • Author
  • Employee
  • Mar 11, 2025 4:29 AM UTC

Yes, so I need to create 2 collectors and set the same destination right?


  • Author
  • Employee
  • Mar 11, 2025 4:29 AM UTC

<@U01J549PR6Y&gt; Do I need to create 2 collectors and set the same destination?


  • Author
  • Employee
  • Mar 11, 2025 4:29 AM UTC

<@U03KND56HLL&gt; <@U01LSBF5953&gt; Can you guys please help me with the above query?


<@U053XJ96R2L&gt; what is the API you are working with?you will most likely need two separate API calls if you want the discovery data passed to the destination


Raanan Dagan
  • 1 .. :thiscribl:

  • Author
  • Employee
  • Mar 11, 2025 4:29 AM UTC

There is one primary API from which I want the response and using one field from the primary API I want to extract other data using secondary API and also want the response of secondary API. So Can I do this using one Rest Collector in Cribl?


<@U053XJ96R2L&gt; , No, you will most likely need two separate REST collectors. One to only grab the discovery data (with the discover request in the collect URL), and one with both the Discover and Collect URL configured.


For an example of using the discover to pass information to the collect URL. Please see the documentation here: https://docs.cribl.io/stream/usecase-rest/#5-http-discover-and-collect-with-login-authentication


  • Author
  • Employee
  • Mar 11, 2025 4:29 AM UTC

Okay, thanks :+1: