Skip to main content
Question

Redis filtering issue with unexpected results and need help on response handling. Any suggestions?

  • March 11, 2025
  • 8 replies
  • 6 views

Hi All,got a question in regards to Redis.We encounter issues, that filtering isn't working as expected. We use Redis as a whitelist, meaning that we extract a field from the raw log and using the EXISTS command to check if the key (build as a combination of strings) exists in Redis. If it does not exists in Redis the return value is `'0'` and we will drop the event.What we discovered is, that events that should be dropped are available at the destination.My questions are:What is / Is there a return value from Redis in case Redis was not able to process the query (EXISTS command)? Proabably not `'0'` , but if there is a response I somehow need to catch it.That was the only thing that came to my mind could cause the issue.Do you have some other ideas?Cheers,Mario

8 replies

Jon Rust
Forum|alt.badge.img
  • Employee
  • March 11, 2025

Can you share your exact filter in the Drop function? Are you comparing with `0` or `'0'`?


  • Author
  • Employee
  • March 11, 2025

Hi Jon,filter is: `__asset_exist=='0'`


Jon Rust
Forum|alt.badge.img
  • Employee
  • March 11, 2025

are you sure it's coming back as a string?


Jon Rust
Forum|alt.badge.img
  • Employee
  • March 11, 2025

i'd validate the type of var


  • Author
  • Employee
  • March 11, 2025

It is a Number, but it shouldn't matter, because:`0 == '0'` > `true``0 === '0'` > `false`


Jon Rust
Forum|alt.badge.img
  • Employee
  • March 11, 2025

can you share the pipeline?


  • Author
  • Employee
  • March 11, 2025

See screenshot.What you can't see is the RegExtract function.Already changed the drop filter to a more conservative one. Before it was as described:`__asset_exist == '0' && __asset_exist_lower == '0'`


  • Author
  • Employee
  • March 11, 2025

Also this route is running for over a month with ~120m events per day, but there are only a few (<20) suspicious events.That is why I was asking this:""" What is / Is there a return value from Redis in case Redis was not able to process the query (EXISTS command)? Proabably not `'0'` , but if there is a response I somehow need to catch it. """