0

uberAgent splunkd Process Utilizing Memory

  • Splunk 8.0.7
  • SUSE Linux 12.5
  • uberAgent 6.0
  • RAM - 128Gb
  • CPU - 48 vCPU

I have Splunk processes utilizing memory up to 85%, the oom-reaper will activate and kill the process, this will create a zombie process, Splunk will report the indexer down, memory will drop back down and this whole process will start over.  This is happening on all 12 indexers and always an uberAgent process.

Below are the screen shots of the PID process information, and TOP screens showing 60% memory usage, 70% memory usage, the oom-reaper process activated and then showing memory after the process has been removed.

5 comments

  • Avatar
    Dominik Britz Official comment

    An upgrade to Splunk 8.2.2 solved a known issue with the pivot command that uberAgent uses heavily.

    https://docs.splunk.com/Documentation/Splunk/8.0.4/ReleaseNotes/Knownissues
    2021-04-14 SPL-204072, SPL-207674, SPL-207675 Using a subsearch on an accelerated datamodel leads to incorrect results

  • 0
    Avatar
    Helge Klein

    Hi Mark,

    Thanks for your detailed description of the issue. We'll analyze the data and get back to you soon.

  • 0
    Avatar
    Dominik Britz

    Forwarded to our support system. We will post the results here, too.

  • 0
    Avatar
    Mark Swenson

    Update and Status

    We upgraded to Splunk 8.2.2.1 and still had the same issue.  We upgraded to 9.0.4.1 and after watching for several weeks, it looks like this issue has been resolved.

  • 0
    Avatar
    Dominik Britz

    Thanks for the update Mark!

Please sign in to leave a comment.