Hi,
We're looking to upgrade to uberAgent 6.1 from uberAgent 5.2.1 in a Distributed Splunk environment with a multi-site index cluster, multiple search heads and a few heavy forwarders running uberAgent Splunk apps.
According to the upgrade documentation (link below) the high level plan seems to be:
https://uberagent.com/docs/uberagent/latest/installation/upgrading-uberagent/
1) Clear KV stores and uninstall Splunk apps in all required locations.
2) Install new Splunk apps in all required locations.
3) Upgrade the endpoint agents.
Once I do step 1) what happens to all the data coming in from uberAgent endpoints before I install the new Splunk apps in step 2)? We have the Splunk index defined in our cluster master managed indexes.conf file so the splunk Index will still exist. This means that, theoretically, data can still write there if required. But if I literally uninstall the indexer app there will be Splunk ingest time parsing config that will no longer be accessible to incoming data. Theoretically this data will still get written to the index, this time without proper parsing, for a period of time until the new ones are installed. Is that ideal?
What happens after I've done 1) and 2) and endpoints are still running 5.2.1? Will the data coming in from them work with the new Splunk apps until which time all the endpoints are upgraded too? Or is the data coming in from 5.2.1 endpoints incompatible with the Splunk ingest time parsing config present in the new 6.1.1 indexer app?
Basically I have some concerns that at various stages of this plan various Splunk components, e.g. Indexers & Heavy Forwarders, will potentially be running different, or no, version of the Splunk apps and am keen to understand what this will mean for the fleet of endpoints running in our environment and the data they are sending in.
Perhaps there is an alternative plan where I disable all inputs collecting the incoming endpoint data so it can't get into Splunk until all the apps are upgraded and re-installed. I'm unsure what that would do the endpoints in terms of them having to buffer that data until the inputs are enabled again. Also that still doesn't solve the second question about the endpoints being different version... unless I didn't re-enable the inputs until they were all upgraded too.
Cheers for any answers/tips!
Michael.
