The ultimate splunk> “Magic 8” for a dramatic performance boost

The magic 6 8 (increased to 8 recently)

Here a well known (for long time splunkers) but hard to find splunk> best practice about how to easily increase efficiency and performance when getting data into Splunk!


These sourcetype changes should always be added into an own TA (Technology Addon) and need to be deployed on your intermediate HF(s) / Indexer(s). For details where things apply in splunk:


The splunk> magic 6 8 need to be configured in your props.conf :

TIME_PREFIX = regex of the text that leads up to the timestamp
MAX_TIMESTAMP_LOOKAHEAD = how many characters for the timestamp
TIME_FORMAT = strftime format of the timestamp
# for multiline events: SHOULD_LINEMERGE should always be set to false as LINE_BREAKER will speed up multiline events
# Wherever the LINE_BREAKER regex matches, Splunk considers the start
# of the first capturing group to be the end of the previous event
# and considers the end of the first capturing group to be the start of the next event.
# Defaults to ([\r\n]+), meaning data is broken into an event for each line
LINE_BREAKER = regular expression for event breaks
TRUNCATE = 999999 (always a high number / not 0)
# Use the following attributes to handle better load balancing from UF.
# Please note the EVENT_BREAKER properties are applicable for Splunk Universal
# Forwarder instances only. Valid with forwarders > 6.5.0
EVENT_BREAKER = regular expression for event breaks