Splunk Book

8/17/2019 Splunk Book

1/275

Splunk Overview Training


2/275

1

Duration: 3 days

Skill Level: Introductory and beyond

Hands-On Format: This hands-on class is approximately 50% hands-on lab to 50% lecture ratio, combining engaging lecture, demos, group activities and

discussions with machine-based practical student labs and project work.

Course Overview

Are you in charge of creating Splunk knowledge objects for your organization? Then you will benefit from this course that walks you through the various

knowledge objects and how to create them. Working with Splunk is a comprehensive hands-on course that teaches students how to search, navigate, tag,

build alerts, create simple reports and dashboards in Splunk, and how to Splunk's Pivot interface.

Working in a hands-on learning environment, students will learn how to use Splunk Analytics to provide an efficient way to search large volumes of data.

Students will learn how to run Basic Searches, Save and Share Search Results, Create Tags and Event Types, Create Reports, Create Different Charts, Perform

Calculations and Format Search Data, and Enrich Data with Lookups. Examples will center around financial institution examples.

What You’ll Learn: Course Objectives

After completion of this Splunk course, you will be able to:

Get insight into Splunk Search App

Learn to save and share Search Results

Understand the use of fields in searching

Learn Search Fundamentals using Splunk

Explore the available visualizations on the software

Create Reports and different Chart Types

Perform Data Analysis, Calculation and Formatting

Understand and execute various techniques of enriching data lookups


3/275

2

Recommended Audience & Pre-Requisites

This is a technical class for technical people, geared for Users, Administrators, Architects, Developers & Support Engineers who are new to Splunk. This course

is ideal for anyone in your organization who need to examine and use IT data.

Ideal attendees would include: Beginners in Splunk who want to enhance their knowledge about this Software usage

System Administrators and Software Developers

Professionals who are eager to learn to search and analyze machine-generated data using a faster and agile software

Course Topics & Agenda

Course Modules 1-4 Day 1 - Morning

Module 1 - Basic Understanding of Architecture (Overview) What are the components?

Discussion on Forwarders- UF/HF Common ports for the set up

License Master/Slave relationship

Understanding of Deployment Server and Indexer

Module 2 - Introduction to Splunk's User Interface Understand the uses of Splunk

Define Splunk Apps

Learn basic navigation in Splunk

Hands on Lab covering: Basic Navigation

End of Module Hands-on Quiz

Module 3 - Searching Run basic searches

Set the time range of a search

Hands on Lab covering: Run basic searches,


Identify the contents of search results

Refine searches


4/275

3

Hands on Lab covering: Identify the contents of search results, Refine searches

Use the timeline

Work with events

Hands on Lab covering: Use the timeline, Work with events

Control a search job

Save search results Hands on Lab covering: Control a search job, Save search results


Module 4 - Using Fields in Searches

Understand fields

Use fields in searches

Use the fields sidebar

Hands on Lab covering: Understand Fields, Use fields in searches, Use the fields sidebar


Course Modules 5-7 Day 1 - Afternoon

Module 5- Creating Reports and Visualizations

Save a search as a report

Edit reports

Create reports that include visualizations such as charts and tables

Hands on Lab covering: Save a search as a report, Edit Reports, Create reports that include visualizations such as charts and tables.

Add reports to a dashboard

Create an instant pivot from a search

Hands on Lab covering: Add reports to a dashboard, Create an instant pivot from a search.

End of Module Hands on Quiz

Module 6 - Working with Dashboards

Create a dashboard

Add a report to a dashboard

Hands on Lab covering: Create a dashboard, Add a report to a dashboard

Add a pivot report to a dashboard


5/275

4

Edit a dashboard

Hands on Lab covering: Add a pivot report to a dashboard, Edit a dashboard.


Module 7 - Search Fundamentals

Review basic search commands and general search practices Examine the anatomy of a search

Use the following commands to perform searches:

Fields

Table

Rename

Rex

Multikv

Hands on Lab covering: Review basic search commands and general search practices, Examine the anatomy of a search, Use the fo llowing commands to

perform searches: Fields, Table, Rename, Rex, Multikv.

End of Module Hands on Quiz.

Course Modules 8-10 Day 2 – Morning (Deep Dive Topics)

Module 8 - Reporting Commands, Part 1

Use the following commands and their functions:

Top

Rare

Hands on Lab covering: Top, Rare

Stats

Add coltotals

Hands on Lab covering: Stats, Add Coltotals


Module 9 - Reporting Commands, Part 2

Explore the available visualizations

Create a basic chart

Split values into multiple series

Hands on Lab covering: Explore the available visualizations, Create a basic chart, Split values into multiple series

Omit null and other values from charts


6/275

5

Create a time chart

Chart multiple values on the same timeline

Hands on Lab covering: Omit null and other values from charts, Create a time chart, Chart multiple values on the same timeline

Format charts

Explain when to use each type of reporting command

Hands on Lab covering: Format Charts, Explain when to use each type of reporting command. End of Module hands on Quiz

Module 10 - Analyzing, Calculating, and Formatting Results

Using the eval command

Perform calculations

Convert values

Hands on Lab covering: Using the eval command, Perform calculations, Convert values.

Round values

Format values

Hands on Lab covering: Round values, Format values

Use conditional statements

Further filter calculated results

Hands on Lab covering: Use conditional statements, Further filter calculated results


Course Modules 11-12 Day 2 – Afternoon (Deep Dive Topics)

Module 11 - Creating Field Aliases and Calculated Fields Define naming conventions

Create and use field aliases

Create and use calculated fields

Hands on Lab covering: Define naming conventions, Create and use field aliases, Create and use calculated fields.


Module 12 - Creating Field Extractions

Perform field extractions using Field Extractor

Hands on Lab covering: Perform f ield extractions using Field Extractor



7/275

6

Course Modules 13-15 Day 3 - Morning

Module 13 - Creating Tags and Event Types

Create and use tags

Describe event types and their uses Create an event type

Hands on Lab covering: Create and use tags, Describe event types and their uses, create and event type.


Module 14 - Creating Workflow Actions

Describe the function of a workflow action

Create a GET workflow action

Hands on Lab covering: Describe the function of a workflow action, Create a GET workflow action

Create a POST workflow action

Create a Search workflow action

Hands on Lab covering: Create a POST workflow action, Create a SEARCH workflow action


Module 15 - Creating and Managing Alerts

Describe alerts

Create alerts

View fired alerts

Hands on Lab covering: Describe alerts, Create alerts, View fired alerts


Course Modules 16-17 Day 3 - Afternoon

Module 16 - Creating and Using Macros Describe macros

Manage macros

Create and use a basic macro

Hands on Lab covering: Describe macros, Manage macros, Create and use a basic macro.

Define arguments and variables for a macro

Add and use arguments with a macro

Hands on Lab covering: Define arguments and variable for a macro, Add and use arguments with a macro.


8/275

7


Module 17 - Using Pivot

Describe Pivot

Understand the relationship between data models and pivot

Select a data model object Hands on Lab covering: Describe Pivot, Understand the relationship between data models and pivot, Select a data model object.

Create a pivot report

Save pivot report as a dashboard

Hands on Lab covering: Create a pivot report, Save pivot report as a dashboard.

End of Module Hands on Quiz.

Post Course Final Quiz

At the end of class, each attendee will take a Post Course Quiz that will gauge the student’s retention of the skills and topics covered throughout the course.

The quiz will be distributed either on paper or online at the end of class and graded promptly.


9/275

8

Module 1 - Basic Understanding of Architecture (Overview) What are the components?

Discussion on Forwarders- UF/HF

Common ports for the set up License Master/Slave relationship

Understanding of Deployment Server and Indexer


10/275

9

Section 1-What are the components?

Splunk Enterprise performs three key functions as it moves data through the data pipeline. First, it consumes data from files, the

network, or elsewhere. Then it indexes the data. (Actually, it first parses and then indexes the data, but for purposes of thisdiscussion, we consider parsing to be part of the indexing process.) Finally, it runs interactive or scheduled searches on the indexeddata.

You can split this functionality across multiple specialized instances of Splunk Enterprise, ranging in number from just a few tothousands, depending on the quantity of data you're dealing with and other variables in your environment. You might, for example,create a deployment with many instances that only consume data, several other instances that index the data, and one or moreinstances that handle search requests. These specialized instances are known collectively as components. There are several types ofcomponents.

For a typical mid-size deployment, for example, you can deploy lightweight versions of Splunk Enterprise, called forwarders, on themachines where the data originates. The forwarders consume data locally and then forward the data across the network to anotherSplunk Enterprise component, called the indexer. The indexer does the heavy lifting; it indexes the data and runs searches. It shouldreside on a machine by itself. The forwarders, on the other hand, can easily co-exist on the machines generating the data, because thedata-consuming function has minimal impact on machine performance.


11/275

10

This diagram shows several forwarders sending data to a single indexer:

As you scale up, you can add more forwarders and indexers. For a larger deployment, you might have hundreds of forwarders sendingdata to a number of indexers. You can use load balancing on the forwarders, so that they distribute their data across some or all of theindexers. Not only does load balancing help with scaling, but it also provides a fail-over capability if one of the indexers goes down.The forwarders automatically switch to sending their data to any indexers that remain alive.


12/275

11

In this diagram, each forwarder load-balances its data across two indexers:


13/275

12

These are the fundamental components and features of a Splunk Enterprise distributed environment:

Indexers.

Forwarders.

Search heads. Deployment server.

Indexer

A Splunk Enterprise instance that indexes data, transforming raw data into events and placing the results into an index. It alsosearches the indexed data in response to search requests.

The indexer also frequently performs the other fundamental Splunk Enterprise functions: data input and search management. Inlarger deployments, forwarders handle data input and forward the data to the indexer for indexing. Similarly, although indexersalways perform searches across their own data, in larger deployments, a specialized Splunk Enterprise instance, called a search head,handles search management and coordinates searches across multiple indexers.

Forwarder

A Splunk Enterprise instance that forwards data to another Splunk Enterprise instance, such as an indexer or another forwarder, or toa third-party system.

There are three types of forwarders:


14/275

13

A universal forwarder is a dedicated, streamlined version of Splunk Enterprise that contains only the essential componentsneeded to send data.

A heavy forwarder is a full Splunk Enterprise instance, with some features disabled to achieve a smaller footprint.

A light forwarder is a full Splunk Enterprise instance, with most features disabled to achieve a small footprint. The universalforwarder supersedes the light forwarder for nearly all purposes. The light forwarder has been deprecated as of SplunkEnterprise version 6.0.0.

The universal forwarder is the best tool for forwarding data to indexers. Its main limitation is that it forwards only unparsed data. Tosend event-based data to indexers, you must use a heavy forwarder.

Search Heads

In a distributed search environment, a Splunk Enterprise instance that handles search management functions, directing searchrequests to a set of search peers and then merging the results back to the user.

A Splunk Enterprise instance can function as both a search head and a search peer. A search head that performs only searching, andnot any indexing, is referred to as a dedicated search head.

Search head clusters are groups of search heads that coordinate their activities.

Deployment Server

A Splunk Enterprise instance that acts as a centralized configuration manager, grouping together and collectively managing anynumber of Splunk Enterprise instances. Instances that are remotely configured by deployment servers are called deployment clients.The deployment server downloads updated content, such as configuration files and apps, to deployment clients. Units of such content

are known as deployment apps.


15/275

14

Section 2-Discussion on Forwarders- UF/HF

Theuniversalforwarder

The universal forwarder is Splunk's new lightweight forwarder. You use it to gather data from a variety of inputs and forward the datato a Splunk Enterprise server for indexing and searching. You can also forward data to another forwarder, as an intermediate step before sending the data onwards to an indexer.

The universal forwarder's sole purpose is to forward data. Unlike a full Splunk Enterprise instance, you cannot use the universalforwarder to index or search data. To achieve higher performance and a lighter footprint, it has several limitations:

The universal forwarder has no searching, indexing, or alerting capability. The universal forwarder does not parse data.

Heavyandlightforwarders While the universal forwarder is generally the preferred way to forward data, you might have reason (legacy-based or otherwise) touse heavy forwarders as well. Unlike the universal forwarder, which is an entirely separate, streamlined executable, both heavy andlight forwarders are actually full Splunk Enterprise instances with certain features disabled.

A heavy forwarder (sometimes referred to as a "regular forwarder") has a smaller footprint than a Splunk Enterprise indexer butretains most of the capability, except that it lacks the ability to perform distributed searches. Much of its default functionality, such asSplunk Web, can be disabled, if necessary, to reduce the size of its footprint. A heavy forwarder parses data before forwarding it andcan route data based on criteria such as source or type of event.


16/275

15

This table summarizes the similarities and differences among the three types of forwarders:

Features and capabilities Universal forwarder Heavy forwarder

Type of Splunk Enterprise instance Dedicated executable Full Splunk Enterprise, with some

features disabled

Footprint (memory, CPU load) Smallest Medium-to-large (depending on enabledfeatures)

Bundles Python? No Yes

Handles data inputs? All types (but scripted inputs mightrequire Python installation)

All types

Forwards to Splunk Enterprise? Yes Yes

Forwards to 3rd party systems? Yes Yes

Serves as intermediate forwarder? Yes Yes

Indexer acknowledgment (guaranteeddelivery)?

Optional Optional (version 4.2+)

Load balancing? Yes Yes

Data cloning? Yes Yes

Per-event filtering? No Yes

Event routing? No Yes

Event parsing? No Yes

Local indexing? No Optional, by setting

indexAndForward attribute inoutputs.conf

Searching/alerting? No Optional

Splunk Web? No Optional


17/275

16

Section 3- Common ports for the set up

Splunk configures two ports at installation time:

The HTTP/HTTPS port. This port provides the socket for Splunk Web. It defaults to 8000.

The management port. This port is used to communicate with the splunkd daemon. Splunk Web talks to splunkd on this port, as does the command line interface and any distributed connections from other servers. This port defaults to 8089.

Let's login to our lab environment

Please go to: http://www.uxcreate.com/guacamole

User name: admin

Password: admin

Your instructor will give you your machine number. Please remember your machine number throughout the training session.

Then please go to Start > All Programs > Splunk Enterprise > Splunk Enterprise

The Splunk web interface should come up.

The login details : username: admin password: admin


18/275

17

Section 4-License Master/Slave relationship

Splunk Enterprise takes in data from sources you designate and processes it so that you can analyze it. We call this process indexing.

Splunk Enterprise licenses specify how much data you can index per calendar day (from midnight to midnight by the clock on thelicense master).

Any host in your Splunk Enterprise infrastructure that performs indexing must be licensed to do so. You can either run a standaloneindexer with a license installed locally, or you can configure one of your Splunk Enterprise instances as a license master and set up alicense pool from which other indexers, configured as license slaves, can draw.

When a license master instance is configured, and license slaves are added to it, the license slaves communicate their usage to thelicense master every minute. If the license master is unreachable for any reason, the license slave starts a 72 hour timer.

If the license slave cannot reach the license master for 72 hours, search is blocked on the license slave (although indexing continues).Users cannot search data in the indexes on the license slave until that slave can reach the license master again.


19/275

18

Section 5-Understanding of Deployment Server and Indexer

The indexer is the Splunk Enterprise component that creates and manages indexes. The primary functions of an indexer are:

Indexing incoming data. Searching the indexed data.

In single-machine deployments consisting of just one Splunk Enterprise instance, the indexer also handles the data input and searchmanagement functions.

For larger-scale needs, indexing is split out from the data input function and sometimes from the search management function as well.

In these larger, distributed deployments, the indexer might reside on its own machine and handle only indexing, along with searchingof its indexed data. In those cases, other Splunk Enterprise components take over the non-indexing roles.

For instance, you might have a set of Windows and Linux machines generating events, which need to go to a central indexer forconsolidation. Usually the best way to do this is to install a lightweight instance of Splunk Enterprise, known as a forwarder, on eachof the event-generating machines. These forwarders handle data input and send the data across the network to the indexer residing onits own machine.

Similarly, in cases where you have a large amount of indexed data and numerous concurrent users searching on it, it can make sense tosplit off the search management function from indexing. In this type of scenario, known as distributed search, one or more searchheads distribute search requests across multiple indexers. The indexers still perform the actual searching of their own indexes, but thesearch heads manage the overall search process across all the indexers and present the consolidated search results to the user.


20/275

19

Here's an example of a scaled-out deployment:

A deployment server uses server classes to determine what content to deploy to groups of deployment clients. The forwardermanagement interface offers an easy way to create, edit, and manage server classes.


21/275

20

Module 2 - Introduction to Splunk's User Interface Understand the uses of Splunk

Define Splunk Apps Learn basic navigation in Splunk




22/275

21

Section 1-Understand the uses of Splunk

Splunk Enterprise makes it simple to collect, analyze and act upon the untapped value of the big data generated by your technologyinfrastructure, security systems and business applications — giving you the insights to drive operational performance and businessresults.

By monitoring and analyzing everything from customer clickstreams and transactions to security events and network activity, SplunkEnterprise helps you gain valuable Operational Intelligence from your machine-generated data.

And with a full range of powerful search, visualization and pre-packaged content for use-cases, any user can quickly discover andshare insights. Just point your raw data at Splunk Enterprise and start analyzing your world.

Collects and indexes log and machine data from any source Powerful search, analysis and visualization capabilities empower users of all types

Apps provide solutions for security, IT ops, business analysis and more Enables visibility across on premise, cloud and hybrid environments

Delivers the scale, security and availability to suit any organization

Available as a software or SaaS ( Software as a Solution) solution


23/275

22

Section 2-Define Splunk Apps

A Splunk App is a prebuilt collection of dashboards, panels and UI elements powered by saved searches and packaged for a specifictechnology or use case to make Splunk immediately useful and relevant to different roles.

As an alternative to using Splunk for searching and exploring, you can use Splunk Apps to gain the specific insights you need fromyour machine data.

You can also apply user/role based permissions and access controls to Splunk Apps, thus providing a level of control when you aredeploying and sharing Apps across your organization.

Apps can be opened from the Splunk Enterprise Home Page, from the App menu, or from the Apps section of Settings.


24/275

23

Section 3-Learn basic navigation in Splunk

AboutSplunkHome

Splunk Home is your interactive portal to the data and apps accessible from this Splunk instance. The main parts of

Home include the Splunk Enterprise navigation bar, the Apps menu, the Explore Splunk Enterprise panel, and acustom default dashboard (not shown here).

Apps

The Apps panel lists the apps that are installed on your Splunk instance that you have permission to view. Select the

app from the list to open it.

For an out-of-the-box Splunk Enterprise installation, you see one App in the workspace: Search & Reporting. When

you have more than one app, you can drag and drop the apps within the workspace to rearrange them.

You can do two actions on this panel:

Click the gear icon to view and manage the apps that are installed in your Splunk instance.


25/275


26/275

25

ReturntoSplunkHome

Click the Splunk logo on the navigation bar to return to Splunk Home from any other view in Splunk Web.

Settingsmenu

The Settings menu lists the configuration pages for Knowledge objects, Distributed environment settings, System and

licensing, Data, and Authentication settings. If you do not see some of these options, you do not have the permissions toview or edit them.


27/275

26

Usermenu

The User menu here is called "Administrator" because that is the default user name for a new installation. You canchange this display name by selecting Edit account and changing the Full name. You can also edit the time zone

settings, select a default app for this account, and change the account's password. The User menu is also where youLogout of this Splunk installation.

Messagesmenu

All system-level error messages are listed here. When there is a new message to review, a notification displays as a countnext to the Messages menu. Click the X to remove the message.

Activitymenu

The Activity menu lists shortcuts to the Jobs, Triggered alerts, and System Activity views.

Click Jobs to open the search jobs manager window, where you can view and manage currently runningsearches.


28/275

27

Click Triggered Alerts to view scheduled alerts that are triggered. This tutorial does not discuss saving andscheduling alerts. See "About alerts" in the Alerting Manual.

Click System Activity to see Dashboards about user activity and status of the system.

Help

Click Help to see links to Video Tutorials, Splunk Answers, the Splunk Support Portal, and online Documentation.

Find

Use Find to search for objects within your Splunk Enterprise instance. Find performs non-case sensitive matches on theID, labels, and descriptions in saved objects. For example, if you type in "error", it returns the saved objects that contain

the term "error".

These saved objects include Reports, Dashboards, Alerts, and Data models. The results appear in the list

separated by the categories where they exist.


29/275

28

You can also run a search for error in the Search & Reporting app by clicking

Open error in search.


Take your time exploring the Splunk Web interface


30/275

29


Please refer to your virtual machine for test


31/275

30

Module 3 - Searching Run basic searches


Hands on Lab covering: Run basic searches,

Set the time range of a search Identify the contents of search results

Refine searches

Hands on Lab covering: Identify the contents of search results, Refine searches

Use the timeline

Work with events

Hands on Lab covering: Use the timeline, Work with events

Control a search job

Save search results

Hands on Lab covering: Control a search job, Save search results



32/275

31

Run basic searches

Typesofsearches

Before delving into the language and syntax of search, you should ask what you are trying to accomplish. Generally, after getting datainto Splunk, you want to:

Investigate to learn more about the data you just indexed or to find the root cause of an issue.

Summarize your search results into a report, whether tabular or other visualization format.

Because of this, you might hear us refer to two types of searches: Raw event searches and Report-generating searches.

Raweventsearches

Raw event searches are searches that just retrieve events from an index or indexes and are typically done when you want to analyze a problem. Some examples of these searches include: checking error codes, correlating events, investigating security issues, and

analyzing failures. These searches do not usually include search commands (except search, itself), and the results are typically a listof raw events.

Transformingsearches

Transforming searches are searches that perform some type of statistical calculation against a set of results. These are searcheswhere you first retrieve events from an index and then pass them into one or more search commands. These searches will alwaysrequire fields and at least one of a set of statistical commands. Some examples include: getting a daily count of error events, counting

the number of times a specific user has logged in, or calculating the 95th percentile of field values.


33/275

32

Informationdensity

Whether you're retrieving raw events or building a report, you should also consider whether you are running a search for sparse ordense information:

Sparse searches are searches that look for single event or an event that occurs infrequently within a large

set of data. You've probably heard these referred to as 'needle in a haystack' or "rare term" searches. Some

examples of these searches include: searching for a specific and unique IP address or error code.

Dense searches are searches that scan through and report on many events. Some examples of these

searches include: counting the number of errors that occurred or finding all events from a specific host.

Searchandknowledge

As you search, you may begin to recognize patterns and identify more information that could be useful as searchable fields. You canconfigure Splunk to recognize these new fields as you index new data or you can create new fields as you search. Whatever you learn,you can use, add, and edit this knowledge about fields, events, and transactions to your event data. This capturing of knowledge helpsyou to construct more efficient searches and build more detailed reports.


34/275

33

Theanatomyofasearch

To better understand how search commands act on your data, it helps to visualize all your indexed data as a table. Each search

command redefines the shape of your table.

For example, let's take a look at the following search.

sourcetype=syslog ERROR | top user | fields - percent

The Disk represents all of your indexed data and it's a table of a certain size with columns represent fields and rows representingevents. The first intermediate results table shows fewer rows--representing the subset of events retrieved from the index that matchedthe search terms "sourcetype=syslog ERROR". The second intermediate results table shows fewer columns, representing the results ofthe top command, "top user", which summarizes the events into a list of the top 10 users and displays the user, count, and percentage.Then, "fields - percent" removes the column that shows the percentage, so you are left with a smaller final results table.

http://docs.splunk.com/File:Anatomy_of_a_search.png


35/275

34

Quotesandescapingcharacters

Generally, you need quotes around phrases and field values that include white spaces, commas, pipes, quotes, and/or brackets. Quotes must be balanced, an opening quote must be followed by an unescaped closing quote. For example:

A search such as error | stats count will find the number of events containing the string error.

A search such as ... | search "error | stats count" would return the raw events containing error, a pipe,

stats, and count, in that order.

Additionally, you want to use quotes around keywords and phrases if you don't want to search for their default meaning, such asBoolean operators and field/value pairs. For example:

A search for the keyword AND without meaning the Boolean operator: error "AND"

A search for this field/value phrase: error "startswith=foo"

The backslash character (\) is used to escape quotes, pipes, and itself. Backslash escape sequences are still expanded inside quotes.For example:

The sequence \| as part of a search will send a pipe character to the command, instead of having the pipe

split between commands.

The sequence \" will send a literal quote to the command, for example for searching for a literal quotation

mark or inserting a literal quotation mark into a field using rex.

The \\ sequence will be available as a literal backslash in the command.

If Splunk does not recognize a backslash sequence, it will not alter it.

For example \s in a search string will be available as \s to the command, because \s is not a known escape

sequence.

However, in the search string \\s will be available as \s to the command, because \\ is a known escape

sequence that is converted to \.


36/275

35

Asterisks, *, can not be searched for using a backslash to escape the character. Splunk treats the asterisk character as a major breaker. Because of this, it will never be in the index. If you want to search for the asterisk character, you will need to run a post-filtering regex search on your data:

index=_internal | regex ".*\*.*"

Examples

Example 1: myfield is created with the value of 6.

... | eval myfield="6"

Example 2: myfield is created with the value of ".

... | eval myfield="\""

Example 3: myfield is created with the value of \.

... | eval myfield="\\"

Example 4: This would produce an error because of unbalanced quotes.

... | eval myfield="\"


37/275

36


Time is crucial for determining what went wrong. You often know when something happened, if not exactly what happened. Lookingat events that happened around the same time can help correlate results and find the root cause.

Searches run with overly-broad time range wastes system resources and produces more results than you can handle.

Selecttimerangestoapplytoyoursearch

Use the time range picker to set time boundaries on your searches. You can restrict a search with preset time ranges, create customtime ranges, specify time ranges based on date or date and time, or work with advanced features in the time range picker. Theseoptions are described in the following sections.

Note: If you are located in a different timezone, time-based searches use the timestamp of the event from the instance that indexed thedata.


38/275

37

SelectfromalistofPresettimeranges

http://docs.splunk.com/File:6.2_timerange_presets.png


39/275

38

DefinecustomRelativetimeranges

Use custom Relative time range options to specify a time range for your search that is relative to Now. You can select from the list oftime range units, "Seconds ago", "Minutes ago", and so on.

http://docs.splunk.com/File:6.2_timerange_preset2.png


40/275

39

The labels for Earliest and Latest update to match your selection.

The preview boxes below the fields update to the time range as you set it.

http://docs.splunk.com/File:6.2_timerange_relative.png


41/275

40

DefinecustomReal-timetimeranges

The custom Real-time option enables you to specify the start time for your real-time time range window.

http://docs.splunk.com/File:6.2_timerange_realtime.png


42/275

41

Define custom Date ranges

Use the custom Date Range option to specify calendar dates in your search. You can choose among options to return events: Between a beginning and end date, Before a date, and Since a date.

For these fields, you can type the date into the text box or select the date from a calendar:

http://docs.splunk.com/File:6.2_timerange_date2.png


43/275

42

http://docs.splunk.com/File:6.2_timerange_date.png


44/275

43

DefinecustomDate&Timeranges

Use the custom Date & Time Range option to specify calendar dates and times for the beginning and ending of your search.

You can type the date into the text box or select the date from a calendar.

http://docs.splunk.com/File:6.2_timerange_datetime.png


45/275

44

UseAdvancedtimerangeoptions

Use the Advanced option to specify the earliest and latest search times. You can write the times in Unix (epoch) time or relative timenotation. The epoch time value you enter is converted to local time. This timestamp is displayed under the text field so that you canverify your entry.

http://docs.splunk.com/File:6.2_timerange_advanced.png


46/275

45

Hands on Lab

Part 1 - Basic Concepts

There are a few concepts in the Splunk world that will be helpful for you to understand. I’ll cover them in a few sentences, so try to pay attention. If you want more details, see the “Concepts” section near the end of this document.

Processing at the time the data is processed: Splunk reads data from a source, such as a file or port, on a host (e.g. "mymachine"), classifies that source into a sourcetype (e.g., "syslog", "access_combined", "apache_error", ...), then extractstimestamps, breaks up the source into individual events (e.g., log events, alerts, …), which can be a single-line or multiple

lines, and writes each event into an index on disk, for later retrieval with a search.

Processing at the time the data is searched: When a search starts, matching indexed events are retrieved from disk, fields (e.g., code=404, user=david,...) are extracted from the event 's text, and the event is classified by matched against eventtype definitions (e.g., 'error', 'login', ...). The events returned from a search can then be powerfully transformed using Splunk's

search language to generate report s that live on dashboards.


47/275

46

Part 2 - Adding Data

Splunk can eat data from just about any source, including files, directories, ports, and scripts, keeping track of changes to

them as they happen. We're going to start simple and just tell Splunk to index a particular file and not monitor it forupdates:

1. Go to the Splunk Web interface (e.g. http://localhost:8000) and log in, if you haven’t already.2. Click Settings in the upper right-hand corner of Splunk Web.3. Under Settings, click Add Data.

4. Click Upload Data to upload file.5. Click Select File.6. Browse and find "websample.log" on your Desktop that we previously saved.7. Accept all the default values and just click Submit. 8. Click Start Searching

Assuming all goes well, websample.log is now indexed, and all the events are timestamped and searchable.


48/275

47

Part3-BasicSearching

Splunk comes with several Apps, but the only relevant one now is the 'Search' app, which is the interface for generic

searching. (More apps can be downloaded and advanced users can built them themselves.) After logging into Splunk,

select the Search app and let's get started in searching. We'll start out simple and work our way up.

To begin your Splunk search, type in terms you might expect to find in your data. For example, if you want to find events

that might be HTTP 404 errors (i.e., webpage not found), type in the keywords:

http 404

You'll get back all the events that have both HTTP and 404 in their text.


49/275

48

Notice that search terms are implicitly AND'd together. The search was the same as "http AND 404". Let's make the search

narrower:

http 404 "like gecko"


50/275

49

Using quotes tells Splunk to search for a literal phrase “like gecko”, which returns more specific results than just searching

for “like” and “gecko” because they must be adjacent as a phrase.

Splunk supports the Boolean operators AND, OR, and NOT (must be capitalized), as well as parentheses to enforce

grouping. To get all HTTP error events (i.e., not 200 error code), not including 403 or 404, use this:

http NOT (200 OR 403 OR 404)

Again, the AND operator is implied; the previous search is the same as

http AND NOT (200 OR 403 OR 404)

Splunk supports the asterisk (*) wildcard for searching. For example, to retrieve events that has 40x and 50xx classes of

HTTP status codes, you could try:

http (40* OR 50*)


51/275

50

When you index data, Splunk automatically adds fields (i.e., attributes) to each of your events. It does this based on some

text patterns commonly found in IT data, and intermediate users can add their own extraction rules for pulling out additional

fields.

To narrow results with a search, just add attribute=value to your search:

sourcetype=access_combined status=404

This search shows a much more precise version of our first search (i.e., "http 404") because it will only return events that

come from access_combined sources (i.e., webserver events) and that have a status code of 404, which is different than just

having a 404 somewhere in the text. The “404” has to be found where a status code is expected on the event and not justanywhere. In addition to =, you can also do != (not equals), and , >=, and


52/275

51

Part5-SearchApp

Now click on Search on the Main toolbar

You will get the following screen:


53/275

52

Click on the Data Summary button, you will get:

Click on the Sources tab, you will get:


54/275

53

Now you can choose websample.log, you will get:


55/275

54

Part6-Let’suploadanothersamplefile:

1. Please upload sampledata.zip, whichh is located on the Desktop2. Notice there is no preview.

3. Please take the defaults and start Searching4. On the Sourcetypes panel, click access_combined_wcookie


56/275

55

You are a member of the Customer Support team for the online Flower & Gift shop. This is your first day on the job. You want tolearn some more about the shop. Some questions you want answered are:

What does the store sell?

How much does each item cost?

How many people visited the site?

How many bought something today?

What is the most popular item that is purchased each day?

It's your first day of work with the Customer Support team for the online Flower & Gift shop. You're just starting to dig into the Webaccess logs for the shop, when you receive a call from a customer who complains about trouble buying a gift forhis girlfriend--he keeps hitting a server error when he tries to complete a purchase. He gives you his IP address, 10.2.1.44.


57/275

56

1. Type the customer's IP address into the search bar:

sourcetype="access_combined_wcookie" 10.2.1.44

As you type into the search bar, Splunk's search assistant opens.

Search assistant shows you typeahead, or contextual matches and completions for each keyword as

you type it into the search bar. These contextual matches are based on what's in your data. The entriesunder matching terms update as you continue to type because the possible completions for your termchange as well.


58/275

57

Part7-TimeRanges

Try different time ranges like the previous week within the search toolbar


59/275

58

Identify the contents of search results and refine searches

Splunk supports the Boolean operators: AND, OR, and NOT. When you include Boolean expressions in your search,the operators have to be capitalized.

Also you can mouse over results to refine searches


60/275

59

Hands on Lab

1. Please choose the Data Source LoanStats3a.csv. Remember click on Search on the Toolbar and then click on

the Data Summary Button.

2. Search for the word : Status

3. Then click on the word Paid and add to the search

4. Click on the word : RENT and exclude from search

BONUS LAB:

1. Without the use of fields, find the status of Not Paid and Not Mortgage


61/275


62/275

61

Format options are located in the Format Timeline menu:

You can hide the timeline (Hidden) and display a Compact or Full view of it. You can also toggle the timeline scale between linear(Linear Scale) or logarithmic (Log Scale).

http://docs.splunk.com/File:6.2_timeline_formatoptions.pnghttp://docs.splunk.com/File:6.2_timeline_compact.png


63/275

62

When Full is selected, the timeline is taller and displays the count on the y-axis and time on the x-axis.

Zoominandzoomouttoinvestigateevents

Zoom and selection options are located above the timeline. At first, only the Zoom Out option is available.

The timeline legend is on the top right corner of the timeline. This indicates the scale of the timeline. For example, 1 minute percolumn indicates that each column represents a count of events during that minute. Zooming in and out changes the time scale. Forexample, if you click Zoom Out the legend will indicate that each column now represents an hour instead of a minute.

When you mouse over and select bars in the timeline, the Zoom to Selection or Deselect options become available.

http://docs.splunk.com/File:6.2_timeline_full.png


64/275

63

Mouse over and click on the tallest bar or drag your mouse over a cluster of bars in the timeline. The events list updates to displayonly the events that occurred in that selected time range. The time range picker also updates to the selected time range. You can cancelthis selection by clicking Deselect.

When you Zoom to Selection, you filter the results of your previous search for your selected time period. The timeline and events listupdate to show the results of the new search.

http://docs.splunk.com/File:6.2_timeline_selectbars.png


65/275

64

You cannot Deselect after you zoomed into a selected time range. But, you can Zoom Out again.

http://docs.splunk.com/File:6.2_timeline_zoomout.pnghttp://docs.splunk.com/File:6.2_timeline_zoomtoselect.png


66/275

65

Work with events

An event is a single piece of data in Splunk software, similar to a record in a log file or other data input. When data

is indexed, it is divided into individual events. Each event is given a timestamp, host, source, and source type.

Often, a single event corresponds to a single line in your inputs, but some inputs (for example, XML logs) have

multiline events, and some inputs have multiple events on a single line. When you run a successful search, you getback events.


67/275

66

Hands on Lab

Back at the Flower & Gift shop, let's continue with the customer (10.2.1.44) you were assisting. He reported an error whilepurchasing a gift for his girlfriend. You confirmed his error, and now you want to find the cause of it.Continue with the last search, which showed you the customer's failed purchase attempts.

1. Type purchase into the search bar and run the search:

sourcetype="access_combined_wcookie" 10.2.1.44 purchase

When you search for keywords, your search is not case-sensitive and Splunk retrieves the events that contain those

keywords anywhere in the raw text of the event's data

Use Boolean operators

If you're familiar with Apache server logs, in this case the access_combined format, you'll notice that

most of these events have an HTTP status of 200, or Successful. These events are not interesting foryou right now, because the customer is reporting a problem.

Splunk supports the Boolean operators: AND, OR, and NOT. When you includeBoolean expressions in your search, the operators have to be capitalized.

2. Use the Boolean NOT operator to quickly remove all of these Successful pagerequests. Type in:


68/275

67

sourcetype="access_combined_wcookie" 10.2.1.44 purchase NOT 200

The AND operator is always implied between search terms. So the search in Step 5 is

the same as:

sourcetype="access_combined_wcookie" AND 10.2.1.44 AND purchase NOT 200

You notice that the customer is getting HTTP server (503) and client (404) errors. But, he specifically

mentioned a server error , so let's quickly remove events that are irrelevant.

Another way to add Boolean clauses quickly and interactively to your search is to use your searchresults. Splunk lets you highlight and select any segment from


69/275

68

Timeline Usage

Continue with the last search, which showed you the customer's failed purchase attempts.

1. Search for:

sourcetype="access_combined_wcookie" 10.2.1.44 purchase NOT 200 NOT 404 In the

last topic, you really just focused on the search results listed in the events viewer area ofthis dashboard. Now, let's take a look at the timeline.

The location of each bar on the timeline corresponds to an instance when the events that match yoursearch occurred. If there are no bars at a time period, no events were found then.

2. Mouse over one of the bars.

A tooltip pops up and displays the number of events that Splunk found during the time span of that bar (1

bar = 1 hour).


70/275

69

The taller the bar, the more events occurred at that time. Often seeing spikes in the number of events

or no events is a good indication that something has happened.

3. Click one of the bars, for example the tallest bar.

This updates your search results to show you only the events at the time span. Splunk does not run thesearch when you click on the bar. Instead, it gives you a preview of the results zoomed-in at the t imerange. You can still select other bars at this point.

One hour is still a wide time period to search, so let's narrow the search down more.

4. Double-click on the same bar.

Splunk runs the search again and retrieves only events during that one hour span you selected.


71/275

70

You should see the same search results in the Event viewer, but, notice that the search overrides thetime range picker and it now shows "Custom time". (You'll see more of the time range picker later.) Also,

each bar now represents one minute of time (1 bar = 1 min).

5. Double-click another bar.

Once again, this updates your search to now retrieve events during that one minute span of time. Each

bar represents the number of events for one second of time.

Now, you want to expand your search to see everything else, if anything, that happened during this

second.

6. Without changing the time range, replace your previous search in the search bar with:

*

Splunk supports using the asterisk (*) wildcard to search for "all" or to retrieve events based on parts


72/275

71

of a keyword. Up to now, you've just searched for Web access logs. This search tells Splunk that you

want to see everything that occurred at this time range:


73/275

72

Control search job progress

After you launch a search, you can access and manage information about the search's job without leaving the Search page. Once yoursearch is running, paused, or finalized, click Job and choose from the available options there.

You can:

Edit the job settings. Select this to open the Job Settings dialog, where you can change the job read permissions, extend the job lifetime, and get a URL for the job that you can use to share the job with others or put a link to the job in your browser's bookmark bar.

Send the job to the background. Select this if the search job is slow to complete and you would like to run the job in the background while you work on other Splunk activities (including running a new search job).

Inspect the job. Opens a separate window and display information and metrics for the search job using the Search JobInspector. You can select this action while the search is running or after it completes.

Delete the job. Use this to delete a job that is currently running, is paused, or which has finalized. After you have deleted the

job you can still save the search as a report.

http://docs.splunk.com/File:6.0_searchjob_progress.png


74/275

73

Change the search mode

The Search mode controls the search experience. You can set it to speed up searches by cutting down on the event data it returns ( Fast mode), or you can set it to return as much event information as possible (Verbose mode). In Smart mode (the default setting) itautomatically toggles search behavior based on the type of search you're running.

http://docs.splunk.com/File:6.0_searchmode.png


75/275


76/275

75

Click Share to share the job. When you select this, the job's lifetime is extended to 7 days and read permissions are set toEveryone.

Click Export to export the results. You can select to output to CSV, raw events, XML, or JSON and specify the number ofresults to export.

Click Print to send the results to a printer that has been configured.


77/275

76

Hands on Lab

1. Using your file LoanStats3a.csv, save your last search as an event type

2. Go to Settings, and click on event types to view your saved event type


78/275

77




79/275

78

Module 4 - Using Fields in Searches

Understand fields



Hands on Lab covering: Understand Fields, Use fields in searches, Use the fields sidebar End of Module Hands-on Quiz


80/275


81/275

80


Use the following syntax to search for a field: fieldname="fieldvalue" . Field names are case sensitive, but field values are not.

1. Go to the Search dashboard and type the following into the search bar:

sourcetype="access_*"

This indicates that you want to retrieve only events from your web access logs and nothing else.

sourcetype is a field name and access_* is a wildcarded field value used to match any Apache web access event. Apache webaccess logs are formatted as access_common, access_combined, or access_combined_wcookie.

2. In the Events tab, scroll through the list of events.

If you are familiar with the access_combined format of Apache logs, you recognize some of the information in each event, such as:

IP addresses for the users accessing the website.

URIs and URLs for the pages requested and referring pages. HTTP status codes for each page request.

GET or POST page request methods.


82/275

81


To the left of the events list is the Fields sidebar. As Splunk Enterprise retrieves the events that match your search, the Fields sidebarupdates with Selected fields and Interesting fields. These are the fields that Splunk Enterprise extracted from your data.

Selected Fields are the fields that appear in your search results. The default fields host, source, and sourcetype are selected.

You can hide and show the fields sidebar by clicking Hide Fields and Show Fields, respectively.

3. Click All Fields.

The Select Fields dialog box opens, where you can edit the fields to show in the events list.

You see the default fields that Splunk defined. Some of these fields are based on each event's timestamp (everything beginning with

date_*), punctuation (punct), and location (index).

http://docs.splunk.com/File:Tutorial_fields_sidebar.png


83/275

82

Other field names apply to the web access logs. For example, there are clientip, method, and status. These are not default fields.They are extracted at search time.

This opens the field summary for the action field.

In this set of search results, Splunk Enterprise found five values for action, and that the action field appears in 49.9% of your searchresults.


84/275

83

Hands on Lab

1. Go back to the Search dashboard and search for web access activity. Select

Other > Yesterday from the t ime range picker:

sourcetype="access_*"You were actually using fields all along! Each time you searched for sourcetype=access_*, you told

Splunk to only retrieve events from your web access logs and nothing else.

To search for a particular field, specify the field name and value:

fieldname="fieldvalue"

sourcetype is a field name and access_combined_wcookie is a field value. Here, the wildcarded value is

used to match all field values beginning with access_ (which would include access_common,access_combined, and access_combined_wcookie) .

Note: Field names are case sensitive, but field values are not!

2. Scroll through the search results.

If you're familiar with the access_combined format of Apache logs, you will recognize some of theinformation in each event, such as:

• IP addresses for the users accessing the website.

•

URIs and URLs for the page request and referring page.• HTTP status codes for each page request.

• Page request methods.


85/275

84

As Splunk retrieves these events, the Fields sidebar updates with selected fieldsand interesting fields. These are the fields that Splunk extracted from your data.

Notice that default fields host, source, and sourcetype are selected fields and are displayed in yoursearch results:

3. Scroll through interesting fields to see what else Splunk extracted.

You should recognize the field names that apply to the Web access logs. For example, there's

clientip, method, and status. These are not default fields; they have (most likely) been extracted atsearch time.

4. Click the Edit link in the fields sidebar.

The Fields dialogue opens and displays all the fields that Splunk extracted.

• Available Fields are the fields that Splunk identified from the events in your

current search (some of these fields were listed under interesting fields).• Selected Fields are the fields you picked (from the available fields) to show in


86/275

85

your search results (by default, host, source, and sourcetype are selected).

5. Scroll through the list of Available Fields.

You're already familiar with the fields that Splunk extracted from the Web access logs based on your

search. You should also see other default fields that Splunk defined--some of these fields are based on

each event's timestamp (everything beginning with date_*), punctuation (punct), and location (index).

But, you should also notice other extracted fields that are related to the online store. For example,

there are action, category_id, and product_id. From conversations with your coworker, you mayknow that these fields are:

Field name Description

action what a user does at the online shop.


87/275

86

category_id the type of product a user is viewing or buying.

product_id the catalog number of the product the user is viewing or buying.

6. From the Available fields list, select action, category_id, and product_id.

7. Click Save.

When you return to the Search view, the fields you selected will be included in your search results if they

exist in that particular event. Different events will have different fields.


88/275

87

The fields sidebar doesn't just show you what fields Splunk has captured from your data. It also displays

how many values exist for each of these fields. For the fields you just selected, there are 2 for action, 5

for category_id, and 9 for product_id. This doesn't mean that these are all the values that exist for eachof the fields--these are just the values that Splunk knows about from the results of your search.

What are some of these values?

8. Under selected fields, click action for the action field.

This opens the field summary for the action field.

This window tells you that, in this set of search results, Splunk found two values for action and they are

purchase and update. Also, it tells you that the action field appears in 71% of your search results. This


89/275

88

means that three-quarters of the Web access events are related to the purchase of an item or an update

(of the item quantity in the cart, perhaps).

9. Close this window and look at the other two fields you selected, category_id (what

types of products the shop sells) and product_id (specific catalog names for products).

Now you know a little bit more about the information in your data relating to the online Flower and Giftshop. The online shop sells a selection of flowers, gifts, plants, candy, and balloons. Let's use these

fields, category_id and product_id, to see what people are buying.

Use fields to run more targeted searches

These next two examples compares the results when searching with and without fields.

Example 1

Return to the search you ran to check for errors in your data. Select Other > Yesterday from the

time range picker:

error OR failed OR severe OR (sourcetype=access_* (404 OR 500 OR 503))


90/275

89

Run this search again, but this time, use fields in your search.

The HTTP error codes are values of the status field. Now your search looks like this:

error OR failed OR severe OR (sourcetype=access_* (status=404 OR

status=500 OR status=503))

Notice the difference in the count of events between the two searches--because it's a more targetedsearch, the second search returns fewer events.

When you run simple searches based on arbitrary keywords, Splunk matches the raw text of your data.

When you add fields to your search, Splunk looks for events that have those specific field/value pairs.

Example 2

Before you learned about the fields in your data, you might have run this search to see how many timesflowers were purchased from the online shop:

sourcetype=access_* purchase flower*

As you typed in "flower", search assistant shows you both "flower" and "flowers' in the typeahead. Since


91/275

90

you don't know which is the one you want, you use the wildcard to match both.

If you scroll through the (many) search results, you'll see that some of the events have

action=update and category_id that have a value other than flowers.These are not events that you wanted!

Run this search instead. Select Other > Yesterday from the time range picker:

sourcetype=access_* action=purchase category_id=flower*

For the second search, even though you still used the wildcarded word "flower*", there is only one value

of category_id that it matches (FLOWERS).


92/275

91

Notice the difference in the number of events that Splunk retrieved for each search; the second search

returns significantly fewer events. Searches with fields are more targeted and retrieves more exact

matches against your data.

Now on your own:

1. Bring up the Loan data file2. Using fields find entries that annual salary is less than 20,000 and they live in the state of

CA. Use addr_state for state3. Refine the search for the field emp_title where it equals Walmart


93/275

92

End of Module Quiz



94/275

93

Module 5- Creating Reports and Visualizations


Edit reports


Hands on Lab covering: Save a search as a report, Edit Reports, Create reports that include visualizations such as chartsand tables.



Hands on Lab covering: Add reports to a dashboard, Create an instant pivot from a search.



95/275

94


To save your search as a report, click on the Report link. This opens the Save As Report dialog:

From here, you need to do the following:

1. Enter a Title (or name) for your report.2. Enter an optional Description to remind users what your report does.3. Indicate if you'd like to include the Splunk Time Range Picker as a part of your report.


96/275

95

Once you click Save, Splunk prompts you to either review Additional Settings for your newly created report (Permissions,Schedule, Acceleration, and Embed), Add (the report) to Dashboard, View the report, or Continue Editing the search:


97/275

96

The additional settings that can be made to the report are given as follows:

Permissions: Allows you to set how the saved report is displayed: by owner, by app, or for all apps. In addition, you can makethe report read only or writeable (can be edited).

Schedule: Allows you to schedule the report (for Splunk to run/refresh it based upon your schedule). For example, an intervallike every week, on Monday at 6 AM, and for a particular time range.

Acceleration: Not all saved reports qualify for acceleration and not all users (not even admins) have the ability to acceleratereports. Generally speaking, Splunk Enterprise will build a report acceleration summary for the report if it determines that thereport would benefit from summarization (acceleration).

Embed: Report embedding lets you bring the results of your reports to large numbers of report stakeholders. With reportembedding, you can embed scheduled reports in external (non-Splunk) websites, dashboards, and portals. Embedded reportscan display results in the form of event views, tables, charts, maps, single values, or any other visualization type. They use thesame formatting as the originating report. When you embed a saved report, you do this by copying a Splunk generated URLinto an HTML-based web page.


98/275


99/275

98


A visualization is a representation of data returned from a search. Most visualizations are graphical representations, however, a

visualization can also be non-graphical. In dashboards, a panel contains one or more visualizations. Visualizations available forsimple XML dashboards include:

chart

event listing map

table

single value

A chart visualization has several types:

area

bar bubble

column filler gauge

line

marker gauge pie

radial gauge scatter


100/275

99

Hands on Lab covering

1. Click Search on the Toolbar, then click the Data Summary button:

2. Choose the SourceType Tab, and click on access_combined_wcookie:


101/275

100

3. Select under Interesting Fields, category_id . Then click under Reports, top values:


102/275

101

4 It h ld i ld t


103/275

102

4. It should yield a report:

5 No click on Statistics notice the table of alues:


104/275

103

5. Now click on Statistics, notice the table of values:

6. Go back to the Visualization tab, under Format , then investigate all the different options

7. Under the Bar Chart drop, investigate all the different chart types as well

Bonus Lab:

Using the LoanStats3a.csv file, create a report from the data that top values across all the states



105/275

104


Once you have created your reports, you can easily add to the dashboard, by clicking Add to Dashboard button



106/275

105


From any search, simply select the Statistics tab and click on the Pivot Icon

Let's take a walkthrough:

1. Make sure to pick make interesting fields to be selected fields

2 Click the Statistics tab after you have the search you want:


107/275

106

2. Click the Statistics tab after you have the search you want:

3. Then click the Pivot Icon

4 Then you can choose the fields you have selected to Pivot and click OK :


108/275

107

4. Then you can choose the fields you have selected to Pivot, and click OK :

5 Then you can choose a field like annual inc with a default of Sum to be part of your Pivot column values:


109/275

108

5. Then you can choose a field like annual_inc with a default of Sum to be part of your Pivot column values:

6. And then pick a field like addr state to the row column


110/275

109

6. And then pick a field like addr_state to the row column


111/275

Hands on Lab


112/275

111

1. Create a report out of LoanStats3a.csv source that looks into the annual income < 70000 and the addr_state of

CA ,FL, NY

2. Create an instant pivot out of the search from #1 above.



113/275

112

Q



114/275

113

Module 6 - Working with Dashboards

Create a dashboard

Add a report to a dashboard

Hands on Lab covering: Create a dashboard, Add a report to a dashboard Add a pivot report to a dashboard

Edit a dashboard

Hands on Lab covering: Add a pivot report to a dashboard, Edit a dashboard.



115/275

114

Create a dashboard

You can create a dashboard from the search OR you can click on the Dashboard option on the Toolbar

OR


116/275


117/275

116

Hands on Lab:

Let's use the flower shop transactions to create a dashboard and add a report to it

Before you learned about the fields in your data, you might have run this search to see how many times flowers werepurchased from the online shop:

sourcetype=access_* purchase flower*| top limit=20 category_id

1. Let's save the report of this search as Flowers Category

2. Click on the view button to view the report

3. Click Add to DashBoard to add report to Dashboard

4. Name the Dashboard, Flowers Dashboard

Bonus Lab:

The report out of LoanStats3a.csv source that looks into the annual income < 70000 and the addr_state of CA ,FL,

NY from the last module and create a dashboard


118/275

117

Add a pivot report to a dashboard

From your pivot , you can save as a dashboard panel


119/275

118

Edit a dashboard

From your dashboard, you can edit your dashboard from the menu

And then you could, for example edit Panels

Hands on Lab:


120/275

119

1. Create an instant pivot, like the one from the previous module out of LoanStats3a.csv source that looks into the

annual income < 70000 and the addr_state of CA ,FL, NY

2. Then add that pivot report to the dashboard

3. Create another report that looks at ALL the annual incomes in the states of CA,FL, NY

4. Add that report to the dashboard created in exercise #1

5. Edit the dashboard panels and add titles to your panels.

Bonus Lab:

1. Create another instant pivot or report and add to the existing dashboard



121/275

120



122/275

R i b i h d d l h ti


123/275

122

Review basic search commands and general search practices

To successfully use Splunk, it is vital that you write effective searches. Using the index efficiently will make your initial discoveriesfaster, and the reports you create will run faster for you and for others. In this chapter, we will cover the following topics:

How to write effective searches

How to search using fields

Understanding time Saving and sharing searches

Usingsearchtermseffectively

The key to creating an effective search is to take advantage of the index. The Splunk index is effectively a huge word index, sliced bytime. The single most important factor for the performance of your searches is how many events are pulled from the disk. Thefollowing few key points should be committed to memory:

Search terms are case insensitive: Searches for error, Error, ERROR, and ErRoR are all the same thing.

Search terms are additive: Given the search item, mary error , only events that contain both words will be found. There areBoolean and grouping operators to change this behavior; we will discuss in this chapter under Boolean and groupingoperators.

Only the time frame specified is queried: This may seem obvious, but it's very different from a database, which wouldalways have a single index across all events in a table. Since each index is sliced into new buckets over time, only the bucketsthat contain events for the time frame in question need to be queried.

Search terms are words, including parts of words : A search for foo will also match foobar .


124/275

123

With just these concepts, you can write fairly effective searches. Let's dig a little deeper, though:

A word is anything surrounded by whitespace or punctuation : For instance, given the log line 2012-02-07T01:03:31.104-0600 INFO AuthClass Hello world. [user=Bobby, ip=1.2.3.3], the "words" indexed are

2012,02, 07T01, 03, 31, 104, 0600, INFO, AuthClass, Hello, world, user, Bobby, ip, 1, 2, 3, and 3 . Thismay seem strange, and possibly a bit wasteful, but this is what Splunk's index is really, really good at — dealing with hugenumbers of words across a huge number of events.

Splunk is not grep with an interface: One of the most common questions is whether Splunk uses regular expressions for yoursearches. Technically, the answer is no. Splunk does use regex internally to extract fields, including the auto generated fields, but most of what you would do with regular expressions is available in other ways. Using the index as it is designed is the bestway to build fast searches. Regular expressions can then be used to further filter results or extract fields.

Numbers are not numbers until after they have been parsed at search time: This means that searching for foo>5 will not

use the index, as the value of foo is not known until it has been parsed out of the event at search time. There are different waysto deal with this behavior, depending on the question you're trying to answer.

Field names are case sensitive: When searching for host=myhost, host must be lowercase. Likewise, any extracted orconfigured fields have case sensitive field names, but the values are case insensitive.

Host=myhost will not work

host=myhost will work

host=MyHost will work Fields do not have to be defined before indexing data: An indexed field is a field that is added to the metadata of an event at

index time. There are legitimate reasons to define indexed fields, but in the vast majority of cases it is unnecessary and isactually wasteful.

Examine the anatomy of a search


125/275

124

Boolean and grouping operators

There are a few operators that you can use to refine your searches (note that these operators must be in uppercase to not be considered

search terms):

AND is implied between terms. For instance, error mary (two words separated by a space) is the same as

error AND mary.

OR allows you to specify multiple values. For instance, error OR mary means find any event that contains

either word.

NOT applies to the next term or group. For example, error NOT mary would find events that contain error but

do not contain mary.

The quote marks ("") identify a phrase. For example, "Out of this world" will find this exact sequence of

words. Out of this world would find any event that contains all of these words, but not necessarily in thatorder.

Parentheses ( ( ) ) is used for grouping terms. Parentheses can help avoid confusion in logic. For instance,

these two statements are equivalent: bob error OR warn NOT debug bob AND (error OR warn)) AND NOT debug

The equal sign (=) is reserved for specifying fields. Searching for an equal sign can be accomplished by

wrapping it in quotes. You can also escape characters to search for them. \= is the same as "=".

Brackets ( [ ] ) are used to perform a subsearch.


126/275

Clickingtomodifyyoursearch


127/275

126

Though you can probably figure it out by just clicking around, it is worth discussing the behavior of the GUI when moving yourmouse around and clicking.

Clicking on any word or field value will give you the option to Add to search or Exclude from search (the

existing search) or (create a) New search:

Clicking on a word or a field value that is already in the query will give you the option to remove it (from the

existing query) or, as above, (create a) new (search):


128/275

To use the field picker, you can click on the link All Fields (see the following image):


129/275

128

Expand the results window by clicking on > in the far-left column. Clicking on a result will append that item to the current search:


130/275

Time


131/275

130

Clicking on the time next to an event will open the _time dialog (shown in the following image) allowing you to change the search toselect Events Before or After a particular time period, and will also have the following choices:

Before this time

After this time At this time

In addition, you can select Nearby Events within plus, minus, or plus or minus, a number of seconds (the default), milliseconds,minutes, hours, days, or weeks:


132/275

131

One search trick is to click on the time of an event, select At this time, and then use the Zoom out (above the timeline) until theappropriate time frame is reached.

Fields command Description


133/275

132

p

Keeps (+) or removes (-) fields from search results based on the field list criteria. If + is specified, only the fields that match one of the

fields in the list are kept. If - is specified, only the fields that match one of the fields in the list are removed. If neither is specified,

defaults to +.

Important: The leading underscore is reserved for all internal Splunk Enterprise field names, such as _raw and _time. By default,

internal fields _raw and _time are included in output. The fields command does not remove internal fields unless explicitlyspecified with:

... | fields - _*

or more explicitly, with:

... | fields - _raw,_time

Note: Be cautious removing the _time field. Statistical commands, such as timechart and chart, cannot display date or time

information without the _time field.

Syntax

fields [+|-]

Required arguments

Syntax: , , ...Description: Comma-delimited list of fields to keep (+) or remove (-). You can use wild card characters in the

field names.

Examples

Example 1:


134/275

133

p

Remove the "host" and "ip" fields.

... | fields - host, ip

Example2:

Keep only the host and ip fields. Remove all of the internal fields. The internal fields begin with an underscore character, for

example _time.

... | fields host, ip | fields - _*

Example3:

Keep only the fields 'source', 'sourcetype', 'host', and all fields beginning with 'error'.

... | fields source, sourcetype, host, error*


135/275

Rename command Description


136/275

135

Use the rename command to rename a specified field or multiple fields. This command is useful for giving fields more meaningfulnames, such as "Product ID" instead of "pid". If you want to rename multiple fields, you can use wildcards.

Use quotes to rename a field to a phrase:

... | rename SESSIONID AS sessionID

Use wildcards to rename multiple fields:

... | rename *ip AS *IPaddress

If both the source and destination fields are wildcard expressions with the same number of wildcards, the renaming will carry over thewildcarded portions to the destination expression. See Example 2, below.

Note: You cannot rename one field with multiple names. For example if you had a field A, you cannot do "A as B, A as C" in onestring.

... | stats first(host) AS site, first(host) AS report

Note: You cannot use this command to merge multiple fields into one field because null, or non-present, fields are

Splunk Book

Documents

Transcript of Splunk Book