Streaming a new class of data into Splunk – Introducing the Splunk App for Stream | Splunk (2024)

Last year in December, we announced the acquisition of Cloudmeter – a company with technology that captures data directly from the network traffic – a rapidly growing source of big data.

Today, I’m stoked to announce the general availability of the Splunk App for Stream v6.0, which stems from that acquisition.

So, why is wire data (data from the network) important? Wire data has the benefit of capturing all data in real-time – it is the communication vehicle for applications and systems to talk to each other, making it a very authoritative source of critical information. It serves a broad range of analytics across different use cases; it is non-intrusive with no impact to workloads and it can be collected without the need for instrumentation and tagging of applications.

The Splunk App for Stream passively captures and sends wire data into the Splunk platform. We’ve always been able to capture wire data. What we’re introducing with the Splunk App for Stream is a simple and elegant mechanism to capture the data. Let me explain why I think this approach is unique and absolutely fantastic.

  1. Delivered as a software solution: This matters because it is often challenging to get wire data from on-premise services and applications e.g. web/email/application services and this problem is compounded in cloud implementations. There are no taps or span ports that can be leveraged. This capability to rapidly deploy wire data collection as software from end points delivers real time network visibility that is otherwise unavailable from cloud implementations and traditional datacenters. For instance, a lot of data compromise happens at end points, which can now be easily monitored or captured on-the-fly. Operationally, IT teams have better control of what is happening in public/hybrid Cloud infrastructures with the ability to deploy this software across all end points in any Cloud service easily. Pretty huge – don’t you think?
  1. Flexibility in deployment: The App can be deployed as an ultra-light, non-intrusive agent to tap into network streams, which clearly makes it cloud friendly (already spoke about this above). Alternatively, the App can also be deployed as an appliance that sits on the mirror/SPAN ports of hardware switches and parse and collect data as it arrives.
  1. Data collection customizability and data volume control: Wire data is voluminous. So, you may be concerned about what this means to your Splunk instance/license. To address this, we’ve included some pretty cool features in the App that provides you a lot of flexibility in configuring and customizing what you want to capture. With the powerful interface, you can define fine-grained filters on protocols and attributes, customize streams and create filters on streams and even aggregate data on the fly. You can whitelist/blacklist IP addresses and subnets. This provides you the ability to capture what is most critical, from endpoints or subnets most important. And of course, control data volumes.
  1. Interface driven deployment and scale-out: The App, being a software solution, can be deployed where you want it when you want it. Let’s say you have some kind of infiltration or malware attack and you want to quickly collect data for forensics. With this App, you can pretty much do this on-the-fly through the interface. You can also manage the deployment of your App across your network with the Splunk deployment server.
  1. Enhances OI with correlated insights: Wire data definitely has a lot of potential, especially around Splunk’s core use-cases – App Mgmt, IT Operations, Security and Business Analytics. When combined with other machine data such as logs, events and metrics, you can gain in-depth insights into performance, availability and usage and enable end-to-end insights across your critical business processes, systems and applications. And Splunk is uniquely positioned to do this.

We have proven beyond doubt that we’re the leading Big Data platform for Operational Intelligence. With the addition of wire data, this only elevates our status. Bernd Harzog, analyst from The Virtualization Practice and CEO and founder of APM Experts, wrote a really nice article about this release and how we’ve now extended the capabilities of the Splunk platform with the addition of wire data. Thanks Bernd.

Wondering how to get started. This is free folks! This App can be downloaded on Splunk Apps right away.

Before I sign off, I also want to introduce you to the Streams Examples App. In order to get you started on how to explore wire data, captured by the Splunk App for Stream, we’ve also authored a Stream Examples App that you can download from Splunk Apps. This App contain searches, examples and instructions for how to enable several use cases using data captured by the Splunk App for Stream. It will cover scenarios like looking for security relevant conversations, looking at a Web Shopping Cart for funnel analysis, using full payload data to track Shopping Cart revenue, analyzing SIP conversations, looking at Application and Database performance metrics, and more. Once you install the App for Stream, I highly recommend you download this as well – it will accelerate the value you get from the data captured by the App for Stream.

Long post, I know, but there is just so much to say about this App. If you have any comments, please do not hesitate to reach out. We’re here to help you and we want to do a good job of it. Let us know what you think and how we can help.

----------------------------------------------------
Thanks!
pbalakrishnan

Streaming a new class of data into Splunk – Introducing the Splunk App for Stream | Splunk (2024)

FAQs

How to setup a Splunk stream? ›

Install the Splunk App for Stream
  1. Click Download. The installation package downloads to your local host.
  2. Log into Splunk Web.
  3. Click Manage Apps > Install app from file.
  4. Upload the installer file.
  5. Restart Splunk Enterprise if prompted.
Feb 17, 2023

What is the difference between streaming and non streaming Splunk? ›

Non-streaming commands run on the search head. By contrast, a streaming command operates on each event as the event is returned by the search. Any command in a search that occurs after a non-streaming command must also be processed on the search head.

What is the Splunk Stream app? ›

Splunk Stream is a great way to monitor network traffic from a host or via a network TAP or SPAN port. The software acts as a network traffic sniffer.

What are streaming commands in Splunk? ›

A streaming command applies a transformation to each event returned by a search. For example, the rex command is streaming because it extracts and adds fields to events at search time.

How do I give access to Splunk app? ›

In Splunk Web, open your app. Go to Settings > Knowledge, then click a category of objects or click All configurations. Click Permissions for the object for which you want to edit permissions. Select an option for the app context, then set read and write permissions for all the roles listed.

How to configure an app in Splunk? ›

You can manage the configurations and properties for apps installed in your Splunk Enterprise instance from the Apps menu. Click on Apps in the User bar to select one of your installed apps or manage an app. From the Manage Apps page, you can do the following: Edit permissions for an app or add-on.

What is the difference between Splunk app and Splunk add-on? ›

Apps: Splunk apps provide user interfaces that let you work with your data. Apps often use one or more add-ons to ingest different types of data. See Apps and add-ons in the Splunk Enterprise Admin Manual. Add-ons: Add-ons enable Splunk Enterprise, or a Splunk app, to ingest or map a particular type of data.

What does the Splunk app consist of? ›

A Splunk app typically contains one or more dashboards with data visualizations, along with saved configurations and knowledge objects such as reports, saved searches, lookups, data inputs, a KV store, alerts, and more.

How do I stream logs to Splunk? ›

How to
  1. In Destination, select Splunk.
  2. In Display name, enter a human-readable description for the destination. ...
  3. In Event collector token, enter the HEC token you created and enabled in Splunk.
  4. If you want to send compressed gzip logs to this destination, check Send compressed data.

What are the 3 modes in Splunk search? ›

Search mode has three settings: Fast, Verbose, and Smart.

What are the Splunk commands? ›

Some of the common distributable streaming commands are: eval, fields, makemv, rename, regex, replace, strcat, typer, and where. For a complete list of distributable streaming commands, see Streaming commands in the Search Reference.

What is command Streaming? ›

Command Streaming sends DATA to draw the image. Read user inputs and run application loop on CPU. Output is drawing commands compressed on the fly – NO GPUs. Data packets are sent via any cloud to the user. User device GPU executes DRAW commands in real time.

How to set up a Splunk environment? ›

  1. Set up your development environment.
  2. Create the app structure.
  3. Create an index and generate sample events.
  4. Create saved searches using different tools.
  5. Set permissions for knowledge objects.
  6. Modify the app bar and create a home page.

How to setup Splunk monitoring console? ›

To configure the monitoring console for a standalone instance:
  1. In Splunk Web, navigate to Monitoring Console > Settings > General Setup.
  2. Check that search head, license manager, and indexer are listed under Server Roles, and nothing else. If not, click Edit to correct.
  3. Click Apply Changes.
Aug 14, 2021

Top Articles
Latest Posts
Recommended Articles
Article information

Author: Jamar Nader

Last Updated:

Views: 5589

Rating: 4.4 / 5 (55 voted)

Reviews: 86% of readers found this page helpful

Author information

Name: Jamar Nader

Birthday: 1995-02-28

Address: Apt. 536 6162 Reichel Greens, Port Zackaryside, CT 22682-9804

Phone: +9958384818317

Job: IT Representative

Hobby: Scrapbooking, Hiking, Hunting, Kite flying, Blacksmithing, Video gaming, Foraging

Introduction: My name is Jamar Nader, I am a fine, shiny, colorful, bright, nice, perfect, curious person who loves writing and wants to share my knowledge and understanding with you.