OSCam/en/Monitoring/Splunk

Aus Streamboard Wiki
Zur Navigation springen Zur Suche springen


Introduction

What is Splunk

Splunk is an universal tool for analyzing big data like logs, syslogs, files +++

Who created Splunk

Splunk is developed and supported by www.splunk.com OSCam add-ons is developed by Jotne only. It was stared as a project to get some control of the server and the help some other also. There are for sure much more that can be done, but this is a way to help other out there to start using Splunk. I have worked with Splunk on professional basis, but still count my self as medium user of it. Lot of work has been done to index and categorize data coming from the OSCam server.

Licensing

What we set up here is free. Splunk Enterprise will use with license named Free License with the following limits:

  • Max 500MB log data pr day. (This is more than you ever need and can easily handle many large OSCam servers)
  • No Alert and Monitoring
  • No distributed search
  • User manager (always logged inn with default user)
  • + some other stuff

What can we do with OSCam add-ons for Splunk

The list of what you can do with this endless, but here is some:

  • See what channels a user/client request from you.
  • See what is requested from a reader/proxy.
  • General status panel with various information.
  • Error logs (by clients or readers)
  • Search logs
  • Count status of any events, averages
  • View events by at any time/time range
  • Manage many servers at once
  • Live view
  • Historical logs

System requirements

What you need to run Splunk depends on OS you select, but the minimum supported is:

Windows

  • 2 GB Ram
  • 32/64 bits
  • Windows
    • Windows Client from Windows 7 and up
    • Windows Server from Windows 2003 and up

Non Windows

  • 1 GB ram
  • 32/64 bits
    • Linux
    • Solaris
    • Mac OS X

Installation

Splunk

Installing Splunk

To download Splunk, you need an account at www.splunk.com It free, so just create one.

Go to:Products -> Overview -> Splunk® Enterprise -> Free download Select your OS -> Windows, Linux, Solaris, Mac OS

Under Windows

Simply execute the downloaded file.

Under Ubuntu 14.04

Check if your Linux runs 32 bits or 64 bits with the command:

uname -a

If in the result you see, i363 or i686 it is a 32 bits system.
Otherwhise if you see, amd64 or x86_64 it is a 64 bits.

Then donwload the right package (in .deb format (debian)), in the sample the name of the selected package is: splunk-6.3.3-f44afce176d0-linux-2.6-intel.deb

Voici la commande pour installer le package:

sudo dpkg -i ./your_selected_package_name.deb

Example log:

Selecting previously unselected package splunk.
(Reading database ... 832902 files and directories currently installed.)
Preparing to unpack .../splunk-6.3.3-f44afce176d0-linux-2.6-intel.deb ...
Unpacking splunk (6.3.3) ...
Setting up splunk (6.3.3) ...
complete

Splunk is installed in the folder: /opt/splunk

To start it:

sudo /opt/splunk/bin/splunk start
Script to auto-start Splunk at boot

Create a script called splunk in the folder /etc/init.d: /etc/init.d/splunk

With this inside:

#!/bin/sh
### BEGIN INIT INFO
# Provides:          splunk
# Required-Start:    
# Required-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: splunk monitoring
# Description: Splunk
### END INIT INFO
splunk_exe="/opt/splunk/bin/splunk"
splunk_start="start"
splunk_stop="stop"

case "$1" in
start)
	exec $splunk_exe $splunk_start
	;;
stop)
	exec $splunk_exe $splunk_stop
	;;
restart|reload)
	$0 stop
	sleep 1
	$0 start
	;;
*)
	echo "Usage: $0 start|stop|restart"
	exit 1
	;;
esac
exit 0

Don't forget to make it executable thanks to:

sudo chmod 755 /etc/init.d/splunk

And to integrate it into the boot process:

sudo update-rc.d splunk defaults

Run Splunk

At first run go to

localhost:8000 or 
your_ip:8000 

Splunk will ask you to change password for the admin account. Do so, but you do not need it more than once, since its gone when converting to free license group.

Convert License

Splunk License -1.jpg

I do recommend that you convert Splunk to Free license group now. But if you like, you can try it out with full license for 30 days. But its important you do convert it before 30 days, or you end up in a situation that it will be locked for use for 30 days if it past the 30 day trail.

To convert it do:

  • Settings -> Licensing.
  • Change license group.
  • Free license
  • Save
  • Restart Now (to activate the new license)

You can no go directly inn to Splunk without been asked for user/password.

Make Splunk accept syslog data

We need to make Splunk accepting syslog packets on UDP port 514.

  • Settings -> Data inputs -> UDP -> New
  • Make sure UDP is selected.
  • Port:514 (default syslog)
  • Next:
  • Source Type: Select
  • In the dropdown box type: syslog and drag scroll bar down until you find: Syslog (Output produced by many syslog daemons....)
  • -> Review
  • -> Submit
  • Start searching

Firewall

Now Splunk is ready to collect data. But first you need to open your firewall, to get UDP:514 to pass trough. If your OSCam is on another site, you need to add port forwarding in your internet router as well.

Windows

Here is an sample on how to open Windows firewall.

  • Open Windows Firewall with Advanced Security (click Windows button, start type firewall)
  • Go to Inbound Rules
  • Right Inbound Rules
  • New Rules...
  • Program (this option is best, since you open all needed port in one go)
  • This program path: %ProgramFiles%\Splunk\bin\splunkd.exe
  • next
  • Allow the connection
  • Next
  • make sure all is checked. (Domain,Private,Public)
  • Next
  • In name type: Splunk
  • Finish

Router

If you traverse internet, you need to port forwarding UDP:514 packages.

  • Public port: 514
  • Internal port: 514
  • Internal ip_of_your_splunk_server_ip

OSCam

Make OSCam sending data

Splunk getting data.jpg

Before we make OSCam send data we go to data view windows in Splunk to see data entering the system

  • In Splunk, select splunk> logo in upper left corner.
  • Click the green >Search & Reporting
  • Click Data Summary

When OSCam start to send data, you should show it a couple seconds later

Head over to your OSCam

  • Config
  • Logging (section)
  • Syslog server: ip_of_your_splunk_server
  • Save
  • Restart
  • Restart (OSCam needs to be restarted)

Splunk should now counting up package coming from OSCam

Adding OSCam files for Splunk

Under Windows

  • Download the oscam_for_splunk.rar file here:

Datei:Oscam for splunk 1.0.rar

  • Unpack it
  • Copy OSCam folder to: C:\Program Files\Splunk\etc\apps (make sure it bcomes C:\Program Files\Splunk\etc\apps\OSCam\...\...\)
  • Restart Splunk: Settings -> Server controls -> Restart Splunk -> OK

Under Ubuntu

Place the downloaded file into /tmp and go into this folder cd /tmp

  • Extract the Splunk add-ons by Jotne, in the sample the file is called Oscam_for_splunk_1.0.rar you need to adapt the name according to the file evolution.
unrar x ./Oscam_for_splunk_1.0.rar
  • Move the files at the right place:
mv ./OSCam/ /opt/splunk/etc/apps/
  • Delete the rar file
rm ./Oscam_for_splunk_1.0.rar
  • Restart Splunk: Settings -> Server controls -> Restart Splunk -> OK

Updating OSCam files for Splunk

You only need to do this part if you like to upgrade the OSCam files.

  • Download the latest version from link above.
  • Unpack it
  • Delete the OSCam folder %splunk_home%/etc/app/OSCam
  • Copy new OSCam file to server in same place as the one you delete.
  • Restart Splunk

Using Splunk

Splunk start.jpg
Splunk User Usage.jpg
Splunk default view.jpg
Splunk detailed view.jpg

Go to your Splunk Server and you get the main page up.

There you can select Splunk> or OSCam view

Splunk view

Here you see all the detailed information coming from OSCam. You can also here go to the Dasboard an see the predefined OSCam view.

OSCam view

This takes you to the predefined view.

OSCam info

This view gives an overall quick status.

OSCam User Usage

Here you see how what channels users request.

OSCam Reader Usage

Here you see what is requested from Readers/Proxies.

OSCam Error

Here can you what errors you get.

Other info

OSCam debug

You can change the debug level from the default 0 to some other.

I have not had time to index the debug message, so they are just received and not categorized.