Blame doc/introduction.rst

Packit 1422b7
Introduction
Packit 1422b7
============
Packit 1422b7
Packit 1422b7
Briefly described, liblognorm is a tool to normalize log data.
Packit 1422b7
Packit 1422b7
People who need to take a look at logs often have a common problem. Logs 
Packit 1422b7
from different machines (from different vendors) usually have different 
Packit 1422b7
formats. Even if it is the same type of log (e.g. from firewalls), the log 
Packit 1422b7
entries are so different, that it is pretty hard to read these. This is 
Packit 1422b7
where liblognorm comes into the game. With this tool you can normalize all 
Packit 1422b7
your logs. All you need is liblognorm and its dependencies and a sample 
Packit 1422b7
database that fits the logs you want to normalize.
Packit 1422b7
Packit 1422b7
So, for example, if you have traffic logs from three different firewalls, 
Packit 1422b7
liblognorm will be able to "normalize" the events into generic ones. Among 
Packit 1422b7
others, it will extract source and destination ip addresses and ports and 
Packit 1422b7
make them available via well-defined fields. As the end result, a common log 
Packit 1422b7
analysis application will be able to work on that common set and so this 
Packit 1422b7
backend will be independent from the actual firewalls feeding it. Even 
Packit 1422b7
better, once we have a well-understood interim format, it is also easy to 
Packit 1422b7
convert that into any other vendor specific format, so that you can use that 
Packit 1422b7
vendor's analysis tool.
Packit 1422b7
Packit 1422b7
By design, liblognorm is constructed as a library. Thus, it can be used by 
Packit 1422b7
other tools.
Packit 1422b7
Packit 1422b7
In short, liblognorm works by:
Packit 1422b7
Packit 1422b7
	1. Matching a line to a rule from predefined configuration;
Packit 1422b7
	2. Picking out variable fields from the line;
Packit 1422b7
	3. Returning them as a JSON hash object.
Packit 1422b7
Packit 1422b7
Then, a consumer of this object can construct new, normalized log line
Packit 1422b7
on its own.