|
Packit |
1422b7 |
Introduction
|
|
Packit |
1422b7 |
============
|
|
Packit |
1422b7 |
|
|
Packit |
1422b7 |
Briefly described, liblognorm is a tool to normalize log data.
|
|
Packit |
1422b7 |
|
|
Packit |
1422b7 |
People who need to take a look at logs often have a common problem. Logs
|
|
Packit |
1422b7 |
from different machines (from different vendors) usually have different
|
|
Packit |
1422b7 |
formats. Even if it is the same type of log (e.g. from firewalls), the log
|
|
Packit |
1422b7 |
entries are so different, that it is pretty hard to read these. This is
|
|
Packit |
1422b7 |
where liblognorm comes into the game. With this tool you can normalize all
|
|
Packit |
1422b7 |
your logs. All you need is liblognorm and its dependencies and a sample
|
|
Packit |
1422b7 |
database that fits the logs you want to normalize.
|
|
Packit |
1422b7 |
|
|
Packit |
1422b7 |
So, for example, if you have traffic logs from three different firewalls,
|
|
Packit |
1422b7 |
liblognorm will be able to "normalize" the events into generic ones. Among
|
|
Packit |
1422b7 |
others, it will extract source and destination ip addresses and ports and
|
|
Packit |
1422b7 |
make them available via well-defined fields. As the end result, a common log
|
|
Packit |
1422b7 |
analysis application will be able to work on that common set and so this
|
|
Packit |
1422b7 |
backend will be independent from the actual firewalls feeding it. Even
|
|
Packit |
1422b7 |
better, once we have a well-understood interim format, it is also easy to
|
|
Packit |
1422b7 |
convert that into any other vendor specific format, so that you can use that
|
|
Packit |
1422b7 |
vendor's analysis tool.
|
|
Packit |
1422b7 |
|
|
Packit |
1422b7 |
By design, liblognorm is constructed as a library. Thus, it can be used by
|
|
Packit |
1422b7 |
other tools.
|
|
Packit |
1422b7 |
|
|
Packit |
1422b7 |
In short, liblognorm works by:
|
|
Packit |
1422b7 |
|
|
Packit |
1422b7 |
1. Matching a line to a rule from predefined configuration;
|
|
Packit |
1422b7 |
2. Picking out variable fields from the line;
|
|
Packit |
1422b7 |
3. Returning them as a JSON hash object.
|
|
Packit |
1422b7 |
|
|
Packit |
1422b7 |
Then, a consumer of this object can construct new, normalized log line
|
|
Packit |
1422b7 |
on its own.
|