python system statistic collection tool
Go to file
2018-10-04 18:50:34 -07:00
bin add python module scripts 2015-12-06 22:41:51 -08:00
examples support for influxdb backend 2018-10-04 18:50:34 -07:00
pymonitor support for influxdb backend 2018-10-04 18:50:34 -07:00
.gitignore elasticsearch 2.x compatibility, bump version number 2016-02-25 22:54:22 -08:00
README.md support for influxdb backend 2018-10-04 18:50:34 -07:00
requirements.txt support for influxdb backend 2018-10-04 18:50:34 -07:00
setup.py support for influxdb backend 2018-10-04 18:50:34 -07:00

python-esmonitor

Modular monitoring tool for logging data to various timeseries databases

Quick start

  • Install: pip3 install -r requirements.txt ; python3 setup.py install
  • Configure: cd examples ; vim config.json
  • Run: pymonitor -c config.json

Configuring

The config file should contain a json object with the keys backend and monitors. Backend contains a key, type, to select what database backend to use. The remaining keys are specific to that database.

For Elasticsearch 6.x, this should be the full url to elasticsearch:

{
    "backend": {
        "type": "elasticsearch"
        "url": "http://192.168.1.210:8297/"
    },

Or for InfluxDB 6.x, several fields describing the connection:

{
    "backend": {
        "type": "influxdb",
        "host": "10.0.0.10",
        "port": "8086",
        "user": "root",
        "password": "root",
        "database": "monitoring"
    },

The monitors key contains a list of monitor modules to run:

    "monitors": [
        {
            "type":"diskspace",
            "freq":"30",
            "args": {
                "filesystems": [
                    "/",
                    "/tmp/monitor"
                ]
            }
        },
        { ... }
    ]
}

The name of the module to run for a monitor is type. The freq option is the frequency, in seconds, that this monitor will check and report data. If the monitor being used takes any options, they can be passed as a object with the args option,

A yaml config can also be used. The data structure must be identical and the filename MUST end in .yml.

Developing Modules

How to create a module:

Add a new python file in pymonitor/monitors/, such as uptime.py. Add a function named the same as the file, accepting any needed params as keyword args:

from pymonitor import Metric


def uptime():

Add your code to retrieve any metrics:

    with open("/proc/uptime", "r") as f:
        uptime_stats = {"uptime":int(float(f.read().split(" ")[0]))}

This function must yield one or more Metric objects. This object will be sent to the database backend, with a type field matching the name if this module ("uptime"). System hostname, ip address, and timestamp will be added automatically.

        yield Metric(uptime_stats)

The module file must set a variable named mapping. For backends that need it, such as Elasticsearch, this contains data mapping information so our data is structured correctly. This value is used verbatim, so any other Elasticsearch options for this type can be specified here.

mapping = {
    "uptime": {
        "type": "integer"
    }
}

Finally, it's often convenient to test your monitor by adding some code so the script can be ran individually:

if __name__ == '__main__':
    for item in uptime():
        print(item["uptime"])

Since this module is named 'uptime' and takes no args, the following added to the monitors array in config.json would activate it:

{
    "type":"uptime",
    "freq":"30",
    "args":{}
}

Roadmap

  • Complete API docs
  • More builtin monitors
  • Local logging in case ES can't be reached