python system statistic collection tool
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
dave 2e5889fd27 support for influxdb backend 2 years ago
bin add python module scripts 5 years ago
examples support for influxdb backend 2 years ago
pymonitor support for influxdb backend 2 years ago
.gitignore elasticsearch 2.x compatibility, bump version number 4 years ago
README.md support for influxdb backend 2 years ago
requirements.txt support for influxdb backend 2 years ago
setup.py support for influxdb backend 2 years ago

README.md

python-esmonitor

Modular monitoring tool for logging data to various timeseries databases

Quick start

  • Install: pip3 install -r requirements.txt ; python3 setup.py install
  • Configure: cd examples ; vim config.json
  • Run: pymonitor -c config.json

Configuring

The config file should contain a json object with the keys backend and monitors. Backend contains a key, type, to select what database backend to use. The remaining keys are specific to that database.

For Elasticsearch 6.x, this should be the full url to elasticsearch:

{
    "backend": {
        "type": "elasticsearch"
        "url": "http://192.168.1.210:8297/"
    },

Or for InfluxDB 6.x, several fields describing the connection:

{
    "backend": {
        "type": "influxdb",
        "host": "10.0.0.10",
        "port": "8086",
        "user": "root",
        "password": "root",
        "database": "monitoring"
    },

The monitors key contains a list of monitor modules to run:

    "monitors": [
        {
            "type":"diskspace",
            "freq":"30",
            "args": {
                "filesystems": [
                    "/",
                    "/tmp/monitor"
                ]
            }
        },
        { ... }
    ]
}

The name of the module to run for a monitor is type. The freq option is the frequency, in seconds, that this monitor will check and report data. If the monitor being used takes any options, they can be passed as a object with the args option,

A yaml config can also be used. The data structure must be identical and the filename MUST end in .yml.

Developing Modules

How to create a module:

Add a new python file in pymonitor/monitors/, such as uptime.py. Add a function named the same as the file, accepting any needed params as keyword args:

from pymonitor import Metric


def uptime():

Add your code to retrieve any metrics:

    with open("/proc/uptime", "r") as f:
        uptime_stats = {"uptime":int(float(f.read().split(" ")[0]))}

This function must yield one or more Metric objects. This object will be sent to the database backend, with a type field matching the name if this module (“uptime”). System hostname, ip address, and timestamp will be added automatically.

        yield Metric(uptime_stats)

The module file must set a variable named mapping. For backends that need it, such as Elasticsearch, this contains data mapping information so our data is structured correctly. This value is used verbatim, so any other Elasticsearch options for this type can be specified here.

mapping = {
    "uptime": {
        "type": "integer"
    }
}

Finally, it’s often convenient to test your monitor by adding some code so the script can be ran individually:

if __name__ == '__main__':
    for item in uptime():
        print(item["uptime"])

Since this module is named ‘uptime’ and takes no args, the following added to the monitors array in config.json would activate it:

{
    "type":"uptime",
    "freq":"30",
    "args":{}
}

Roadmap

  • Complete API docs
  • More builtin monitors
  • Local logging in case ES can’t be reached