14 Mart 2018 Çarşamba

How to Create a Metricbeat Module?

Metricbeat is written in Golang like all Beats. If you want to create to Metricbeat Module, you must install the latest version of Golang.
I recommend you to install vim-go.

What is the Metricbeat?

Metricbeat is a lightweight shipper that you can install on your servers to periodically collect metrics from the operating system and from services running on the server. Metricbeat takes the metrics and statistics that it collects and ships them to the output that you specify, such as Elasticsearch or Logstash.

Metricbeat consists of modules and metricsets.  A Metricbeat module defines a basic logic for collecting data from a specific service. You can configure the frequency at which Metricbeat collects the metrics and what specific metric metrics to collect using these modules and metricsets.

Contributing a Metricbeat Module

At first, you should get enough information about the service whose statistics you would like to collect such as how you can have the statistics or what kind of outputs it produces. For a module to be accepted, it should focus on fetching service metrics directly from the service itself.

Metricbeat Module and Community Beats are different, but it's a good idea to create your own beat before you start working on the Metricbeat Module. 
For more information about Community Beats check [1] and [2].

First Part of Modules: Metricsets

A metricset is the part of a Metricbeat module that fetches and structures the data from the remote service.

To create a new metricsets you can run the following command inside Metricbeat directory:
make create-metricset
When you run this command, it creates all the basic files for your metricset, along with the required module files if the module doesn't already exist. The metricset that you created is already a functioning metricset and can be compiled.

And then compile your new metricset by running the following command:
make collect
The first command updates all generated files with the most recent files, data, and meta information from the metricset. The second command compiles your source code.

For example:
$ make create-metricset 
Module name: kripton                   
Metricset name: metricwork           
Module kripton created.                 
Metricset metricwork created.         
$ make update
$ make
This should now have created a directory kripton inside metricbeat/module with several files. Let's change to this directory and list up files automatically created.
$ cd kripton
$ tree
├── doc.go
├── _meta
│   ├── config.yml
│   ├── docs.asciidoc
│   └── fields.yml
└── metricwork
    ├── _meta
    │   ├── data.json
    │   ├── docs.asciidoc
    │   └── fields.yml
    └── metricwork.go
3 directories, 8 files

The {metricset}.go file contains the logic on how to fetch data from the service and convert it for sending to the output.

The generated file looks like [3].
For more details, you can check Beats Developer Guide and Metricset Details.

Second Part of Modules: Testing

It's very important to add tests for your metricset. 
There are three different types of tests:
  • unit tests
  • integration tests
  • system tests
You should use a combination of the three types to test your metricsets.
You have to ensure the 80% test coverage. 
For more details about testing you can check [4].

Last Part of Modules: Module Files

Metricbeat modules are used to group multiple metricsets together and to implement shared functionality of the metricsets.
It's very important to complete documentation and configuration files for a module. When you create new metricset, default versions these files are generated in the _meta directory.

For example: 

We already have kripton module and skipping creating module kripton when we running make create-metricset command. Just new metricset created.

$ make create-metricset
Module name: kripton
Metricset name: metrickripton
Module already exists.
Skipping creating module kripton.
Metricset metrickripton created. 
$ tree _meta 
├── config.yml
├── docs.asciidoc
└── fields.yml

The config.yml file contains the basic configuration options. Looks like this:

- module: kripton   metricsets: ["metricwork","metrickripton"]   enabled: false   period: 10s   hosts: ["localhost"]

We have multiple metricsets in kripton module, we have to make sure that extend metricset array.

The docs.asciidoc file contains the documentation about your module. Looks like this:

== kripton module
This is the kripton module.

Use this file to describe your module in more detail and to document specific configuration options.

The fields.yml file contains the top level structure for the fields in your metricset. Looks like this:
- key: kripton
  title: "kripton"
  description: >
    kripton module
    - name: kripton
      type: group
      description: >

This blog post is included basic information about Metricbeat Modules. For more information, you can look into Metricbeat Modules.

Hiç yorum yok:

Yorum Gönder