2 Ekim 2019 Çarşamba

Mesosbeat: Elastic Beats Based on Metricbeat

What is the Mesosbeat


Mesosbeat is a beat based on Metricbeat which read metrics from Apache Mesos HTTP API and indexes them into Elasticsearch or Logstash.

What is the Apache Mesos?


Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications or frameworks.


Run Mesosbeat


First, setup Golang environment (if you don't have it already)
To create a binary run the make command. This will create the binary in your beats directory.
cd $GOPATH
mkdir -p src/github.com/berfinsari
cd src/github.com/berfinsari
git clone https://github.com/berfinsari/mesosbeat.git
cd mesosbeat
make

To run mesosbeat, execute the binary. This will automatically load the default configuration which was generated by make update.

./mesosbeat -e -d "*"
This will run the beat with debug output enabled to the console to directly see what is happening. Stop the beat with CTRL-C.

If you can not see the Mesos module output, you need to enable the Mesos module:
./mesosbeat modules enable mesos
Now Mesos module is enabled and ready to collects metrics.


Configuration


Mesosbeat has some settings. If you want to change some config options adjust the modules.d/mesos.yml file to your needs.

Here is a sample configuration:
- module: mesos
  metricsets: ["master"]
  period: 10s
  hosts: ["localhost:5050"]

module: The name of the module to run.
metricsets: A list of metricsets to execute.(For now, mesos module have one metricset)
period: How often the metricsets are executed. If a system is not reachable, Metricbeat returns an error for each period.
hosts: A list of hosts of fetch information from.


Mesos Stats


Mesosbeat collects metrics for Mesos master and returns all available metrics from the /metrics/snapshot API endpoint on Mesos master. Master metricset provides information regarding the current metrics tracked by the system.

Metrics Collected


Resources
These metrics provide information about the total resources available in the cluster and their current usage.

    "master": {
      "gpus_used": 0,
      "gpus_revocable_total": 0,
      "mem_percent": 0,
      "disk_revocable_percent": 0,
      "invalid_status_update_acknowledgements": 0,
      "event_queue_dispatches": 8,
      "gpus_revocable_used": 0,
      "mem_total": 14886,
      "disk_revocable_used": 0,
      "cpus_revocable_total": 0,
      "cpus_revocable_used": 0,
      "disk_percent": 0,
      "cpus_used": 0,
      "gpus_revocable_percent": 0,
      "cpus_total": 12,
      "disk_total": 172196,
      "cpus_revocable_percent": 0,
      "cpus_percent": 0,
      "mem_used": 0,
      "disk_revocable_total": 0,
      "gpus_total": 0,
      "mem_revocable_total": 0,
      "disk_used": 0,
      "gpus_percent": 0,
      "mem_revocable_percent": 0,
      "mem_revocable_used": 0,
      },


Master
These metrics provide information about whether a master is currently elected and how long it has been running.

    "master": {
      "elected": 1,
      "uptime_secs": 2113.715932928,
      },


System
These metrics provide information about the resources available on this master node and their current usage.

    "system": {
      "cpus_total": 12,
      "load_15min": 0.37,
      "load_5min": 0.52,
      "load_1min": 0.63,
      "mem_free_bytes": 9097818112,
      "mem_total_bytes": 16683503616
    },


Agents
These metrics provide information about agent events, agent counts, and agent states.

   "master": {
     "slave_registrations": 0,
      "slave_unreachable_scheduled": 0,
      "slaves_inactive": 0,
      "slave_unreachable_completed": 0,
      "slaves_unreachable": 0,
      "slaves_connected": 1,
      "slave_removals": 0,
      "slaves_active": 1,
      "slaves_disconnected": 0,
      "slave_unreachable_canceled": 0,
      "slave_reregistrations": 1,
    },


Frameworks
These metrics provide information about the registered frameworks in the cluster.

  "master": {
      "frameworks_active": 0,
      "frameworks_connected": 0,
      "frameworks_disconnected": 0,
      "frameworks_inactive": 0,
      "outstanding_offers": 0,
   },


Tasks
These metrics provide information about active and terminated tasks.

  "master": {
      "tasks_error": 0,
      "tasks_lost": 0,
      "tasks_running": 0,
      "tasks_killed": 0,
      "tasks_killing": 0,
      "tasks_failed": 0,
      "tasks_finished": 0,
      "tasks_starting": 0,
      "tasks_unreachable": 0,
      "tasks_staging": 0,
   },


Messages
These metrics provide information about messages between the master and the agents and between the framework and the executors.

  "master": {
      "invalid_executor_to_framework_messages": 0,
      "invalid_framework_to_executor_messages": 0,
      "invalid_operation_status_update_acknowledgements": 0,
      "invalid_status_updates": 0,
      "dropped_messages": 0,
      "messages_authenticate": 0,
      "messages_deactivate_framework": 0,
      "messages_decline_offers": 0,
      "messages_executor_to_framework": 0,
      "messages_exited_executor": 0,
      "messages_framework_to_executor": 0,
      "messages_kill_task": 0,
      "messages_launch_tasks": 0,
      "messages_operation_status_update_acknowledgement": 0,
      "messages_reconcile_operations": 0,
      "messages_reconcile_tasks": 0,
      "messages_register_framework": 0,
      "messages_register_slave": 0,
      "messages_reregister_framework": 0,
      "messages_reregister_slave": 1,
      "messages_resource_request": 0,
      "messages_revive_offers": 0,
      "messages_status_update": 0,
      "messages_status_update_acknowledgement": 0,
      "messages_unregister_framework": 0,
      "messages_unregister_slave": 0,
      "messages_update_slave": 1,
      "recovery_slave_removals": 0,
      "slave_Removals": {
        "reason_registered": 0,
        "reason_unhealthy": 0,
        "reason_unregistered": 0
      },
      "valid_framework_to_executor_messages": 0,
      "valid_operation_status_update_acknowledgements": 0
      "valid_status_update_acknowledgements": 0,
      "valid_status_updates": 0,
      "valid_executor_to_framework_messages": 0,
   },


Event queue
These metrics provide information about different types of events in the event queue.

  "master": {
      "event_queue_dispatches": 6,
      "event_queue_http_requests": 0,
      "event_queue_messages": 0,
      "operator_event_stream_subscribers": 0,
   },


Registrar
These metrics provide information about read and write latency to the agent registrar and replicate log metrics provide information about the replicated log underneath the registrar, which is the persistent store for masters.

    "registrar": {
      "log": {
        "ensemble_size": 1,
        "recovered": 1
      },
      "state_fetch_ms": 28.32896,
      "state_store_ms": 18.193152,
      "state_store_MS": {}
    }


Allocator
These metrics provide information about performance and resource allocations in the allocator.

    "allocator": {
      "mesos": {
        "allocation_run_latency_ms": 0.166912,
        "allocation_run_latency_MS": {
          "p50": 0.153088,
          "p90": 0.165888,
          "p999": 0.34411084799999886,
          "p95": 0.175872,
          "min": 0.036096,
          "max": 0.390912,
          "count": 1000,
          "p99": 0.19791872,
          "p9999": 0.3862318848000031
        },
        "resources": {
          "disk": {
            "offered_or_allocated": 0,
            "total": 172196
          },
          "mem": {
            "total": 14886,
            "offered_or_allocated": 0
          },
          "cpus": {
            "offered_or_allocated": 0,
            "total": 12
          }
        },
        "allocation_run_ms": 0.231936,
        "allocation_run_MS": {
          "max": 0.323072,
          "p50": 0.228096,
          "p90": 0.2439936,
          "count": 1000,
          "p95": 0.26039039999999997,
          "p999": 0.3100290559999997,
          "min": 0.070144,
          "p9999": 0.3217677056000009,
          "p99": 0.28597248
        },
        "allocation_runs": 2110
      }


You can check Mesos documentation for more information about Mesos metrics.

Hiç yorum yok:

Yorum Gönder