Alerting and monitoring system

Out of date

Existing systems

Within NYC Mesh only 

• #monitoring-unms/UISP
  

Grafana/Prometheus

public, setup 4 years ago:UISP:  https://stats.nycmesh.net10.70.76.21/nms/login
Mesh only, Omni's etc:Grafana: http://10.70.90.82:3000/dashboards3000

General link (Backbone) Omni stats
LHG60 Prometheus General:  http://10.70.90.82:9090 Prometheus Omni only: http://10.70.90.142:9090

Omni port5 at 100Mbps
Omni memory used 75% or above  snmp-exporter: http://10.70.90.82:9116

 

• support report generator

Zabbix

• IP:http://10.70.73.58/
• Details: Runs on Quincy's server, connected to Beta Slack

Requirements

• Must:
  • alert Slack team when key infrasture goes offline within 5 minutes
• Should:
  • be easy to update for new equipment
  • be easy to configure to notify new volunteers
  • be easy to deploy
  • be reliable
  • be configurable though a version controlled config to enable easy updates
  • be editable by multiple volunteers

Questions

• Major
  • What key metrics should we alert based on?
• Minor
  • frequency? ~1 point/hour

Proposed software

• Zabbix
• Nagios
• Grafana
• [add your suggestion here]

Next Steps

Log

• prompted by this Slack discussion on Grand St. outage
• added Zabbix server during Hack night,