DevOps Daily

/home

/articles

/topics

Home > Articles > Self-Hosting Monitoring: Adding Alertmanager, Uptime Kuma, and Prometheus Blackbox Exporter

Self-Hosting Monitoring: Adding Alertmanager, Uptime Kuma, and Prometheus Blackbox Exporter

  ·   4 min read

#self-hosting #homelab #docker #monitoring #prometheus #alertmanager

In the previous articles of this series, we set up a self-hosting environment with Raspberry Pi, Docker, and Nginx Proxy Manager. We then installed a monitoring stack consisting of Node Exporter, Prometheus, and Grafana. Now, we will enhance our monitoring capabilities by adding Alertmanager, Uptime Kuma, and Prometheus Blackbox Exporter.

These tools will allow you to monitor the health of your websites, receive alerts for critical issues, and track the status of services and SSL certificates. We will show how to deploy all three services as Docker containers and demonstrate how to set up website monitoring and TLS certificate expiry checks.

Step 1: Install Alertmanager for Notifications

Alertmanager handles alerts sent by Prometheus and can send notifications through various channels like email, Slack, and more. It’s essential for ensuring that you are notified when there’s an issue with your self-hosted services.

  1. Create an Alertmanager Configuration File (alertmanager.yml):

    global:
  resolve_timeout: 5m

route:
  receiver: 'email-notifications'

receivers:
  - name: 'email-notifications'
    email_configs:
      - to: '[email protected]'
        from: '[email protected]'
        smarthost: 'smtp.example.com:587'
        auth_username: 'username'
        auth_password: 'password'
        require_tls: true

  2. Create a Docker Compose File for Alertmanager:

    version: '3'
services:
  alertmanager:
    image: prom/alertmanager:latest
    container_name: alertmanager
    restart: unless-stopped
    volumes:
      - ./alertmanager.yml:/etc/alertmanager/alertmanager.yml
    ports:
      - "9093:9093"

  3. Deploy Alertmanager:

    docker-compose -f alertmanager.yml up -d

  4. Configure Prometheus to Use Alertmanager: Add the following to your prometheus.yml file:

    alerting:
  alertmanagers:
    - static_configs:
        - targets: ['alertmanager:9093']

rule_files:
  - "alert_rules.yml"

Step 2: Install Prometheus Blackbox Exporter

The Blackbox Exporter is a versatile tool for monitoring website availability, checking HTTP response codes, and testing TLS certificate expiry. It allows you to configure probes that will periodically test your websites and services.

  1. Create a Docker Compose File for Blackbox Exporter:

    version: '3'
services:
  blackbox_exporter:
    image: prom/blackbox-exporter:latest
    container_name: blackbox_exporter
    restart: unless-stopped
    ports:
      - "9115:9115"

  2. Deploy Blackbox Exporter:

    docker-compose -f blackbox_exporter.yml up -d

  3. Configure Prometheus to Scrape Blackbox Exporter: Add the following to your prometheus.yml:

    scrape_configs:
  - job_name: 'blackbox'
    metrics_path: /probe
    params:
      module: [http_2xx]  # Use 'http_2xx' to check HTTP response
    static_configs:
      - targets:
        - https://example.com
        - https://anotherwebsite.com
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: blackbox_exporter:9115  # Blackbox Exporter container

Example: Monitoring Website Availability and TLS Expiry

Create a Prometheus Alert Rule (alert_rules.yml):

groups:
  - name: website_alerts
    rules:
      - alert: WebsiteDown
        expr: probe_success == 0
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Website {{ $labels.instance }} is down"
          description: "The website {{ $labels.instance }} is unreachable or returned a non-2xx status."

      - alert: TLSCertificateExpiry
        expr: probe_ssl_earliest_cert_expiry - time() < 86400 * 7
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "TLS Certificate for {{ $labels.instance }} is expiring soon"
          description: "The TLS certificate for {{ $labels.instance }} will expire in less than 7 days."

Step 3: Install Uptime Kuma for Service Monitoring

Uptime Kuma is an open-source, self-hosted monitoring tool that provides a beautiful dashboard to track uptime and response time for your websites, APIs, and services. It’s similar to services like UptimeRobot, but it runs locally.

  1. Create a Docker Compose File for Uptime Kuma:

    version: '3'
services:
  uptime-kuma:
    image: louislam/uptime-kuma:latest
    container_name: uptime_kuma
    restart: unless-stopped
    ports:
      - "3001:3001"
    volumes:
      - ./uptime-kuma:/app/data

  2. Deploy Uptime Kuma:

    docker-compose -f uptime_kuma.yml up -d

  3. Access Uptime Kuma: Open a browser and go to http://<your-raspberry-pi-ip>:3001. Set up your account, then you can start adding monitors for your services.

Step 4: Configure Domain Names with Nginx Proxy Manager

We’ll use Nginx Proxy Manager to set domain names like uptime.example.com, blackbox.example.com, and alerts.example.com for easy access.

  1. Open Nginx Proxy Manager: Navigate to http://<your-raspberry-pi-ip>:81 and log in.

  2. Add Proxy Hosts:

    • For Alertmanager:
      • Domain Names: alerts.example.com
      • Forward Hostname / IP: <your-raspberry-pi-ip>
      • Forward Port: 9093
      • SSL: Check Request a new SSL Certificate.
      • Click Save.
    • For Blackbox Exporter:
      • Domain Names: blackbox.example.com
      • Forward Hostname / IP: <your-raspberry-pi-ip>
      • Forward Port: 9115
      • SSL: Check Request a new SSL Certificate.
      • Click Save.
    • For Uptime Kuma:
      • Domain Names: uptime.example.com
      • Forward Hostname / IP: <your-raspberry-pi-ip>
      • Forward Port: 3001
      • SSL: Check Request a new SSL Certificate.
      • Click Save.

  3. Verify Domain Names:

    • Visit http://alerts.example.com to access Alertmanager.
    • Visit http://blackbox.example.com to access Blackbox Exporter.
    • Visit http://uptime.example.com to access Uptime Kuma.

Conclusion

With the addition of Alertmanager, Prometheus Blackbox Exporter, and Uptime Kuma, your homelab’s monitoring capabilities are now robust and comprehensive. You can monitor websites, track service uptime, and receive alerts for potential issues. Using Nginx Proxy Manager, you can easily manage access to these services with custom domain names and SSL certificates.

In the next article, we will explore how to create advanced dashboards in Grafana to visualize metrics and monitor your self-hosted services effectively. Stay tuned!

References

By following these steps, you can efficiently monitor your self-hosted services and gain insights into their performance using Docker containers. Happy monitoring!

←
Useful Tips for Working with RabbitMQ
Self-Hosting Monitoring Stack: Installing Node Exporter, Prometheus, and Grafana
→
back to top