siri chandana's blog
Day 74: Connecting EC2 with Grafana☁☁

Day 74: Connecting EC2 with Grafana☁☁

Siri Chandana's photo
Siri Chandana
·

4 min read

Table of contents

You guys did an amazing job last day setting up Grafana on Local 🎉🎉 Now, let's take one step ahead.

Introduction:

Grafana can be effectively utilized in conjunction with AWS EC2 instances for monitoring various aspects of your AWS infrastructure, applications, and services. Here's how Grafana can be explained in the context of AWS EC2

Grafana can also ingest custom metrics and logs from EC2 instances using plugins or agents like Prometheus, Telegraf, or Fluentd. This enables you to monitor application-specific metrics, log data, and other custom performance indicators alongside AWS-native metrics.

Grafana offers robust alerting capabilities that allow you to define threshold-based alerts on your EC2 instance metrics. You can configure alerts to trigger notifications via email, Slack, PagerDuty, or other channels when certain conditions are met, helping you proactively identify and respond to performance issues or anomalies.

what is prometheus?

Prometheus is an open-source monitoring and alerting toolkit originally built at SoundCloud. It's designed for reliability, scalability, and flexibility in monitoring dynamic, cloud-native environments. Prometheus collects metrics from monitored targets by scraping HTTP endpoints on those targets. These targets can be anything from application instances, services, databases, to even the infrastructure itself.

Here are some key aspects of Prometheus and why it's widely used:

  1. Multi-dimensional Data Model: Prometheus uses a powerful multi-dimensional data model with time series data identified by metric name and key-value pairs called labels.

  2. Scalability: Prometheus is designed to be highly scalable both in terms of the number of targets it can monitor and the volume of metrics it can handle.

  3. Alerting: Prometheus has a built-in alerting system that allows users to define alerting rules based on the collected metrics. When a rule is triggered, Prometheus can send alerts to various alerting integrations such as email, Slack, PagerDuty, etc.

  4. Integration with Grafana: While Prometheus has its own basic visualization capabilities, it is commonly used in conjunction with Grafana, a popular open-source visualization tool. Grafana provides more advanced visualization features and allows users to create custom dashboards for monitoring and analyzing Prometheus metrics.

Task:

Connecting EC2 Instances with Grafana

Now that we have a brief overview of the tools we’ll be using, let’s go ahead and connect our Linux EC2 instances with Grafana. We’ll follow these steps:

  1. Set Up Grafana Server (Already Covered): Day73

    • As mentioned in the previous response, you should have Grafana installed and running on a Linux EC2 instance. Make sure it's accessible via a web browser.

We have already setup Grafana on the Ubuntu EC2 "Grafana-Server" using shell-script using the below code:

#!/bin/bash

# Update the system for the first time:
sudo apt update

# Install the prerequisite packages:
sudo apt-get install -y apt-transport-https software-properties-common wget

# Import the GPG key:
sudo mkdir -p /etc/apt/keyrings/
wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null

# To add a repository for stable releases, run the following command:
echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

# Updates the list of available packages
sudo apt-get update

# To install Grafana OSS, run the following command:
sudo apt-get install grafana -y

# To start and enable the service, run the following commands:
sudo systemctl start grafana-server.service
sudo systemctl enable grafana-server.service

# To verify that the service is running, run the following command:
sudo systemctl status grafana-server.service

Install Prometheus on Linux EC2 (Monitoring Server):

  • Prometheus is a popular monitoring and alerting toolkit that can scrape metrics from various targets, including EC2 instances.

    • Install Prometheus on your Linux EC2 instance. You can download it from the Prometheus website and follow the installation instructions.

        sudo apt-get update
  sudo apt-get install -y prometheus
  sudo systemctl enable prometheus
  sudo systemctl start prometheus

Configure Prometheus:

  • Access your Grafana web interface and log in.

  • Configure Prometheus as a data source in Grafana.

  • Go to "Configuration" > "Data Sources" > "Add data source."

  • Choose "Prometheus" as the data source type.

    Create CPU Usage Dashboard in Grafana

    1. In Grafana, go to the "+" menu on the left sidebar and choose "Dashboard."

    2. Add a new panel.

    3. Select the Prometheus data source.

    4. Use the following query to monitor user CPU usage:

         rate(node_cpu_seconds_total{mode="user"}[1m])

  • Configure visualization settings and save the panel.

  • Explore and Analyze Metrics

Explore the Grafana dashboard to visualize and analyze CPU usage metrics from both Ubuntu instances. Customize panels and queries based on your monitoring requirements.

Congratulations! You’ve successfully connected Linux EC2 instances with Grafana and created a dashboard to monitor their performance metrics. Monitoring is a crucial aspect of any DevOps practice, and Grafana along with Prometheus and Node Exporter provides a powerful combination to gain insights into the health of your infrastructure.

Thank you for reading. 👍 Happy Learning😊😊

If you like this article , then click on 👏👏 do follow for more interesting 📜articles. Hope you find it helpful✨✨

Connect on 👉 chandana LinkedIn

#90daysofdevopschallengeTrainWithShubhamDevops articlesGrafanamonitoring