The unified CloudWatch agent enables you to do the following:

  • Collect more system-level metrics from Amazon EC2 instances across operating systems. The metrics can include in-guest metrics, in addition to the metrics for EC2 instances. The additional metrics that can be collected are listed in Metrics Collected by the CloudWatch Agent.

  • Collect system-level metrics from on-premises servers. These can include servers in a hybrid environment as well as servers not managed by AWS.

  • Retrieve custom metrics from your applications or services using the StatsD and collectd protocols. StatsD is supported on both Linux servers and servers running Windows Server. collectd is supported only on Linux servers.

  • Collect logs from Amazon EC2 instances and on-premises servers, running either Linux or Windows Server.

BUT, because the Raspberry Pi’s architecture is “armhf” or ARM Hard Float and not arm64, CloudWatch does not provide a suitable architecture for installation.

  • Run the command “dpkg –print-architecture” to determine your systems architecture.

In order to get around this limitation we look to AWSLabs on GitHub to provide us with the python scripted C programmed variant known as “CloudWatch collectd plugin” [2]

“The CloudWatch collectd plugin is a publishing extension for collectd, an open source statistic gathering daemon. With our extension all configured collectd metrics are automatically published to CloudWatch. This plugin allows monitoring of servers and applications within and outside of EC2 instances. The plugin offers additional functionality for EC2 customers such as automatic discovery of Instance ID and AWS region.ls”

Here are the steps to use it

Install the CloudWatch collectd plugin

  1. Download collectd

    sudo apt-get collectd -y
    
  2. Download the CloudWatch collectd plugin

    wget https://raw.githubusercontent.com/awslabs/collectd-cloudwatch/master/src/setup.py
    

    REQUIRED You will need to modify setup.py by adding ““Debian GNU”: APT_INSTALL_COMMAND,” to “DISTRIBUTION_TO_INSTALLER” to get it to work on the Raspberry Pi.

    DISTRIBUTION_TO_INSTALLER = {
        "Debian GNU": APT_INSTALL_COMMAND,
        "Ubuntu": APT_INSTALL_COMMAND,
        "Red Hat Enterprise Linux Server": YUM_INSTALL_COMMAND,
        "Amazon Linux AMI": YUM_INSTALL_COMMAND,
        "Amazon Linux": YUM_INSTALL_COMMAND,
        "CentOS Linux": YUM_INSTALL_COMMAND,
    }
    
    • The reason for this modication is because the Raspberry Pi Raspbian OS “$ cat /etc/*-release” returns “NAME=“Debian GNU/Linux”“” and several functions within the Setup.py rely on the distribution to determine appropriate commands.
  3. Configure and run the now modified Setup.py

    chmod u+x setup.py
    sudo ./setup.py
    
  4. Configuration options to set via the script

  • You will see errors because it cannot contact the metadata EC2 server to scrape configuration options, but will be prompted to provide them, thus it will still work!

    Installing dependencies ... OK
    Installing python dependencies ... OK
    Downloading plugin ... OK
    Extracting plugin ... OK
    Moving to collectd plugins directory ... OK
    Copying CloudWatch plugin include file ... OK
    DEBUG:urllib3.util.retry:Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
    DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 169.254.169.254:80
    DEBUG:urllib3.util.retry:Incremented Retry for (url='/latest/meta-data/placement/availability-zone/'): Retry(total=0, connect=None, read=None, redirect=None, status=None)
    WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x75f4b610>, 'Connection to 169.254.169.254 timed out. (connect timeout=0.3)')': /latest/meta-data/placement/availability-zone/
    DEBUG:urllib3.connectionpool:Starting new HTTP connection (2): 169.254.169.254:80
    
    AWS region could not be automatically detected. Cause:Cannot access metadata service. Cause: HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /latest/meta-data/placement/availability-zone/ (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x75f4b790>, 'Connection to 169.254.169.254 timed out. (connect timeout=0.3)'))
    Enter one of the available regions from: http://docs.aws.amazon.com/general/latest/gr/rande.html#cw_region
    Enter region: us-west-2
    DEBUG:urllib3.util.retry:Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
    DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 169.254.169.254:80
    DEBUG:urllib3.util.retry:Incremented Retry for (url='/latest/meta-data/instance-id/'): Retry(total=0, connect=None, read=None, redirect=None, status=None)
    WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x75f4b850>, 'Connection to 169.254.169.254 timed out. (connect timeout=0.3)')': /latest/meta-data/instance-id/
    DEBUG:urllib3.connectionpool:Starting new HTTP connection (2): 169.254.169.254:80
        
    EC2 instance id could not be automatically detected.
    Enter hostname [raspberrypi]: 
    DEBUG:urllib3.util.retry:Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
    DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 169.254.169.254:80
    DEBUG:urllib3.util.retry:Incremented Retry for (url='/latest/meta-data/iam/security-credentials/'): Retry(total=0, connect=None, read=None, redirect=None, status=None)
    WARNING:urllib3.connectionpool:Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x75f4bb30>, 'Connection to 169.254.169.254 timed out. (connect timeout=0.3)')': /latest/meta-data/iam/security-credentials/
    DEBUG:urllib3.connectionpool:Starting new HTTP connection (2): 169.254.169.254:80
        
    IAM Role could not be automatically detected.
    Enter absolute path to AWS credentials file [/root/.aws/credentials]: /home/pi/.aws/credentials
        
    Enter proxy server name:
      1. None
      2. Custom
    Enter choice [1]: 
        
    Enter proxy server port:
      1. None
      2. Custom
    Enter choice [1]: 
        
    Include the Auto-Scaling Group name as a metric dimension:
      1. No
      2. Yes
    Enter choice [1]: 
        
    Include the FixedDimension as a metric dimension:
      1. No
      2. Yes
    Enter choice [1]: 
        
    Enable high resolution:
      1. Yes
      2. No
    Enter choice [2]: 
        
    Enter flush internal:
      1. Default 60s
      2. Custom
    Enter choice [1]: 
        
    Choose how to install CloudWatch plugin in collectd:
      1. Do not modify existing collectd configuration
      2. Add plugin to the existing configuration
      3. Use CloudWatch recommended configuration (4 metrics)
    Enter choice [3]: 2
    Plugin configuration written successfully.
    Stopping collectd process ... OK
    Starting collectd process ... OK
    

Configure CloudWatch metrics to publish

5. Determine which metrics are publishable

  • The file /opt/collectd-plugins/cloudwatch/config/blocked_metrics, contains a list of metrics that have been collected by collectd but have not been published yet.

    $ cat /opt/collectd-plugins/cloudwatch/config/blocked_metrics 
        # This file is automatically generated - do not modify this file.    
        # Use this file to find metrics to be added to the whitelist file instead.
        interface-wlan0-if_packets-rx
        interface-wlan0-if_packets-tx
        interface-wlan0-if_octets-rx
        interface-wlan0-if_octets-tx
        df-root-df_complex-free
        entropy--entropy-
        ...etc.
    

6. “Whitelist” metrics to publish

  • Add any metric from “blocked_metrics” to /opt/collectd-plugins/cloudwatch/config/whitelist.conf

  • I added these:

    df-root--percent_bytes-used
    memory--percent-used
    swap--percent-used
    cpu--percent-active
    

7. Restart collectd to begin pushing metrics to CloudWatch

```
sudo service collectd restart
```

8. Login to CloudWatch to see your Raspberry Pi metrics!

Raspberry_Pi_Metrics


AWS Guides

AWS CloudWatch agent

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html

AWS Labs Cloudwatch collectd plugin

https://github.com/awslabs/collectd-cloudwatch

AWS Blog Post collectd plugin

https://aws.amazon.com/blogs/aws/new-cloudwatch-plugin-for-collectd/