In the realm of Kubernetes, ensuring your clusters are observable and that logs are efficiently managed can be pivotal for understanding the behavior of your applications and for troubleshooting issues. Amazon Elastic Kubernetes Service (EKS) users have a robust tool at their disposal for this purpose: Fluent Bit. This lightweight log processor and forwarder is designed for the cloud, and when configured correctly, can provide deep insights into your applications running on Kubernetes. Today, we’ll dive into setting up Fluent Bit using a Kubernetes ConfigMap to enhance your EKS cluster’s observability.

Introduction to ConfigMaps

Before we delve into the specifics, let’s understand what a ConfigMap is. In Kubernetes, a ConfigMap is a key-value store used to store configuration data. This data can be consumed by pods or used to store configuration files. It’s an ideal way to manage configurations and make them available to your applications without hardcoding them into your application’s code.

Setting Up Fluent Bit for Logging in EKS

The goal here is to configure Fluent Bit to forward logs from your EKS cluster to AWS CloudWatch, allowing you to monitor, store, and access your logs. The configuration involves creating a ConfigMap that Fluent Bit will use to understand where and how to process and forward your logs.

Here’s an overview of the ConfigMap for setting up Fluent Bit for logging:

kind: ConfigMap
apiVersion: v1
metadata:
  name: aws-logging
  namespace: aws-observability
data:
  output.conf: |
    [OUTPUT]
        Name cloudwatch_logs
        Match   *
        region us-east-1
        log_group_name eks/sandbox-cluster
        log_group_template eks/$kubernetes['namespace_name']
        log_stream_prefix pod-logs-
        log_retention_days 15
        auto_create_group true
        log_key log
  parsers.conf: |
    [PARSER]
        Name crio
        Format Regex
        Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>P|F) (?<log>.*)$
        Time_Key    time
        Time_Format %Y-%m-%dT%H:%M:%S.%L%z
  filters.conf: |
    [FILTER]
        Name parser
        Match *
        Key_name log
        Parser crio

    [FILTER]
        Name             kubernetes
        Match            kube.*
        Kube_Tag_Prefix  kube.var.log.containers.
        Merge_Log        On
        Merge_Log_Key    log_processed

Understanding the Configuration

• Metadata: The metadata section names our ConfigMap aws-logging and places it within the aws-observability namespace.
• Data: Contains the configurations for Fluent Bit’s operation. It’s divided into three parts:
• output.conf: Defines how logs are forwarded to AWS CloudWatch. It specifies the log group name, region, retention policies, and more.
• parsers.conf: Contains parser definitions that help Fluent Bit understand the format of your logs. The example provided uses a regex parser for logs coming from crio (a lightweight container runtime).
• filters.conf: Filters allow Fluent Bit to process the logs before forwarding them. The provided configuration parses logs and enriches them with Kubernetes metadata.

Applying the ConfigMap

To apply this ConfigMap to your EKS cluster, save the YAML to a file and use kubectl apply -f <filename.yaml>. This command instructs Kubernetes to create the ConfigMap based on your file. After applying, Fluent Bit will use this configuration to process and forward logs from your cluster to AWS CloudWatch.

Conclusion

Setting up Fluent Bit with a properly configured ConfigMap can significantly enhance the observability of your EKS clusters. By leveraging AWS CloudWatch, you gain a powerful tool for log management and analysis, helping you keep your applications healthy and performant. Remember, the key to effective Kubernetes management lies in understanding the tools at your disposal and configuring them to meet your specific needs.