How to run filebeat in kubernetes ?

Introduction

Filebeat is a lightweight shipper for forwarding and centralizing log data. Deploying Filebeat on Kubernetes allows you to collect logs from containerized applications running within your cluster. This document provides a step-by-step guide for deploying Filebeat on Kubernetes and configuring it to collect container logs.

Prerequisites

Before proceeding with the deployment of Filebeat on Kubernetes, ensure the following prerequisites are met:

1. Access to a Kubernetes cluster

2. kubectl or oc command-line tool installed

3. Knowledge of basic Kubernetes concepts and operations

Procedure

Download Manifest File: Obtain the Kubernetes deployment manifest file for Filebeat by running the following command:

  curl -L -O https://raw.githubusercontent.com/elastic/beats/8.9/deploy/kubernetes/filebeat-kubernetes.yaml

Modify Container Spec: Update the DaemonSet container specification in the manifest file to include security context settings:

  securityContext:

  runAsUser: 0

  privileged: true

Grant Permissions: Grant the Filebeat service account access to the privileged Security Context Constraints (SCC) with the following command:

  oc adm policy add-scc-to-user privileged system:serviceaccount:kube-system:filebeat

Override Node Selector: Override the default node selector for the kube-system namespace (or your custom namespace) to allow scheduling on any node:

  oc patch namespace kube-system -p '{"metadata": {"annotations": {"openshift.io/node-selector": ""}}}'

Deploy Filebeat: Deploy Filebeat to Kubernetes by running the following command:

  kubectl create -f filebeat-kubernetes.yaml

Check Status: Verify the status of the Filebeat DaemonSet by running:

  kubectl --namespace=kube-system get ds/filebeat

Apply Configuration: Apply any custom configurations to Filebeat by creating and applying a configuration YAML file:

apiVersion: v1
kind: ServiceAccount
metadata:
 name: filebeat
 namespace: kube-system
 labels:
   k8s-app: filebeat
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
 name: filebeat
 labels:
   k8s-app: filebeat
rules:
- apiGroups: [""]
 resources:
 - namespaces
 - pods
 - nodes
 verbs: ["get", "watch", "list"]
- apiGroups: ["apps"]
 resources:
   - replicasets
 verbs: ["get", "list", "watch"]
- apiGroups: ["batch"]
 resources:
   - jobs
 verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
 name: filebeat
 namespace: kube-system
 labels:
   k8s-app: filebeat
rules:
 - apiGroups: ["coordination.k8s.io"]
   resources: ["leases"]
   verbs: ["get", "create", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
 name: filebeat-kubeadm-config
 namespace: kube-system
 labels:
   k8s-app: filebeat
rules:
 - apiGroups: [""]
   resources:
     - configmaps
   resourceNames:
     - kubeadm-config
   verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 name: filebeat
subjects:
- kind: ServiceAccount
 name: filebeat
 namespace: kube-system
roleRef:
 kind: ClusterRole
 name: filebeat
 apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: filebeat
 namespace: kube-system
subjects:
 - kind: ServiceAccount
   name: filebeat
   namespace: kube-system
roleRef:
 kind: Role
 name: filebeat
 apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: filebeat-kubeadm-config
 namespace: kube-system
subjects:
 - kind: ServiceAccount
   name: filebeat
   namespace: kube-system
roleRef:
 kind: Role
 name: filebeat-kubeadm-config
 apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ConfigMap
metadata:
 name: filebeat-config
 namespace: kube-system
 labels:
   k8s-app: filebeat
data:
 filebeat.yml: |-
   filebeat.inputs:
   - type: container
     paths:
      - /var/log/containers/advance-website-*.log
     fields:
      environment: eks-prod-advance-website
   - type: container
     paths:
      - /var/log/containers/advance-backend-*.log
     fields:
      environment: eks-prod-advance-backend
   - type: container
     paths:
      - /var/log/containers/leap-backend-*.log
     fields:
      environment: eks-prod-leap-backend
   - type: container
     paths:
      - /var/log/containers/finance-frontend-*.log
     fields:
      environment: eks-prod-finance-frontend
   - type: container
     paths:
      - /var/log/containers/leap-portal-*.log
     fields:
      environment: eks-prod-leap-portal
   cloud.id: ${ELASTIC_CLOUD_ID}
   cloud.auth: ${ELASTIC_CLOUD_AUTH}
   output.logstash:
       hosts: ["20.0.2.28:5045"]
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
 name: filebeat
 namespace: kube-system
 labels:
   k8s-app: filebeat
spec:
 selector:
   matchLabels:
     k8s-app: filebeat
 template:
   metadata:
     labels:
       k8s-app: filebeat
   spec:
     serviceAccountName: filebeat
     terminationGracePeriodSeconds: 30
     hostNetwork: true
     dnsPolicy: ClusterFirstWithHostNet
     containers:
     - name: filebeat
       securityContext:
          runAsUser: 0
          privileged: true
       image: docker.elastic.co/beats/filebeat:8.8.1
       args: [
         "-c", "/etc/filebeat.yml",
         "-e",
       ]
       resources:
         limits:
           memory: 200Mi
         requests:
           cpu: 100m
           memory: 100Mi
       volumeMounts:
       - name: config
         mountPath: /etc/filebeat.yml
         readOnly: true
         subPath: filebeat.yml
       - name: varlibdockercontainers
         mountPath: /var/lib/docker/containers
         readOnly: true
       - name: varlog
         mountPath: /var/log
         readOnly: true
     volumes:
     - name: config
       configMap:
         defaultMode: 0640
         name: filebeat-config
     - name: varlibdockercontainers
       hostPath:
         path: /var/lib/docker/containers
     - name: varlog
       hostPath

   kubectl apply -f filebeat.yaml

Load Kibana Dashboard: Login to the Kibana dashboard and create an index pattern with the pod or application name. Navigate to Stack Management and create an index pattern. Once the index pattern is created, logs can be viewed in the Discover option.

Conclusion

By following this guide, you have successfully deployed Filebeat on Kubernetes, configured it to collect container logs, and loaded the logs into Kibana for visualization and analysis. Filebeat will now continuously monitor and ship logs from your Kubernetes cluster, providing valuable insights into the health and performance of your applications.