Splitting Kubernetes Logs by Namespaces With Fluent Bit

Today, One of the easiest ways to do log shipping from a Kubernetes cluster is by using fluent bit. Fluent Bit Fluent Bit is like the little brother of fluentd and is written in C and takes less resources, so it is the best fit for running as a Daemonset in Kubernetes for log shipping pod logs. Fluent bit also enriches the logs it is collecting from pods in kubernetes using a built-in filter called kubernetes which enriches the logs with the following information:...

May 5, 2020 · 5 min · 854 words · Omer

A Simple way to do log rotate on an AWS Elasticsearch using AWS Lambda

This is a short post on log rotating AWS Elasticsearch indices easily using curator In the past I’ve made a simple docker container with curator and with a simple actions file for managing indices. This worked great on my own installation of Elasticsearch that managed our logs on a small scale. The actions file was: --- actions: 1: action: delete_indices description: >- Rolling indeces accoring to filters options: ignore_empty_list: True #disable_action: True disable_action: False filters: - filtertype: pattern kind: prefix value: filebeat- - filtertype: age source: name direction: older timestring: '%Y....

July 16, 2019 · 2 min · 417 words · Omer