Posts

Today, One of the easiest ways to do log shipping from a Kubernetes cluster is by using fluent bit. Fluent Bit Fluent Bit is like the little brother of fluentd and is written in C and takes less resources, so it is the best fit for running as a Daemonset in Kubernetes for log shipping pod logs. Fluent bit also enriches the logs it is collecting from pods in kubernetes using a built-in filter called kubernetes which enriches the logs with the following information:...

Recently I’ve joined a project with real time machine learning inference. The project was set to run on Kubernetes 1.12 on AWS while development and training was made on premise, and some were made using Docker. When the application was deployed to production we started to see poor performance and started to investigate. and it was weird, on the developers machines and in docker everything worked faster. After debugging it for a while I understood that the jvm doesn’t see all available cores inside the pod....

When you choose to run your kubernetes cluster on AWS, there are 2 easy ways: The almost valina way using Kops The amazon way using EKS The common thing between both ways, is using Amazon ECR for storing the docker images and have a worry free push, pull to and from Amazon ECR which requires a IAM Role that allows the worker nodes pulling the images saftly. Amazon ECR Amazon Elastic Container Service is one of the cheapst ways to store docker images and safer due to the nature of Amazon IAM....

This is a short post on log rotating AWS Elasticsearch indices easily using curator In the past I’ve made a simple docker container with curator and with a simple actions file for managing indices. This worked great on my own installation of Elasticsearch that managed our logs on a small scale. The actions file was: --- actions: 1: action: delete_indices description: >- Rolling indeces accoring to filters options: ignore_empty_list: True #disable_action: True disable_action: False filters: - filtertype: pattern kind: prefix value: filebeat- - filtertype: age source: name direction: older timestring: '%Y....

Using AWS Lambda has become very popular. Most of the time using it is fast and just (but not always). When you start having a lot of lambda functions the complexity of managing them can be hard. When a developer can just edit the code inline, it can become the worse manage service use ever chose. So again, the most important thing to do when you start working with Lambda is to have a proper way to deliver the code from git to aws....

Docker has become the preferable way and the most easy way to run continuous integration, due to its reproducible nature and fast learning curve. There are multiple ways approaching CI processes using docker. Using docker build, which is the easiest way FROM node:12 ARG NODE_ENV=production COPY . /src RUN npm install WORKDIR /src CMD [ "node", "app.js" ] Using docker multi step build, this usually helps making the final image smaller and safer without all the build binaries FROM maven AS Builder COPY ....

In this post I am going to describe how to maintain access to Amazon AWS EKS - kubernetes cluster with only attaching an IAM Role as an instance profile, without configuring access keys on the ec2 instance. The benefit of course is not storing any Amazon IAM credentials on the ec2 instance. Having your infrastructure more secure. In my use case, I wanted to have my Jenkins CI server have access to EKS cluster for adding continuance delivery using Jenkins declarative pipeline, to my continuance integration process....

This blog will run using hugo static site generator project. I am using the theme hugo-cactus-theme by digitalcraftsman which hopefully I wont edit it, its the cleanest, simplest and SEO friendly that I found. The entire blog will be made by using markdown language. The blog will be hosted on Netlify. Netlify offers CI/CD, CDN, Let’s Encrypt SSL and an optional custom domain for free. The code will be saved on Github for free....