Posts

Today, one of the easiest ways to ship logs from a Kubernetes cluster is by using Fluent Bit. Fluent Bit Fluent Bit is the lightweight sibling of Fluentd. It is written in C, uses fewer resources, and is a great fit for running as a DaemonSet in Kubernetes to ship pod logs. Fluent Bit also enriches logs it collects from pods in Kubernetes using a built-in filter called kubernetes, which adds the following information: Pod Name Namespace Container Name Container ID Pod ID Labels Annotations The first four are collected from the pod tag and the last three from the Kubernetes API server. The data is stored in Fluent Bit’s cache, so there isn’t a big overhead on the API server. These records are appended to each log collected from the pods, making them easier to search. ...

Recently I’ve joined a project with real time machine learning inference. The project was set to run on Kubernetes 1.12 on AWS while development and training was made on-premise, and some were made using Docker. When the application was deployed to production we started to see poor performance and began to investigate. It was weird, on the developers machines and in docker everything worked faster. After debugging it for a while I understood that the jvm doesn’t see all available cores inside the pod. while searching for some solutions I found that when running java 8 with and adding some extra JVM_OPTS along with all the rest of the opts it would solve my issue. So I’ve added it to the deployment.yaml file and deployed it again. ...

When you choose to run your kubernetes cluster on AWS, there are 2 easy ways: The almost valina way using Kops The amazon way using EKS The common thing between both ways, is using Amazon ECR for storing the docker images and have a worry free push, pull to and from Amazon ECR which requires a IAM Role that allows the worker nodes pulling the images saftly. Amazon ECR Amazon Elastic Container Service is one of the cheapst ways to store docker images and safer due to the nature of Amazon IAM. To pull from ECR you first need to authenticate using you AWS credentials, or role, get a token, do docker login to your ECR with the server address, and pull from the repository the docker image. ...

This is a short post on log rotating AWS Elasticsearch indices easily using curator In the past I’ve made a simple docker container with curator and with a simple actions file for managing indices. This worked great on my own installation of Elasticsearch that managed our logs on a small scale. The actions file was: --- actions: 1: action: delete_indices description: >- Rolling indeces according to filters options: ignore_empty_list: True #disable_action: True disable_action: False filters: - filtertype: pattern kind: prefix value: filebeat- - filtertype: age source: name direction: older timestring: '%Y.%m.%d' unit: days unit_count: ${DAYS_TO_KEEP:14} And it was controlled in the docker run command, which host to work on and what old indices should be deleted, with environment variables. ...

Using AWS Lambda has become very popular. Most of the time using it is fast and just (but not always). When you start having a lot of lambda functions the complexity of managing them can be hard. When a developer can just edit the code inline, it can become the worse manage service use ever chose. So again, the most important thing to do when you start working with Lambda is to have a proper way to deliver the code from git to aws. ...

Docker has become the preferable way and the most easy way to run continuous integration, due to its reproducible nature and fast learning curve. There are multiple ways approaching CI processes using docker. Using docker build, which is the easiest way FROM node:12 ARG NODE_ENV=production COPY . /src RUN npm install WORKDIR /src CMD [ "node", "app.js" ] Using docker multi step build, this usually helps making the final image smaller and safer without all the build binaries FROM maven AS Builder COPY . /src RUN [ "mvn" , "clean" , "install" ] FROM openjdk:12-alpine WORKDIR /srv COPY --from=Builder /src/target ./srv/ CMD [ "java", "-jar", "app.jar" ] Using docker run with a pre-built build image on a volume mount this allows having cache directories and have some more functionality during the build, and will allow multiple commands docker run -v `$pwd`:/src -v `~/.m2`:/root/.m2 maven mvn clean install Using docker in docker, like gitlab, bitbucket etc… This method is more complex than the rest and require you to mount docker socket to the as a volume to the docker container, and can be preformed in 2 ways: There is an official docker in docker image by docker inc that is tagged as docker:dind ...

Overview In this post, I am going to describe how to maintain access to Amazon AWS EKS - Kubernetes cluster with only attaching an IAM Role as an instance profile, without configuring access keys on the EC2 instance. The benefit of course is not storing any Amazon IAM credentials on the EC2 instance, having your infrastructure more secure. Use Case: Jenkins CI/CD Pipeline In my use case, I wanted to have my Jenkins CI server have access to EKS cluster for adding continuous delivery using Jenkins declarative pipeline to my continuous integration process. By doing that, I achieved a full CI/CD process for all the microservices that Jenkins handled. ...

This blog will run using hugo static site generator project. I am using the theme hugo-PaperMod by adityatelange which is clean, modern, and SEO friendly. The entire blog will be made by using markdown language. The blog will be hosted on Netlify. Netlify offers CI/CD, CDN, Let’s Encrypt SSL and an optional custom domain for free. The code will be saved on Github for free. That is why I think that everyone should blog for free, not using any platform that offers you the same functionality or less, but doesn’t share its revenue on the content you write. ...