Kubernetes

Today, one of the easiest ways to ship logs from a Kubernetes cluster is by using Fluent Bit. Fluent Bit Fluent Bit is the lightweight sibling of Fluentd. It is written in C, uses fewer resources, and is a great fit for running as a DaemonSet in Kubernetes to ship pod logs. Fluent Bit also enriches logs it collects from pods in Kubernetes using a built-in filter called kubernetes, which adds the following information: Pod Name Namespace Container Name Container ID Pod ID Labels Annotations The first four are collected from the pod tag and the last three from the Kubernetes API server. The data is stored in Fluent Bit’s cache, so there isn’t a big overhead on the API server. These records are appended to each log collected from the pods, making them easier to search. ...

Recently I’ve joined a project with real time machine learning inference. The project was set to run on Kubernetes 1.12 on AWS while development and training was made on-premise, and some were made using Docker. When the application was deployed to production we started to see poor performance and began to investigate. It was weird, on the developers machines and in docker everything worked faster. After debugging it for a while I understood that the jvm doesn’t see all available cores inside the pod. while searching for some solutions I found that when running java 8 with and adding some extra JVM_OPTS along with all the rest of the opts it would solve my issue. So I’ve added it to the deployment.yaml file and deployed it again. ...

When you choose to run your kubernetes cluster on AWS, there are 2 easy ways: The almost valina way using Kops The amazon way using EKS The common thing between both ways, is using Amazon ECR for storing the docker images and have a worry free push, pull to and from Amazon ECR which requires a IAM Role that allows the worker nodes pulling the images saftly. Amazon ECR Amazon Elastic Container Service is one of the cheapst ways to store docker images and safer due to the nature of Amazon IAM. To pull from ECR you first need to authenticate using you AWS credentials, or role, get a token, do docker login to your ECR with the server address, and pull from the repository the docker image. ...

Overview In this post, I am going to describe how to maintain access to Amazon AWS EKS - Kubernetes cluster with only attaching an IAM Role as an instance profile, without configuring access keys on the EC2 instance. The benefit of course is not storing any Amazon IAM credentials on the EC2 instance, having your infrastructure more secure. Use Case: Jenkins CI/CD Pipeline In my use case, I wanted to have my Jenkins CI server have access to EKS cluster for adding continuous delivery using Jenkins declarative pipeline to my continuous integration process. By doing that, I achieved a full CI/CD process for all the microservices that Jenkins handled. ...