Linux Tips

Welcome to the latest edition of LHB Linux Digest. I don't know if you have noticed but I have changed the newsletter day from Wednesday to Friday so that you can enjoy your Fridays learning something new and discovering some new tool. Enjoy 😄

Here are the highlights of this edition :

• Creating a .deb package from Python app
• Quick Vim tip on indentation
• Pushing Docker image to Hub
• And more tools, tips and memes for you
• This edition of LHB Linux Digest newsletter is supported by PikaPods.

❇️ Self-hosting without hassle

PikaPods allows you to quickly deploy your favorite open source software. All future updates are handled automatically by PikaPods while you enjoy using the software. I use it to self host Umami analytics.

Oh! You get $5 free credit, so try it out and see if you could rely on PikaPods.

PikaPods - Instant Open Source App Hosting

Run the finest Open Source web apps from $1.20/month, fully managed, no tracking, no ads, full privacy. Self-hosting was never this convenient.

Instant Open Source App Hosting

Docker has changed the way we package and distribute applications, but I only truly appreciated its power when I needed to share a project with a friend.

Initially, we used docker save and docker load to transfer the image, which worked fine but was cumbersome.

Then, while browsing the Docker documentation, I discovered how easy it was to push images to Docker Hub.

That was a game-changer! Now, I push my final builds to Docker Hub the moment they're done, allowing my clients and collaborators to pull and run them effortlessly.

In this guide, I’ll walk you through building, tagging, pushing, and running a Docker image.

To keep things simple, we’ll create a minimal test image.

Writing a simple Dockerfile

A Dockerfile defines how to build your image. It contains a series of instructions that tell Docker how to construct the image layer by layer.

Let’s create a minimal one:

# Use an official lightweight image
FROM alpine:latest

# Install a simple utility
RUN apk add --no-cache figlet

# Set the default command
CMD ["/usr/bin/figlet", "Docker is Fun!"]

• FROM alpine:latest – This sets the base image to Alpine Linux, a minimal and lightweight distribution.
• RUN apk add --no-cache figlet – Installs the figlet package using Alpine's package manager (apk), with the --no-cache option to keep the image clean.
• CMD ["/usr/bin/figlet", "Docker is Fun!"] – Specifies the default command that will run when a container is started.

Save this file as Dockerfile in an empty directory.

Building the docker image

To build the image, navigate to the directory containing the Dockerfile and run:

docker build -t <cool-image-name> .

• docker build – The command to build an image.
• -t cool-image-name – The -t flag assigns a tag (cool-image-name) to the image, making it easier to reference later.
• . – The dot tells Docker to look for the Dockerfile in the current directory.

Once completed, list your images to confirm:

docker images

Running the docker image

To run the container and see the output:

docker run <cool-image-name>

You should see an ASCII text saying, “Docker is fun!”

Tagging the Image

Before pushing to a registry, we need to tag the image with our Docker Hub username:

docker tag <cool-image-name> your-dockerhub-username/cool-image-name:latest

• docker tag – Creates an alias for the image.
• your-dockerhub-username/cool-image-name:latest – This follows the format username/repository-name:tag. The latest tag is used as a default version identifier.

List images again to see the updated tag:

docker images

Pushing to Docker Hub

First, log in to Docker Hub:

docker login

You will be prompted to enter your Docker Hub username and password.

Once authenticated, you can push the image:

docker push your-dockerhub-username/cool-image-name:latest

And that’s it! Your image is now live on Docker Hub.

Anyone can pull and run it with:

docker pull your-dockerhub-username/cool-image-name:latest
docker run your-dockerhub-username/cool-image-name

Feels great, doesn’t it?

Alternatives to Docker Hub

Docker Hub is not the only place to store images. Here are some alternatives:

• GitHub Container Registry (GHCR.io) – If you already use GitHub, this is a great option as it integrates with GitHub Actions and repositories seamlessly.
• Google Container Registry (GCR.io) – Ideal for those using Google Cloud services. It allows private and public image hosting with tight integration into GCP workloads.
• Amazon Elastic Container Registry (ECR) – Part of AWS, ECR is an excellent option for those using AWS infrastructure, providing strong security and IAM-based authentication.

Self-hosted Docker Registry

If you need complete control over your images, you can set up your own registry by running:

docker run -d -p 5000:5000 --name registry registry:2

This starts a private registry on port 5000, allowing you to store and retrieve images without relying on external providers.

You can read more about this in docker's official documentation to host your own docker registry.

Final thoughts

Building and pushing Docker images has completely streamlined how I distribute my projects.

What once felt like a tedious process is now as simple as writing a Dockerfile, tagging an image, and running a single push command.

No more manual file transfers or complex setup steps, it’s all automated and ready to be pulled anywhere.

However, Docker Hub's free tier limits private repositories to just one. For personal projects, that’s a bit restrictive, which is why I’m more inclined toward self-hosting my own Docker registry.

It gives me complete control, avoids limitations, and ensures I don’t have to worry about third-party policies.

What about you? Which container registry do you use for your projects? Have you considered self-hosting your own registry? Drop your thoughts in the comments.

Que: How do I go to the root directory in Linux command line?

The simple answer is, you type the cd command like this:

cd /

That will put you in the root directory, which is the starting point for the Linux directory structure.

If you want to go to the /root directory (i.e. home directory of the root user), you'll have to use:

cd /root

I know that a new Linux users can be confused with the notation of root directory (/) and the /root directory.

Understand the difference between / and /root

New Linux users often confuse two important concepts: the root directory (/) and the root user's home directory (/root). They sound similar but serve different purposes:

The root directory (/):

• This is the primary directory that contains all other directories and files on your Linux system
• It's the starting point of the filesystem hierarchy
• All paths in Linux begin from this location
• You can access it using cd / regardless of your current location

The root user's home directory (/root):

• This is the home directory for the root user (the superuser with all the access)
• It's located at /root (a directory inside the root directory)
• Regular users may or may not have permission to access this directory

Learn more about the Linux directory structure in the GNU/Linux filesystem documentation.

Understanding / as root directory vs directory separator

The forward slash (/) in Linux serves dual purposes, which can be confusing for newcomers:

As the root directory:

• When used alone or at the beginning of a path, it refers to the root directory
• Example: cd / or cd /home/user

As a directory separator:

• When used between directory names, it separates different levels in the path
• Example: In /home/user/Documents, the slashes separate the directories

This dual usage is important to understand for proper navigation. When you see a path like /var/log/syslog:

• The first / indicates we're starting from the root directory
• The subsequent / characters are separators between directories

# Go to a directory using an absolute path (starting from root)
cd /var/log

# Go to a directory using a relative path (from current location)
cd Documents/Projects

💡Use special navigation shortcuts

Linux provides handy shortcuts for directory navigation:

cd /         # Go to root directory
cd ~         # Go to your home directory
cd -         # Go to previous directory
cd ..        # Go up one level
cd ../..     # Go up two levels

These shortcuts save time and make navigation more efficient.

Conclusion

Understanding how to navigate to and from the root directory is fundamental to working effectively in Linux. The root directory (/) serves as the foundation of your filesystem, distinct from the root user's home directory (/root).

By mastering the concepts of absolute vs relative paths and understanding the dual role of the forward slash, you'll be well-equipped to navigate your Linux system confidently.

Kubernetes is a powerful tool for managing containerized applications, and one of its key features is the ability to run specific workloads across your cluster. One such workload is the DaemonSet, a Kubernetes API object designed to ensure that a copy of a Pod runs on every Node in your cluster.

In this article, we’ll explore what DaemonSets are, how they work, and when to use them.

What is a Kubernetes DaemonSet?

A DaemonSet is a Kubernetes object that ensures a specific Pod runs on every Node in your cluster. When new Nodes are added, the DaemonSet automatically schedules the Pod on them. Similarly, when Nodes are removed, the Pods are cleaned up. This makes DaemonSets ideal for running background services that need to be present on every Node, such as monitoring agents, log collectors, or backup tools.

Key Features of DaemonSets:

• Automatic Pod Scheduling: DaemonSets ensure that a Pod runs on every Node, even as Nodes are added or removed.
• Tolerations: DaemonSets can schedule Pods on Nodes with resource constraints or other restrictions that would normally prevent scheduling.
• Node-Specific Customization: You can configure DaemonSets to run Pods only on specific Nodes using labels and selectors.

When Should You Use a DaemonSet?

DaemonSets are particularly useful for workloads that need to run on every Node in your cluster. Here are some common use cases:

• Node Monitoring Agents: Tools like Prometheus Node Exporter or Datadog agents need to run on every Node to collect metrics.
• Log Collection: Services like Fluentd or Logstash can be deployed as DaemonSets to collect logs from each Node.
• Backup Tools: Backup agents that need to interact with Node-level data can be deployed as DaemonSets to ensure all Nodes are covered.
• Network Plugins: Tools like Calico or Weave Net that provide networking functionality often run as DaemonSets to ensure they’re present on every Node.

Unlike ReplicaSets or Deployments, which schedule Pods based on resource availability, DaemonSets are tied to the number of Nodes in your cluster.

Example: Deploying a DaemonSet

Let’s walk through a simple example of deploying a DaemonSet in your Kubernetes cluster. For this tutorial, we’ll use Filebeat, a lightweight log shipper that collects logs and forwards them to Elasticsearch or Logstash.

You can use Minikube to create a local cluster with three Nodes:

minikube start --nodes=3

Step 1: Create a DaemonSet Manifest

Here’s a basic DaemonSet manifest for Filebeat:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: filebeat
spec:
  selector:
    matchLabels:
      name: filebeat
  template:
    metadata:
      labels:
        name: filebeat
    spec:
      containers:
        - name: filebeat
          image: docker.elastic.co/beats/filebeat:8.10.0
          volumeMounts:
            - name: varlog
              mountPath: /var/log
            - name: varlibdockercontainers
              mountPath: /var/lib/docker/containers
              readOnly: true
      volumes:
        - name: varlog
          hostPath:
            path: /var/log
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers

Step 2: Apply the Manifest

Save the manifest to a file filebeat.yaml and apply it to your cluster:

kubectl apply -f filebeat.yaml

Step 3: Verify the DaemonSet

Check the status of the DaemonSet and the Pods it created:

kubectl get daemonsets

Output:

NAME       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
filebeat   3         3         3       3            3           <none>          10s

For detailed information, run:

kubectl get pods -o wide

Output:

NAME             READY   STATUS    RESTARTS   AGE   IP           NODE
filebeat-abc12   1/1     Running   0          30s   10.244.1.2   minikube-m02
filebeat-def34   1/1     Running   0          30s   10.244.2.2   minikube-m03
filebeat-ghi56   1/1     Running   0          30s   10.244.0.3   minikube

Scoping DaemonSets to Specific Nodes

Sometimes, you may want to run DaemonSet Pods only on specific Nodes. You can achieve this using nodeSelectors or affinity rules. For example, to run Filebeat only on Nodes labeled with log-collection-enabled=true, update the DaemonSet manifest:

spec:
  template:
    spec:
      nodeSelector:
        log-collection-enabled: "true"

Then, label the desired Node:

kubectl label node <node-name> log-collection-enabled=true

Apply the updated manifest, and the DaemonSet will only schedule Pods on the labeled Node.

kubectl apply -f filebeat.yaml

Check the DaemonSet status:

kubectl get daemonsets

Output:

NAME       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                 AGE
filebeat   1         1         1       1            1           log-collection-enabled=true   1m

View the Pod list to confirm the Pod is running on the labeled Node:

kubectl get pods -o wide

Output:

NAME             READY   STATUS    RESTARTS   AGE   IP           NODE
filebeat-abc12   1/1     Running   0          2m    10.244.1.2   minikube-m02

Scaling a DaemonSet

DaemonSets are automatically scaled based on the number of Nodes in your cluster. To scale a DaemonSet:

• Add Nodes: New Nodes will automatically run the DaemonSet Pods.
• Remove Nodes: Pods on removed Nodes will be cleaned up.

If you need to temporarily scale a DaemonSet to 0 (e.g., for maintenance), you can patch the DaemonSet with a dummy nodeSelector:

kubectl patch daemonset <daemonset-name> -p '{"spec": {"nodeSelector": {"dummy": "true"}}}'

To scale it back up, remove the dummy selector.

kubectl patch daemonset filebeat -p '{"spec": {"template": {"spec": {"nodeSelector": {}}}}}'

DaemonSet Best Practices

Use DaemonSets for Node-Specific Workloads: Only use DaemonSets when your Pods need to run on every Node or a subset of Nodes.

• Set Restart Policies Correctly: Ensure Pods have a restartPolicy of Always to ensure they restart with the Node.
• Avoid Manual Pod Management: Don’t manually edit or delete DaemonSet Pods, as this can lead to orphaned Pods.
• Leverage Rollbacks: Use Kubernetes’ rollback feature to revert DaemonSet changes quickly if something goes wrong.

Conclusion

Whether you’re collecting logs with Filebeat, monitoring Nodes with Prometheus, or managing backups, DaemonSets provide a reliable and scalable solution. By understanding how to create, configure, and manage DaemonSets, you can ensure that your Node-level workloads are always running where they’re needed most.

You know that moment when you dive into a project, thinking, "This should be easy," and then hours later, you're buried under obscure errors, outdated forum posts, and conflicting advice?

Yeah, that was me while trying to package my Python app into a .deb file.

It all started with my attempt to revive an old project, which some of our long-time readers might remember - Compress PDF.

Since I’ve been learning Python these days, I thought, why not customize the UI, tweak the logic, and give it a fresh start?

The python app was working great when running inside a virtual environment but I was more interested in shipping this app as a .deb binary, making installation as simple as dpkg -i app.deb.

Every tutorial I read online, covered bits and pieces, but none walked me through the entire process. So here I am, documenting my experience while packaging my script into a .deb file.

Choosing the right packaging tool

For turning a Python script into an executable, I am using PyInstaller. Initially, I tried using py2deb, a tool specifically meant for creating .deb packages.

Bad idea. Turns out, py2deb hasn’t been maintained in years and doesn’t support newer Python versions.

PyInstaller takes a Python script and bundles it along with its dependencies into a single standalone executable. This means users don’t need to install Python separately, it just works out of the box.

Step 1: Install PyInstaller

First, make sure you have PyInstaller installed. If not, install it using pip:

pip install pyinstaller

Check if it's installed correctly:

pyinstaller --version

Step 2: Create the .deb package structure

To keep things clean and structured, .deb packages follows a specific folder structure.

compressor/
├── pdf-compressor/                
│   ├── DEBIAN/                     
│   │   ├── control
│   │   ├── postinst
│   ├── usr/                         
│   │   ├── bin/
│   │   ├── share/
│   │   │   ├── applications/
│   │   │   ├── icons/
│   │   │   ├── pdf-compressor/

Let’s create it:

mkdir -p pdf-compressor/DEBIAN
mkdir -p pdf-compressor/usr/bin
mkdir -p pdf-compressor/usr/share/applications
mkdir -p pdf-compressor/usr/share/icons/
mkdir -p pdf-compressor/usr/share/pdf-compressor/

What each directory is for?

• usr/bin/: Stores the executable file.
• usr/share/applications/: Contains the .desktop file (so the app appears in the system menu).
• usr/share/icons/: Stores the app icon.
• DEBIAN/: Contains metadata like package info and dependencies.

Optional: Packaging dependencies

Before packaging the app, I wanted to ensure it loads assets and dependencies correctly whether it's run as a script or a standalone binary.

Initially, I ran into two major problems:

1. The in-app logo wasn’t displaying properly because asset paths were incorrect when running as a packaged executable.
2. Dependency errors occurred when running the app as an executable.

To keep everything self-contained and avoid conflicts with system packages, I created a virtual environment inside: pdf-compressor/usr/share/pdf-compressor

python3 -m venv venv
source venv/bin/activate

Then, I installed all the dependencies inside it:

pip install -r requirements.txt
deactivate

This ensures that dependencies are bundled properly and won’t interfere with system packages.

Now to ensure that the app correctly loads assets and dependencies, I modified the script as follows:

import sys
import os

# Ensure the virtual environment is used
venv_path = "/usr/share/pdf-compressor/venv"
if os.path.exists(venv_path):
    sys.path.insert(0, os.path.join(venv_path, "lib", "python3.10", "site-packages"))

# Detect if running as a standalone binary
if getattr(sys, 'frozen', False):
    app_dir = sys._MEIPASS  # PyInstaller's temp folder
else:
    app_dir = os.path.dirname(os.path.abspath(__file__))

# Set correct paths for assets
icon_path = os.path.join(app_dir, "assets", "icon.png")
logo_path = os.path.join(app_dir, "assets", "itsfoss-logo.webp")
pdf_icon_path = os.path.join(app_dir, "assets", "pdf.png")

print("PDF Compressor is running...")

What’s happening here?

• sys._MEIPASS → When the app is packaged with PyInstaller, assets are extracted to a temporary folder. This ensures they are accessible.
• Virtual environment path (/usr/share/pdf-compressor/venv) → If it exists, it is added to sys.path, so installed dependencies can be found.
• Assets paths → Dynamically assigned so they work in both script and standalone modes.

After making these changes, my issue was mostly resolved.

Step 3: Compiling python script into executable binary

Now comes the exciting part, turning the Python script into a standalone executable. Navigate to the root directory where the main Python script is located. Then run:

pyinstaller --name=pdf-compressor --onefile --windowed --add-data "assets:assets" pdf-compressor.py

• --onefile: Packages everything into a single executable file
• --windowed: Hides the terminal (useful for GUI apps)
• --name=compress-pdf: Sets the output filename
• --add-data "assets:assets" → Ensures images/icons are included.

After this, PyInstaller will create a dist/ , inside, you'll find compress-pdf . This is the standalone app!

Try running it:

./dist/pdf-compressor

If everything works as expected, you’re ready to package it into a .deb file.

Step 4: Move the executable to the correct location

Now, move the standalone executable into the bin directory:

mv dist/compress-pdf pdf-compressor/usr/bin/pdf-compressor

Step 5: Add an application icon

I don't know about you but to me, an app without an icon or just generic gear icons feels incomplete. Icon gives the vibe to your app.

Let’s place the assets directory which contains the icon and logo files inside the right directory:

cp assets/ pdf-compressor/usr/share/pdf-compressor

Step 6: Create a desktop file

To make the app appear in the system menu, we need a .desktop file. Open a new file:

nano pdf-compressor/usr/share/applications/pdf-compressor.desktop

Paste this content:

[Desktop Entry]
Name=PDF Compressor
Comment=Compress PDF files easily
Exec=/usr/bin/pdf-compressor
Icon=/usr/share/icons/pdf-compressor.png
Terminal=false
Type=Application
Categories=Utility

• Exec → Path to the executable.
• Icon → App icon location.
• Terminal=false → Ensures it runs as a GUI application.

Save and exit (CTRL+X, then Y, then Enter).

Step 7: Create the control file

At the heart of every .deb package is a metadata file called control.

This file is what tells the Debian package manager (dpkg) what the package is, who maintains it, what dependencies it has, and a brief description of what it does.

That’s why defining them here ensures a smooth experience for users.

Inside the DEBIAN/ directory, create a control file:

nano pdf-compressor/DEBIAN/control

then I added the following content in it:

Package: pdf-compressor
Version: 1.0
Section: utility
Priority: optional
Architecture: amd64
Depends: python3, ghostscript
Recommends: python3-pip, python3-venv
Maintainer: Your Name <your@email.com>
Description: A simple PDF compression tool.
 Compress PDF files easily using Ghostscript.

Step 8: Create the postinst script

The post-installation (postinst) script as the name suggests is executed after the package is installed. It ensures all dependencies are correctly set up.

nano pdf-compressor/DEBIAN/postinst

Add this content:

#!/bin/bash
set -e  # Exit if any command fails

echo "Setting up PDF Compressor..."
chmod +x /usr/bin/pdf-compressor

# Install dependencies inside a virtual environment
python3 -m venv /usr/share/pdf-compressor/venv
source /usr/share/pdf-compressor/venv/bin/activate
pip install --no-cache-dir pyqt6 humanize

echo "Installation complete!"
update-desktop-database

What’s happening here?

• set -e → Ensures the script stops on failure.
• Creates a virtual environment → This allows dependencies to be installed in an isolated way.
• chmod +x /usr/bin/pdf-compressor → Ensures the binary is executable.
• update-desktop-database → Updates the system’s application database.

Setting up the correct permission for postinst is important:

chmod 755 pdf-compressor/DEBIAN/postinst

Step 9: Build & Install the deb package

After all the hard work, it's finally time to bring everything together. To build the package, we’ll use dpkg-deb --build, a built-in Debian packaging tool.

This command takes our structured pdf-compressor directory and turns it into a .deb package that can be installed on any Debian-based system.

dpkg-deb --build pdf-compressor

If everything goes well, you should see output like:

dpkg-deb: building package 'pdf-compressor' in 'pdf-compressor.deb'.

Now, let’s install it and see our application in action!

sudo dpkg -i pdf-compressor.deb

This installs pdf-compressor onto your system just like any other Debian package. To verify, you can either launch it from the Applications menu or directly via terminal:

pdf-compressor

Final thoughts

Packaging a Python application isn’t as straightforward as I initially thought. During my research, I couldn't find any solid guide that walks you through the entire process from start to finish.

So, I had to experiment, fail, and learn, and that’s exactly what I’ve shared in this guide. Looking back, I realize that a lot of what I struggled with could have been simplified had I known better. But that’s what learning is all about, right?

I believe that this write-up will serve as a good starting point for new Python developers like me who are still struggling to package their projects.

That said, I know this isn’t the only way to package Python applications, there are probably better and more efficient approaches out there. So, I’d love to hear from you!

Also, if you found this guide helpful, be sure to check out our PDF Compressor project on GitHub. Your feedback, contributions, and suggestions are always welcome!

Happy coding! 😊

The whereis command helps users locate the binary, source, and manual page files for a given command. And in this tutorial, I will walk you through practical examples to help you understand how to use whereis command.

Unlike other search commands like find that scan the entire file system, whereis searches predefined directories, making it faster and more efficient.

It is particularly useful for system administrators and developers to locate files related to commands without requiring root privileges.

whereis Command Syntax

To use any command to its maximum potential, it is important to know its syntax and that is why I'm starting this tutorial by introducing the syntax of the whereis command:

whereis [OPTIONS] FILE_NAME...

Here,

• OPTIONS: Flags that modify the search behavior.
• FILE_NAME: The name of the file or command to locate.

Now, let's take a look at available options of the whereis command:

1. Locate all files related to a command

To find all files (binary, source, and manual) related to a command, all you have to do is append the command name to the whereis command as shown here:

whereis command

For example, if I want to locate all files related to bash, then I would use the following:

whereis bash

Here,

• /usr/bin/bash: Path to the binary file.
• /usr/share/man/man1/bash.1.gz: Path to the manual page.

2. Search for binary files only

To locate only the binary (executable) file of a command, use the -b flag along with the target command as shown here:

whereis -b command

If I want to search for the binary files for the ls command, then I would use the following:

whereis -b ls

3. Search for the manual page only

To locate only the manual page for a specific command, use the -m flag along with the targeted command as shown here:

whereis -m command

For example, if I want to search for the manual page location for the grep command, then I would use the following:

whereis -m grep

As you can see, it gave me two locations:

• /usr/share/man/man1/grep.1.gz: A manual page which can be accessed through man grep command.
• /usr/share/info/grep.info.gz: An info page that can be accessed through info grep command.

4. Search for source files only

To find only source code files associated with a command, use the -s flag along with the targeted command as shown here:

whereis -s command

For example, if I want to search source files for the gcc, then I would use the following:

whereis -s gcc

My system is fresh and I haven't installed any packages from the source so I was given a blank output.

5. Specify custom directories for searching

To limit your search to specific directories, use options like -B, -S, or -M. For example, if I want to limit my search to the /bin directory for the cp command, then I would use the following command:

whereis -b -B /bin -f cp

Here,

• -b: This flag tells whereis to search only for binary files (executables), ignoring source and manual files.
• -B /bin: The -B flag specifies a custom directory (/bin in this case) where whereis should look for binary files. It also limits the search to the /bin directory instead of searching all default directories.
• -f cp: Without -f, the whereis command would interpret cp as another directory.

6. Identify commands missing certain files (unusual files)

The whereis command can help you find commands that are missing one or more associated files (binary, source, or manual). This is particularly useful for troubleshooting or verifying file completeness.

For example, to search for commands in the /bin directory that is missing manual pages, you first have to change your directory to /bin and then use the -u flag to search for unusual files:

cd /bin
whereis -u -m *

Wrapping Up...

This was a quick tutorial on how you can use the whereis command in Linux including practical examples and syntax. I hope you will find this guide helpful.

If you have any queries or suggestions, leave us a comment.

Kubernetes is a powerful platform designed to manage and automate the deployment, scaling, and operation of containerized applications. In simple terms, it helps you run and manage your software applications in an organized and efficient way.

kubectl is the command-line tool that helps you manage your Kubernetes cluster. It allows you to deploy applications, manage resources, and get information about your applications. Simply put, kubectl is the main tool you use to communicate with Kubernetes and get things done.

In this article, we will explore essential kubectl commands that will make managing your Kubernetes cluster easier and more efficient.

Essential Kubernetes Concepts

Before diving into the commands, let's quickly review some key Kubernetes concepts to ensure a solid understanding.

• Pod: The smallest deployable unit in Kubernetes, containing one or more containers that run together on the same node.
• Node: A physical or virtual machine in the Kubernetes cluster where Pods are deployed.
• Services: An abstraction that defines a set of Pods and provides a stable network endpoint to access them.
• Deployment: A controller that manages the desired state and lifecycle of Pods by creating, updating, and deleting them.
• Namespace: A logical partition in a Kubernetes cluster to isolate and organize resources for different users or teams.

General Command Line Options

This section covers various optional flags and parameters that can be used with different kubectl commands. These options help customize the output format, specify namespaces, filter resources, and more, making it easier to manage and interact with your Kubernetes clusters.

The get command in kubectl is used to retrieve information about Kubernetes resources. It can list various resources such as pods, services, nodes, and more.

To retrieve a list of all the pods in your Kubernetes cluster in JSON format,

kubectl get pods -o json

List all the pods in the current namespace and output their details in YAML format.

kubectl get pods -o yaml

Output the details in plain-text format, including the node name for each pod,

kubectl get pods -o wide

List all the pods in a specific namespace using the -n option:

kubectl get pods -n <namespace_name>

To create a Kubernetes resource from a configuration file, us the command:

kubectl create -f <filename>

To filter logs by a specific label, you can use:

kubectl logs -l <label_filter>

For example, to get logs from all pods labeled app=myapp, you would use:

kubectl logs -l app=myapp

For quick command line help, always use the -h option.

kubectl -h

Create and Delete Kubernetes Resources

In Kubernetes, you can create resources using the kubectl create command, update or apply changes to existing resources with the kubectl apply command, and remove resources with the kubectl delete command. These commands allow you to manage the lifecycle of your Kubernetes resources effectively and efficiently.

The apply and create are two different approaches to create resources in Kubernetes. While the apply follows a declarative approach, create follows an imperative approach.

Learn about these different approaches in our dedicated article.

kubectl apply vs create: What’s the Difference?

Two different approaches for creating resources in Kubernetes cluster. What’s the difference?

Linux HandbookRakesh Jain

To apply a configuration file to a pod, use the command:

 kubectl apply -f <JSON/YAML configuration file>

If you have multiple JSON/YAML configuration files, you can use glob pattern matching here:

 kubectl apply -f '*.json'

To create a new Kubernetes resource using a configuration file,

kubectl create -f <configuration file>

The -f option can receive directory values or configuration file URL to create resource.

kubectl create -f <directory>

OR

kubectl create -f <URL to files>

The delete option is used to delete resources by file names, resources and names, or by resources and label selector.

To delete resources using the type and name specified in the configuration file,

 kubectl delete -f <configuration file>

Cluster Management and Context Commands

Cluster management in Kubernetes refers to the process of querying and managing information about the Kubernetes cluster itself. According to the official documentation, it involves various commands to display endpoint information, view and manage cluster configurations, list API resources and versions, and manage contexts.

The cluster-info command displays the endpoint information about the master and services in the cluster.

kubectl cluster-info

To print the client and server version information for the current context, use:

kubectl version

To display the merged kubeconfig settings,

kubectl config view

To extract and display the names of all users from the kubeconfig file, you can use a jsonpath expression.

kubectl config view -o jsonpath='{.users[*].name}'

Display the current context that kubectl is using,

kubectl config current-context

You can display a list of contexts with the get-context option.

kubectl config get-contexts

To set the default context, use:

kubectl config use-context <context-name>

Print the supported API resources on the server.

kubectl api-resources

It includes core resources like pods, services, and nodes, as well as custom resources defined by users or installed by operators.

You can use the api-versions command to print the supported API versions on the server in the form of "group/version". This command helps you identify which API versions are available and supported by your Kubernetes cluster.

kubectl api-versions

The --all-namespaces option available with the get command can be used to list the requested object(s) across all namespaces. For example, to list all pods existing in all namespaces,

kubectl get pods --all-namespaces

Daemonsets

A DaemonSet in Kubernetes ensures that all (or some) Nodes run a copy of a specified Pod, providing essential node-local facilities like logging, monitoring, or networking services. As nodes are added or removed from the cluster, DaemonSets automatically add or remove Pods accordingly. They are particularly useful for running background tasks on every node and ensuring node-level functionality throughout the cluster.

You can create a new DaemonSet with the command:

kubectl create daemonset <daemonset_name>

To list one or more DaemonSets, use the command:

kubectl get daemonset

The command,

kubectl edit daemonset <daemonset_name>

will open up the specified DaemonSet in the default editor so you can edit and update the definition.

To delete a daemonset,

kubectl delete daemonset <daemonset_name>

You can check the rollout status of a daemonset with the kubectl rollout command:

kubectl rollout status daemonset

The command below provides detailed information about the specified DaemonSet in the given namespace:

kubectl describe ds <daemonset_name> -n <namespace_name>

Deployments

Kubernetes deployments are essential for managing and scaling applications. They ensure that the desired number of application instances are running at all times, making it easy to roll out updates, perform rollbacks, and maintain the overall health of your application by automatically replacing failed instances.

In other words, Deployment allows you to manage updates for Pods and ReplicaSets in a declarative manner. By specifying the desired state in the Deployment configuration, the Deployment Controller adjusts the actual state to match at a controlled pace. You can use Deployments to create new ReplicaSets or replace existing ones while adopting their resources seamlessly. For more details, refer to StatefulSet vs. Deployment.

To list one or more deployments:

kubectl get deployment

To display detailed information about the specified deployment, including its configuration, events, and status,

kubectl describe deployment <deployment_name>

The below command opens the specified deployment configuration in the default editor, allowing you to make changes to its configuration:

kubectl edit deployment <deployment_name>

To create a deployment using kubectl, specify the image to use for the deployment:

kubectl create deployment <deployment_name> --image=<image_name>

You can delete a specified deployment and all of its associated resources, such as Pods and ReplicaSets by using the command:

kubectl delete deployment <deployment_name>

To check the rollout status of the specified deployment and providing information about the progress of the deployment's update process,

kubectl rollout status deployment <deployment_name>

Perform a rolling update in Kubernetes by setting the container image to a new version for a specific deployment.

kubectl set image deployment/<deployment name> <container name>=image:<new image version>

To roll back the specified deployment to the previous revision (undo),

kubectl rollout undo deployment/<deployment name>

The command below will forcefully replace a resource from a configuration file:

kubectl replace --force -f <configuration file>

Retrieving and Filtering Events

In Kubernetes, events are a crucial component for monitoring and diagnosing the state of your cluster. They provide real-time information about changes and actions happening within the system, such as pod creation, scaling operations, errors, and warnings.

Use the command:

kubectl get events

To retrieve and list recent events for all resources in the system, providing valuable information about what has happened in your cluster.

To filter and list only the events of type "Warning," thereby providing insights into any potential issues or warnings in your cluster,

kubectl get events --field-selector type=Warning

You can retrieve and list events sorted by their creation timestamp. This allows you to view events in chronological order.

kubectl get events --sort-by=.metadata.creationTimestamp

To lists events, excluding those related to Pod events,

kubectl get events --field-selector involvedObject.kind!=Pod

This helps you focus on events for other types of resources.

To list events specifically for a node with the given name,

kubectl get events --field-selector involvedObject.kind=Node, involvedObject.name=<node_name>

You can filter events, excluding those that are of the "Normal" type, allowing you to focus on warning and error events that may require attention:

kubectl get events --field-selector type!=Normal

Managing Logs

Logs are essential for understanding the real-time behavior and performance of your applications. They provide a record of activity and outputs generated by containers and pods, which can be invaluable for debugging and monitoring purposes.

To print the logs for the specified pod:

kubectl logs <pod_name>

To print the logs for the specified pod since last hour:

kubectl logs --since=1h <pod_name>

You can read the most recent 50 lines of logs for the specified pod using the --tail option.

kubectl logs --tail=50 <pod_name>

The command below streams and continuously displays the logs of the specified pod, optionally filtered by the specified container:

kubectl logs -f <pod_name> [-c <container_name>]

For example, as per the official documentation,

kubectl logs -f -c ruby web-1

Begin streaming the logs of the ruby container in pod web-1.

To continuously display the logs of the specified pod in real-time,

kubectl logs -f <pod_name>

You can fetch the logs up to the current point in time for a specific container within the specified pod using the command:

kubectl logs -c <container_name> <pod_name>

To save the logs for the specified pod to a file,

kubectl logs <pod_name> > pod.log

To print the logs for the previous instance of the specified pod:

kubectl logs --previous <pod_name>

This is particularly useful for troubleshooting and analyzing logs from a previously failed pod.

Namespaces

In Kubernetes, namespaces are used to divide and organize resources within a cluster, creating separate environments for different teams, projects, or applications. This helps in managing resources, access permissions, and ensuring that each group or application operates independently and securely.

To create a new namespace with the specified name in your Kubernetes cluster:

kubectl create namespace <namespace_name>

To list all namespaces in your Kubernetes cluster, use the command:

kubectl get namespaces

You can get a detailed description of the specified namespace, including its status, resource quotas using the command:

kubectl describe namespace <namespace_name>

To delete the specified namespace along with all the resources contained within it:

kubectl delete namespace <namespace_name>

The command

kubectl edit namespace <namespace_name>

opens the default editor on your machine with the configuration of the specified namespace, allowing you to make changes directly.

To display resource usage (CPU and memory) for all pods within a specific namespace, you can use the following command:

kubectl top pods --namespace=<namespace_name>

Nodes

In Kubernetes, nodes are the fundamental building blocks of the cluster, serving as the physical or virtual machines that run your applications and services.

To update the taints on one or more nodes,

kubectl taint node <node_name>

List all nodes in your Kubernetes cluster:

kubectl get node

Remove a specific node from your Kubernetes cluster,

kubectl delete node <node_name>

Display resource usage (CPU and memory) for all nodes in your Kubernetes cluster:

kubectl top nodes

List all pods running on a node with a specific name:

kubectl get pods -o wide | grep <node_name>

Add or update annotations on a specific node:

kubectl annotate node <node_name> <key>=<value>

Mark a node as unschedulable (no new pods will be scheduled on the specified node).

kubectl cordon node <node_name>

Mark a previously cordoned (unschedulable) node as schedulable again:

kubectl uncordon node <node_name>

Safely evict all pods from the specified node in preparation for maintenance or decommissioning:

kubectl drain node <node_name>

Add or update labels on a specific node in your Kubernetes cluster:

kubectl label node <node_name> <key>=<value>

Pods

A pod is the smallest and simplest unit in the Kubernetes object model that you can create or deploy. A pod represents a single instance of a running process in your cluster and can contain one or more containers. These containers share the same network namespace, storage volumes, and lifecycle, allowing them to communicate with each other easily and share resources.

Pods are designed to host tightly coupled application components and provide a higher level of abstraction for deploying, scaling, and managing applications in a Kubernetes environment. Each pod is scheduled on a node, where the containers within it are run and managed together as a single, cohesive unit.

List all pods in your Kubernetes cluster:

kubectl get pods

List all pods in your Kubernetes cluster and sort them by the restart count of the first container in each pod:

kubectl get pods --sort-by='.status.containerStatuses[0].restartCount'

List all pods in your Kubernetes cluster that are currently in the "Running" phase:

kubectl get pods --field-selector=status.phase=Running

Delete a specific pod from your Kubernetes cluster:

kubectl delete pod <pod_name>

Display detailed information about a specific pod in your Kubernetes cluster:

kubectl describe pod <pod_name>

Create a pod using the specifications provided in a YAML file:

kubectl create -f pod.yaml

OR

kubectl apply -f pod.yaml

To execute a command in a specific container within a pod in your Kubernetes cluster:

kubectl exec <pod_name> -c <container_name> <command>

Start an interactive shell session in a container within a specified pod:

# For Single Container Pods
kubectl exec -it <pod_name> -- /bin/sh

# For Multi-container pods,
kubectl exec -it <pod_name> -c <container_name> -- /bin/sh

Display resource (CPU and memory) usage statistics for all pods in your Kubernetes cluster:

kubectl top pods

Add or update annotations on a specific pod:

kubectl annotate pod <pod_name> <key>=<value>

To add or update the label of a pod:

kubectl label pod <pod_name> new-label=<label name>

List all pods in your Kubernetes cluster and display their labels:

kubectl get pods --show-labels

Forward one or more local ports to a pod in your Kubernetes cluster, allowing you to access the pod's services from your local machine:

kubectl port-forward <pod_name> <port_number_to_listen_on>:<port_number_to_forward_to>

Replication Controllers

Replication Controller (RC) ensures that a specified number of pod replicas are running at any given time. If any pod fails or is deleted, the Replication Controller automatically creates a replacement. This self-healing mechanism enables high availability and scalability of applications.

To list all Replication Controllers in your Kubernetes cluster

kubectl get rc

List all Replication Controllers within a specific namespace:

kubectl get rc --namespace=”<namespace_name>”

ReplicaSets

ReplicaSet is a higher-level concept that ensures a specified number of pod replicas are running at any given time. It functions similarly to a Replication Controller but offers more powerful and flexible capabilities.

List all ReplicaSets in your Kubernetes cluster.

kubectl get replicasets

To display detailed information about a specific ReplicaSet:

kubectl describe replicasets <replicaset_name>

Scale the number of replicas for a specific resource, such as a Deployment, ReplicaSet, or ReplicationController, in your Kubernetes cluster.

kubectl scale --replicas=<number_of_replicas> <resource_type>/<resource_name>

Secrets

Secrets are used to store and manage sensitive information such as passwords, tokens, and keys.

Unlike regular configuration files, Secrets help ensure that confidential data is securely handled and kept separate from application code.

Secrets can be created, managed, and accessed within the Kubernetes environment, providing a way to distribute and use sensitive data without exposing it in plain text.

To create a Secret,

kubectl create secret (docker-registry | generic | tls)

List all Secrets in your Kubernetes cluster:

kubectl get secrets

Display detailed information about a specific Secret:

kubectl describe secret <secret_name>

Delete a specific Secret from your Kubernetes cluster:

kubectl delete secret <secret_name>

Services

Services act as stable network endpoints for a group of pods, allowing seamless communication within the cluster. They provide a consistent way to access pods, even as they are dynamically created, deleted, or moved.

By using a Service, you ensure that your applications can reliably find and interact with each other, regardless of the underlying pod changes.

Services can also distribute traffic across multiple pods, providing load balancing and improving the resilience of your applications.

To list all Services in your Kubernetes cluster:

kubectl get services

To display detailed information about a specific Service:

kubectl describe service <service_name>

Create a Service that exposes a deployment:

kubectl expose deployment <deployment_name> --port=<port> --target-port=<target_port> --type=<type>

Edit the configuration of a specific Service:

kubectl edit service <service_name>

Service Accounts

Service Accounts provide an identity for processes running within your cluster, enabling them to interact with the Kubernetes API and other resources. By assigning specific permissions and roles to Service Accounts, you can control access and limit the actions that pods and applications can perform, enhancing the security and management of your cluster.

Service Accounts are essential for managing authentication and authorization, ensuring that each component operates with the appropriate level of access and adheres to the principle of least privilege.

To list all Service Accounts in your Kubernetes cluster:

kubectl get serviceaccounts

Display detailed information about a specific Service Account:

kubectl describe serviceaccount <serviceaccount_name>

Next is replacing a service account. Before replacing, you need to export the existing Service Account definition to a YAML file.

kubectl get serviceaccount <serviceaccount_name> -o yaml > serviceaccount.yaml

Once you made changes to the YAML file, replace the existing Service Account with the modified one:

kubectl replace -f serviceaccount.yaml

Delete a specific Service Account from your Kubernetes cluster:

kubectl delete serviceaccount <service_account_name>

StatefulSet

StatefulSet is a specialized workload controller designed for managing stateful applications. Unlike Deployments, which are suitable for stateless applications, StatefulSets provide guarantees about the ordering and uniqueness of pods.

Each pod in a StatefulSet is assigned a unique, stable identity and is created in a specific order. This ensures consistency and reliability for applications that require persistent storage, such as databases or distributed systems.

StatefulSets also facilitate the management of pod scaling, updates, and rollbacks while preserving the application's state and data.

To list all StatefulSets in your Kubernetes cluster:

kubectl get statefulsets

To delete a specific StatefulSet from your Kubernetes cluster without deleting the associated pods:

kubectl delete statefulset <stateful_set_name> --cascade=false

💬 Hope you like this quick overview of the kubectl commands. Please let me know if you have any questions or suggestions.

What career opportunities are available for someone starting with Linux? I am talking about entering this field and that's why I left out roles like SRE from this list. I would appreciate your feedback on it if you are already working in the IT industry. Let's help out our juniors.

What Kind of Job Can You Get if You Learn Linux?

While there are tons of job roles created around Linux, here are the ones that you can choose for an entry level career.

Linux HandbookAbhishek Prakash

Here are the other highlights of this edition of LHB Linux Digest:

• Zed IDE
• Essential Docker commands
• Self hosted project management tool
• And more tools, tips and memes for you
• This edition of LHB Linux Digest newsletter is supported by PikaPods.

📖 Linux Tips and Tutorials

• Finding the biggest files and directories so that you can see what's taking all that disk space
• List only directories, not files (sometimes, you need to do that)
• Checking Crontab logs so that you can troubleshoot

Learn to increase (or perhaps decrease) swap on Ubuntu Linux. This should work on other distros too if they use swap file instead of swap partition.

How to Increase Swap Size on Ubuntu Linux

In this quick tip, you’ll learn to increase the swap size on Ubuntu and other Linux distributions.

Linux HandbookAbhishek Prakash

Linux is the foundation of many IT systems, from servers to cloud platforms. Mastering Linux and related tools like Docker, Kubernetes, and Ansible can unlock career opportunities in IT, system administration, networking, and DevOps.

I mean, that's one of the reasons why many people use Linux.

The next question is, what kinds of job roles can you get if you want to begin a career with Linux?

Let me share the job roles, required skills, certifications, and resources to help you transition into a Linux-based career.

1. IT Technician

IT Technicians are responsible for maintaining computer systems, troubleshooting hardware/software issues, and supporting organizational IT needs.

They ensure smooth daily operations by resolving technical problems efficiently. So if you are a beginner and just want to get started in IT field, IT technician is one of the most basic yet important roles.

Responsibilities:

• Install and configure operating systems, software, and hardware.
• Troubleshoot system errors and repair equipment.
• Provide user support for technical issues.
• Monitor network performance and maintain security protocols.

Skills Required:

• Basic Linux knowledge (file systems, permissions).
• Networking fundamentals (TCP/IP, DNS).
• Familiarity with common operating systems like Windows and MacOS.

Certifications:

• CompTIA Linux+ (XK0-005): Validates foundational Linux skills such as system management, security, scripting, and troubleshooting. Recommended for entry-level roles.
• CompTIA A+: Focuses on hardware/software troubleshooting and is ideal for beginners.

2. System Administrator

System administrators manage servers, networks, and IT infrastructure and on a personal level, this is my favourite role.

Being a System admin, you are supposed to ensure system reliability, security, and efficiency by configuring software/hardware and automating repetitive tasks.

Responsibilities:

• Install and manage operating systems (e.g., Linux).
• Set up user accounts and permissions.
• Monitor system performance and troubleshoot outages.
• Implement security measures like firewalls.

Skills Required:

• Proficiency in Linux commands and shell scripting.
• Experience with configuration management tools (e.g., Ansible).
• Knowledge of virtualization platforms (e.g., VMware).

Certifications:

• Red Hat Certified System Administrator (RHCSA): Focuses on core Linux administration tasks such as managing users, storage configuration, basic container management, and security.
• LPIC-1: Linux Administrator: Covers fundamental skills like package management and networking.

3. Network Engineer

Being a network engineer, you are responsible for designing, implementing, and maintaining an organization's network infrastructure. In simple terms, you will be called first if there is any network-related problem ranging from unstable networks to misconfigured networks.

Responsibilities:

• Configure routers, switches, firewalls, and VPNs.
• Monitor network performance for reliability.
• Implement security measures to protect data.
• Document network configurations.

Skills Required:

• Advanced knowledge of Linux networking (firewalls, IP routing).
• Familiarity with protocols like BGP/OSPF.
• Scripting for automation (Python or Bash).

Certifications:

• Cisco Certified Network Associate (CCNA): Covers networking fundamentals such as IP connectivity, network access, automation, and programmability. It’s an entry-level certification for networking professionals.
• CompTIA Network+: Focuses on troubleshooting network issues and implementing secure networks.

4. DevOps Engineer

DevOps Engineers bridge development and operations teams to streamline software delivery. This is more of an advanced role where you will be focusing on automation tools like Docker for containerization and Kubernetes for orchestration.

Responsibilities:

• Automate CI/CD pipelines using tools like Jenkins.
• Deploy containerized applications using Docker.
• Manage Kubernetes clusters for scalability.
• Optimize cloud-based infrastructure (AWS/Azure).

Skills Required:

• Strong command-line skills in Linux.
• Proficiency in DevOps tools (e.g., Terraform).
• Understanding of cloud platforms.

Certifications:

• Certified Kubernetes Administrator (CKA): Validates expertise in managing Kubernetes clusters by covering topics like installation/configuration, networking, storage management, and troubleshooting.
• AWS Certified DevOps Engineer – Professional: Focuses on automating AWS deployments using DevOps practices.

Linux for DevOps: Essential Knowledge for Cloud Engineers

Learn the essential concepts, command-line operations, and system administration tasks that form the backbone of Linux in the DevOps world.

Linux HandbookAbdullah Tarek

Recommended certifications:

Skills required across roles

Here, I have listed the skills that are required for all the 4 roles listed above:

Core skills:

1. Command-line proficiency: Navigating file systems and managing processes.
2. Networking basics: Understanding DNS, SSH, and firewalls.
3. Scripting: Automating tasks using Bash or Python.

Advance skills:

1. Configuration management: Tools like Ansible or Puppet.
2. Containerization: Docker for packaging applications.
3. Orchestration: Kubernetes for managing containers at scale.

Free resources to Learn Linux

For beginners:

1. Bash Scripting for Beginners: Our in-house free course for command-line basics.
2. Linux Foundation Free Courses: Covers Linux basics like command-line usage.
3. LabEx: Offers hands-on labs for practising Linux commands.
4. Linux for DevOps: Essential Linux knowledge for cloud and DevOps engineers.
5. Learn Docker: Our in-house effort to provide basic Docker tutorials for free.

For advanced topics:

1. KodeKloud: Interactive courses on Docker/Kubernetes with real-world scenarios.
2. Coursera: Free trials for courses like "Linux Server Management."
3. RHCE Ansible EX294 Exam Preparation Course: Our editorial effort is to provide a free Ansible course covering basic to advanced Ansible.

Conclusion

I would recommend you start by mastering the basics of Linux commands before you dive into specialized tools like Docker or Kubernetes.

We have a complete course on Linux command line fundamentals. No matter which role you are preparing for, you cannot ignore the basics.

Linux for DevOps: Essential Knowledge for Cloud Engineers

Learn the essential concepts, command-line operations, and system administration tasks that form the backbone of Linux in the DevOps world.

Linux HandbookAbdullah Tarek

Use free resources to build your knowledge base and validate your skills through certifications tailored to your career goals. With consistent learning and hands-on practice, you can secure a really good role in the tech industry!

If you've ever wanted a secure way to access your home network remotely, whether for SSH access, private browsing, or simply keeping your data encrypted on public Wi-Fi, self-hosting a VPN is the way to go.

While commercial VPN services exist, hosting your own gives you complete control and ensures your data isn't being logged by a third party.

I’m using a Linode VPS for this guide, but if you're running this on your home network, make sure your router allows port forwarding.

Customer Referral Landing Page - $100

Cut Your Cloud Bills in Half Deploy more with Linux virtual machines, global infrastructure, and simple pricing. No surprise bills, no lock-in, and the

Linode

What is PiVPN?

PiVPN is a lightweight, open-source project designed to simplify setting up a VPN server on a Raspberry Pi or any Debian-based system.

It supports WireGuard and OpenVPN, allowing you to create a secure, private tunnel to your home network or VPS.

The best part? PiVPN takes care of the heavy lifting with a one-command installer and built-in security settings.

With PiVPN, you can:

• Securely access your home network from anywhere
• Encrypt your internet traffic on untrusted networks (coffee shops, airports, etc.)
• Avoid ISP snooping by routing traffic through a VPS
• Run it alongside Pi-hole for an ad-free, secure browsing experience

PiVPN makes self-hosting a VPN accessible, even if you’re not a networking expert. Now, let’s get started with setting it up.

Installing PiVPN

Now that we've handled the prerequisites, it's time to install PiVPN. The installation process is incredibly simple.

Open a terminal on your server and run:

curl -L https://install.pivpn.io | bash

This command will launch an interactive installer that will guide you through the setup.

1. Assign a Static IP Address

You'll be prompted to ensure your server has a static IP. If your local IP changes, your port forwarding rules will break, rendering the VPN useless.

If running this on a VPS, the external IP is usually static.

2. Choose a User

Select the user that PiVPN should be installed under. If this is a dedicated server for VPN use, the default user is fine.

3. Choose a VPN Type: WireGuard or OpenVPN

PiVPN supports both WireGuard and OpenVPN. For this guide, I'll go with WireGuard, but you can choose OpenVPN if needed.

4. Select the VPN Port

You'll need to specify the port, for WireGuard, this defaults to 51820 (this is the same port which you need to forward on your router)

5. Choose a DNS Provider

PiVPN will ask which DNS provider to use. If you have a self-hosted DNS, select "Custom" and enter the IP. Otherwise, pick from options like Google, Cloudflare, or OpenDNS.

6. Public IP vs. Dynamic DNS

If you have a static public IP, select that option. If your ISP gives you a dynamic IP, use a Dynamic DNS (DDNS) service to map a hostname to your changing IP.

7. Enable Unattended Upgrades

For security, it's a good idea to enable automatic updates. VPN servers are a crucial entry point to your network, so keeping them updated reduces vulnerabilities.

After these steps, PiVPN will complete the installation.

Creating VPN profiles

Now that the VPN is up and running, we need to create client profiles for devices that will connect to it.

Run the following command:

pivpn add

You'll be asked to enter a name for the client profile.

Once created, the profile file will be stored in:

/home/<user>/configs/

Connecting devices

On Mobile (WireGuard App)

1. Install the WireGuard app from the Play Store or App Store.
2. Transfer the .conf file to your phone (via email, Airdrop, or a file manager).
3. Import it into the WireGuard app and activate the connection.

On Desktop (Linux)

• Install the WireGuard client for your OS.
• Copy the .conf file into the /etc/wireguard directory.

• Connect to the VPN.

Conclusion

And just like that, we now have our own self-hosted VPN up and running! No more sketchy public Wi-Fi risks, no more ISP snooping, and best of all, full control over our own encrypted tunnel.

Honestly, PiVPN makes the whole process ridiculously easy compared to manually setting up WireGuard or OpenVPN from scratch.

It took me maybe 15–20 minutes from start to finish, and that’s including the time spent debating whether I should stick to my usual WireGuard setup or try OpenVPN just for fun.

If you’ve been thinking about rolling your own VPN, I’d say go for it. It’s a great weekend project that gives you actual privacy, plus it’s a fun way to dive into networking without things getting overwhelming.

Now, I’m curious, do you already use a self-hosted VPN, or are you still sticking with a paid service?

And hey, if you’re looking for a simpler “click-and-go” solution, We've also put together a list of the best VPN services, check it out if self-hosting isn’t your thing!

Guess who's rocking Instagram? That's right. It's Linux Handbook ;)

If you are an active Instagram user, do follow us as I am posting interesting graphics and video memes.

Here are the other highlights of this edition of LHB Linux Digest:

• Vi vs Vim
• Flexpilot IDE
• Self hosted time tracker
• And more tools, tips and memes for you
• This edition of LHB Linux Digest newsletter is supported by PikaPods.

❇️ Self-hosting without hassle

PikaPods allows you to quickly deploy your favorite open source software. All future updates are handled automatically by PikaPods while you enjoy using the software. I use it to self host Umami analytics.

Oh! You get $5 free credit, so try it out and see if you could rely on PikaPods.

PikaPods - Instant Open Source App Hosting

Run the finest Open Source web apps from $1/month, fully managed, no tracking, no ads, full privacy. Self-hosting was never this convenient.

Instant Open Source App Hosting

📖 Linux Tips and Tutorials

• Learn the difference between Vi and Vim.
• Take full advantage of the man pages in Linux.
• Complete beginner's guide to understanding different types of virtualization.
• Small but useful. Learn why BusyBox is being popular.

Get more out of your bash history with these tips.

5 Simple Bash History Tricks Every Linux User Should Know

Effectively using bash history will save you plenty of time in the Linux terminal.

Linux HandbookAbhishek Prakash

I have encountered situations where I had executed vi and it still runs Vim instead of the program that I had requested (Vi). That was just one part of the confusion.

I have seen people using Vi and Vim interchangeably, even though they are not the same editors.

Sure, many of you might know that Vim is an improved version of Vi (that is so obvious as Vim reads for Vi Improved) but there are still many differences and scenarios where you might want to use Vi over Vim.

Vi vs Vim: Why the confusion?

The confusion between Vi and Vim starts from their shared history and overlapping functionality. Vi, short for Visual Editor, was introduced in 1976 as part of the Unix operating system. It became a standard text editor on Unix systems, renowned for its efficiency and minimalism.

Vim, on the other hand, stands for Vi IMproved, and was developed in 1991 as an enhanced version of Vi, offering additional features like syntax highlighting, plugin support, and advanced editing capabilities.

Adding to the confusion is a common practice among Linux distributions: many create an alias or symlink that maps vi to Vim by default. This means that when users type vi in a terminal, they are often unknowingly using Vim instead of the original Vi. As a result, many users are unaware of where Vi ends and Vim begins.

While both editors share the same core functionality, Vim extends Vi with numerous modern features that make it more versatile for contemporary workflows. For most users, this aliasing works in their favour since Vim’s expanded feature set is generally more useful.

However, it has also led to widespread misunderstanding about what distinguishes the two editors.

Key differences between Vi and Vim

Now, let's take a look at the key differences between Vi and Vim:

Is anything still better about Vi?

While I was not sure if anything was still positive about Vi, when I talked to some sysadmins and power users, I came across some surprising points which prove that Vi is still relevant:

• Minimalism: Vi’s simplicity makes it extremely lightweight on system resources. This can be advantageous on older hardware or when working in minimalistic environments.
• Universality: As a default editor on all POSIX-compliant systems, Vi is guaranteed to be available without installation. This makes it a reliable fallback editor when working on constrained systems or during system recovery.
• Consistency: Vi adheres strictly to its original design philosophy, avoiding potential quirks or bugs introduced by newer features in Vim.

Who should choose Vi?

You might wonder that based on the points I made, the userbase for Vi will be close to nothing but that is not true. I know multiple users who use Vi over anything modern.

Here are groups of people who can benefit from Vi:

• System administrators on legacy systems: If you work on older Unix systems or environments where only basic tools are available, learning Vi is a dependable choice.
• Minimalists: Those who value simplicity and prefer minimal resource usage may find Vi sufficient for their needs.

Who should choose Vim?

For most users, however, Vim is the better choice:

• Learning the basics: Beginners aiming to understand core text-editing concepts might benefit from starting with Vim as the lack of features in Vi could be even more overwhelming.
• Developers and programmers: With features like syntax highlighting, plugin support, and advanced navigation tools, Vim is ideal for coding tasks.
• Power users: Those who require multilevel undo, visual mode for block selection, or split windows for multitasking will find Vim indispensable.
• Cross-platform users: Vim’s availability across multiple platforms ensures a consistent experience regardless of the operating system.

In fact, unless you’re working in an environment where minimalism is critical or resources are highly constrained, you’re almost certainly better off using Vim. Its additional features make it far more versatile while still retaining the efficiency of its predecessor.

Start Learning Vim [Tutorial Series]

Start learning Vim by following these Vim tips for beginners and advanced users.

Linux HandbookAbhishek Prakash

Vi vs Vim: which one should I use?

Conclusion

Vi and Vim cater to different needs despite their shared lineage. While Vi remains a lightweight, universal editor suitable for basic tasks or constrained environments, Vim extends its capabilities significantly, making it a powerful tool for modern development workflows.

The choice ultimately depends on your specific requirements—whether you value simplicity or need advanced functionality.

Which one do you use? Let us know in the comments.

As an alert Linux sysadmin, you may want to monitor web traffic for specific services. Here's why?

• Telemetry detection: Some tools with sensitive user data go online when they shouldn't. Good examples are offline wallet or note-taking applications.
• Application debugging when something goes wrong.
• High traffic usage: 4G or 5G connections are usually limited, so it's better for the wallet to stay within the limits.

The situation becomes complicated on servers due to the popularity of containers, mostly Docker or LXC.

How to identify an application's traffic within this waterfall?

Httpat from Monastic Academy is a great solution for this purpose. It works without root access, you only need write access to /dev/net/tun to be able to work with TUN virtual device used for traffic interception.

Installing Httpat

The application is written in Go and the binary can be easily downloaded from the Github release page using these three commands one by one:

wget -c https://github.com/monasticacademy/httptap/releases/latest/download/httptap_linux_$(uname -m).tar.gz
tar xf httptap_linux_$(uname -m).tar.gz
sudo mv httptap /usr/bin && rm -rf httptap_linux_$(uname -m).tar.gz

Install with Go:

go install github.com/monasticacademy/httptap@latest

Another way is to check your Linux distribution repository for the httptap package. The Repology project is great to see which distributions currently have a Httptap package.

On Ubuntu 24.04 and later, the next AppArmor restrictions should be disabled:

sudo sysctl -w kernel.apparmor_restrict_unprivileged_unconfined=0
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0

Practical examples for common use-cases

For a quick start, let's load the website "linuxhandbook.com" using curl:

httptap -- curl -s -o /dev/null https://linuxhandbook.com

Looks great, it tells us that curl used 141714 bytes for a GET request with code 200, which is OK. We use -s -o /dev/null to prevent any output from the curl to see what Httptap does.

---> GET https://linuxhandbook.com/
<--- 200 https://linuxhandbook.com/ (141714 bytes)

Let's try google.com website, which use redirects:

httptap -- python -c "import requests; requests.get('https://google.com')"

---> GET https://google.com/
<--- 301 https://google.com/ (220 bytes)
decoding gzip content
---> GET https://www.google.com/
<--- 200 https://www.google.com/ (20721 bytes)

It works and notifies us about 301 redirects and archived content. Not bad at all.
Let's say we have a few instances in the Google Cloud, managed by the cli tool called gcloud. What HTTP endpoints does this command use? Let's take a look:

httptap -- gcloud compute instances list
---> POST https://oauth2.googleapis.com/token
<--- 200 https://oauth2.googleapis.com/token (997 bytes)
---> GET https://compute.googleapis.com/compute/v1/projects/maple-public-website/aggregated/instances?alt=json&includeAllScopes=True&maxResults=500&returnPartialSuccess=True
<--- 200 https://compute.googleapis.com/compute/v1/projects/maple-public-website/aggregated/instances?alt=json&includeAllScopes=True&maxResults=500&returnPartialSuccess=True (19921 bytes)

The answer is compute.googleapis.com.

OK, we have Dropbox storage and the rclone tool to manage it from the command line. What API endpoint uses Dropbox?

$ httptap -- rclone lsf dropbox:
decoding gzip content
---> POST https://api.dropboxapi.com/2/files/list_folder
<--- 200 https://api.dropboxapi.com/2/files/list_folder (2119 bytes)

The answer is loud and clear again: api.dropboxapi.com.
Let's play a bit with DoH - encrypted DNS, DNS-over-HTTPS. We will use Quad9, a famous DNS service which supports DoH via https://dns.quad9.net/dns-query endpoint.

$ httptap -- curl -sL --doh-url https://dns.quad9.net/dns-query https://linuxhandbook.com -o /dev/null
---> POST https://dns.quad9.net/dns-query
<--- 200 https://dns.quad9.net/dns-query (83 bytes)
---> POST https://dns.quad9.net/dns-query
<--- 200 https://dns.quad9.net/dns-query (119 bytes)
---> GET https://linuxhandbook.com/
<--- 200 https://linuxhandbook.com/ (141727 bytes)

Now we can see that it makes two POST requests to the Quad9 DoH endpoint, and one GET request to the target - linuxhandbook.com/ to check if it works correctly, all with success.
Let's take a look under the hood - print the payloads of the DNS-over-HTTPS requests with --head and --body flags:

./httptap --head --body -- curl -sL --doh-url https://dns.quad9.net/dns-query https://linuxhandbook.com -o /dev/null---> POST https://dns.quad9.net/dns-query
> Accept: */*
> Content-Type: application/dns-message
> Content-Length: 35
linuxhandbookcom
<--- 200 https://dns.quad9.net/dns-query (83 bytes)
< Content-Type: application/dns-message
< Cache-Control: max-age=300
< Content-Length: 83
< Server: h2o/dnsdist
< Date: Sun, 09 Feb 2025 15:43:37 GMT
linuxhandbookcom
                ,he
                   ,he
                      ,CI�
                          ---> POST https://dns.quad9.net/dns-query
> Accept: */*
> Content-Type: application/dns-message
> Content-Length: 35
linuxhandbookcom
<--- 200 https://dns.quad9.net/dns-query (119 bytes)
< Server: h2o/dnsdist
< Date: Sun, 09 Feb 2025 15:43:38 GMT
< Content-Type: application/dns-message
< Cache-Control: max-age=300
< Content-Length: 119
linuxhandbookcom
                ,&G CI�
                       ,&G he
                             ,&G he
---> GET https://linuxhandbook.com/
> User-Agent: curl/8.11.1
> Accept: */*
<--- 200 https://linuxhandbook.com/ (141742 bytes)
< Cache-Control: private, max-age=0, must-revalidate, no-cache, no-store
< Pagespeed: off
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-Origin-Cache-Control: public, max-age=0
< Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=bAHIntCPfaGgoUwEwhk5QWPETFvnq5K9Iw60TGIAcnTisEfo%2BjKulz%2FJP7rTPgmyznVSc%2BSwIOKtajz%2BZTg71To4BuapDd%2BKdgyar%2FpIGT76XWH9%2FVNMyliYqgceD7DwuBmiPr3F77zxa7b6ty8J"}],"group":"cf-nel","max_age":604800}
< Server: cloudflare
< Cf-Ray: 90f4fa286f9970bc-WAW
< X-Middleton-Response: 200
< X-Powered-By: Express
< Cf-Cache-Status: DYNAMIC
< Alt-Svc: h3=":443"; ma=86400
< Date: Sun, 09 Feb 2025 15:43:48 GMT
< Display: orig_site_sol
< Expires: Sat, 08 Feb 2025 15:43:48 GMT
< Response: 200
< Set-Cookie: ezoictest=stable; Path=/; Domain=linuxhandbook.com; Expires=Sun, 09 Feb 2025 16:13:48 GMT; HttpOnly
< Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
< X-Middleton-Display: orig_site_sol
< Server-Timing: cfL4;desc="?proto=TCP&rtt=0&min_rtt=0&rtt_var=0&sent=0&recv=0&lost=0&retrans=0&sent_bytes=0&recv_bytes=0&delivery_rate=0&cwnd=0&unsent_bytes=0&cid=0a7f5fbffa6452d4&ts=351&x=0"
< Content-Type: text/html; charset=utf-8
< Vary: Accept-Encoding,User-Agent
< X-Ezoic-Cdn: Miss
< X-Sol: orig
< Nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
<!DOCTYPE html><html lang="en" class="group/html min-h-screen has-inline-code-block





         has-gray-scale-Slate
    " data-prismjs-copy="Copy" data-prismjs-copy-error="Error" data-prismjs-copy-success="Copied"><head><meta charset="UTF-8"/>
...

Fantastic! Httptap just intercepted the HTTP headers thanks to the --head option and the payloads because the --body option was used.

HAR

To work more comfortably with HTTP requests and responses, Httptap supports HAR format:

httptap --dump-har out.har -- curl -Lso /dev/null https://linuxhandbook.com

There are many HAR viewer applications, let's open it in Google HAR Analyzer:

More useful Httptap options:

• --no-new-user-namespace - run as root without user namespace.
• --subnet and --gateway - subnet and gateway of network inteface visible for subprocess.
• --dump-tcp - dump all TCP packets
  --http HTTP - list of TCP ports to intercept HTTPS traffic on (default: 80)
  --https HTTPS - list of TCP ports to intercept HTTP traffic on (default: 443)

Httptap runs the process in an isolated network namespace and also mounts an overlay filesystem for /etc/resolv.conf to make sure the correct DNS is used. The Linux namespace is a list of network interfaces and routing rules, and httptap uses it to not affect network traffic on the system.

It also injects a Certificate Authority to be able to decrypt HTTPS traffic. Httptap creates a TUN device and runs the subprocess in an environment where all network traffic is routed through this device, just like OpenVPN.

Httptap parses the IP packets, including inner TCP and UDP packets, and writes back raw IP packets using a software implementation of the TCP/IP protocol.

Advanced - modifying requests and responses

Currently there are no interface or command line options to do this, but it's possible with simple source code modification. Basic Go programming skills are required, of course.

The code that handles HTTP requests is here, and the code that handles responses is a few lines below that. So it's very easy to modify outgoing traffic in the same way as a normal GO HTTP request modification. Real expamples: modify or randomize application telemetry by inserting random data to make it less readable.

Conclusion

There are a few related tools that I find interesting and would like to share with you:

• Wireshark - if you want to know what's going on your network interfaces, the real must-have tool.
• OpenSnitch - interactive application firewall inspired by Little Snitch for macOS.
• Douane - personal firewall that protects a user's privacy by allowing a user to control which applications can connect to the internet from their GNU/Linux computer.
• Adnauseam - "clicking ads, so you don't have to".

I hope you enjoy using Httptap as much as I do 😄

I’ve been using Cloudflare Tunnel for over a year, and while it’s great for hosting static HTML content securely, it has its limitations.

For instance, if you’re running something like Jellyfin, you might run into issues with bandwidth limits, which can lead to account bans due to their terms of service.

Cloudflare Tunnel is designed with lightweight use cases in mind, but what if you need something more robust and self-hosted?

Let me introduce you to some fantastic open-source alternatives that can give you the freedom to host your services without restrictions.

1. ngrok (OSS Edition)

ngrok is a globally distributed reverse proxy designed to secure, protect, and accelerate your applications and network services, regardless of where they are hosted.

Acting as the front door to your applications, ngrok integrates a reverse proxy, firewall, API gateway, and global load balancing into one seamless solution.

Although the original open-source version of ngrok (v1) is no longer maintained, the platform continues to contribute to the open-source ecosystem with tools like Kubernetes operators and SDKs for popular programming languages such as Python, JavaScript, Go, Rust, and Java.

Key features:

• Securely connect APIs and databases across networks without complex configurations.
• Expose local applications to the internet for demos and testing without deployment.
• Simplify development by inspecting and replaying HTTP callback requests.
• Implement advanced traffic policies like rate limiting and authentication with a global gateway-as-a-service.
• Control device APIs securely from the cloud using ngrok on IoT devices.
• Capture, inspect, and replay traffic to debug and optimize web services.
• Includes SDKs and integrations for popular programming languages to streamline workflows.

2. frp (Fast Reverse Proxy)

frp (Fast Reverse Proxy) is a high-performance tool designed to expose local servers located behind NAT or firewalls to the internet.

Supporting protocols like TCP, UDP, HTTP, and HTTPS, frp enables seamless request forwarding to internal services via custom domain names.

It also includes a peer-to-peer (P2P) connection mode for direct communication, making it a versatile solution for developers and system administrators.

Key features:

• Expose local servers securely, even behind NAT or firewalls, using TCP, UDP, HTTP, or HTTPS protocols.
• Provide token and OIDC authentication for secure connections.
• Support advanced configurations such as encryption, compression, and TLS for enhanced security.
• Enable efficient traffic handling with features like TCP stream multiplexing, QUIC protocol support, and connection pooling.
• Facilitate monitoring and management through a server dashboard, client admin UI, and Prometheus integration.
• Offer flexible routing options, including URL routing, custom subdomains, and HTTP header rewriting.
• Implement load balancing and service health checks for reliable performance.
• Allow for port reuse, port range mapping, and bandwidth limits for granular control.
• Simplify SSH tunneling with a built-in SSH Tunnel Gateway.

3. localtunnel

Localtunnel is an open-source, self-hosted tool that simplifies the process of exposing local web services to the internet.

By creating a secure tunnel, Localtunnel allows developers to share their local resources without needing to configure DNS or firewall settings.

It’s built on Node.js and can be easily installed using npm.

While Localtunnel is straightforward and effective, the project hasn't seen active maintenance since 2022, and the default Localtunnel.me server's long-term reliability is uncertain.

However, you can host your own Localtunnel server for better control and scalability.

Key features

• Secure HTTPS for all tunnels, ensuring safe connections.
• Share your local development environment with a unique, publicly accessible URL.
• Test webhooks and external API callbacks with ease.
• Integrate with cloud-based browser testing tools for UI testing.
• Restart your local server seamlessly, as Localtunnel automatically reconnects.
• Request a custom subdomain or proxy to a hostname other than localhost for added flexibility.

4. boringproxy

boringproxy is a reverse proxy and tunnel manager designed to simplify the process of securely exposing self-hosted web services to the internet.

Whether you're running a personal website, Nextcloud, Jellyfin, or other services behind a NAT or firewall, boringproxy handles all the complexities, including HTTPS certificate management and NAT traversal, without requiring port forwarding or extensive configuration.

It’s built with self-hosters in mind, offering a simple, fast, and secure solution for remote access.

Key features

• 100% free and open source under the MIT license, ensuring transparency and flexibility.
• No configuration files required—boringproxy works with sensible defaults and simple CLI parameters for easy adjustments.
• No need for port forwarding, NAT traversal, or firewall rule configuration, as boringproxy handles it all.
• End-to-end encryption with optional TLS termination at the server, client, or application, integrated seamlessly with Let's Encrypt.
• Fast web GUI for managing tunnels, which works great on both desktop and mobile browsers.
• Fully configurable through an HTTP API, allowing for automation and integration with other tools.
• Cross-platform support on Linux, Windows, Mac, and ARM devices (e.g., Raspberry Pi and Android).
• SSH support for those who prefer using a standard SSH client for tunnel management.

5. zrok

zrok is a next-generation, peer-to-peer sharing platform built on OpenZiti, a programmable zero-trust network overlay.

It enables users to share resources securely, both publicly and privately, without altering firewall or network configurations.

Designed for technical users, zrok provides frictionless sharing of HTTP, TCP, and UDP resources, along with files and custom content.

• Share resources with non-zrok users over the public internet or directly with other zrok users in a peer-to-peer manner.
• Works seamlessly on Windows, macOS, and Linux systems.
• Start sharing within minutes using the zrok.io service. Download the binary, create an account, enable your environment, and share with a single command.
• Easily expose local resources like localhost:8080 to public users without compromising security.
• Share "network drives" publicly or privately and mount them on end-user systems for easy access.
• Integrate zrok’s sharing capabilities into your applications with the Go SDK, which supports net.Conn and net.Listener for familiar development workflows.
• Deploy zrok on a Raspberry Pi or scale it for large service instances. The single binary contains everything needed to operate your own zrok environment.
• Leverages OpenZiti’s zero-trust principles for secure and programmable network overlays.

6. Pagekite

PageKite is a veteran in the tunneling space, providing HTTP(S) and TCP tunnels for more than 14 years. It offers features like IP whitelisting, password authentication, and supports custom domains.

While the project is completely open-source and written in Python, the public service imposes limits, such as bandwidth caps, to prevent abuse.

Users can unlock additional features and higher bandwidth through affordable payment plans.

The free tier provides 2 GB of monthly transfer quota and supports custom domains, making it accessible for personal and small-scale use.

Key features

• Enables any computer, such as a Raspberry Pi, laptop, or even old cell phones, to act as a server for hosting services like WordPress, Nextcloud, or Mastodon while keeping your home IP hidden.
• Provides simplified SSH access to mobile or virtual machines and ensures privacy by keeping firewall ports closed.
• Supports embedded developers with features like naming and accessing devices in the field, secure communications via TLS, and scaling solutions for both lightweight and large-scale deployments.
• Offers web developers the ability to test and debug work remotely, interact with secure APIs, and run webhooks, API servers, or Git repositories directly from their systems.
• Utilizes a global relay network to ensure low latency, high availability, and redundancy, with infrastructure managed since 2010.
• Ensures privacy by routing all traffic through its relays, hiding your IP address, and supporting both end-to-end and wildcard TLS encryption.

7. Chisel

Chisel is a fast and efficient TCP/UDP tunneling tool transported over HTTP and secured using SSH.

Written in Go (Golang), Chisel is designed to bypass firewalls and provide a secure endpoint into your network.

It is distributed as a single executable that functions as both client and server, making it easy to set up and use.

Key features

• Offers a simple setup process with a single executable for both client and server functionality.
• Secures connections using SSH encryption and supports authenticated client and server connections through user configuration files and fingerprint matching.
• Automatically reconnects clients with exponential backoff, ensuring reliability in unstable networks.
• Allows clients to create multiple tunnel endpoints over a single TCP connection, reducing overhead and complexity.
• Supports reverse port forwarding, enabling connections to pass through the server and exit via the client.
• Provides optional SOCKS5 support for both clients and servers, offering additional flexibility in routing traffic.
• Enables tunneling through SOCKS or HTTP CONNECT proxies and supports SSH over HTTP using ssh -o ProxyCommand.
• Performs efficiently, making it suitable for high-performance requirements.

8. Telebit

Telebit has quickly become one of my favorite tunneling tools, and it’s easy to see why. It's still fairly new but does a great job of getting things done.

By installing Telebit Remote on any device, be it your laptop, Raspberry Pi, or another device, you can easily access it from anywhere.

The magic happens thanks to a relay system that allows multiplexed incoming connections on any external port, making remote access a breeze.

Not only that, but it also lets you share files and configure it like a VPN.

Key features

• Share files securely between devices
• Access your Raspberry Pi or other devices from behind a firewall
• Use it like a VPN for additional privacy and control
• SSH over HTTPS, even on networks with restricted ports
• Simple setup with clear documentation and an installer script that handles everything

9. tunnel.pyjam.as

As a web developer, one of my favorite tools for quickly sharing projects with clients is tunnel.pyjam.as.

It allows you to set up SSL-terminated, ephemeral HTTP tunnels to your local machine without needing to install any custom software, thanks to Wireguard.

It’s perfect for when you want to quickly show someone a project you’re working on without the hassle of complex configurations.

Key features

• No software installation required, thanks to Wireguard
• Quickly set up a reverse proxy to share your local services
• SSL-terminated tunnels for secure connections
• Simple to use with just a curl command to start and stop tunnels
• Ideal for quick demos or temporary access to local projects

Final thoughts

When it comes to tunneling tools, there’s no shortage of options and each of the projects we’ve discussed here offers something unique.

Personally, I’m too deeply invested in Cloudflare Tunnel to stop using it anytime soon. It’s become a key part of my workflow, and I rely on it for many of my use cases.

However, that doesn’t mean I won’t continue exploring these open-source alternatives. I’m always excited to see how they evolve.

For instance, with tunnel.pyjam.as, I find it incredibly time-saving to simply edit the tunnel.conf file and run its WireGuard instance to quickly share my projects with clients.

I’d love to hear what you think! Have you tried any of these open-source tunneling tools, or do you have your own favorites? Let me know in the comments.

What's in a name? Sometimes the name can be deceptive.

For example, in the Linux Tips and Tutorials section of this newsletter, I am sharing a few commands that have nothing to do with what their name indicates 😄

Here are the other highlights of this edition of LHB Linux Digest:

• Nice and renice commands
• ReplicaSet in Kubernetes
• Self hosted code snippet manager
• And more tools, tips and memes for you
• This edition of LHB Linux Digest newsletter is supported by RELIANOID.

❇️Comprehensive Load Balancing Solutions For Modern Networks

RELIANOID’s load balancing solutions combine the power of SD-WAN, secure application delivery, and elastic load balancing to optimize traffic distribution and ensure unparalleled performance.

With features like a robust Web Application Firewall (WAF) and built-in DDoS protection, your applications remain secure and resilient against cyber threats. High availability ensures uninterrupted access, while open networking and user experience networking enhance flexibility and deliver a seamless experience across all environments, from on-premises to cloud.

Free Load Balancer Download | Community Edition by RELIANOID

Discover our Free Load Balancer | Community Edition | The best Open Source Load Balancing software for providing high availability and content switching services

RELIANOIDAdmin

📖 Linux Tips and Tutorials

• There is an install command in Linux but it doesn't install anything
• There is a hash command in Linux but it doesn't have anything to do with hash-password
• There is a tree command in Linux but it has nothing to do with plants
• There is a wc command in Linux and it has nothing to do with washrooms 🚻 (I understand that you know what wc stands for in the command but I still find it amusing)

Using nice and renice commands to change process priority.

Change Process Priority WIth nice and renice Commands

You can modify if a certain process should get priority in consuming CPU with nice and renice commands.

Linux HandbookHelder

Kubernetes is a powerful container orchestration platform that enables developers to manage and deploy containerized applications with ease. One of its key components is the ReplicaSet, which plays a critical role in ensuring high availability and scalability of applications.

In this guide, we will explore the ReplicaSet, its purpose, and how to create and manage it effectively in your Kubernetes environment.

What is a ReplicaSet in Kubernetes?

A ReplicaSet in Kubernetes is a higher-level abstraction that ensures a specified number of pod replicas are running at all times. If a pod crashes or becomes unresponsive, the ReplicaSet automatically creates a new pod to maintain the desired state. This guarantees high availability and resilience for your applications.

The key purposes of a ReplicaSet include:

• Scaling Pods: ReplicaSets manage the replication of pods, ensuring the desired number of replicas are always running.
• High Availability: Ensures that your application remains available even if one or more pods fail.
• Self-Healing: Automatically replaces failed pods to maintain the desired state.
• Efficient Workload Management: Helps distribute workloads across nodes in the cluster.

How Does a ReplicaSet Work?

ReplicaSets rely on selectors to match pods using labels. It uses these selectors to monitor the pods and ensures the actual number of pods matches the specified replica count. If the number is less than the desired count, new pods are created. If it’s greater, excess pods are terminated.

Creating a ReplicaSet

To create a ReplicaSet, you define its configuration in a YAML file. Here’s an example:

Example YAML Configuration

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: nginx-replicaset
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.21
        ports:
        - containerPort: 80

In this YAML file:

• replicas: Specifies the desired number of pod replicas.
• selector: Matches pods with the label app=nginx.
• template: Defines the pod’s specifications, including the container image and port.

Deploying a ReplicaSet

Once you have the YAML file ready, follow these steps to deploy it in your Kubernetes cluster.

Apply the YAML configuration to create the ReplicaSet:

kubectl apply -f nginx-replicaset.yaml

Verify that the ReplicaSet was created and the pods are running:

kubectl get replicaset

Output:

NAME                DESIRED   CURRENT   READY   AGE
nginx-replicaset    3         3         3       5s

View the pods to check the pods created by the ReplicaSet:

kubectl get pods

Output:

NAME                      READY   STATUS    RESTARTS   AGE
nginx-replicaset-xyz12    1/1     Running   0          10s
nginx-replicaset-abc34    1/1     Running   0          10s
nginx-replicaset-lmn56    1/1     Running   0          10s

Scaling a ReplicaSet

You can easily scale the number of replicas in a ReplicaSet. For example, to scale the above ReplicaSet to 5 replicas:

kubectl scale replicaset nginx-replicaset --replicas=5

Verify the updated state:

kubectl get replicaset

Output:

NAME                DESIRED   CURRENT   READY   AGE
nginx-replicaset    5         5         5       2m

Learn Kubernetes Operator

Learn to build, test and deploy Kubernetes Opeartor using Kubebuilder as well as Operator SDK in this course.

Linux HandbookTeam LHB

Conclusion

A ReplicaSet is an essential component of Kubernetes, ensuring the desired number of pod replicas are running at all times. By leveraging ReplicaSets, you can achieve high availability, scalability, and self-healing for your applications with ease.

Whether you’re managing a small application or a large-scale deployment, understanding ReplicaSets is crucial for effective workload management.

A few years ago, we witnessed a shift to containers and in current times, I believe containers have become an integral part of the IT infrastructure for most companies.

Traditional monitoring tools often fall short in providing the visibility needed to ensure performance, security, and reliability.

According to my experience, monitoring resource allocation is the most important part of deploying containers and that is why I found the top container monitoring solutions offering real-time insights into your containerized environments.

Top Container Monitoring Solutions

Before I jump into details, here's a brief of all the tools which I'll be discussing in a moment:

Here, "N/A (trial only)" means that the tool does not offer a permanent free tier but provides a limited-time free trial for users to test its features. After the trial period ends, users must subscribe to a paid plan to continue using the tool. Essentially, there is no free version available for long-term use—only a temporary trial.

1. Middleware

Middleware is an excellent choice for teams looking for a free or scalable container monitoring solution. It provides pre-configured dashboards for Kubernetes environments and real-time visibility into container health.

With a free tier supporting up to 100GB of data and a pay-as-you-go model at $0.3/GB thereafter, it’s ideal for startups or small teams.

Key features:

• Pre-configured dashboards for Kubernetes
• Real-time metrics tracking
• Alerts for critical events
• Correlation of metrics with logs and traces

Pros:

• Free tier available
• Easy setup with minimal configuration
• Scalable pricing model

Cons:

• Limited advanced features compared to premium tools

2. Datadog

Datadog is a premium solution offering observability across infrastructure, applications, and logs. Its auto-discovery feature makes it particularly suited for dynamic containerized environments.

The free plan supports up to five hosts with limited retention. Paid plans start at $15 per host per month.

Key features:

• Real-time performance tracking
• Anomaly detection using ML
• Auto-discovery of new containers
• Distributed tracing and APM

Pros:

• Extensive integrations (750+)
• User-friendly interface
• Advanced visualization tools

Cons:

• High cost for small teams
• Pricing can vary based on usage spikes

3. Prometheus & Grafana

This open-source duo provides powerful monitoring and visualization capabilities. Prometheus has an edge in metrics collection with its PromQL query language, while Grafana offers stunning visualizations.

This eventually makes it perfect for teams seeking customization without licensing costs.

Key features:

• Time-series data collection
• Flexible query language (PromQL)
• Customizable dashboards
• Integrated alerting system

Pros:

• Free to use
• Highly customizable
• Strong community support

Cons:

• Requires significant setup effort
• Limited out-of-the-box functionality

4. Dynatrace

Dynatrace is an AI-powered observability platform designed for large-scale hybrid environments. It automates topology discovery and offers you deep insights into containerized workloads. Pricing starts at $0.04/hour for infrastructure-only monitoring.

Key features:

• AI-powered root cause analysis
• Automatic topology mapping
• Real-user monitoring
• Cloud-native support (Kubernetes/OpenShift)

Pros:

• Automated configuration
• Scalability for large environments
• End-to-end visibility

Cons:

• Expensive for smaller teams
• Proprietary platform limits flexibility

5. Sematext

Sematext is a lightweight tool that allows users to monitor metrics and logs across Docker and Kubernetes platforms. Its free plan supports basic container monitoring with limited retention and alerting rules. Paid plans start at just $0.007/container/hour.

Key features:

• Unified dashboard for logs and metrics
• Real-time insights into containers and hosts
• Auto-discovery of new containers
• Anomaly detection and alerting

Pros:

• Affordable pricing plans
• Simple setup process
• Full-stack observability features

Cons:

• Limited advanced features compared to premium tools

7. SolarWinds

SolarWinds offers an intuitive solution for SMBs needing straightforward container monitoring. While it doesn’t offer a permanent free plan, its pricing starts at around $27.50/month or $2995 as a one-time license fee.

Key features:

• Pre-built Docker templates
• Application-centric performance tracking
• Hardware health monitoring
• Dependency mapping

Pros:

• Easy deployment and setup
• Out-of-the-box templates
• Suitable for smaller teams

Cons:

• Limited flexibility compared to open-source tools

8. Splunk

Splunk not only provides log analysis but also provides strong container monitoring capabilities through its Observability Cloud suite. Pricing starts at $15/host/month on annual billing.

Key features:

• Real-time log and metrics analysis
• AI-based anomaly detection
• Customizable dashboards and alerts
• Integration with OpenTelemetry standards

Pros:

• Powerful search capabilities
• Scalable architecture
• Extensive integrations

Cons:

• High licensing costs for large-scale deployments

9. MetricFire

It simplifies container monitoring by offering customizable dashboards and seamless integration with Kubernetes and Docker. MetricFire is ideal for teams looking for a reliable hosted solution without the hassle of managing infrastructure. Pricing starts at $19/month.

Key features:

• Hosted Graphite and Grafana dashboards
• Real-time performance metrics
• Integration with Kubernetes and Docker
• Customizable alerting systems

Pros:

• Easy setup and configuration
• Scales effortlessly as metrics grow
• Transparent pricing model
• Strong community support

Cons:

• Limited advanced features compared to proprietary tools
• Requires technical expertise for full customization

10. SigNoz

SigNoz is an open-source alternative to proprietary APM tools like Datadog and New Relic. It offers a platform for logs, metrics, and traces under one interface.

With native OpenTelemetry support and a focus on distributed tracing for microservices architectures, SigNoz is perfect for organizations seeking cost-effective yet powerful observability solutions.

Key features:

• Distributed tracing for microservices
• Real-time metrics collection
• Centralized log management
• Customizable dashboards
• Native OpenTelemetry support

Pros:

• Completely free if self-hosted
• Active development community
• Cost-effective managed cloud option
• Comprehensive observability stack

Cons:

• Requires infrastructure setup if self-hosted
• Limited enterprise-level support compared to proprietary tools

Evaluate your infrastructure complexity and budget to select the best tool that aligns with your goals!

Imagine this: You’ve deployed a handful of Docker containers to power your favorite applications, maybe a self-hosted Nextcloud for your files, a Pi-hole for ad-blocking, or even a media server like Jellyfin.

Everything is running like a charm, but then you hit a common snag: keeping those containers updated.

When a new image is released, you’ll need to manually pull it, stop the running container, recreate it with the updated image, and hope everything works as expected.

Multiply that by the number of containers you’re running, and it’s clear how this quickly becomes a tedious and time-consuming chore.

But there’s more at stake than just convenience. Skipping updates or delaying them for too long can lead to outdated software running in your containers, which often means unpatched vulnerabilities.

These can become a serious security risk, especially if you’re hosting services exposed to the internet.

This is where Watchtower steps in, a tool designed to take the hassle out of container updates by automating the entire process.

Whether you’re running a homelab or managing a production environment, Watchtower ensures your containers are always up-to-date and secure, all with minimal effort on your part.

What is Watchtower?

Watchtower is an open-source tool that automatically monitors your Docker containers and updates them whenever a new version of their image is available.

It keeps your setup up-to-date, saving time and reducing the risk of running outdated containers.

But it’s not just a "set it and forget it" solution, it’s also highly customizable, allowing you to tailor its behavior to fit your workflow.

Whether you prefer full automation or staying in control of updates, Watchtower has you covered.

How does it work?

Watchtower works by periodically checking for updates to the images of your running containers. When it detects a newer version, it pulls the updated image, stops the current container, and starts a new one using the updated image.

The best part? It maintains your existing container configuration, including port bindings, volume mounts, and environment variables.

If your containers depend on each other, Watchtower handles the update process in the correct order to avoid downtime.

Deploying watchtower

Since you’re reading this article, I’ll assume you already have some sort of homelab or Docker setup where you want to automate container updates. That means I won’t be covering Docker installation here.

When it comes to deploying Watchtower, it can be done in two ways:

Docker run

If you’re just trying it out or want a straightforward deployment, you can run the following command:

docker run -d --name watchtower -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower

This will spin up a Watchtower container that monitors your running containers and updates them automatically.

But here’s the thing, I’m not a fan of the docker run command.

It’s quick, sure, but I prefer stack approach rather than cramming everything into a single command.

Docker compose

If you facny using Docker Compose to run Watchtower, here’s a minimal configuration that replicates the docker run command above:

version: "3.8"

services:
  watchtower:
    image: containrrr/watchtower
    container_name: watchtower
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

To start Watchtower using this configuration, save it as docker-compose.yml and run:

docker-compose up -d

This will give you the same functionality as the docker run command, but in a cleaner, more manageable format.

Customizing watchtower with environment variables

Running Watchtower plainly is all good, but we can make it even better with environment variables and command arguments.

Personally, I don’t like giving full autonomy to one service to automatically make changes on my behalf.

Since I have a pretty decent homelab running crucial containers, I prefer using Watchtower to notify me about updates rather than updating everything automatically.

This ensures that I remain in control, especially for containers that are finicky or require a perfect pairing with their databases.

Sneak peak into my homelab

Take a look at my homelab setup: it’s mostly CMS containers for myself and for clients, and some of them can behave unpredictably if not updated carefully.

So instead of letting Watchtower update everything, I configure it to provide insights and alerts, and then I manually decide which updates to apply.

To achieve this, we’ll add the following environment variables to our Docker Compose file:

Here’s how the updated docker-compose.yml file would look:

version: "3.8"

services:
  watchtower:
    image: containrrr/watchtower
    container_name: watchtower
    restart: always
    environment:
      WATCHTOWER_CLEANUP: "true"
      WATCHTOWER_POLL_INTERVAL: "3600"
      WATCHTOWER_LABEL_ENABLE: "true"
      WATCHTOWER_DEBUG: "true"
      WATCHTOWER_NOTIFICATIONS: "email"
      WATCHTOWER_NOTIFICATION_EMAIL_FROM: "admin@example.com"
      WATCHTOWER_NOTIFICATION_EMAIL_TO: "notify@example.com"
      WATCHTOWER_NOTIFICATION_EMAIL_SERVER: "smtp.example.com"
      WATCHTOWER_NOTIFICATION_EMAIL_SERVER_PORT: "587"
      WATCHTOWER_NOTIFICATION_EMAIL_USERNAME: "your_email_username"
      WATCHTOWER_NOTIFICATION_EMAIL_PASSWORD: "your_email_password"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

Once you run the Watchtower container for the first time, you'll receive an initial email confirming that the service is up and running.

Here's an example of what that email might look like:

After some time, as Watchtower analyzes your setup and scans the running containers, it will notify you if it detects any updates available for your containers.

These notifications are sent in real-time and look something like this:

This feature ensures you're always in the loop about potential updates without having to check manually.

Final thoughts

I’m really impressed by Watchtower and have been using it for a month now.

I recommend, if possible, to play around with it in an isolated environment first, that’s what I did before deploying it in my homelab.

The email notification feature is great, but my inbox now looks totally filled with Watchtower emails, so I might create a rule to manage them better. Overall, no complaints so far! I find it better than the Docker Compose method we discussed earlier.

Updating Docker Containers With Zero Downtime

A step by step methodology that can be very helpful in your day to day DevOps activities without sacrificing invaluable uptime.

Linux HandbookAvimanyu Bandyopadhyay

What about you? What do you use to update your containers?

If you’ve tried Watchtower, share your experience, anything I should be mindful of?

Let us know in the comments!

The pwd command in Linux, short for Print Working Directory, displays the absolute path of the current directory, helping users navigate the file system efficiently.

It is one of the first commands you use when you start learning Linux. And if you are absolutely new, take advantage of this free course:

Learn the Basic Linux Commands in an Hour [With Videos]

Learn the basics of Linux commands in this crash course.

Linux HandbookAbhishek Prakash

pwd command syntax

Like other Linux commands, pwd also follows this syntax.

pwd [OPTIONS]

Here, you have [OPTIONS], which are used to modify the default behavior of the pwd command. If you don't use any options with the pwd command, it will show the physical path of the current working directory by default.

Unlike many other Linux commands, pwd does not come with many flags and has only two important flags:

Now, let's take a look at the practical examples of the pwd command.

1. Display the current location

This is what the pwd command is famous for, giving you the name of the directory where you are located or from where you are running the command.

pwd

2. Display the logical path including symbolic links

If you want to display logical paths and symbolic links, all you have to do is execute the pwd command with the -L flag as shown here:

pwd -L

To showcase its usage, I will need to go through multiple steps so stay with me. First, go to the tmp directory using the cd command as shown here:

cd /tmp

Now, let's create a symbolic link which is pointing to the /var/log directory:

ln -s /var/log log_link

Finally, change your directory to log_link and use the pwd command with the -L flag:

pwd -L

In the above steps, I went to the /tmp directory and then created a symbolic link which points to a specific location (/var/log) and then I used the pwd command and it successfully showed me the symbolic link.

3. Display physical path resolving symbolic links

The pwd command is one of the ways to resolve symbolic links. Meaning, you'll see the destination directory where soft link points to.

Use the -P flag:

pwd -P

I am going to use the symbolic link which I had created in the 2nd example. Here's what I did:

• Navigate to /tmp.
• Create a symbolic link (log_link) pointing to /var/log.
• Change into the symbolic link (cd log_link)

Once you perform all the steps, you can check the real path of the symbolic link:

pwd -P

4. Use pwd command in shell scripts

To get the current location in a bash shell script, you can store the value of the pwd command in a variable and later on print it as shown here:

current_dir=$(pwd)
echo "You are in $current_dir"

Now, if you execute this shell script in your home directory like I did, you will get similar output to mine:

Bonus: Know the previous working directory

This is not exactly the use of the pwd command but it is somewhat related and interesting. There is an environment variable in Linux called OLDPWD which stores the previous working directory path.

This means you can get the previous working directory by printing the value of this environment variable:

echo "$OLDPWD"

Conclusion

This was a quick tutorial on how you can use the pwd command in Linux where I went through syntax, options, and some practical examples of it.

I hope you will find them helpful. If you have any queries or suggestions, leave us a comment.

This is the first newsletter of the year 2025. I hope expanding your Linux knowledge is one of your New Year's resolution, too. I am looking to learn and use Ansible in homelab setup. What's yours?

The focus of Linux Handbook in 2025 will be on self-hosting. You'll see more tutorials and articles on open source software you can self host on your cloud server or your home lab.

Of course, we'll continue to create new content on Kubernetes, Terraform, Ansible and other DevOps tools.

Here are the other highlights of this edition of LHB Linux Digest:

• Extraterm terminal
• File descriptors
• Self hosting mailing list manager
• And more tools, tips and memes for you
• This edition of LHB Linux Digest newsletter is supported by PikaPods.

❇️Self-hosting without hassle

PikaPods allows you to quickly deploy your favorite open source software. All future updates are handled automatically by PikaPods while you enjoy using the software. I use it to self host Umami analytics.

Oh! You get $5 free credit, so try it out and see if you could rely on PikaPods.

PikaPods - Instant Open Source App Hosting

Run the finest Open Source web apps from $1/month, fully managed, no tracking, no ads, full privacy. Self-hosting was never this convenient.

Instant Open Source App Hosting

As a tech-enthusiast content creator, I'm always on the lookout for innovative ways to connect with my audience and share my passion for technology and self-sufficiency.

But as my newsletter grew in popularity, I found myself struggling with the financial burden of relying on external services like Mailgun - a problem many creators face when trying to scale their outreach efforts without sacrificing quality.

That's when I discovered Listmonk, a free and open-source mailing list manager that not only promises high performance but also gives me complete control over my data.

In this article, I'll walk you through how I successfully installed and deployed Listmonk locally using Docker, sharing my experiences and lessons learned along the way.

I used Linode's cloud server to test the scenario. You may try either of Linode or DigitalOcean or your own servers.

Customer Referral Landing Page - $100

Cut Your Cloud Bills in Half Deploy more with Linux virtual machines, global infrastructure, and simple pricing. No surprise bills, no lock-in, and the

Linode

DigitalOcean – The developer cloud

Helping millions of developers easily build, test, manage, and scale applications of any size – faster than ever before.

Explore our products

Prerequisites

Before diving into the setup process, make sure you have the following:

• Docker and Docker Compose installed on your server.
• A custom domain that you want to use for Listmonk.
• Basic knowledge of shell commands and editing configuration files.

If you are absolutely new to Docker, we have a course just for you:

Learn Docker: Complete Beginner’s Course

Learn Docker, an important skill to have for any DevOps and modern sysadmin. Learn all the essentials of Docker in this series.

Linux HandbookAbdullah Tarek

Step 1: Set up the project directory

The first thing you need to do is create the directory where you'll store all the necessary files for Listmonk, I like an organized setup (helps in troubleshooting).

In your terminal, run:

mkdir listmonk
cd listmonk

This will set up a dedicated directory for Listmonk’s files.

Step 2: Create the Docker compose file

Listmonk has made it incredibly easy to get started with Docker. Their official documentation provides a detailed guide and even a sample docker-compose.yml file to help you get up and running quickly.

Download the sample file to the current directory:

curl -LO https://github.com/knadh/listmonk/raw/master/docker-compose.yml

Here is the sample docker-compose.yml file, I tweaked some default environment variables:

For most users, this setup should be sufficient but you can always tweak settings to your own needs.

then run the container in the background:

docker compose up -d

Once you've run these commands, you can access Listmonk by navigating to http://localhost:9000 in your browser.

Setting up SSL

By default, Listmonk runs over HTTP and doesn’t include built-in SSL support. It is kinda important if you are running any service these days. So the next thing we need to do is to set up SSL support.

While I personally prefer using Cloudflare Tunnels for SSL and remote access, this tutorial will focus on Caddy for its straightforward integration with Docker.

Start by creating a folder named caddy in the same directory as your docker-compose.yml file:

mkdir caddy

Inside the caddy folder, create a file named Caddyfile with the following content:th the following contents:

listmonk.example.com {
    reverse_proxy app:9000
}

Replace listmonk.example.com with your actual domain name. This tells Caddy to proxy requests from your domain to the Listmonk service running on port 9000.

Ensure your domain is correctly configured in DNS. Add an A record pointing to your server's IP address (in my case, the Linode server's IP).

If you’re using Cloudflare, set the proxy status to DNS only during the initial setup to let Caddy handle SSL certificates.

Next, add the Caddy service to your docker-compose.yml file. Here’s the configuration to include:

  caddy:
    image: caddy:latest
    restart: unless-stopped
    container_name: caddy
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./caddy/Caddyfile:/etc/caddy/Caddyfile
      - ./caddy/caddy_data:/data
      - ./caddy/caddy_config:/config
    networks:
      - listmonk

This configuration sets up Caddy to handle HTTP (port 80) and HTTPS (port 443) traffic, automatically obtain SSL certificates, and reverse proxy requests to the Listmonk container.

Finally, restart your containers to apply the new settings:

docker-compose restart

Once the containers are up and running, navigate to your domain (e.g., https://listmonk.example.com) in a browser.

Caddy will handle the SSL certificate issuance and proxy the traffic to Listmonk seamlessly.

Step 3: Accessing Listmonk webUI

Once Listmonk is up and running, it’s time to access the web interface and complete the initial setup.

Open your browser and navigate to your domain or IP address where Listmonk is hosted. If you’ve configured HTTPS, the URL should look something like this:

https://listmonk.yourdomain.com

and you’ll be greeted with the login page. Click Login to proceed.

Creating the admin user

On the login screen, you’ll be prompted to create an administrator account. Enter your email address, a username, and a secure password, then click Continue.

This account will serve as the primary admin for managing Listmonk.

Configure general settings

Once logged in, navigate to Settings > Settings in the left sidebar. Under the General tab, customize the following:

• Site Name: Enter a name for your Listmonk instance.
• Root URL: Replace the default http://localhost:9000 with your domain (e.g., https://listmonk.yourdomain.com).
• Admin Email: Add an email address for administrative notifications.

Click Save to apply these changes.

Configure SMTP settings

To send emails, you’ll need to configure SMTP settings:

1. Click on the SMTP tab in the settings.
2. Fill in the details:
   • Host: smtp.emailhost.com
   • Port: 465
   • Auth Protocol: Login
   • Username: Your email address
   • Password: Your email password (or Gmail App password, generated via Google’s security settings)
   • TLS: SSL/TLS
3. Click Save to confirm the settings.

Create a new campaign list

Now, let’s create a list to manage your subscribers:

1. Go to All Lists in the left sidebar and click + New.
2. Give your list a name, set it to Public, and choose between Single Opt-In or Double Opt-In.
3. Add a description, then click Save.

Your newsletter subscription form will now be available at:

https://listmonk.yourdomain.com/subscription/form

With everything set up and running smoothly, it’s time to put Listmonk to work.

You can easily import your existing subscribers, customize the look and feel of your emails, and even change the logo to match your brand.

Final thoughts

And that’s it! You’ve successfully set up Listmonk, configured SMTP, and created your first campaign list. From here, you can start sending newsletters and growing your audience.

I’m currently testing Listmonk for my own newsletter solution on my website, and while it’s a robust solution, I’m curious to see how it performs in a production environment.

That said, I’m genuinely impressed by the thought and effort that Kailash Nadh and the contributors have put into this software, it’s a remarkable achievement.

For any questions or challenges you encounter, the Listmonk GitHub page is an excellent resource and the developers are highly responsive.

Finally, I’d love to hear your thoughts! Share your feedback, comments, or suggestions below. I’d love to hear about your experience with Listmonk and how you’re using it for your projects.

Happy emailing! 📨

File descriptors are a core concept in Linux and other Unix-like operating systems. They provide a way for programs to interact with files, devices, and other input/output (I/O) resources.

Simply put, a file descriptor is like a "ticket" or "handle" that a program uses to access these resources. Every time a program opens a file or creates an I/O resource (like a socket or pipe), the operating system assigns it a unique number called a file descriptor.

This number allows the program to read, write, or perform other operations on the resource.

And as we all know, in Linux, almost everything is treated as a file—whether it's a text file, a keyboard input, or even network communication. File descriptors make it possible to handle all these resources in a consistent and efficient way.

What Are File Descriptors?

A file descriptor is a non-negative integer assigned by your operating system whenever a program opens a file or another I/O resource. It acts as an identifier that the program uses to interact with the resource.

For example:

• When you open a text file, the operating system assigns it a file descriptor (e.g., 3).
• If you open another file, it gets the next available file descriptor (e.g., 4).

These numbers are used internally by the program to perform operations like reading from or writing to the resource.

This simple mechanism allows programs to interact with different resources without needing to worry about how these resources are implemented underneath.

For instance, whether you're reading from a keyboard or writing to a network socket, you use file descriptors in the same way!

The three standard file descriptors

Every process in Linux starts with three predefined file descriptors: Standard Input (stdin), Standard Output (stdout), and Standard Error (stderr).

Here's a brief summary of their use:

Now, let's address each file descriptor with details.

1. Standard Input (stdin)- Descriptor: 0

The purpose of the standard input stream is to receive input data. By default, it reads input from the keyboard unless redirected to another source like a file or pipe. Programs use stdin to accept user input interactively or process data from external sources.

When you type something into the terminal and press Enter, the data is sent to the program's stdin. This stream can also be redirected to read from files or other programs using shell redirection operators (<).

One simple example of stdin would be a script that takes input from the user and prints it:

#!/bin/bash

# Prompt the user to enter their name
echo -n "Enter your name: "

# Read the input from the user
read name

# Print a greeting message
echo "Hello, $name!"

Here's what the output looks like:

But there is another way of using the input stream–redirecting the input itself. You can create a text file and redirect the input stream.

For example, here I have created a sample text file named input.txt which contains my name Satoshi. Later I redirected the input stream using <:

As you can see, rather than waiting for my input, it took data from the text file and we somewhat automated this.

2. Standard Output (stdout)- Descriptor: 1

The standard output stream is used for displaying normal output generated by programs. By default, it writes output to the terminal screen unless redirected elsewhere.

In simple terms, programs use stdout to print results or messages. This stream can be redirected to write output to files or other programs using shell operators (> or |).

Let's take a simple script that prints a greeting message:

#!/bin/bash

# Print a message to standard output
echo "This is standard output."

Here's the simple output (nothing crazy but a decent example):

Now, if I want to redirect the output to a file, rather than showing it on the terminal screen, then I can use > as shown here:

./stdout.sh > output.txt

Another good example can be the redirecting output of a command to a text file:

ls > output.txt

3. Standard Error (stderr)- Descriptor: 2

The standard error stream is used for displaying error messages and diagnostics. It is separate from stdout so that errors can be handled independently of normal program output.

For better understanding, I wrote a script that will trigger the stderr signal as I have used the exit 1 to mimic a faulty execution:

#!/bin/bash

# Print a message to standard output
echo "This is standard output."

# Print an error message to standard error
echo "This is an error message." >&2

# Exit with a non-zero status to indicate an error
exit 1

But if you were to execute this script, it would simply print "This is an error message." To understand better, you can redirect the output and error to different files.

For example, here, I have redirected the error message to stderr.log and the normal output will go into stdout.log:

./stderr.sh > stdout.log 2> stderr.log

Bonus: Types of limits on file descriptors

Linux kernel puts a limit on the number of file descriptors a process can use. These limits help manage system resources and prevent any single process from using too many. There are different types of limits, each serving a specific purpose.

• Soft Limits: The default maximum number of file descriptors a process can open. Users can temporarily increase this limit up to the hard limit for their session.
• Hard Limits: The absolute maximum number of file descriptors a process can open. Only the system admin can increase this limit to ensure system stability.
• Process-Level Limits: Each process has its own set of file descriptor limits, inherited from its parent process, to prevent any single process from overusing resources.
• System-Level Limits: The total number of file descriptors available across all processes on the system. This ensures fairness and prevents global resource exhaustion.
• User-Level Limits: Custom limits set for specific users or groups to allocate resources differently based on their needs.

Wrapping Up...

In this explainer, I went through what file descriptors are in Linux and shared some practical examples to explain their function. I tried to cover the types of limits in detail but then I had to drop the "detail" to stick to the main idea of this article.

But if you want, I can surely write a detailed article on the types of limits on file descriptors. Also, if you have any questions or suggestions, leave us a comment.

In Bash version 4, associative arrays were introduced, and from that point, they solved my biggest problem with arrays in Bash—indexing. Associative arrays allow you to create key-value pairs, offering a more flexible way to handle data compared to indexed arrays.

In simple terms, you can store and retrieve data using string keys, rather than numeric indices as in traditional indexed arrays.

But before we begin, make sure you are running the bash version 4 or above by checking the bash version:

echo $BASH_VERSION

If you are running bash version 4 or above, you can access the associative array feature.

Using Associative arrays in bash

Before I walk you through the examples of using associative arrays, I would like to mention the key differences between Associative and indexed arrays:

Now, let's take a look at what you are going to learn in this tutorial on using Associative arrays:

• Declaring an Associative array
• Assigning values to an array
• Accessing values of an array
• Iterating over an array's elements

1. How to declare an Associative array in bash

To declare an associative array in bash, all you have to do is use the declare command with the -A flag along with the name of the array as shown here:

declare -A Array_name

For example, if I want to declare an associative array named LHB, then I would use the following command:

declare -A LHB

2. How to add elements to an Associative array

There are two ways you can add elements to an Associative array: You can either add elements after declaring an array or you can add elements while declaring an array. I will show you both.

Adding elements after declaring an array

This is quite easy and recommended if you are getting started with bash scripting. In this method, you add elements to the already declared array one by one.

To do so, you have to use the following syntax:

my_array[key1]="value1"

In my case, I have assigned two values using two key pairs to the LHB array:

LHB[name]="Satoshi"
LHB[age]="25"

Adding elements while declaring an array

If you want to add elements while declaring the associative array itself, you can follow the given command syntax:

declare -A my_array=(
    [key1]="value1"
    [key2]="value2"
    [key3]="value3"
)

For example, here, I created a new associated array and added three elements:

declare -A myarray=(
    [Name]="Satoshi"
    [Age]="25"
    [email]="satoshi@xyz.com"
)

3. Create a read-only Associative array

If you want to create a read-only array (for some reason), you'd have to use the -r flag while creating an array:

declare -rA my_array=(
    [key1]="value1"
    [key2]="value2"
    [key3]="value3"
)

Here, I created a read-only Associative array named MYarray:

declare -rA MYarray=(
    [City]="Tokyo"
    [System]="Ubuntu"
    [email]="satoshi@xyz.com"
)

Now, if I try to add a new element to this array, it will throw an error saying "MYarray: read-only variable":

4. Print keys and values of an Associative array

If you want to print the value of a specific key (similar to printing the value of a specific indexed element), you can simply use the following syntax for that purpose:

echo ${my_array[key1]}

For example, if I want to print the value of email key from the myarray array, I would use the following:

echo ${myarray[email]}

Print the value of all keys and elements at once

The method of printing all the keys and elements of an Associative array is mostly the same. To print all keys at once, use ${!my_array[@]} which will retrieve all the keys in the associative array:

echo "Keys: ${!my_array[@]}"

If I want to print all the keys of myarray, then I would use the following:

echo "Keys: ${!myarray[@]}"

On the other hand, if you want to print all the values of an Associative array, use ${my_array[@]} as shown here:

echo "Values: ${my_array[@]}"

To print values of the myarray, I used the below command:

echo "Values: ${myarray[@]}"

5. Find the Length of the Associative Array

The method for finding the length of the associative array is exactly the same as you do with the indexed arrays. You can use the ${#array_name[@]} syntax to find this count as shown here:

echo "Length: ${#my_array[@]}"

If I want to find a length of myarray array, then I would use the following:

echo "Length: ${#myarray[@]}"

6. Iterate over an Associative array

Iterating over an associative array allows you to process each key-value pair. In Bash, you can loop through:

• The keys using ${!array_name[@]}.
• The corresponding values using ${array_name[$key]}.

This is useful for tasks like displaying data, modifying values, or performing computations. For example, here I wrote a simple for loop to print the keys and elements accordingly:

for key in "${!myarray[@]}"; do
    echo "Key: $key, Value: ${myarray[$key]}"
done

7. Check if a key exists in the Associative array

Sometimes, you need to verify whether a specific key exists in an associative array. Bash provides the -v operator for this purpose.

Here, I wrote a simple if else script that uses the -v flag to check if a key exists in the myarray array:

if [[ -v myarray["username"] ]]; then
    echo "Key 'username' exists"
else
    echo "Key 'username' does not exist"
fi

8. Clear Associative array

If you want to remove specific keys from the associative array, then you can use the unset command along with a key you want to remove:

unset my_array["key1"]

For example, if I want to remove the email key from the myarray array, then I will use the following:

unset myarray["email"]

9. Delete the Associative array

If you want to delete the associative array, all you have to do is use the unset command along with the array name as shown here:

unset my_array

For example, if I want to delete the myarray array, then I would use the following:

unset myarray

Wrapping Up...

In this tutorial, I went through the basics of the associative array with multiple examples. I hope you will find this guide helpful.

If you have any questions or suggestions, leave us a comment.