Linux Tips

I told you about the AWK tutorial series in the previous newsletter. Well, it has an awkward start. I thought I would be able to finish, but I could only complete the first three chapters. Accept my apologies. I have the additional responsibilities of a month-old child now 😊

Still, please feel free to explore this work in progress and share your feedback.

Mastering AWK as a Linux System Administrator

Transform from basic text processing to advanced data manipulation in 10 comprehensive chapters.

Linux HandbookAbhishek Prakash

For some reasons, the SVGs of command replay are not displaying properly. I'll be reuploading them as GIFs/videos over the weekend.

You already saw a few built-in variables in the first chapter. Let's have a look at some other built-in variables along with the ones you already saw. Repitition is good for reinforced learning.

Sample Data Files

Let me create some sample files for you to work with. Save these to follow along the tutorial on your system:

Create access.log:

192.168.1.100 - alice [29/Jun/2024:10:15:22] "GET /index.html" 200 1234
192.168.1.101 - bob [29/Jun/2024:10:16:45] "POST /api/login" 200 567
192.168.1.102 - charlie [29/Jun/2024:10:17:10] "GET /images/logo.png" 404 0
10.0.0.50 - admin [29/Jun/2024:10:18:33] "GET /admin/panel" 403 892
192.168.1.100 - alice [29/Jun/2024:10:19:55] "GET /profile" 200 2456

Create inventory.csv:

laptop,Dell,XPS13,1299.99,5
desktop,HP,Pavilion,899.50,3
tablet,Apple,iPad,599.00,8
monitor,Samsung,27inch,349.99,12
keyboard,Logitech,MX Keys,99.99,15

FS (Field Separator): How you split your data

You have already used FS before. FS tells AWK how to slice each line into fields - think of it as choosing the right places to cut your data.

Default whitespace splitting

By default, the field separator is white space (space, tab etc).

Let's extract user information from our access log:

awk '{print "IP:", $1, "User:", $3, "Status:", $7}' access.log

It automatically splits on spaces and extracts IP address, username, and HTTP status code.

Output:

IP: 192.168.1.100 User: alice Status: 200
IP: 192.168.1.101 User: bob Status: 200
IP: 192.168.1.102 User: charlie Status: 404
IP: 10.0.0.50 User: admin Status: 403
IP: 192.168.1.100 User: alice Status: 200

Custom field separators

Now let's process our CSV inventory. Here we define that we have to cut the data at every comma with -F,:

awk -F, '{print $1, "by", $2, "costs $" $4}' inventory.csv

In this example, it uses comma as a separator to extract product type, manufacturer, and price from CSV.

laptop by Dell costs $1299.99
desktop by HP costs $899.50
tablet by Apple costs $599.00
monitor by Samsung costs $349.99
keyboard by Logitech costs $99.99

💡 You can also handle multiple separators.

Create mixed_data.txt:

server01::cpu::75::memory::4096
web02|admin|active|192.168.1.10
db-server,mysql,running,8192,16
cache:redis:online:1024

Now let's work on it.

awk -F'[:|,]' '{print "Server:", $1, "Service:", $2, "Info:", $4}' mixed_data.txt

It uses a character class to split on colons, pipes, or commas, thus handling inconsistent delimiters.

Server: server01 Service:  Info: 75
Server: web02 Service: admin Info: 192.168.1.10
Server: db-server Service: mysql Info: 8192
Server: cache Service: redis Info: 1024

OFS (Output Field Separator): How you join your data

OFS controls how fields appear in your output - it's like choosing the glue between your data pieces.

Let's convert our space-separated log to CSV:

awk 'BEGIN {OFS=","} {print $3, $1, $7}' access.log

It will set the output separator to comma and create CSV with username, IP, and status.

alice,192.168.1.100,200
bob,192.168.1.101,200
charlie,192.168.1.102,404
admin,10.0.0.50,403
alice,192.168.1.100,200

Of course, you can simply use awk '{print $3 "," $1 "," $7}' access.log to achieve the same output, but that's not the point here.

Similarly, let's change our inventory csv to a pipe-delimited report:

awk -F, 'BEGIN {OFS="|"} {print $2, $3, $4, $5}' inventory.csv

Here's what it would look like:

Dell|XPS13|1299.99|5
HP|Pavilion|899.50|3
Apple|iPad|599.00|8
Samsung|27inch|349.99|12
Logitech|MX Keys|99.99|15

Note that the original files are not touched. You see the output on STDOUT. They are not written on the input file.

RS (Record Separator): How you define records

RS tells AWK where one record ends and another begins.

We'll use a new sample file multiline_records.txt:

Name: John Smith
Age: 35
Department: Engineering
Salary: 75000

Name: Mary Johnson
Age: 42
Department: Marketing
Salary: 68000

Name: Bob Wilson
Age: 28
Department: Sales
Salary: 55000

Process these paragraph-style records with:

awk 'BEGIN {RS=""; FS="\n"} {
    name = substr($1, 7)
    age = substr($2, 6) 
    dept = substr($3, 13)
    salary = substr($4, 9)
    print name, age, dept, salary
}' multiline_records.txt

It is a bit complicated, but assuming that you are terating data files, it will be worth the effort. Here, awk treats empty lines as record separators and each line (\n) within a record as a field, then extracts the values after the colons.

Look at the formatted output now:

John Smith 35 Engineering 75000
Mary Johnson 42 Marketing 68000
Bob Wilson 28 Sales 55000

ORS (Output Record Separator): How you end records

ORS controls what goes at the end of each output record - think of it as choosing your punctuation mark.

For example, if you use this command with inventory.csv file:

awk -F, 'BEGIN {ORS=" | "} {print $1}' inventory.csv

It will replace newlines with " | " to create a continuous horizontal list of product types.

laptop | desktop | tablet | monitor | keyboard | 

A more practical, real-world use case would be to add HTML line breaks to your log output so that it is displayed properly in a web browser:

awk 'BEGIN {ORS="<br>\n"} {print $3, "accessed at", $2}' access.log

Here's the output and feel free to parse it as HTML

alice accessed at -<br>
bob accessed at -<br>
charlie accessed at -<br>
admin accessed at -<br>
alice accessed at -<br>

NR (Number of Records): Your line counter

Honestly, I like to remember it as number of rows. NR tracks which record you're currently processing - like a page number, I mean line number ;)

Add line numbers to the inventory file:

awk '{printf "%2d: %s\n", NR, $0}' inventory.csv

It prints a formatted line number followed by the original line. Deja Vu? We have seen this in the first chapter, too.

 1: laptop,Dell,XPS13,1299.99,5
 2: desktop,HP,Pavilion,899.50,3
 3: tablet,Apple,iPad,599.00,8
 4: monitor,Samsung,27inch,349.99,12
 5: keyboard,Logitech,MX Keys,99.99,15

Now a better idea would to use this information to dela with specific lines only.

awk -F, 'NR >= 2 && NR <= 4 {print "Item " NR ":", $1, $3}' inventory.csv

So now, AWK will process only lines 2-4, extracting product type and model.

Item 2: desktop Pavilion
Item 3: tablet iPad
Item 4: monitor 27inch

NF (Number of Fields): Your column counter

NF tells you how many fields are in each record (row/line). This is excellent when you have to loop on data (discussed in later chapters) or have to get the last column/field for processing.

Create variable_fields.txt:

web01 active 
db02 maintenance scheduled friday
cache01 offline
backup01 running full-backup nightly
api-server online load-balanced

Let's work on this data file and make it display the number of fields in each line:

awk '{print "Server " $1 " has " NF " fields:", $0}' variable_fields.txt

As you can see, it displays the number of fields:

Server web01 has 2 fields: web01 active 
Server db02 has 4 fields: db02 maintenance scheduled friday
Server cache01 has 2 fields: cache01 offline
Server backup01 has 4 fields: backup01 running full-backup nightly
Server api-server has 3 fields: api-server online load-balanced

Let's take another example where it always prints the last field irrespective of the number of fields:

awk '{print $1 ":", $NF}' variable_fields.txt

Works fine, right?

web01: active
db02: friday
cache01: offline
backup01: nightly
api-server: load-balanced

FILENAME: Your file tracker

FILENAME shows which file is being processed. This is essential when you handle multiple files.

Create these log files:

server1.log:

ERROR: Database connection failed
WARN: High memory usage
INFO: Backup completed

server2.log:

ERROR: Network timeout
INFO: Service restarted  
ERROR: Disk space low

Track errors across multiple files but also include from which file the output line is coming from by printing FILENAME:

awk '/ERROR/ {print FILENAME ":", $0}' server1.log server2.log

As you can see, it finds all ERROR lines and shows which file they came from.

server1.log: ERROR: Database connection failed
server2.log: ERROR: Network timeout
server2.log: ERROR: Disk space low

FNR (File Number of Records): Your per-file counter

Another in-built AWK variable that helps while dealing with multiple files. FNR resets to 1 for each new file.

Imagine a situation where you have two files to deal with AWK. If you use NR, it will count the number of rows from both files together. FNR on the other hand, will give you the number of records from each file.

Let's take an example:

awk '{print FILENAME, "line", FNR, "(overall line", NR "):", $0}' server1.log server2.log

It shows both the line number within each file (FNR) and the overall line number (NR) across all files.

server1.log line 1 (overall line 1): ERROR: Database connection failed
server1.log line 2 (overall line 2): WARN: High memory usage
server1.log line 3 (overall line 3): INFO: Backup completed
server2.log line 1 (overall line 4): ERROR: Network timeout
server2.log line 2 (overall line 5): INFO: Service restarted
server2.log line 3 (overall line 6): ERROR: Disk space low

Field Manipulation: Changing Your Data

Modifying Existing Fields

Apply a 10% discount to all prices:

awk -F, 'BEGIN {OFS=","} {$4 = $4 * 0.9; print}' inventory.csv

What it does: Multiplies the price field (column 4) by 0.9 and rebuilds the line with commas.

Output:

laptop,Dell,XPS13,1169.991,5
desktop,HP,Pavilion,809.55,3
tablet,Apple,iPad,539.1,8
monitor,Samsung,27inch,314.991,12
keyboard,Logitech,MX Keys,89.991,15

Adding New Fields

Calculate total inventory value:

awk -F, 'BEGIN {OFS=","} {
    total_value = $4 * $5
    print $0, total_value
}' inventory.csv

What it does: Multiplies price by quantity and adds the result as a new field.

Output:

laptop,Dell,XPS13,1299.99,5,6499.95
desktop,HP,Pavilion,899.50,3,2698.5
tablet,Apple,iPad,599.00,8,4792
monitor,Samsung,27inch,349.99,12,4199.88
keyboard,Logitech,MX Keys,99.99,15,1499.85

Working with Multi-Character Delimiters

Create complex_log.txt:

2024-06-29::10:15:22::INFO::System started successfully
2024-06-29::10:16:45::ERROR::Database connection timeout  
2024-06-29::10:17:10::WARN::Low disk space detected
2024-06-29::10:18:33::ERROR::Network unreachable

Parse double-colon separated data:

awk -F'::' '{print $1, $2, $3 ":", $4}' complex_log.txt

What it does: Uses double-colon as field separator to create readable timestamp and message format.

Output:

2024-06-29 10:15:22 INFO: System started successfully
2024-06-29 10:16:45 ERROR: Database connection timeout
2024-06-29 10:17:10 WARN: Low disk space detected
2024-06-29 10:18:33 ERROR: Network unreachable

🪧 Time to recall

You now have powerful tools for data manipulation:

• FS/OFS: Control how you split input and join output
• RS/ORS: Define what constitutes records
• NR/FNR: Track line numbers globally and per-file
• NF: Count fields and access the last column
• FILENAME: Identify source files

These variables work together to give you complete control over how AWK processes your data.

Practice Exercises

Try these exercises with the sample files I've provided:

1. Convert the access.log to CSV format with just IP, user, and status
2. Add a 10% discount to the items in inventory.csv
3. Find all inventory items with quantity less than 10
4. Add a new field in inventory.csv that shows inventory value by multiplying sock with pricing
5. Add line numbers only to ERROR entries in the server logs
6. Calculate the average price of all inventory items
7. Process the variable_fields.txt file and show only lines with exactly 3 fields

In the next chapter, you'll learn mathematical operations and string functions that will turn AWK into your personal calculator and text processor!

Think of AWK patterns like a security guard at a nightclub - they decide which lines get past the velvet rope and which actions get executed. Master pattern matching, and you control exactly what AWK processes.

Pattern matching fundamentals

AWK patterns work like filters: they test each line and execute actions only when conditions are met. No match = no action.

Here are some very basic examples of pattern matching:

awk '/ERROR/ {print $0}' logfile.txt       # Find lines containing "ERROR"
awk '/^root/ {print $1}' /etc/passwd       # Lines starting with "root"
awk '/ssh$/ {print NR}' processes.txt      # Lines ending with "ssh"
awk '/^ERROR$/ {print}' logfile.txt        # Lines containing only ERROR

Regular expressions in AWK use the same syntax as grep and sed. The pattern sits between forward slashes /pattern/.

You must have some basic understanding of regex to use the pattern matching.

Tip: Instead of multiple AWK calls:

awk '/ERROR/ file && awk '/WARNING/ file

Use one call with OR:

awk '/ERROR|WARNING/ {print}' file

Conditional operations: making decisions with if-else

AWK's if-else statements work like traffic lights - they direct program flow based on conditions.

Create this file as performance.txt:

server01 cpu 75 memory 60 disk 45
server02 cpu 45 memory 30 disk 85  
server03 cpu 95 memory 85 disk 70
server04 cpu 25 memory 40 disk 20
server05 cpu 65 memory 75 disk 90

And we shall see how you can use if-else to print output that matches a certain pattern.

Simple if statement: Binary decisions

Think of if like a bouncer - one condition, one action.

Let's use this command with the performance.txt we created previously:

awk '{if ($3 > 80) print $1, "CPU ALERT"}' performance.txt

It will show the lines that have CPU usage ($3=3rd column) greater than 80 but print the server name ($1=first column).

server03 CPU ALERT

Only server03 exceeds the 80% CPU threshold, so only it 'triggers the alert'.

if-else structure: Either-or logic

Think of if-else like a fork in the road - two paths, always take one.

Let's label the servers based on the disk usage.

awk '{
    if ($5 > 70) 
        print $1, "HIGH DISK"
    else 
        print $1, "DISK OK"
}' performance.txt

Output:

server01 DISK OK
server02 HIGH DISK
server03 DISK OK  
server04 DISK OK
server05 HIGH DISK

Every server gets classified - no line escapes without a label.

if-else if-else chain: Multi-tier classification

Think of nested conditions like a sorting machine - items flow through multiple gates until they find their category.

awk '{
    if ($5 > 80) 
        status = "CRITICAL"
    else if ($5 > 60) 
        status = "WARNING" 
    else if ($5 > 40)
        status = "MODERATE"
    else 
        status = "OK"
    print $1, "disk:", $5"%", status
}' performance.txt

Output:

server01 disk: 45% MODERATE
server02 disk: 85% CRITICAL
server03 disk: 70% WARNING
server04 disk: 20% OK
server05 disk: 90% CRITICAL

Each server cascades through conditions until it hits its classification tier.

Complex multi-field analysis

Let's make it a bit more complicated by combining CPU, memory, and disk metrics and create a monitoring script:

awk '{
    cpu = $3; mem = $5; disk = $7
    
    if (cpu > 90 || mem > 90 || disk > 90)
        alert = "CRITICAL"
    else if (cpu > 70 && mem > 70)
        alert = "HIGH_LOAD" 
    else if (cpu > 80 || mem > 80 || disk > 80)
        alert = "WARNING"
    else
        alert = "NORMAL"
        
    printf "%-10s CPU:%2d%% MEM:%2d%% DISK:%2d%% [%s]\n", $1, cpu, mem, disk, alert
}' performance.txt

It should show this output.

server01   CPU:75% MEM:60% DISK:45% [NORMAL]
server02   CPU:45% MEM:30% DISK:85% [WARNING]
server03   CPU:95% MEM:85% DISK:70% [CRITICAL]
server04   CPU:25% MEM:40% DISK:20% [NORMAL]
server05   CPU:65% MEM:75% DISK:90% [CRITICAL]

This tiered approach mimics real monitoring systems - critical issues trump warnings, combined load factors matter. You can combine it with proc data and cron to convert it into an actual system resource alert system.

Comparison operators

AWK comparison operators work like mathematical symbols - they return true and false for conditions. This gives you greater control to put login in place.

We will use the following data files for our testing in this section.

A server_stats.txt file that has the hostname, cpu_cores, memory_mb, cpu_usage, status fields.

web01 8 4096 75 online
web02 4 2048 45 maintenance  
db01 16 8192 90 online
db02 8 4096 65 online
cache01 2 1024 0 offline
backup01 4 2048 100 online

And a network_ports.txt file that has ip, service, port, protocol and state fields.

192.168.1.10 ssh 22 tcp open
192.168.1.10 http 80 tcp open
192.168.1.15 ftp 21 tcp closed
192.168.1.20 mysql 3306 tcp open  
192.168.1.25 custom 8080 tcp open
192.168.1.30 ssh 22 tcp open

Numeric comparisons

Numeric comparisons are simple. You use the regular <,>,= etc symbols for comparing numbers.

Greater than - Like checking if servers exceed CPU thresholds:

awk '$4 > 70 {print $1, "high CPU:", $4"%"}' server_stats.txt

Output:

web01 high CPU: 75%
db01 high CPU: 90%
backup01 high CPU: 100%

Less than or equal - Like finding servers with limited resources:

awk '$2 <= 4 {print $1, "low core count:", $2}' server_stats.txt

Output:

web02 low core count: 4
cache01 low core count: 2
backup01 low core count: 4

Equals - Like finding servers with zero usage (probably offline):

awk '$4 == 0 {print $1, "zero CPU usage"}' server_stats.txt

Output:

cache01 zero CPU usage

Not equals - Like finding non-standard ports:

awk '$3 != 22 && $3 != 80 && $3 != 443 {print $1, "unusual port:", $3}' network_ports.txt

Output:

192.168.1.15 unusual port: 21
192.168.1.20 unusual port: 3306
192.168.1.25 unusual port: 8080

String comparisons

You have different operators for comparing strings. They are quite easy to use and understand.

Exact string match (==)

Let's check servers with running status:

awk '$5 == "online" {print $1, "is running"}' server_stats.txt

Output:

web01 is running
db01 is running
db02 is running
backup01 is running

Pattern match (~)

Let's find ports that are running a database like sql:

awk '$2 ~ /sql/ {print $1, "database service on port", $3}' network_ports.txt

Output:

192.168.1.20 database service on port 3306

Does NOT match (!~):

When you want to exclude the matches. For example,

echo -e "# This is a comment\nweb01 active\n# Another comment\ndb01 running" | awk '$1 !~ /^#/ {print "Valid config:", $0}'

The output will omit lines starting with #:

Valid config: web01 active
Valid config: db01 running

Logical operators: &&, || and !

Logical operators work like sentence connectors - they join multiple conditions into complex tests. You'll be using them as well to add complex logic to your scripts.

Here's the test file process_list.txt for this section:

nginx 1234 www-data 50 2.5 running
mysql 5678 mysql 200 8.1 running
apache 9012 www-data 30 1.2 stopped
redis 3456 redis 15 0.8 running
postgres 7890 postgres 150 5.5 running
sshd 2468 root 5 0.1 running

It has process, pid, user, memory_mb, cpu_percent and status fields.

AND Operator (&&) - Both conditions must be true

Let's find processes that are both memory AND CPU hungry in our input file. Let's filter the lines that have RAM more than 100 and CPU usage greater than 5.

awk '$4 > 100 && $5 > 5.0 {print $1, "resource hog:", $4"MB", $5"%"}' process_list.txt

Here's the output:

mysql resource hog: 200MB 8.1%
postgres resource hog: 150MB 5.5%

OR Operator (||) - Either condition can be true

The command filters out important services like mysql or postgres or services with CPU usage greater than 7:

awk '$1 == "mysql" || $1 == "postgres" || $5 > 7.0 {print $1, "needs attention"}' process_list.txt

Output:

mysql needs attention
postgres needs attention

NOT Operator (!) - Reverses the condition

Let's find all the services that are not active in our test file:

awk '!($6 == "running") {print $1, "not running, status:", $6}' process_list.txt

Here's the output:

apache not running, status: stopped

Complex combined Logic

You can combine them to test multiple criteria. Try and figure out what this command does:

awk '($3 == "root" && $4 > 10) || ($5 > 2.0 && $6 == "running") {
    printf "Monitor: %-10s User:%-8s Mem:%3dMB CPU:%.1f%%\n", $1, $3, $4, $5
}' process_list.txt

The output should help you understand it:

Monitor: nginx      User:www-data Mem: 50MB CPU:2.5%
Monitor: mysql      User:mysql    Mem:200MB CPU:8.1%
Monitor: postgres   User:postgres Mem:150MB CPU:5.5%

Practical examples for system administrators

Now, let's see some real-world scenarios where you can use these operators. It will also have some elements from the previous chapters.

Example 1: Failed SSH login analysis

Find failed SSH attempts with IP addresses. Please note that this may not output anything if you are on a personal system that does not accept SSH connections.

awk '/Failed password/ && /ssh/ {
    for(i=1; i<=NF; i++) 
        if($i ~ /^[0-9]+\.[0-9]+/) 
            print $1, $2, $i
}' /var/log/auth.log

Example 2: Process memory monitoring

Let's create a script that will display processes with high memory consumption at the time when the script was run.

ps aux | awk 'NR > 1 {
    if ($4 > 5.0) 
        printf "HIGH MEM: %s using %.1f%%\n", $11, $4
    else if ($4 > 2.0)
        printf "MEDIUM: %s using %.1f%%\n", $11, $4
}'

There are better ways to monitor and setting up alert system though.

Example 3: Disk space alerts

Check for filesystems with over 80% full space.

df -h | awk 'NR > 1 {
    gsub(/%/, "", $5)  # Remove % symbol
    if ($5 > 80)
        printf "WARNING: %s is %s%% full\n", $6, $5
}'

Example 4: Log level filtering

Filter logs based on the severity levels. This is a dummy example, as you'll need some services running that have these logs level.

awk '{
    if ($3 ~ /ERROR|FATAL/) 
        print "CRITICAL:", $0
    else if ($3 ~ /WARNING|WARN/)
        print "ATTENTION:", $0  
    else if ($3 ~ /INFO/)
        print "INFO:", $4, $5, $6
}' application.log

Example 5: Network connection analysis

Analyze netstat output for suspicious connections:

netstat -an | awk '
    $1 ~ /tcp/ && $6 == "ESTABLISHED" {
        if ($4 !~ /:22$|:80$|:443$/)
            print "Unusual connection:", $4, "->", $5
    }
'

Works better on servers.

🪧 Time to recall

In this chapter, you've learned:

• Patterns filter which lines get processed
• Comparison operators test numeric and string conditions
• Logical operators combine multiple tests
• Regular expressions provide flexible string matching
• Field-specific patterns offer precision control

Practice Exercises

1. Find all users with UID greater than 1000 in /etc/passwd
2. Extract error and warning messages from a log file (if you have one)
3. Show processes using more than 50% CPU from ps aux output
4. Find /etc/ssh/sshd_config configuration lines that aren't comments
5. Identify network connections on non-standard ports (see one of the examples below for reference)

In the next chapter, learn about built-in variables and field manipulation - where AWK transforms from a simple filter into a data processing powerhouse.

If you're a Linux system administrator, you've probably encountered situations where you need to extract specific information from log files, process command output, or manipulate text data.

While tools like grep and sed are useful, there's another much more powerful tool in your arsenal that can handle complex text processing tasks with remarkable ease: AWK.

What is AWK and why should You care about it?

AWK is not just a UNIX command, it is a powerful programming language designed for pattern scanning and data extraction. Named after its creators (Aho, Weinberger, and Kernighan), AWK excels at processing structured text data, making it invaluable for system administrators who regularly work with log files, configuration files, and command output.

Here's why AWK should be in your and every sysadmin's toolkit:

• Built for text processing: AWK automatically splits input into fields and records, making column-based data trivial to work with.
• Pattern-action paradigm: You can easily define what to look for (pattern) and what to do when found (action).
• No compilation needed: Unlike most other programming languages, AWK scripts do not need to be compiled first. AWK scripts run directly and thus making them perfect for quick one-liners and shell integration.
• Handles complex logic: Unlike simple Linux commands, AWK supports variables, arrays, functions, and control structures.
• Available everywhere: Available on virtually every Unix-like system and all Linux distros by default.

AWK vs sed vs grep: When to use which tool

grep, sed and awk all three deal with data processing and that may make you wonder whether you should use sed or grep or AWK.

In my opinion, you should

Use grep when:

• You need to find lines matching a pattern
• Simple text searching and filtering
• Binary yes/no decisions about line content

# Find all SSH login attempts
grep "ssh" /var/log/auth.log

Use sed when:

• You need to perform find-and-replace operations
• Simple text transformations
• Stream editing with minimal logic

# Replace all occurrences of "old" with "new"
sed 's/old/new/g' file.txt

Use AWK when:

• You need to work with columnar data
• Complex pattern matching with actions
• Mathematical operations on data
• Multiple conditions and logic branches
• Generating reports or summaries

# Print username and home directory from /etc/passwd
awk -F: '{print $1, $6}' /etc/passwd

Now that you are a bit more clear about when to use AWK, let's see the basics of AWK command structure.

Basic AWK syntax and structure

AWK follows a simple but powerful syntax:

awk 'pattern { action }' file

• Pattern: Defines when the action should be executed (optional)
• Action: What to do when the pattern matches (optional)
• File: Input file to process (can also read from stdin)

Let's get started with using AWK for some simple but interesting use cases.

Your first AWK Command: Printing specific columns

Let's start with a practical example. Suppose you want to see all users in Linux and their home directories from /etc/passwd file:

awk -F: '{print $1, $6}' /etc/passwd

The output should be something like this:

root /root
daemon /usr/sbin
bin /bin
sys /dev
sync /bin
games /usr/games
...

Let's break this doww:

• -F: sets the field separator to colon (since /etc/passwd uses colons)
• $1 refers to the first field (username)
• $6 refers to the sixth field (home directory)
• print outputs the specified fields

Understanding AWK's automatic field splitting and

AWK automatically splits each input line into fields based on whitespace (i.e. spaces and tabs) by default. Each field is accessible through variables:

• $0 - The entire line
• $1 - First field
• $2 - Second field
• $NF - Last field (NF = Number of Fields)
• $(NF-1) - Second to last field

Let's see it in action by extracting process information from the ps command output.

ps aux | awk '{print $1, $2, $NF}'

This prints the user, process ID, and command for each running process.

NF is one of the several built-in variables

Know built-in variables: Your data processing toolkit

AWK provides several built-in variables that make text processing easier.

FS (Field separator)

By default, AWK uses white space, i.e. tabs and spaces as field separator. With FS, you can define other field separators in your input.

For example, /etc/passwd file contains lines that have values separated by colon : so if you define file separator as : and extract the first column, you'll get the list of the users on the system.

awk -F: '{print $1}' /etc/passwd

You'll get the same result with the following command.

awk 'BEGIN {FS=":"} {print $1}' /etc/passwd

More about BEGIN in later part of this AWK series.

NR (Number of records/lines)

NR keeps track of the current line number. It is helpful when you have to take actions for certain lines in a file.

For example, the command below will output the content of the /etc/passwd file but with line numbers attached at the beginning.

awk '{print NR, $0}' /etc/passwd

NF (Number of fields)

NF contains the number of fields in the current record. Which is basically number of columns when separated by FS.

For example, the command below will print exactly 5 fields in each line.

awk -F: 'NF == 5 {print $0}' /etc/passwd

Practical examples for system administrators

Let's see some practical use cases where you can utilize the power of AWK.

Example 1: Analyzing disk usage

The command below shows disk usage percentages and mount points, sorted by usage where as skipping the header line with NR > 1.

df -h | awk 'NR > 1 {print $5, $6}' | sort -nr

Example 2: Finding large files

I know that find command is more popular for this but you can also use AWK to print file sizes and names for files larger than 1MB.

ls -l | awk '$5 > 1000000 {print $5, $NF}'

Example 3: Processing log files

awk '/ERROR/ {print $1, $2, $NF}' /var/log/syslog

This extracts timestamp and error message from log files containing "ERROR".

Example 4: Memory usage summary

Use AWK to calculate and display memory usage percentage.

free -m | awk 'NR==2 {printf "Memory Usage: %.1f%%\n", $3/$2 * 100}'

Pattern matching in AWK works like a smart bouncer - it evaluates conditions and controls access to actions. Master these concepts:It is slightly complicated than the other examples, so let me break it down for you.

Typical free command output looks like this:

              total        used        free      shared  buff/cache   available
Mem:           7974        3052         723         321        4199        4280
Swap:          2048          10        2038

With NR==2 we only take the second line from the above output. $2 (second column) gives total memory and $3 gives used memory.

Next, in printf "Memory Usage: %.1f%%\n", $3/$2 * 100 , the printf command prints formatted output, %.1f%% shows one decimal place, and a % symbol and $3/$2 * 100 calculates memory used as a percentage.

So in the example above, you get the output as Memory Usage: 38.3% where 3052 is ~38.3% of 7974.

You'll learn more on arithmetical operation with AWK later in this series.

🪧 Time to recall

In this introduction, you've learned:

• AWK is a pattern-action language perfect for structured text processing
• It automatically splits input into fields, making columnar data easy to work with
• Built-in variables like NR, NF, and FS provide powerful processing capabilities
• AWK excels where grep and sed fall short: complex data extraction and manipulation

AWK's real power becomes apparent when you need to process data that requires logic, calculations, or complex pattern matching.

In the next part of this series, we'll dive deeper into pattern matching and conditional operations that will transform how you handle text processing tasks.

🏋️ Practice exercises

Try these exercises to reinforce what you've learned:

1. Display only usernames from /etc/passwd
2. Show the last field of each line in any file
3. Print line numbers along with lines containing "root" in /etc/passwd
4. Extract process names and their memory usage from ps aux output
5. Count the number of fields in each line of a file

The solutions involve combining the concepts we've covered: field variables, built-in variables, and basic pattern matching.

In the next tutorial, we'll explore these patterns in much more detail.

Is it too 'AWKward' to use AWK in the age of AI? I don't think so. AWK is so underrated despite being so powerful for creating useful automation scripts.

We have had a very good intro to AWK and now I am working on a series that covers the basics of AWK, just like our Bash series.

Hopefully, you'll see it in the next newsletter. Stay tuned 😊

In this technologically rich era, businesses deploy servers in no time and also manage hundreds of devices on the cloud. All this is possible with the assistance of Ansible-like automation engines.

Ansible is an automation server that manages multiple remote hosts and can deploy applications, install packages, troubleshoot systems remotely, perform network automation, configuration management, and much more, all at once or one by one.

In today’s guide, we’ll elaborate on the steps to install, configure, and automate Linux in minutes. This guide is broadly divided into 2 categories:

• Install and Configure Ansible → Practical demonstration of installing and configuring Ansible on Control Node.
• Ansible Playbooks | Automate Linux in Minutes → Creating Ansible Playbooks and implementing playbooks on managed nodes.

Let’s have a look at the brief outline:

• Install and Configure Ansible | Control Node and Host Nodes
  • Step 1: Install and Configure Ansible on the Control Node
  • Step 2: Create an Inventory/Hosts File on the Control Node
  • Step 3: Install and Configure SSH on the Host Nodes
  • Step 4: Create an Ansible User for Remote Connections
  • Step 5: Set up SSH Key | Generate and Copy
  • Step 6: Test the Connection | Control Node to Host Nodes
• Ansible Playbooks | Automate Linux in Minutes
  • YAML Basics
  • Step 1: Create an Ansible Playbook
  • Step 2: Automate the Tasks
• Conclusion

Install and Configure the Ansible | Control Node and Host Nodes

As already discussed, Ansible is the automation server that has the control node and some managed nodes to manage the overall server. In this section, we’ll demonstrate how you can install and configure Ansible to work properly.

Prerequisites: Understanding the Basics | Control Node, Managed Nodes, Inventory File, Playbook

Before proceeding to the real-time automation, let’s have a look at the list of components that we need to understand before proceeding:

• Control Node: The system where Ansible is installed. In this guide, the Ansible Server is set up on OpenSUSE Linux.
• Managed Nodes: The servers that are being managed by the Ansible control node.
• Inventory/Host File: The inventory file contains a list of host(s) IP(s) that a control node will manage.
• Playbook: Playbook is an automated script based on YAML that Ansible utilizes to perform automated tasks on the Managed Nodes.

Let’s now start the initial configuration:

Step 1: Install and Configure Ansible on the Control Node

Let’s set up Ansible on the control node, i.e., installing Ansible on the Control node:

The command will automatically select the required essentials (Python and its associated dependencies, especially):

Here are the commands to install Ansible on other Linux distros:

Let’s check the installed version:

Step 2: Create an Inventory/Hosts File on the Control Node

The inventory file is by default located in the “/etc/ansible/hosts”. However, if it is not available, we can create it manually:

Here, the [main] is the group representing specific servers. Similarly, we can create multiple groups in the same pattern to access the servers and perform the required operation on the group as a whole.

Step 3: Install and Configure SSH on the Host Nodes

Ansible communicates with the host nodes via SSH. Now, we’ll set up SSH on the host nodes (managed nodes). The process in this “Step” is performed on all the “Managed Nodes”.

Let’s first install SSH on the system:

If you have managed nodes other than Ubuntu/Debian, you can use one of the following commands, as per your Linux distribution, to install SSH:

Since we have only one “Control Node”, for better security, we add a rule that only the SSH port can be accessed from the Control Node:

Note: If you have changed the SSH default port, then you have to mention the port name to open that specific port.

• Allow Specific IP on SSH Port: When configuring the Firewall on the Managed Nodes, you can only allow a specific IP to interact with the managed node on SSH. For instance, the command below will only allow the IP “192.168.140.142” to interact over the SSH port.

Let’s reload the firewall:

Confirming the firewall status:

Step 4: Create an Ansible User for Remote Connections

Let’s use the “adduser” command to create a new user for Ansible. The Control Node only communicates through the Ansible user:

Adding it to a sudo group:

Creating a no-password login for this user only. Open the “/etc/sudoers” file and add the following line at the end of the file:

Step 5: Set up SSH Key | Generate and Copy

Let’s generate the SSH keys on the control node:

Now, copy these keys to the remote hosts:

Note: There are multiple ways of generating and copying the SSH keys. Read our dedicated guide on “How to Set up SSH Keys” to have a detailed overview of how SSH keys work.

Step 6: Test the Connection | Control Node to Host Nodes

Once every step is performed error-free, let’s test the connection from the control node to the managed hosts. There are two ways to test the connection, i.e., one-to-one connection and one-to-many.

The Ansible command below uses its “ping” module to test the connection from the Control Node to one of the hosts, i.e., linuxhint.

Here, the following Ansible command pings all the hosts that a Control Node has to manage:

The success status paves the way to proceed further.

Ansible Playbooks | Automate Linux in Minutes

Ansible Playbook is an automated script that runs on the managed nodes (either all or the selected ones). Ansible Playbooks follow the YAML syntax that needs to be followed strictly to avoid any syntax errors. Let’s first have a quick overview of the YAML Syntax:

Prerequisites: Understanding the YAML Basics

YAML is the primary requirement for writing an Ansible playbook. Since it is a markup language thus its syntax must be followed properly to have an error-free playbook and execution. The main components of the YAML that need to be focused on at the moment to get started with Ansible playbooks are:

• Indentation → Defines the hierarchy and the overall structure. Only 2 spaces. Don’t use Tab.
• Key:Value Pairs → Defines the settings/parameters/states to assist the tasks in the playbook.
• Lists → In YAML, a list contains a series of actions to be performed. The list may act as an independent or can assist with any task.
• Variables → Just like other scripting/programming languages. The variables in YAML define dynamic values in a playbook for reusability.
• Dictionaries → Groups relevant “key:value” pairs under a single key, often for module parameters.
• Strings → Represents text values such as task names, messages, and optional quotes. The strings also have the same primary purpose, just like in other scripting/programming languages.

That’s what helps you write Ansible playbooks.

Variable File | To be used in the Ansible Playbook

Here, we will be using a variable file, which is used in the Playbook for variable calling/assignment. The content of the Vars.yml file is as follows:

There are three variables in this file, i.e., the package contains only one package, and the other two variables are “server_packages” and “other_utils,” which contain a group of packages.

Step 1: Create an Ansible Playbook

Let’s create a playbook file:

Here, the variables file named “vars.yml” is linked to this Playbook. At our first run, we will use the first variable named “package”:

Here:

• The “hosts: all” states that this playbook will be implemented on all the hosts listed in the hosts/inventory file.
• “become: yes” elevates the permissions, i.e., useful when running the commands that require root privileges.
• “Vars_file” calls the variable files.
• The “tasks” contain the tasks to be implemented in this playbook. There is only one task in this playbook:
  • The task is named “Install package”, with the “apt” module, and the variable “name” to be used from the variable file.

Step 2: Automate the Tasks

Before implementing this playbook, we can have a dry run of the playbook on all the servers to check for its successful execution. Here’s the command to do so:

Let’s run the newly created playbook with the created user:

Note: We can also provide the hosts/inventory file location (if it is not at the default location, i.e., /etc/ansible/) here as well, i.e., using the “-i” option and providing the path of the inventory file.

Similarly, we can use other variable groups mentioned in the variable file as well.

For instance, the following playbook now calls the “server_packages” variable and installs the server as per their availability:

Here, the “become: yes” is used for the root permissions. This is used when the tasks require root privileges. The task in this playbook utilizes different variables from the variable file.

Let’s dry-run the playbook on the managed nodes using the below command:

All green states that the playbook will be successfully implemented. Remove the “–check” flag from the above command to implement the playbook.

That’s all about the main course of this article. Since Ansible is backed up by a list of commands, we have compiled a list of commands necessary for beginners to understand while using Ansible.

Bonus: Ansible 101 Commands

Ansible is an essential automation server with a long list of its own commands to manage the overall server operations. Here’s the list of Ansible commands that would be useful for all those using Ansible or aiming to use Ansible in the future:

For a complete demonstration of the Ansible CLI Cheat Sheet, please see the Ansible documentation – Ansible CLI Cheat Sheet.

Conclusion

To get started with Ansible, first, install Ansible on one system (Control Node), then install and configure SSH on the remote hosts (Managed Nodes). Now, generate the SSH keys on the Control Node and copy the key to the Managed Nodes.

Once the connectivity is resolved, configure the inventory file and write the playbook. That’s it. The Ansible will be configured and ready to run.

All these steps are practically demonstrated in this guide. Just go through the guide and let us know if you have any questions or anything that is difficult to understand. We would assist with Ansible’s installation and configuration.

We have converted our text-based Docker course into an eBook; Learn Docker Quickly.

It is available for free to LHB Pro members along with all the other eBooks in the resources section.

If you are not a Pro member, you can either opt for the Pro membership or purchase just this ebook from our Gumroad page.

I am working on the next series and hopefully, you'll see it in July. Stay tuned 😄

Ever wondered how to make your bash scripts more robust and professional? The declare command in bash is your secret weapon for proper variable management!

Alright! So, variables in bash don't have any types and you can simply use them as name=value . That's not surprising.

What you might not know is that you can control variable types, scope, and behavior by using declare with your variables.

Interesting, right?

What is Declare in Bash?

The declare built-in command in bash allows you to explicitly declare variables with specific attributes. Think of it as giving your variables special properties that control how they behave.

The syntax for declare is simple:

declare [options] [variable[=value]]

If you use it without any options, it will be the same as regular variable assighnment.

# Simple variable assignment
name="John"

# Using declare (equivalent but more explicit)
declare name="John"

The magic happens with the options that define variable attributes, as they open up a world of possibilities! Stay with me to see the incredible power of this lesser known bash command.

Making variables read-only (-r)

Want to create constants that can't be accidentally modified? Use the -r flag:

declare -r var=value

Here's an example:

declare -r API_URL="https://api.example.com"
declare -r MAX_RETRIES=3

# This will fail with an error because API_URL is readonly variable
API_URL="https://malicious.com" 

Define integer variables (-i)

Force variables to behave as integers for mathematical operations in bash:

declare -i int_var=123

Here's a proper example:

declare -i counter=10
declare -i result

counter=counter+5    # Works without $ or (( ))
echo $counter        # Output: 15

result=counter*2     # Mathematical operations work directly
echo $result         # Output: 30

This is a good way to validate user input of your bash script.

declare -i user_input
read -p "Enter a number: " user_input

if [[ $user_input -eq 0 ]] && [[ "$user_input" != "0" ]]; then
    echo "Invalid input! Please enter a number."
else
    echo "You entered: $user_input"
fi

⚠️ Don't mix strings and integers. You won't see an error but you won't get the intended result as well.

declare -i number="abc"
echo $number           # Output: 0 (not an error!)

Use array variables (-a)

You can create indexed arrays explicitly with option -a:

declare -a array_var=("get" "LHB" "Pro" "Membership")

A better example that shows things in action:

declare -a fruits=("apple" "banana" "orange")
declare -a numbers

# Add elements
fruits[3]="grape"
numbers[0]=100
numbers[1]=200

echo ${fruits[@]}    # Output: apple banana orange grape
echo ${#fruits[@]}   # Output: 4 (array length)

⚠️ Beware of the index gap issues. In the example below, the element was added at 11th position but length of the array is still counted as 2. Basically, bash is not a high-level programming language. So, be careful of the pitfalls.

declare -a sparse_array
sparse_array[0]="first"
sparse_array[10]="eleventh"

echo ${#sparse_array[@]}    # Output: 2 (not 11!)

Declare associative arrays (-A)

I hope you are familiar with the concept of associative arrays in bash. Basically, Associative arrays let you create structured data with key-value pairs, offering a more flexible way to handle data compared to indexed arrays.

declare -A user_info
user_info["name"]="Alice"
user_info["age"]=30
user_info["role"]="developer"

echo ${user_info["name"]}           # Output: Alice
echo ${!user_info[@]}               # Output: name age role (keys)
echo ${user_info[@]}                # Output: Alice 30 developer (values)

Create exported variables (-x)

By default, the variables you create are not available to the child

Make variables available to child processes with option -x.

In the screenshot below, you can see that the exported variable with declare was available in the subshell while the normal variable was not.

This is useful when you have scripts with the variables that need to be available beyond the current shell. An example with pseudocode:

declare -x DATABASE_URL="postgresql://localhost/mydb"
declare -x -r CONFIG_FILE="/etc/myapp.conf"  # Read-only AND exported

# Now DATABASE_URL is available to any command you run
python my_script.py  # Can access DATABASE_URL

Unsetting attributes

You can also remove specific attributes from variables by using + in the option.

In the example below, I have the variable set as an integer first and then I change it to a string.

declare -i number=42
declare +i number          # Remove integer attribute
number="hello"             # Now this works (was previously integer-only)

Just letting you know that this option is also there if the situation demands it.

When and where should you use declare?

Use declare when you want to:

• Create constants with -r
• Work with arrays (-a or -A)
• Ensure variables are integers (-i)
• Make variables available to child processes (-x)
• Create more readable, self-documenting code

Stick with simple assignment when:

• Creating basic string variables
• Working with temporary values
• Writing quick one-liners

I can think of some practical, real-world use cases.

Let's say you are creating a script for system monitoring (pseudocode for example):

#!/bin/bash

# System thresholds (read-only)
declare -r -i CPU_THRESHOLD=80
declare -r -i MEMORY_THRESHOLD=85
declare -r -i DISK_THRESHOLD=90

# Current values (will be updated)
declare -i current_cpu
declare -i current_memory
declare -i current_disk

# Arrays for storing historical data
declare -a cpu_history
declare -a memory_history

# Function to check system resources
check_system() {
    current_cpu=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | cut -d'%' -f1)
    current_memory=$(free | grep Mem | awk '{printf("%.0f", $3/$2 * 100.0)}')
    
    # Add to history
    cpu_history+=($current_cpu)
    memory_history+=($current_memory)
    
    # Alert if thresholds exceeded
    if (( current_cpu > CPU_THRESHOLD )); then
        echo "⚠️  CPU usage high: ${current_cpu}%"
    fi
    
    if (( current_memory > MEMORY_THRESHOLD )); then
        echo "⚠️  Memory usage high: ${current_memory}%"
    fi
}

Or, configuration management of your web service:

#!/bin/bash

# Application configuration
declare -r APP_NAME="MyWebApp"
declare -r APP_VERSION="2.1.0"
declare -r CONFIG_DIR="/etc/myapp"

# Runtime settings (can be modified)
declare -i PORT=8080
declare -i MAX_CONNECTIONS=100

# Export for child processes
declare -x DATABASE_URL="postgresql://localhost/myapp"
declare -x LOG_LEVEL="INFO"

echo "Starting $APP_NAME v$APP_VERSION on port $PORT"

Wrapping Up

The declare command transforms bash from a simple scripting language into a more robust programming environment. It's not just about creating variables - it's about creating reliable, maintainable scripts that handle data properly.

That being said, there are still a few pitfalls you should avoid. I mentioned a few of them in the article but it is always good to double test your scripts.

Let me know if you have questions on the usage of declare shell built-in.

Bash (Bourne Again Shell) is a free and open-source shell and scripting language. Its journey started in the late 80s, and since then, the Bash has been adopted by routine Linux users and Linux SysAdmins.

Bash has automated the daily tasks of a Linux System Administrator. A Linux SysAdmin has to spend hours running scripts and commands. Not only the SysAdmins, but the simplicity and easy-to-learn capability of Bash have automated the tasks of a normal Linux user as well.

Inspired by this, we have today demonstrated the 10 most useful Bash scripts for Linux SysAdmins. These are chosen based on the general working of any Linux System Administrator (from a small scale to a larger scale).

• 10 Bash Scripts to Automate Daily Linux SysAdmin Tasks
• Prerequisite 1: Running a Bash Script | To be Done Before Running Each Script in This Post
• Prerequisite 2: Package Management Commands for Distros Other Than Debian/Ubuntu
• Script 1: Update and Upgrade the System Repositories/Packages Index
• Script 2: Install a Package on Linux
• Script 3: Remove a Package
• Script 4: Monitoring Systems Performance
• Script 5: Log Monitoring
• Script 6: User Management | Adding a New User, Adding a User to a Group
• Script 7: Disk Management
• Script 8: Service Management
• Script 9: Process Management
• Script 10: Allow or Deny Services Over the Firewall
• Bonus: Automating the Scripts using Cron
• Conclusion

10 Bash Scripts to Automate Daily Linux SysAdmin Tasks

A System Administrator can create as many scripts as required. We have automated some of the most common and most used tasks through Bash scripts. Let’s go through the prerequisites first and then the Scripts:

Prerequisite 1: Running a Bash Script | To be Done Before Running Each Script in This Post

Before we get into the scripts, let’s quickly go through the process to run a bash script.

Step 1: Make the Script Executable

A bash script is useless until it is made executable. Here, the scripts refer to the Linux sys admin, so we use the “u+x” with “sudo” to make the scripts executable for admin only:

Step 2: Execute the Script

Once the script is executable, it can now be run from the terminal using the command:

Click here to get more details on running a Bash script.

Prerequisite 2: Package Management Commands for Distros Other Than Debian/Ubuntu

To assist with Script 1, Script 2, and Script 3, we prepared a command cheat sheet for managing the packages on Linux distros other than Debian/Ubuntu and their derivatives. Here’s the table that lists the commands referring to each package manager of the Linux distro:

Script 1: Update and Upgrade the System Repositories/Packages Index

“Update and upgrade” commands are the most used commands by any Linux SysAdmin or a routine user.

Here, the below script updates, upgrades, and autoremoves packages:

Note: Please refer to the table (Prerequisites 2) for Linux package management commands.

Let’s make it executable:

Permission Denied: Since the script belongs to the SysAdmin, we strictly kept the permissions to the sudo user only:

Here’s the update, upgrade, and autoremoval of packages:

Script 2: Install a Package on Linux

A Linux SysAdmin has to install and remove packages from the systems and keep an eye on this process. Each package installation requires a few commands to effectively install that package.

Note: Please refer to the table (Prerequisites 2) for Linux package management commands.

Update and upgrade package repositories, followed by installing a specific package ($1, specify the package name while running the script):

Here, we choose $1=ssh and run the script:

Script 3: Remove a Package

A complete removal of a package involves multiple commands. Let’s manage it through a single script:

Note: Go through the table (Prerequisites 2) for the commands of other Linux package managers:

Let’s execute it, i.e.,”$1=ssh”:

Script 4: Monitoring Systems Performance

A Linux sysadmin has to monitor and keep an eye on measurable components (CPU, RAM) of the system. These preferences vary from organization to organization.

Here’s the Bash script that checks the RAM status, Uptime, and CPU/memory stats, which are the primary components to monitor:

• free -h: RAM status in human-readable form.
• uptime: how long the system has been running.
• vmstat 2: Live CPU/memory stats, i.e., records every 2 seconds.

Once we run the command, the output shows the “Ram Status”, the “Uptime”, and the “CPU/Memory” status:

Script 5: Log Monitoring

A Linux SysAdmin has to go through different log files to effectively manage the system. For instance, the “/var/log/auth.log” file contains the user logins/logouts, SSH access, sudo commands, and other authentication mechanisms.

Here’s the Bash script that allows filtering these logs based on a search result.

The $1 positional parameter shows that this script would be run with one argument:

We use “UID=0” as the variable’s value for this script. Thus, only those records are shown that contain UID=0:

The log file can be changed in the script as per the requirement of the SysAdmin. Here are the log files associated with different types of logs in Linux:

There are a few more. Let’s open the “/var/log” directory and look at the logs that SysAdmin can use for fetching details inside each log file:

Script 6: User Management | Adding a New User, Adding a User to a Group

Adding a new user is one of the key activities in a Linux sysadmin’s daily tasks. There are numerous ways to add a new user with a Bash script. We have created the following Bash Script that demonstrates the user creation:

• 2 positional parameters are initialized, i.e., $1 for user and $2 for group.
• First, the required group is created. Then, the user is created. Lastly, the newly created user is added to the group.

Since the script has positional parameters, let’s execute it with the required 2 arguments (one for username and the other for groupname):

Similarly, the system administrator can create scripts to delete users as well.

Script 7: Disk Management

Disk management involves multiple commands, such as listing and checking the number of block devices. We run the “lsblk” command. To “mount” or “unmount” any filesystem, we run the “mount” and “umount” commands.

Let’s incorporate a few commands in a Bash script to view some data about disks:

$1 positional parameter refers to the address of the directory whose disk usage is to be checked:

Let’s run the script:

Also, remember to provide the argument value, i.e., here, the “$1=/home/adnan/Downloads”:

Script 8: Service Management

To manage any service, the SysAdmin has to run multiple commands for each service. Like, to start a service, the SysAdmin uses the “systemctl start” command and verifies its status through “systemctl status”. Let’s make this task easy for Linux SysAdmins:

Start a Service

The following Bash script refers to only one service, i.e., every time the script only manages the NGINX service:

For a more diverse use case, we declare a positional parameter to manage different services with each new run:

Now, pass the value of the positional parameter at the time of executing the script:

The “apache2” is the argument on which the script would run:

Stop a Service

In this case, we use the positional parameter to make it more convenient for the Linux SysAdmins or the regular users:

The $1 positional parameter refers to the specific service that is mentioned when executing a command:

Let’s execute the command:

Script 9: Process Management

A Linux System Administrator has a keen eye on the processes and manages each category of process as per the requirement. A simple script can kill the specific processes. For instance, the script demonstrated here fetches the Zombie and Defunct processes, identifies the parent IDs of these processes:

• Zombie and Defunct process IDs are fetched and stored in a variable.
• The parent process IDs of the Zombie and defunct processes are fetched.
• Then, the parent processes can be killed

Let’s execute it:

Script 10: Allow or Deny Services Over the Firewall

A firewall is a virtual wall between your system and the systems connecting to your system. We can set the firewall rules to allow or deny what we want. Firewall has a significant role in managing the system. Let’s automate to allow or deny any service on your system:

Allow a Service Through the Firewall

The following script enables SSH through the firewall:

Let’s execute the script.

We can also include a positional parameter here to use the same script for multiple services to be allowed on the firewall. For instance, the script below has only one positional parameter. This parameter’s value is to be provided at the time of executing the script.

While executing, just specify the name of the service as an argument:

Deny a Service or Deny All:

We can either deny one service or deny all the services attempting to reach our system. The below script updates the default incoming policy to deny, disables the firewall as well.

Note: These kinds of denial scripts are run when the overall system is in trouble, and we just need to make sure there is no service trying to approach our system.

Running the script:

Now that you have learned the 10 Bash scripts to automate daily SysAdmin tasks.

Let’s learn how we can schedule the scripts to run them as per our schedule.

Bonus: Automating the Scripts Using Cron

A cron job allows the SysAdmin to execute a specific script at a specific time, i.e., scheduling the execution of the script. It is managed through the crontab file.

First, use the “crontab -e” command to enter the edit mode of the crontab file:

To put a command on a schedule with the cron file, you have to use a specific syntax to put it in the cron file. The below script will run on the 1st minute of each hour.

There are a total of 5 parameters to be considered for each of the commands:

• m: minute of a specific hour, i.e., choose between 1-59 minutes.
• h: hour of the day, i.e., choose between 0-23.
• dom: date of the month → Choose between 1-31.
• mon: foreach month → Choose between 1- 12
• dow: day of the week → Choose between 1-7

You can check the crontab listings using:

Important: Do you want a Linux Commands Cheat Sheet before you start using Bash? Click here to get a detailed commands cheat sheet.

Conclusion

Bash has eased the way of commanding in Linux. Usually, we can run a single command each session on a terminal. With Bash scripts, we can automate the command/s execution process to accomplish tasks with the least human involvement. We have to write it once and then keep on repeating the same for multiple tasks.

With this post, we have demonstrated the 10 Bash scripts to automate daily Linux System Administrator.

FAQs

How to run a Bash script as an Admin?

Use the “sudo /path/to/script” command to run the Bash script as an Admin. It is recommended to restrict the executable permissions of the Bash scripts to only authorized persons.

What is #!/bin/bash in Bash?

The “#!/bin/bash” is the Bash shebang. It tells the system to use the “bash” interpreter to run this script. If we don’t use this, our script is a simple shell script.

How do I give permissions to run a Bash script?

The “chmod” command is used to give permissions to run the Bash script. For a Linux sysadmin script, use the “sudo chmod u+x /path/of/script” command to give permissions.

What does $1 mean in Bash?

In Bash, $1 is a positional parameter. While running a Bash script, the first argument refers to $1, the second argument refers to $2, and so on.

Basic Workflow of Ansible | What components are necessary

Ansible Control Node IP: 192.168.140.139 (Where Ansible is configured)

Ansible Host IPs: {

Server 1 [172.17.33.7]

Server2 [192.168.18.140]

}

Inventory File:

Default inventory file location: /etc/ansible/hosts. Usually, it is not available when we install Ansible from the default repositories of the distro, so we need to create it anywhere in the filesystem. If we create it in the default location, then no need to direct Ansible to the location of the file.

However, when we create the inventory file other than the default, we need to tell Ansible about the location of the inventory file.

Inventory listing (Verifying the Inventory Listing):

SSH (as it is the primary connection medium of Ansible with its hosts):

Allow port 22 through the firewall on the client side:

Let’s check the status of the firewall:

Step 2: Establish a No-Password Login on a Specific Username | At the Host End

Create a new dedicated user for the Ansible operations:

Adding the Ansible user to the sudo group:

Add the user to the sudo group (open the sudoers file):

SSH Connection (From Ansible Control Node to one Ansible Host):

SSH key generation and copying the public key to the remote host:

Note: Copy the public key to the user that you will be using to control the hosts on various machines.

Test All the Servers Listed in the Inventory File:

Testing the Ansible Connection to the Ansible host (remember to use the username who is trusted at the host or has a passwordless login). I have the user “adnan” as the trusted user in the Ansible user list.

Same with the different username configured on the host side:

We can ping a specific group, i.e., in our case, we have a group named [servers] in the inventory.

Here’s your curated dose of Linux news, tutorials, and updates to keep you informed and productive in your open-source journey.

• Find with exec
• Named and unnamed pipes in Linux
• Container lifeycycle commands
• Clickhouse and Dockman
• And regular dose of important news, tips and memes

Another week, another chance to pretend you're fixing something important by typing furiously in the terminal. You do that, right? Or is it just me? 😉

This week's highlights are:

• lsattr, chatter and grep commands
• brace expansions
• VERT converter
• And your regular dose of news, memes and tips

❇️ Explore DigitalOcean with $100 free credit

DigitalOcean is my favorite alternative to the likes of AWS and Azure and Google Cloud. I use it to host Linux Handbook and pretty happy with their performance and ease of deployment. Try their servers and marketplace apps for free with $100 credit which is applicable to new accounts.

DigitalOcean – The developer cloud

Helping millions of developers easily build, test, manage, and scale applications of any size – faster than ever before.

Explore our products

Note-taking has come a long way from crumpled sticky notes and scattered .txt files. Today, we want our notes to be searchable, linked, visualized, and ideally, available anywhere. That’s where Obsidian shines.

Built around plain-text Markdown files, Obsidian offers local-first knowledge management with powerful graph views, backlinks, and a thriving plugin ecosystem.

For many, it has become the go-to app for personal knowledge bases and second brains.

While Obsidian does offer Obsidian Sync, a proprietary syncing service that lets you keep your notes consistent across devices, it’s behind a paywall.

That’s fair for the convenience, but I wanted something different:

A central Obsidian server, running in my homelab, accessible via browser, no desktop clients, no mobile apps, just one self-hosted solution available from anywhere I go.

And yes, that’s entirely possible.

Thanks to LinuxServer.io, who maintain some of the most stable and well-documented Docker images out there, setting this up was a breeze.

I’ve been using their containers for various services in my homelab, and they’ve always been rock solid.

Let me walk you through how I deployed Obsidian this way.

Prerequisites

We assume you have:

• A Linux system with Docker and Docker Compose installed.
• A basic understanding of terminal commands.
• Familiarity with editing YAML files.

Setting up Obsidian

If you prefer keeping your self-hosted apps neatly organized (like I do), it's a good idea to create separate folders for each container.

This not only helps with manageability, but also makes it easier to back up or migrate later.

1. Create a data directory for Obsidian

Let’s start by creating a folder for Obsidian data:

mkdir -p ~/docker/obsidian
cd ~/docker/obsidian

You can name it whatever you like, but I’m sticking with obsidian to keep things clear.

2. Create a docker-compose.yml File

Now, we’ll set up a Docker Compose file, this is the file that tells Docker how to run Obsidian, what image to use, which ports to open, and other important stuff.

You don’t need to write the whole thing from scratch. I’m using the official example from the LinuxServer.io image page, but with a few changes tailored to my system.

Just copy the following into a new file named docker-compose.yml:

version: "3.8"
services:
  obsidian:
    image: ghcr.io/linuxserver/obsidian:latest
    container_name: obsidian
    security_opt:
      - no-new-privileges:false
      - seccomp:unconfined
    healthcheck:
      test: timeout 10s bash -c ':> /dev/tcp/127.0.0.1/3000' || exit 1
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 90s
    ports:
      - "3000:3000"
    shm_size: "2gb"
    volumes:
      - ./config:/config:rw
    environment:
      CUSTOM_USER: yourusername
      PASSWORD: yourpassword
      PUID: 1000
      PGID: 1000
      TZ: Asia/Kolkata
    restart: unless-stopped

Let’s break down a few important parts of this:

• image: We're using the latest Obsidian image provided by LinuxServer.io.
• volumes: Maps a config folder in your current directory to Obsidian’s internal config directory, this is where all your Obsidian data and settings will live.
• ports: The app will be available on port 3000 of your machine. You can change this if you prefer a different port.
• shm_size: Allocates shared memory; useful for apps with a UI like Obsidian.
• environment: This is where you set up your user, password, timezone, and file ownership.

Make sure you replace the following placeholders with your own values:

• yourusername: The username you'll use to log in to Obsidian.
• yourpassword: Choose a strong password.
• TZ: Use your local timezone. (Example: Asia/Kolkata)
• PUID and PGID: These should match your user’s UID and GID on the host system. To find them, run:

id yourusername

You'll get something like this:

uid=1000(yourusername) gid=1000(yourusername) groups=1000(yourusername),27(sudo),...

Use those values in your Compose file.

3. Deploy the Container

Once the docker-compose.yml file is ready and the values are customized, go ahead and start the container:

docker-compose up -d

This command tells Docker to:

• Pull the Obsidian image (if it’s not already downloaded)
• Create a container using the settings we defined
• Run it in detached mode (-d), so it continues running in the background

Give it a minute or two, the first time you run this, Docker needs to download the entire image and set everything up. After that, it’ll be much faster on subsequent restarts.

Accessing Obsidian in your browser

Once it's done, you should be able to open Obsidian in your browser at:

http://localhost:3000

Or replace localhost with your server's IP if you’re not running it locally.

Log in using the CUSTOM_USER and PASSWORD you set earlier.

Once inside, it will look like this:

Here you can:

• Create a new vault.
• Open an existing vault in the config volume.
• Explore the graph view, plugins, and everything else, right from the browser.

Creating new vault

For this tutorial, we’ll keep things simple, I’m just going to create a new vault to get started.

Click on "Create", give your vault a name (anything you like - "secondbrain", "mynotes", "vault", etc.), and Obsidian will take care of the rest.

It’ll create a new folder inside the mounted config directory we set up in Docker earlier. This means all your notes and settings will be saved persistently on your machine, even if the container is stopped or restarted.

After you name and create the vault, Obsidian will drop you straight into the note-taking interface. And that’s it, you’re in!

You can now start writing notes, creating folders, and playing around with features like:

• Graph view to visualize links between notes
• Command palette to quickly access features
• Themes and plugin settings to customize your environment

Everything is accessible from the left sidebar, just like in the desktop app. No extra setup needed, just start typing and let your ideas flow.

Final thoughts

Setting up Obsidian inside Docker was surprisingly easy, it didn’t take much time, and before I knew it, I had the full desktop-like experience running in my browser.

This setup is especially great for people on the go or students like me who love using Obsidian but can’t always afford the Sync feature just yet.

Now, I personally don’t mind paying for good software and I think Obsidian Sync is a solid service but those little costs start stacking up fast.

I’ve also seen quite a few Reddit threads where folks have built their own syncing setups using Syncthing to keep notes in sync across devices, and that seems like a solid workaround as well.

For me, this self-hosted browser version of Obsidian fits somewhere in the middle. It gives you the full experience without the limitations of a mobile app or the need to sync through someone else’s servers.

And if you're already in the self-hosting ecosystem, it’s just another powerful tool you can add to your stack.

Lesser known... that's the theme of this week's newsletter. Hope you like it 😄

Here are the highlights of this edition :

• Lesser known mouse mode in Vim
• Lesser known dir command in Linux
• Lesser known special file permissions
• And your regular dose of better known memes, tips and news ;)

🚀 Level up your coding skills and build your own bots

Harness the power of machine learning to create digital agents and more with hot courses like Learning LangChain, The Developer's Playbook for Large Language Model Security, Designing Large Language Model Applications, and more.

Part of the purchase goes to Code for America! Check out the ebook bundle here.

Humble Tech Book Bundle: Machine Learning, AI, and Bots by O’Reilly 2025

Master machine learning with this comprehensive library of coding and programming courses from the pros at O’Reilly.

Humble Bundle

Think of Vim tabs like browser tabs for your code editor - each tab holds one or more windows, letting you organize multiple files into logical workspaces.

Unlike window splits that divide your screen, tabs stack contexts you can flip between instantly.

Let's see how you can use tabs in Vim.

Essential Vim tab commands at a glance

Here are the most common actions you can use while dealing with tabs in Vim.

Interesting, right? Let's see it in details.

Opening files in tabs in Vim

Let's start by opening files in tabs first.

Start Vim with multiple files opened in tabs

Launch Vim with multiple tabs instantly:

vim -p file1.py file2.py file3.py

How can you open just one file in tab? Well... if it's one file, what's the point of tab, right?

Open a file in a new tab in the current Vim session

When you are already inside Vim and want to open a file in a new tab, switch to normal mode by pressing Esc key and use the command:

:tabnew filename

This will load the file in a new tab. If the file doesn't exist, it will create a new one.

Filename is optional. If you don't provide it, it will open a new file without any name:

:tabnew

Search for files and open them in tabs

Search the current directory for filename matching the given pattern and open it in a new tab:

:tabf filename*

This only works if the search results into a single file. If there are more than one file matched, it will throw an error:

E77: Too many file names

Navigating between tabs

You can move between opened tabs using:

• :tabn: for next tab
• :tabp: for previous tab

Typing the commands could be tedious, so you can use the following key combinations in the nomral mode:

• gt: To go to the next tab
• gT (i.e. press g and shift and t keys together) To go to the previous tab

If there are too many tabs opened, you can use:

• :tabfirst: Jump to first tab
• :tablast: Jump to last tab

In many distributions these days, Vim is preconfigured to show the tab labels on the top. If that's not the case, add this to your vimrc:

set showtabline=2

You can list all the opened tabs with:

:tabs

Closing tabs

How do you close a tab? If the tab has a single filed opened, the regular save/exit Vim commands work.

But it will be an issue if you have multiple split windows opened in a tab.

• :tabclose: Close current tab
• :tabonly: Only keep the current tab opened, close all others

Bulk tab operations

With :tabdo command, you can run the same operations in all the tabs.

For example, :tabdo w will save file changes in all tabs, :tabdo normal! gg=G auto-indents every file.

Similarly, tabdo %s/oldvar/newvar/g executes search-replace across every tab simultaneously. Parallel processing for repetitive changes.

You get the idea. tabdo is the key here.

Conclusion

While it is good to enjoy tabs in Vim, don't create dozens of them - they become harder to navigate than helpful. Use buffers for file switching and tabs for context switching.

As you can see, with the tab feature, you get one inch closer to having the IDE like experience in Vim.

This tool lets you generate the SSH config entries quickly. Fill the fields and hit the generate button and copy the entries to your SSH config file.

With the ~/.ssh/config file, you can save the details about various servers you regularly connect to via SSH.

It generates one set of entries for a server at a time. If you have multiple servers, you need to manually add the generated entries one after another in ~/.ssh/config file.

Learn more about using SSH config file in detail.

How to Use SSH Config File [Beginner’s Guide]

Using SSH profiles can help you in cases where you regularly connect to various servers. No need to remember the IP address and other such details for SSH connection.

Linux HandbookAbhishek Prakash

I hope I am not committing blasphemy but you can use the mouse in Vim.

Press Esc to go to command mode in Vim and use:

:set mouse=a

It will enable mouse mode immediately in all Vim modes, i.e. normal, insert and visual. To disable the mouse mode, use this:

:set mouse=

If you want to use mouse mode all the time, I am not judging you, add this entry to your ~/.vimrc file:

set mouse=a

Save, restart Vim, and your mouse now works for clicking, scrolling, and selecting text.

What does mouse mode do in Vim?

First thing first, the mouse mode has limited usage in Vim. Don't expect to get a notepad like experience just because you enabled mouse mode.

It does add some functionality and ease of use like:

• Jump to exact locations without counting lines or using search
• Wheel scrolling while dealing with long files
• Easy text selection: double click to select a word, triple click to select an entire line, use mouse click and drag for the desired text selection
• Click between panes and resize them easily while using split windows in Vim
• While using tabs, click on other tabs to switch, click on X (should be visible at top right in mouse mode) to close a tab, double click on tab bar to create new empty files in new tabs
• Use middle click to paste from system clipboard

The video below shows some of the above discussed featured in action:

Mouse mode doesn't replace Vim's keyboard efficiency - it complements it. Think of it as having both a sports car's manual controls and cruise control available.

Understanding mouse mode Options

Vim's mouse configuration works like a permission system:

set mouse=a     " All modes (normal, visual, insert, command)
set mouse=n     " Normal mode only
set mouse=v     " Visual mode only  
set mouse=i     " Insert mode only
set mouse=c     " Command mode only
set mouse=nv    " Normal and visual modes

Mostly, you would want mouse=a - which is like having universal access rather than mode-specific restrictions.

Conclusion

I am not a Vim purist and I won't judge you if you use mouse in Vim, which is known for its keyboard-centric approach to productivity.

Mouse mode doesn't diminish Vim's keyboard efficiency - it provides additional options for specific scenarios.

If you feel too uncomfortable with Vim at the beginning, opt for the hybrid approach where you use mouse for navigation and positioning, keyboard for editing operations.

I welcome your comments.

Windows Subsystem for Linux (WSL) allows you to run Linux distros within the Windows operating system. WSL is available in multiple versions: WSL1 (older but still supported) and WSL2 (newer with continuous development support).

Recently, on May 19, 2025, Microsoft conducted a 2025 Build Conference where they announced the open-source nature of WSL for Windows, which is huge news for Linux users and the open-source community.

In today’s guide, I will elaborate on the recent development in the WSL project, the open-source nature of the WSL, how mature its open-source nature is, and Microsoft’s journey towards open-source. So, let’s get started:

• Windows Subsystem for Linux (WSL) is now Open Source
• Road to Open Source WSL
• How can I Contribute to Open-Source WSL?
  • Basic Understanding of the WSL Architecture
  • Bonus: Keep Your WSL Updated
  • Conclusion

Windows Subsystem for Linux (WSL) is now Open Source

Yes, you heard that right, Microsoft has recently declared WSL as an open-source component of Windows. Microsoft has officially released the source code of WSL on May 19, 2025, at the Microsoft Build 2025 Conference.

As per the announcement of Microsoft, the WSL is completely open-sourced except for the following three components:

• “Lxcore.sys”: Kernel-side driver that powers WSL1, i.e., component that translates Linux commands for Windows in WSL1.
• “P9rdr.sys” and “p9n9.dll”: These 9P protocol components refer to the filesystem redirection from Windows to Linux.

The above three components are tightly integrated with Windows core processes. For instance, the “Lxcore.sys” has to interact with the Windows kernel. Similarly, the 9P protocol components handle Windows files and directories. These deep integrations of WSL’s components with Windows might be the reasons behind keeping them closed-source.

This was about Microsoft’s recent announcement. Now, let me take you through Microsoft’s path towards open-source WSL.

Road to Open Source WSL

In 2019, Microsoft started maintaining its own Linux Kernel specifically for WSL, and it was declared an open-source component.

Since then, the WSL community has started growing and contributing to the WSL project. Later on, the WSLg component became open source, which provides the graphical applications support for the Linux applications in the WSL.

Microsoft gradually moved WSL to an open-source project. In 2021, Microsoft separated the Windows code base from the WSL code base. The newly released WSL was then available on the Microsoft Store for Windows 11 users. However, in 2022, it was made available for Windows 10 users too.

Apart from that, the development and upgradation of WSL keep going, i.e., the latest available WSL is “2.5.7.0”.

How can I Contribute to Open-Source WSL?

Microsoft has provided the source code Git Repo of WSL, where you can contribute to reporting the bugs/issues or developing the source code from scratch. A Linux enthusiast may love to look into the source code and contribute according to their interest.

If you are facing any technical issues relevant to WSL, you can ask for assistance on WSL’s Git repo. Similarly, if you are a regular user, you must keep on visiting the WSL repo and contribute to any other issues of the users in the comments.

Basic Understanding of the WSL Architecture

Since two different operating systems are collaborating, the component functionalities would differ in both environments. Let’s have a bird’s eye view of the WSL architecture:

Windows Host: The components hosted on the Windows side are the entry points to interact with the WSL.

• wsl.exe: Command line utility to launch or manage the WSL.
• wslconfig.exe: This mostly belongs to WSL 1 and is used to configure WSL 1.
• wslg.exe: Launch and manage the GUI apps from Windows.
• wslservice.exe: This is the background process, i.e., launch the VM, start/stop the distros attached, mount the filesystem, etc.

WSL VM Service: These components work inside the WSL VM and are triggered when the user executes Windows components.

• init: The initial process that starts in the Linux environment. It initializes the necessary processes and manages the distribution at the WSL end.
• gns (Networking): Manages DNS and network integration with Windows.
• localhost (port forwarding): The localhost running inside the Linux VM is the port forwarding.
• Plan9 Server: The server is responsible for file sharing from Windows to Linux.

Bonus: Keep Your WSL Updated

When you install WSL from the command line, it might fetch and install a bit lower version. Or even if it is up to date as per your installation, you must check for updates and get the latest available version:

Before updating:

Updating WSL:

Both the Linux Kernel and WSL versions are updated:

That’s all from this post.

Conclusion

On May 19, 2025, Microsoft announced the WSL as an open-source project. Not 100% open-source, yet a major advancement in the open-source world. Microsoft has long been working on this. Microsoft has held three components closed-source because of their deep integration with the Windows core processes.

In today’s post, I have talked about the open-source nature of WSL and Microsoft’s path to open-source WSL development.

FAQs:

Q1: Is WSL completely open source now?

Yes, except for only three components, which are “Lxcore.sys”, “P9rdr.sys”, and “p9n9.dll”.

Q2: How can I contribute to WSL?

Microsoft has provided the source code Git Repo of WSL, where you can contribute to reporting the bugs/issues or developing the source code from scratch.

Q3: Is WSL a replacement for Linux?

WSL cannot be referred to as a full replacement for Linux. WSL is just a subsystem within Windows.

Q4: Can WSL open a GUI?

Yes, WSL can open a Linux GUI application with the help of WSLg, i.e., introduced in 2021 for Windows 11 and in 2022 for Windows 10.

Time to practice your Bash script and develop a simple game.

Exercise

Create a bash script that implements a number guessing game. The script should:

1. Generate a random number between 1 and 100
2. Allow the user to make guesses
3. Provide feedback whether the guess is too high, too low, or correct
4. Count the number of attempts
5. Allow the user to play again after winning

The game should continue until the user guesses correctly, and then ask if they want to play another round.

💡 Hints

• Use $RANDOM to generate random numbers
• The modulo operator % can help limit the range
• Use a while loop for the main game logic
• Consider using a nested loop structure for replay functionality
• Input validation is important - check if the user enters a valid number

Test data

Test Case 1: Basic game flow

$ ./guessing_game.sh
Welcome to the Number Guessing Game!
I'm thinking of a number between 1 and 100.

Enter your guess: 50
Too low! Try again.
Enter your guess: 75
Too high! Try again.
Enter your guess: 62
Too low! Try again.
Enter your guess: 68
Congratulations! You guessed it in 4 attempts!

Do you want to play again? (yes/no): no
Thanks for playing! Goodbye!

Test Case 2: Input validation

$ ./guessing_game.sh
Welcome to the Number Guessing Game!
I'm thinking of a number between 1 and 100.

Enter your guess: abc
Invalid input! Please enter a number between 1 and 100.
Enter your guess: 150
Your guess must be between 1 and 100!
Enter your guess: 0
Your guess must be between 1 and 100!
Enter your guess: 42
Too high! Try again.

Test Case 3: Multiple rounds

$ ./guessing_game.sh
Welcome to the Number Guessing Game!
I'm thinking of a number between 1 and 100.

Enter your guess: 30
Congratulations! You guessed it in 1 attempts!

Do you want to play again? (yes/no): yes

Starting new game...
I'm thinking of a number between 1 and 100.

Enter your guess: 50
Too low! Try again.
Enter your guess: 60
Congratulations! You guessed it in 2 attempts!

Do you want to play again? (yes/no): no
Thanks for playing! Goodbye!

Solution 1: Basic implementaion

This is a straightforward implementation using nested while loops. The outer loop controls whether the player wants to play again, while the inner loop handles the guessing game logic. Key features:

• Uses $RANDOM % 100 + 1 to generate numbers between 1-100
• Input validation with regex pattern matching
• Simple counter for tracking attempts

#!/bin/bash

# Number Guessing Game - Basic Implementation

# Main game loop
play_again="yes"

echo "Welcome to the Number Guessing Game!"

while [[ "$play_again" == "yes" ]]; do
    # Generate random number between 1 and 100
    secret_number=$((RANDOM % 100 + 1))
    attempts=0
    guessed=false
    
    echo "I'm thinking of a number between 1 and 100."
    echo
    
    # Game loop for current round
    while [[ "$guessed" == "false" ]]; do
        # Read user input
        echo -n "Enter your guess: "
        read guess
        
        # Validate input - check if it's a number
        if ! [[ "$guess" =~ ^[0-9]+$ ]]; then
            echo "Invalid input! Please enter a number between 1 and 100."
            continue
        fi
        
        # Check if guess is in valid range
        if [[ $guess -lt 1 || $guess -gt 100 ]]; then
            echo "Your guess must be between 1 and 100!"
            continue
        fi
        
        # Increment attempt counter
        ((attempts++))
        
        # Check the guess
        if [[ $guess -eq $secret_number ]]; then
            echo "Congratulations! You guessed it in $attempts attempts!"
            guessed=true
        elif [[ $guess -lt $secret_number ]]; then
            echo "Too low! Try again."
        else
            echo "Too high! Try again."
        fi
    done
    
    # Ask if player wants to play again
    echo
    echo -n "Do you want to play again? (yes/no): "
    read play_again
    play_again=$(echo "$play_again" | tr '[:upper:]' '[:lower:]')
    
    if [[ "$play_again" == "yes" ]]; then
        echo
        echo "Starting new game..."
    fi
done

echo "Thanks for playing! Goodbye!"

Solution 2: Using functions

In this solution, I break down the game into modular functions, making the code more organized and reusable:

• generate_random(): Generates the secret number
• validate_input(): Handles all input validation
• play_round(): Contains the core game logic
• ask_replay(): Manages the replay prompt
• main(): Orchestrates the overall game flow

#!/bin/bash

# Number Guessing Game - Function-based Implementation

# Function to generate random number
generate_random() {
    echo $((RANDOM % 100 + 1))
}

# Function to validate input
validate_input() {
    local input=$1
    
    # Check if input is a number
    if ! [[ "$input" =~ ^[0-9]+$ ]]; then
        echo "Invalid input! Please enter a number between 1 and 100."
        return 1
    fi
    
    # Check if number is in range
    if [[ $input -lt 1 || $input -gt 100 ]]; then
        echo "Your guess must be between 1 and 100!"
        return 1
    fi
    
    return 0
}

# Function to play one round
play_round() {
    local secret_number=$(generate_random)
    local attempts=0
    local guess
    
    echo "I'm thinking of a number between 1 and 100."
    echo
    
    while true; do
        # Get user input
        echo -n "Enter your guess: "
        read guess
        
        # Validate input
        if ! validate_input "$guess"; then
            continue
        fi
        
        # Increment attempts
        ((attempts++))
        
        # Check guess
        if [[ $guess -eq $secret_number ]]; then
            echo "Congratulations! You guessed it in $attempts attempts!"
            break
        elif [[ $guess -lt $secret_number ]]; then
            echo "Too low! Try again."
        else
            echo "Too high! Try again."
        fi
    done
}

# Function to ask for replay
ask_replay() {
    local response
    echo
    echo -n "Do you want to play again? (yes/no): "
    read response
    response=$(echo "$response" | tr '[:upper:]' '[:lower:]')
    
    [[ "$response" == "yes" ]]
}

# Main program
main() {
    echo "Welcome to the Number Guessing Game!"
    
    while true; do
        play_round
        
        if ! ask_replay; then
            break
        fi
        
        echo
        echo "Starting new game..."
    done
    
    echo "Thanks for playing! Goodbye!"
}

# Run the main program
main

Solution 3: Enhanced version with difficulty levels

I added a few extra features:

• Difficulty levels with different number ranges
• Performance feedback based on attempts
• Hints when guesses are very far off
• Better user experience with clear menu options

#!/bin/bash

# Number Guessing Game - Enhanced Version with Difficulty Levels

# Function to display menu
show_menu() {
    echo "Select difficulty level:"
    echo "1. Easy (1-50)"
    echo "2. Medium (1-100)"
    echo "3. Hard (1-200)"
    echo -n "Enter your choice (1-3): "
}

# Function to get range based on difficulty
get_range() {
    case $1 in
        1) echo 50 ;;
        2) echo 100 ;;
        3) echo 200 ;;
        *) echo 100 ;;  # Default to medium
    esac
}

# Main game
echo "Welcome to the Number Guessing Game!"
echo

play_again="yes"

while [[ "$play_again" == "yes" ]]; do
    # Select difficulty
    show_menu
    read difficulty
    
    # Validate difficulty selection
    if ! [[ "$difficulty" =~ ^[1-3]$ ]]; then
        echo "Invalid choice! Using medium difficulty."
        difficulty=2
    fi
    
    # Get range for selected difficulty
    max_number=$(get_range $difficulty)
    
    # Generate random number
    secret_number=$((RANDOM % max_number + 1))
    attempts=0
    
    echo
    echo "I'm thinking of a number between 1 and $max_number."
    echo
    
    # Game loop
    while true; do
        echo -n "Enter your guess: "
        read guess
        
        # Validate input
        if ! [[ "$guess" =~ ^[0-9]+$ ]]; then
            echo "Invalid input! Please enter a number."
            continue
        fi
        
        if [[ $guess -lt 1 || $guess -gt $max_number ]]; then
            echo "Your guess must be between 1 and $max_number!"
            continue
        fi
        
        ((attempts++))
        
        # Check guess
        if [[ $guess -eq $secret_number ]]; then
            echo "Congratulations! You guessed it in $attempts attempts!"
            
            # Give performance feedback
            if [[ $attempts -le 5 ]]; then
                echo "Excellent! You're a mind reader!"
            elif [[ $attempts -le 10 ]]; then
                echo "Good job! That was quick!"
            else
                echo "Well done! Practice makes perfect!"
            fi
            break
        elif [[ $guess -lt $secret_number ]]; then
            echo "Too low! Try again."
            # Give hint for very far guesses
            if [[ $((secret_number - guess)) -gt $((max_number / 4)) ]]; then
                echo "(Hint: You're quite far off!)"
            fi
        else
            echo "Too high! Try again."
            # Give hint for very far guesses
            if [[ $((guess - secret_number)) -gt $((max_number / 4)) ]]; then
                echo "(Hint: You're quite far off!)"
            fi
        fi
    done
    
    # Ask for replay
    echo
    echo -n "Do you want to play again? (yes/no): "
    read play_again
    play_again=$(echo "$play_again" | tr '[:upper:]' '[:lower:]')
    
    if [[ "$play_again" == "yes" ]]; then
        echo
        echo "Starting new game..."
        echo
    fi
done

echo "Thanks for playing! Goodbye!"

📖 Concepts to revise

The solutions discussed here use some terms, commands and concepts and if you are not familiar with them, you should learn more about them.

• Variables: secret_number, attempts, guess
• Arithmetic operations: $RANDOM % 100 + 1, numeric comparisons
• Conditional statements: if/elif/else for guess validation and checking
• Loops: while loops, nested loops, continue statement
• Regular expressions: [[ "$guess" =~ ^[0-9]+$ ]] for number validation
• Functions: Modular code organization (Solutions 2 & 3)
• Input/Output: read command, echo with -n flag
• String manipulation: tr command for case conversion
• Command substitution: $(command) syntax

📚 Further reading

If you are new to bash scripting, we have a streamlined tutorial series on Bash that you can use to learn it from scratch or use it to brush up the basics of bash shell scripting.

Bash Scripting Tutorial Series for Beginners [Free]

Get started with Bash Shell script learning with practical examples. Also test your learning with practice exercises.

Linux HandbookAbhishek Prakash

Tails is a Tor-based project inspired by the Debian Linux distro. Tails works on Tor Foundation, i.e., the onion router. It is supposed to be a secure OS and difficult to trace with ordinary tools/tricks.

Tails is used to perform privacy-focused tasks without leaving digital traces. These tasks include accessing or monitoring the target, i.e., it is recommended to access the identity type of data ethically (with consent of the target).

Today, this guide brings a brief walkthrough of the Tails OS and also lists the process to set up Tails.

• Tails OS | A Quick Walkthrough
  • Tails Development and Merger With Tor
  • Release Cycle
  • Top Features of Tails OS
  • Built-in Security and Privacy Tools for Maximum Anonymity
  • Persistent Vs Non-Persistent Storage in Tails
  • How to Install and Set up Tails on a USB Drive (Step-by-Step Guide)
    • Hardware Requirements for Installing Tails
    • Step 1: Download and Verify the Tails OS Disk Image (For USB or Virtual Machines)
    • Step 2: Install Tails
    • Should You Use Tails? | Technical Pros and Cons
    • Conclusion

Tails OS | A Quick Walkthrough

Let’s have a quick walkthrough of the features, some facts, and tools that Tails has:

Tails Development and Merger With Tor

• Tails was first released in 2009 as an iteration of Incognito, a Gentoo-based Linux distribution.
• The Tails project was originally named “Amnesia,” and now “Amnesia” is the default username/hostname of the Tails OS.
• In 2023, the Tails submitted a request to merge with Tor, which was completed in 2024, and now Tails is a Tor project.

Release Cycle

• Tails usually release the update after every 6-8 weeks. However, the major release (Like 5.0, 6.0, 7.0) is expected approximately every 2 years.

Top Features of Tails OS

• Communication via Tails is done using Tor (The Onion Router).
• Direct or non-Tor traffic is blocked by default.
• Can create persistent (keeping the data) and non-persistent (temporary) storage.
• Tails verifies and authenticates the Tails version you acquire.
• Wipes out the memory (RAM) after every reboot or shutdown, nullifying the chance of accessing the sensitive data, i.e., passwords, logins, etc, after restarts/ shutdowns.
• MAC Address Spoofing

Built-in Security and Privacy Tools for Maximum Anonymity

• • Tor Network: An Anonymous network to route traffic anonymously.
  • Thunderbird: An email client to send/receive encrypted emails.
  • KeePassXc: A password manager to securely save passwords.
  • Electrum: Bitcoin wallet
  • OnionShare: Sharing files/media anonymously
  • MAT2: A tool to remove metadata from files, i.e., PDF.
  • GnuPG (GPG): Tool for encrypting, signing, and verifying emails, using Public-Key cryptography.

Persistent Vs Non-Persistent Storage in Tails

Before proceeding to its setup, let‘s have a look at the difference between Persistent and Non-Persistent.

How to Install and Set up Tails on a USB Drive (Step-by-Step Guide)

Why are you installing Tails? If the purpose is anonymity, you need to create a non-persistent Live USB. However, if the purpose is to learn and implement, then you can create a persistent one to keep the history and some data from the previous sessions.

• Recommended: Use Tails as a Live OS, i.e., a non-persistent OS.
• Not Recommended: Avoid installing it on your primary host or primary virtual machine, where traces can persist.

Hardware Requirements for Installing Tails

• 64-bit processor
• Minimum 2GB RAM (Recommended is 4GB)
• Minimum 8 GB USB drive (Recommended is 16GB)

Step 1: Download and Verify the Tails OS Disk Image (For USB or Virtual Machines)

Download the Tails (either for USB sticks or the ISO file for Virtual machines).

Download Tails:

Once downloaded, verify your downloaded version:

Confirm that the verification is successful before proceeding further:

Step 2: Install Tails

Once done, you must have a USB bootable tool like balenaEtcher (if using Linux), Rufus (if using Windows), etc. Here, I have listed two possibilities:

• Installation Option 1: Installing Tails Using the GNOME Disks App
• Installation Option 2: Installing Tails Using the Balena Etcher

• Installation Option 1: GNOME Disks App

If you are using a GNOME desktop environment, you can make the Tails USB using the GNOME Disks Desktop application. Here’s the process.

Attach the USB with the Linux OS, and open the “GNOME Disks” app. Now, select the USB and click the “vertical ellipsis”.

It will then ask you to choose the Disk Image:

Confirm to start writing Tails on USB:

This will take a few minutes. Once all done, you can eject the drive and boot from the USB to run Tails in an isolated environment.

• Installation Option 2: Installing Tails Using the balenaEtcher

First, you must download and install the Balena Etcher on your respective Linux distribution.

Once installed, you have to make the USB bootable with the Tails disk image that you have already downloaded.

Follow this guide to briefly understand how to install and use Balena Etcher

Setting Up Persistent Storage and Administrator Password

When you boot from Tails, you will have to pass through the following interface, where you can create persistent storage and set the administrator password.

• Persistent Storage: To keep the data, history for further sessions.
• Administrator Password: To handle the permissions or admin password required for installing applications.

Now, you have to set the Tor connection, where you have two options, i.e., choose “Connect to Tor automatically”.

This way, you can set anonymity through Tails.

• What is Tor/Onion Circuits: The Path that the Tor network uses to route traffic from a user’s device to the destination server with anonymity.

Here’s the terminal look of the Tails:

Here you go with the Tails:

Should You Use Tails? | Technical Pros and Cons

If you still have questions about whether Tails is a perfect fit for me or not, don’t worry, I have compiled a technical Pros/Cons of Tails to assist you in this regard:

Pro-Tip: For maximum security, always use the non-persistent USB for Tails and enjoy this state-of-the-art tool in the tech era.

Conclusion

Tails OS is one of the best anonymity tools based on the Debian distro and is a part of the Tor project. With one end in Open Source (Linux-based) and the other in anonymity (Tor project).

Tails OS is installed and used as a Live non-persistent USB, which minimizes the risk of any traces. Today, I have drafted an in-depth review of the Tails OS, including its installation on a USB as well.

FAQs:

Q1: Is Tails OS Legal?

Yes, Tails is legal. Tails is an open-source Debian-inspired Tor project. The Onion Routing (Tor) nature of Tails makes it suspicious but not illegal.

Q2: Does Tails hide your IP?

Tails is a Tor project, and when routing, it hides your IP and server information.

Q3: Does Tails run in RAM?

Since it’s an amnestic, no data or history is kept. This anonymity is due to RAM. It temporarily stores data and refreshes it after each session.

Q4: What is the Tails filesystem?

Tails works on the Ext4 file system, i.e., Fourth Extended Filesystem.

An interesting development came from Microsoft as it released a new terminal-based editor with open source license.

I kind of liked it at first glance until I tried my hands on a shell script written in this editor and then I ran into:

The issue is that it added the classic Windows-style line endings, which is not liked by UNIX-like systems.

I knew it was too good to be true to have something perfect for Linux from Microsoft 🤦

Here are the highlights of this edition :

• Open-source notification Inbox infrastructure
• Listmonk newsletter
• Historical view of system resource utilization
• Bang bang... shebang
• And memes, news and tools to discover

🚀 Level up your coding skills and build your own bots

Harness the power of machine learning to create digital agents and more with hot courses like Learning LangChain, The Developer's Playbook for Large Language Model Security, Designing Large Language Model Applications, and more.

Part of the purchase goes to Code for America! Check out the ebook bundle here.

Humble Tech Book Bundle: Machine Learning, AI, and Bots by O’Reilly 2025

Master machine learning with this comprehensive library of coding and programming courses from the pros at O’Reilly.

Humble Bundle

Ubuntu 25.04, codenamed Plucky Puffin, released in April 2025, is an interim release supported for 9 months (until Jan 2026). Ubuntu 25.04 is equipped with experimental features that will be tested until the next LTS, 26.04, and if declared stable, these features will be carried forward and may be part of Ubuntu 26.04, the next Ubuntu LTS in line.

In today’s guide, I’ll give you a brief overview of Ubuntu 25.04, what it looks like, and what other features are included in the development and testing.

Outline:

• What’s New in Ubuntu 25.04 Codenamed Plucky Puffin?
• GNOME 48
• Kernel .14
• Security Center
• Updated apt Installation Interface
• Well-Being
• HDR Display – High Dynamic Range
• Other Notable Updates in Ubuntu 25.04
• How to Upgrade to Ubuntu 25.04
• Conclusion

What’s New in Ubuntu 25.04 Codenamed Plucky Puffin?

With every interim release (just like Ubuntu 25.04), there comes a list of new features to be tested and some improvements to existing functionalities. This time we are focusing on Ubuntu 25.04, some major as well as minor updates will be provided:

GNOME 48

Ubuntu 24.04 is based on GNOME 46, whereas at the moment of writing this post, Ubuntu 25.04 is being experimented with GNOME 48. As of now, GNOME 48 is more modern and graphics-friendly, which is always, i.e., the latest version is supposed to overcome the deficiency of the previous GNOME version and improve over time.

Kernel 6.14

The kernel is the central nervous system of Linux, i.e., a bridge between the hardware and the software. Ubuntu 25.04 comes with a Kernel 6.14 (Upstream), i.e., developed and maintained by Linus Torvalds and the Linux kernel maintainers.

The first official release of Ubuntu 24.04 contained the Kernel 6.8. However, Ubuntu 24.04.2 is now updated to the Linux Kernel 6.11.

Security Center

Although Ubuntu is an open-source OS and is more secure than other OSs. However, to align with this top-tech era, Ubuntu might be seeking some additional application support. These applications require some permissions that a user has to give for smooth functionality. To deal with such permissions, Ubuntu has released a Security Center in this release, so that users may turn on or off the permissions.

Here’s the initial interface of the Security Center, where you can see that the feature is experimental at the moment.

If you enable it, the strictly confined apps request permissions. The app permissions can be checked in the settings, i.e., “Settings > Apps”

Updated apt Installation Interface

An interactive UI for the apt-based installations and uninstallations:

Uninstalling:

Well-Being

This is about the well-being of the Ubuntu lovers. The users can enable it and set the following features:

• Screen Time: Set the average screen time usage.
• Eyesight Reminder: A reminder to look away from the screen.
• Movement Reminder: A reminder to move around.

Document Viewer (PDF Viewer)

Ubuntu 25.04 is now equipped with a built-in Document Viewer for various types of files. You can open a variety of files on this viewer, i.e., PDF, comic books, DjVU, and TIFF documents. Here’s the document viewer:

HDR Display – High Dynamic Range

High Dynamic Range (HDR) is a state-of-the-art technology to provide better display with advanced coloring sense of the HDR technology. This is one of the significant additions in this update list. If you have an HDR monitor, now, you can attach it to your Ubuntu 25.04 to experience HDR displays.

Head over to “Settings > Display > HDR” to manage it.

Other Notable Updates in Ubuntu 25.04

Color to Color Management:

The Color section in the “Settings” has been replaced with Color Management in the Settings.

Timezone in Events:

Ubuntu 25.04 provides timezone support while creating events in the calendar. Here’s how you can locate it in the Calendar app of Ubuntu 25.04:

JPEG XL Image Support:

JPEG XL is an image type (an enhanced JPEG), and now it is supported by Ubuntu and providing a better experience for the users.

Notification Grouping:

Ubuntu 25.04 has now offered a notification grouping inside the notifications, making it easier for users to navigate through multiple notifications.

Yaru Theme:

The icon and theme experience is better than the previous releases. The icons are now more dynamic and are well integrated with the accent color support.

Updated Network Manager:

Ubuntu 25.04 has an updated Network Manager 1.52, whereas Ubuntu 24.04.2 (released parallel to Ubuntu 25.04) has 1.46. The significant change is that Network Manager 1.52 is more aligned towards IPv6 as compared to the 1.46 version.

Chrony (Network Time Protocol):

Ubuntu 25.04 has adopted Chrony as its Network Time Protocol client (SUSE and RHEL inspired), which synchronizes the system time as per the NTP servers, i.e., a GPS receiver.

Until now, Ubuntu has been using “systemd-timesync” as its Network Time Protocol client, which is also known as SNTP (Simple Network Time Protocol). The SNTP synchronizes the system clock with the remote server and has less accuracy when compared with Chrony (Full NTP).

• What is NTP? The purpose of the NTP is to synchronize the clocks of the systems over a network to ensure the security, performance, event coordination, and logging. NTP ensures the time sync is as accurate as possible, i.e., in milliseconds / submilliseconds.

Developer Tools and Libraries:

Since Ubuntu is well-liked in the developer community, the Ubuntu contributors continuously work on providing updated tools. Ubuntu 25.04 is equipped with updated tools, i.e., Python, GCC, Rust, and Go.

Similarly, a few of the developers associated libraries are also upgraded, i.e., glibc, binutils, and OpenSSL.

Gaming Support (NVIDIA Dynamic Boost):

The NVIDIA dynamic boost enables the gamer to manage the power between the CPU and GPU. This is now enabled by default in Ubuntu 25.04.

System Monitor’s Interface:

Ubuntu’s system monitor shows information about the processes, resources, and file systems. In Ubuntu 25.04, there is a slight change in the interface of the Ubuntu System Monitor. For instance, the info inside the processes tab is restricted to, i.e., ID, CPU, Memory, Disk Write, and Disk Read. Here’s the interface where you can see this.

However, in older versions, the Processes tab has some additional info for each process:

That’s all from the notable features of Ubuntu 25.04.

Would you like to upgrade your Ubuntu to Ubuntu 25.04?

How to Upgrade to Ubuntu 25.04 Plucky Puffin

If you are using any other release of Ubuntu (Ubuntu 25.10 or Ubuntu 24.04), you can easily upgrade to Ubuntu 25.04. Let’s go through the steps to upgrade:

Important Note: If you are using a Ubuntu LTS release other than Ubuntu 24.04, then you have to first upgrade to Ubuntu 24.04:

• Upgrade Ubuntu to the Latest LTS

Once upgraded to Ubuntu 24.04, you are now ready to follow the steps below and upgrade to Ubuntu 24.10.

Step 1: Upgrade Your Ubuntu to Ubuntu 24.10

Since it is an interim release, you must have the previous release installed to get Ubuntu 25.04. Here’s how you can upgrade to Ubuntu 24.10:

• Update and upgrade the system repositories:

Note: It is recommended to use “sudo apt autoremove” after update/upgrade, to clean up the system from any useless dependencies/packages that are not required.

• If you are using Ubuntu 24.04 LTS, then you have to enable the non-LTS release upgrade. For that, open the release-upgrader file in an editor:

Now, change the “Prompt” parameter’s value from “lts” to “normal”, as can be seen below:

• Start the upgrade using the do-release-upgrade command:

Here you go:

Press “y” to proceed with the installation:

While upgrading, you will be prompted several times asking for acceptance of the changes being processed:

Step 2: Upgrade to Ubuntu 25.04

Once you are in Ubuntu 24.10, use the do-release command again to upgrade to Ubuntu 25.04:

Note: If you get any prompt like “please install all updates available for your release”, then use the command “sudo apt dist-upgrade” and reboot to fix it.

Here’s the Ubuntu 25.04:

That’s all from this guide.

Conclusion

Ubuntu 25.04, codenamed “Plucky Puffin”, is an interim Ubuntu release supported for 9 months. Ubuntu 25.04, released in April 2025, features the updated GNOME (48), updated Kernel (6.14), an improved apt version (3.0), and a security center. Other features include the HDR display, enhanced color management, timezone support in events, etc.

This post briefly lists the notable features of Ubuntu 25.04 and also explains the process to upgrade to Ubuntu 25.04.

FAQS

How Long Will Ubuntu 25.04 be Supported?

Ubuntu 25.04 will be supported until January 2026. Since Ubuntu 25.04 is an interim release and an Ubuntu interim release is supported for 9 months after its release.

Is Ubuntu 25.04 LTS?

No, Ubuntu 25.04 is an interim release, not an LTS. The current latest LTS is Ubuntu 24.04 codenamed Noble Numba, and the next in line LTS is Ubuntu 26.04.

How to Upgrade to Ubuntu 25.04?

First, upgrade to Ubuntu 24.04, then to 24.10, and from there, you can upgrade to Ubuntu 25.10.

Paste your YAML content or upload a file to validate syntax. Scroll down to see the details on the errors, if any.

I see a lot of posts on my Twitter (or X) feed debating the merits of ditching cloud services in favor of homelab self-hosted setups just like I tried hosting Wikipedia and the Arch wiki. Some even suggest using bare-metal servers for professional environments.

While these posts often offer intriguing tools and perspectives, I can't help but notice a pattern: companies lean heavily on cloud services until something goes catastrophically wrong, like Google Cloud accidentally deleting customer data.

However, let’s be real, human error can occur anywhere. Whether in the cloud or on-premises, mistakes are universal.

So, no, I’m not here to tell you to migrate your production services to a makeshift homelab server and become the next Antoine from Silicon Valley.

But if you’re wondering why people even homelab in the era of AWS and Hetzner, I’m here to make the case: it’s fun, empowering, and yes, sometimes even practical.

1. Cost control over time

Cloud services are undeniably convenient, but they often come with hidden costs. I still remember during my initial days here at It's FOSS, and Abhishek messaged me reminding me to delete any unused or improperly configured Linode instances.

That’s the thing with cloud services, you pay for convenience, and if you’re not meticulous, it can snowball.

A homelab, on the other hand, is a one-time investment. You can repurpose an old PC or buy retired enterprise servers at a fraction of their original cost.

Sure, there are ongoing power costs, but for many setups, especially with efficient hardware like Raspberry Pi clusters, this remains manageable.

I'll take this opportunity to share my favorite AWS meme.

2. Learning and experimentation

If you’re in tech, be it as a sysadmin, developer, or DevOps engineer, having a homelab is like owning a personal playground.

Want to deploy Kubernetes? Experiment with LXC containers? Test Ansible playbooks? You can break things, fix them, and learn along the way without worrying about production outages or cloud charges.

For me, nothing beats the thrill of running Proxmox on an old Laptop with a Core i5, 8 GB of RAM, and a 1 TB hard drive.

It’s modest (you might've seen that poor machine in several of my articles), but I’ve used it to spin up VMs, host Docker containers, and even test self-hosted alternatives to popular SaaS tools.

3. Privacy and ownership

When your data resides in the cloud, you trust the provider with its security and availability. But breaches happen, and privacy-conscious individuals might balk at the idea of sensitive information being out of their direct control.

With a homelab, you own your data. Want a cloud backup? Use tools like Nextcloud. Need to share documents easily? Host your own FileBrowser. This setup isn’t just practical, it’s liberating.

Sure, there’s a learning curve and it could be steep for many. Thankfully, we also have plug-and-play solutions like CasaOS, which we covered in a previous article. All you need to do is head to the app store, select 'install,' and everything will be taken care of for you.

4. Practical home uses

Homelabs aren’t just for tech experiments. They can serve real-world purposes, often replacing expensive commercial solutions:

• Media servers: Host your own movie library with Jellyfin or Plex. No subscription fees, no geo-restrictions, and no third-party tracking, as long as you have the media on your computer.
• Home security: Set up a CCTV network with open-source tools like ZoneMinder. Add AI-powered object detection, and you’ve built a system that rivals professional offerings at a fraction of the cost.
• Family productivity: Create centralized backups with Nextcloud or run remote desktop environments for family members. You become the go-to tech person for your household, but in a rewarding way.

For my parents, I host an Immich instance for photo management and a Jellyfin instance for media streaming on an old family desktop. Since the server is already running, I also use it as my offsite backup solution, just to be safe. 😅

9 Dashboard Tools to Manage Your Homelab Effectively

See which server is running what services with the help of a dashboard tool for your homelab.

It's FOSSAbhishek Kumar

What about renting a VPS for cloud computing?

I completely understand that renting a VPS can be a great choice for many. It offers flexibility, ease of use, and eliminates the need to manage physical hardware.

These days, most VPS providers like AWS, Oracle, and others offer a 1-year free tier to attract users and encourage them to explore their platforms. This is a fantastic opportunity for beginners or those testing the waters with cloud hosting.

I’ve also heard great things about Hetzner, especially their competitive pricing, which makes them an excellent option for those on a budget.

In fact, I use Nanode myself to experiment with a DNS server. This setup spares me the hassle of port forwarding, especially since I’m behind CGNAT.

If you’re interested in hosting your own projects or services but face similar limitations, I’ve covered a guide on Cloudflare Tunnels that you can consult, it’s a handy workaround for these challenges.

Personally, I believe gatekeeping is one of the worst things in the tech community. No one should dictate how or where you host your projects. Mix and match! Host some services on your own systems, build something from scratch, or rent a VPS for convenience.

Just be mindful of what you’re hosting like ensuring no misconfigured recursive function is running loose and keep exploring what suits your needs best.

My journey into homelab

My journey into homelabbing and self-hosting started out of necessity. When I was in university, I didn’t have the budget for monthly cloud server subscriptions, domain hosting, or cloud storage.

That limitation pushed me to learn how to piece things together: finding free tools, configuring them to fit my needs, diving into forum discussions, helping others solve specific issues, and eagerly waiting for new releases and features.

This constant tinkering became an endless cycle of learning, often without even realizing it. And that’s the beauty of it. Whether you’re self-hosting on a VPS or your homelab, every step is a chance to explore, experiment, and grow.

So, don’t feel constrained by one approach. The freedom to create, adapt, and learn is what makes this space so exciting.

Wrapping up

At the end of the day, whether you choose to build a homelab, rent a VPS, or even dabble in both, it’s all about finding what works best for you. There’s no one-size-fits-all approach here.

For me, homelabbing started as a necessity during my university days, but over time, it became a passion, a way to learn, experiment, and create without boundaries.

Sure, there are challenges, misconfigured services, late nights debugging, and the occasional frustration, but those moments are where the real learning happens.

Renting a VPS has its perks too. It’s quick, convenient, and often more practical for certain projects.

I’ve come to appreciate the balance of using both approaches, hosting some things locally for the sheer fun of it and using VPS providers when it makes sense. It’s not about choosing sides; it’s about exploring the possibilities and staying curious.

If you’re someone who enjoys tinkering, building, and learning by doing, I’d encourage you to give homelabbing a try. Start small, experiment, and let your curiosity guide you.

And if you prefer the convenience of a VPS or a mix of both, that’s perfectly fine too. At the end of the day, it’s your journey, your projects, and your learning experience.

So, go ahead, spin up that server, configure that service, or build something entirely new. The world of self-hosting is vast, and the possibilities are endless. Happy tinkering!