A Comprehensive Guide to Creating a Linux Performance Monitoring Script

In today’s digital landscape, maintaining optimal server performance is crucial for ensuring the reliability and efficiency of applications and services. Whether you’re a system administrator, a DevOps engineer, or a developer, having the right tools to monitor server performance can save you time and help you troubleshoot issues effectively. In this blog post, we will walk through the process of creating a shell script that analyzes basic server performance statistics on a Linux server. We will cover essential metrics such as CPU usage, memory usage, disk usage, and more, while also addressing potential pitfalls and compatibility issues.

Why Monitor Server Performance?

Monitoring server performance is essential for several reasons:

1. Resource Management: Understanding how resources are utilized helps in optimizing performance and planning for future capacity needs.
2. Troubleshooting: Identifying performance bottlenecks can help in diagnosing issues before they escalate into critical failures.
3. Security: Monitoring failed login attempts and unusual activity can help in identifying potential security threats.
4. Compliance: Many industries require regular performance audits to ensure compliance with regulations.

Overview of the Script

The script we will create, server-stats.sh, will provide the following metrics:

• Total CPU usage
• Total memory usage (used vs. free, including percentage)
• Total disk usage (used vs. free, including percentage)
• Top 5 processes by CPU usage
• Top 5 processes by memory usage
• Additional stats such as OS version, uptime, load average, logged-in users, and failed login attempts

Step 1: Setting Up the Script

To get started, open your terminal and create a new file named server-stats.sh:

touch server-stats.sh
chmod +x server-stats.sh

Next, open the file in your preferred text editor:

nano server-stats.sh

Step 2: Writing the Script

Below is the complete code for the server-stats.sh script, along with explanations for each section:

#!/bin/bash

# Function to get total CPU usage
get_cpu_usage() {
    if ! command -v mpstat &> /dev/null; then
        echo "Error: mpstat not found. Install sysstat package."
        return
    fi
    echo "Total CPU Usage:"
    mpstat | awk '$3 ~ /[0-9.]+/ { printf "%.1f%%\n", 100 - $12 }'
}

Explanation: This function checks if the mpstat command is available (part of the sysstat package). It then calculates the total CPU usage by subtracting the idle percentage from 100.

# Function to get total memory usage
get_memory_usage() {
    echo "Total Memory Usage:"
    free -h | awk 'NR==2{printf "Used: %s (%.2f%%)\nFree: %s\n", $3, $3*100/$2, $4}'
}

Explanation: This function uses the free -h command to display memory usage in a human-readable format. It calculates the percentage of used memory and displays both used and free memory.

# Function to get total disk usage
get_disk_usage() {
    echo "Total Disk Usage:"
    df -h | awk '$NF=="/"{printf "Used: %s (%.2f%%)\nFree: %s\n", $3, $3*100/$2, $4}'
}

Explanation: This function retrieves disk usage statistics using the df -h command, focusing on the root filesystem (/). It calculates and displays used and free disk space.

# Function to get top 5 processes by CPU usage
get_top_cpu_processes() {
    echo "Top 5 Processes by CPU Usage:"
    ps -eo pid,comm,%cpu --sort=-%cpu | head -n 6
}

Explanation: This function lists the top 5 processes consuming the most CPU resources using the ps command, sorted by CPU usage.

# Function to get top 5 processes by memory usage
get_top_memory_processes() {
    echo "Top 5 Processes by Memory Usage:"
    ps -eo pid,comm,%mem --sort=-%mem | head -n 6
}

Explanation: Similar to the CPU function, this function lists the top 5 processes by memory usage.

# Function to get failed login attempts
get_failed_logins() {
    local log_file
    if [ -f /var/log/auth.log ]; then
        log_file="/var/log/auth.log"
    elif [ -f /var/log/secure ]; then
        log_file="/var/log/secure"
    else
        echo "Failed Login Attempts: Log file not found."
        return
    fi
    echo "Failed Login Attempts: $(sudo grep 'Failed password' "$log_file" 2>/dev/null | wc -l || echo "Permission denied")"
}

Explanation: This function checks for the existence of the log files for failed login attempts. It handles both Debian-based and Red Hat-based systems. If the user does not have permission to read the log file, it gracefully reports the issue.

# Function to get additional stats
get_additional_stats() {
    echo "Additional Stats:"
    echo "OS Version: $(lsb_release -ds 2>/dev/null || cat /etc/os-release | grep PRETTY_NAME | cut -d '"' -f2)"
    echo "Uptime: $(uptime | awk -F'( |,|:)+' '{print $6 " " $7 " hours"}')"
    echo "Load Average: $(cat /proc/loadavg | awk '{print $1, $2, $3}')"
    echo "Logged In Users: $(who | wc -l)"
    get_failed_logins
}

Explanation: This function gathers additional system statistics, including the OS version, uptime, load average, and the number of logged-in users. It also calls the get_failed_logins function to include that information.

Step 3: Main Function

The main function orchestrates the execution of all the other functions:

# Main function
main() {
    get_cpu_usage
    echo
    get_memory_usage
    echo
    get_disk_usage
    echo
    get_top_cpu_processes
    echo
    get_top_memory_processes
    echo
    get_additional_stats
}

# Execute
main

Explanation: The main function calls each of the previously defined functions in order, providing a structured output of the server’s performance metrics.

Here’s the revised script addressing all issues:

#!/bin/bash

#Function to get total CPU usage

get_cpu_usage() {
if ! command -v mpstat &> /dev/null; then
echo “Error: mpstat not found. Install sysstat package.”
return
fi
echo “Total CPU Usage:”
mpstat | awk ‘$3 ~ /[0-9.]+/ { printf “%.1f%%\n”, 100 - $12 }’
}

#Function to get total memory usage
get_memory_usage() {
echo “Total Memory Usage:”
free -h | awk ‘NR==2{printf “Used: %s (%.2f%%)\nFree: %s\n”, $3, $3*100/$2, $4}’
}

#Function to get total disk usage
get_disk_usage() {
echo “Total Disk Usage:”
df -h | awk ‘$NF=="/"{printf “Used: %s (%.2f%%)\nFree: %s\n”, $3, $3*100/$2, $4}’
}

#Function to get top 5 processes by CPU usage
get_top_cpu_processes() {
echo “Top 5 Processes by CPU Usage:”
ps -eo pid,comm,%cpu --sort=-%cpu | head -n 6
}

#Function to get top 5 processes by memory usage
get_top_memory_processes() {
echo “Top 5 Processes by Memory Usage:”
ps -eo pid,comm,%mem --sort=-%mem | head -n 6
}

#Function to get failed login attempts
get_failed_logins() {
local log_file
if [ -f /var/log/auth.log ]; then
log_file="/var/log/auth.log"
elif [ -f /var/log/secure ]; then
log_file="/var/log/secure"
else
echo “Failed Login Attempts: Log file not found.”
return
fi
echo “Failed Login Attempts: $(sudo grep ‘Failed password’ “$log_file” 2>/dev/null | wc -l || echo “Permission denied”)”
}

#Function to get additional stats
get_additional_stats() {
echo “Additional Stats:”
echo “OS Version: $(lsb_release -ds 2>/dev/null || cat /etc/os-release | grep PRETTY_NAME | cut -d '”’ -f2)"
echo “Uptime: $(uptime | awk -F’( |,|:)+’ '{print $6 " " $7 " hours”}’)"
echo “Load Average: $(cat /proc/loadavg | awk ‘{print $1, $2, $3}’)”
echo “Logged In Users: $(who | wc -l)”
get_failed_logins
}

#Main function
main() {
get_cpu_usage
echo
get_memory_usage
echo
get_disk_usage
echo
get_top_cpu_processes
echo
get_top_memory_processes
echo
get_additional_stats
}

#Execute
main

Step 4: Running the Script

To run the script, ensure you have the necessary permissions and dependencies installed. You can execute the script with:

sudo ./server-stats.sh

Conclusion

By following this guide, you have created a comprehensive performance monitoring script for Linux servers. This script not only provides essential metrics but also includes error handling and compatibility checks to ensure it runs smoothly across different distributions. Regularly monitoring server performance can help you maintain optimal resource utilization, troubleshoot issues proactively, and enhance the overall security of your systems.

Additional Considerations

• Dependencies: Ensure that the sysstat package is installed for the mpstat command to work. You can install it using your package manager:

  sudo apt install sysstat   # For Debian/Ubuntu
  sudo yum install sysstat    # For Red Hat/CentOS
  

• Permissions: Some commands require elevated privileges (e.g., reading log files). Always run the script with sudo to get complete results.

• Customization: Feel free to modify the script to include additional metrics or to format the output according to your preferences.

By implementing this script, you can gain valuable insights into your server’s performance and make informed decisions to optimize your infrastructure.