Friday, 29 November 2024

Using grep --exclude/--include Syntax to Skip Certain Files

 When you need to search for a specific string across files in a directory structure, but wish to exclude or include certain types of files (such as excluding binary files or including only certain file types), you can leverage grep's --exclude and --include options.

Scenario: Searching for foo= in Text Files While Excluding Binary Files

Consider the task of searching for foo= in text files but excluding binary files such as images (JPEG, PNG) to speed up the search and avoid irrelevant results. You can use the following command to achieve this:

Read more »

Labels:

Tuesday, 2 March 2021

Top 10 Example of lsof commands in Linux and UNIX

In a Linux or UNIX operating system, there are many processes running simultaneously. These processes use files, sockets, and other resources that are managed by the operating system. To get information about these resources, we can use the "lsof" (list open files) command. Lsof is a powerful tool that can help us understand the system's state, monitor and debug applications, and troubleshoot issues. In this article, we will look at ten examples of using the lsof command in Linux and UNIX.

List all open files:

To list all the open files in the system, we can simply run the lsof command without any arguments. This will give us a comprehensive list of all the files and their corresponding processes.

lsof

Read more »

Labels: , ,

Sunday, 24 March 2024

The Essential 70 Linux Commands for DevOps

In the world of DevOps, efficiency and automation are kings. This is where Linux, with its vast array of command-line tools, shines. The command line is a powerful ally, providing direct control over the operating system and the machinery that runs your applications. Here, we introduce the top 70 Linux commands that are indispensable for DevOps engineers and system administrators. These commands form the backbone of many automated tasks, troubleshooting, and daily management of systems.

File and Directory Operations

  1. ls: Unveil the contents of directories.
  2. cd: Navigate through directories.
  3. pwd: Display the current directory.
  4. mkdir: Forge new directories.
  5. touch: Create files without content.
  6. cp: Duplicate files or directories.
  7. mv: Relocate or rename files/directories.
  8. rm: Eliminate files or directories.
  9. find: Seek out files or directories.
  10. grep: Filter patterns within files.
Read more »

Labels:

Friday, 8 November 2024

Reading a File into a Java String: Modern Techniques

Java provides multiple ways to read the entire contents of a file into a single String. From traditional methods using BufferedReader to modern approaches with NIO utilities, there’s a technique to suit different Java versions and project requirements. Here’s a guide to some of the most popular methods, including options for Java 7 and newer, as well as some third-party alternatives.

1. Using Files.readString() in Java 11+

For Java 11 and later, the Files.readString() method offers a simple and efficient way to read the entire content of a file into a String. This method preserves line terminators.

Read more »

Labels:

Wednesday, 8 July 2020

perl program to Renaming Files with a Prefix

This Perl script renames all the files in the current directory with a given prefix. The prefix can be passed as a command-line argument.

Method 1: Using rename function

#!/usr/bin/perl

my $prefix = shift @ARGV || '';

opendir my $dh, '.' or die "Couldn't open current directory: $!";

while (my $filename = readdir $dh) {

    next if $filename =~ /^\./; # Skip dotfiles

    my $newname = $prefix . $filename;

    rename $filename, $newname or warn "Couldn't rename $filename: $!";

}

closedir $dh;


or


#!/usr/bin/perl

use strict;

use warnings;

use File::Glob;


my $prefix = 'new_';

my $pattern = '*.txt';


foreach my $file (glob $pattern) {

    my $new_file = $prefix . $file;

    rename $file, $new_file or die "Can't rename $file to $new_file: $!";

}

Save the script as rename.pl and run it with perl rename.pl newprefix_. This will add the prefix "newprefix_" to all the files in the current directory.


Method 2: Using File::Copy

This method uses the File::Copy module to copy files with a new name and then delete the old file. We can use the File::Glob module to get a list of files matching a pattern and then use the File::Copy module to copy each file with the new name.

#!/usr/bin/perl

use strict;

use warnings;

use File::Glob;

use File::Copy;

my $prefix = 'new_';

my $pattern = '*.txt';

foreach my $file (glob $pattern) {

    my $new_file = $prefix . $file;

    copy $file, $new_file or die "Can't copy $file to $new_file: $!";

    unlink $file or die "Can't delete $file: $!";

}

Save the script as rename_files_method2.pl and run it with perl rename_files_method2.pl. This will rename all .txt files in the current directory with the prefix new_.


Method 3: Using regular expressions

This method uses regular expressions to modify the filenames with a prefix. We can use the File::Glob module to get a list of files matching a pattern and then use regular expressions to modify each filename.

use strict;

use warnings;


use File::Glob;

my $prefix = 'new_';

my $pattern = '*.txt';

foreach my $file (glob $pattern) {

    my $new_file = $file;

    $new_file =~ s/^/$prefix/;

    rename $file, $new_file or die "Can't rename $file to $new_file: $!";

}

Save the script as rename_files_method3.pl and run it with `perl rename_files

all .txt files in the current directory with the prefix new_.


Method 4: Using File::Find

This method uses the File::Find module to recursively find and rename files with a prefix. The File::Find module provides a way to traverse a directory tree and apply a function to each file found. We can use the rename function within the wanted function to rename each file with the specified prefix.

#!/usr/bin/perl

use strict;

use warnings;

use File::Find;

my $prefix = 'new_';


sub rename_file {

    if (-f && /^.*\.txt$/) {

        my $new_file = $prefix . $_;

        rename $_, $new_file or warn "Can't rename $_ to $new_file: $!";

    }

}

find(\&rename_file, '.');

Save the script as rename_files_method4.pl and run it with perl rename_files_method4.pl. This will recursively find and rename all .txt files in the current directory and its subdirectories with the prefix new_.


Method 5: Using opendir and readdir

This method uses the opendir and readdir functions to find and rename files with a prefix. The opendir function opens a directory, and the readdir function reads the contents of the directory. We can use regular expressions to match the filenames and then use the rename function to rename each file with the specified prefix.

#!/usr/bin/perl

use strict;

use warnings;

my $prefix = 'new_';

opendir(my $dh, '.') or die "Can't open current directory: $!";

while (my $file = readdir($dh)) {

    next unless $file =~ /^.*\.txt$/;

    my $new_file = $prefix . $file;

    rename $file, $new_file or warn "Can't rename $file to $new_file: $!";

}

closedir $dh;

Save the script as rename_files_method5.pl and run it with perl rename_files_method5.pl. This will find and rename all .txt files in the current directory with the prefix new_.


Method 6: Using File::Copy

This method uses the File::Copy module to copy files with a prefix to a new directory and then rename the files. The File::Copy module provides functions for copying and moving files. We can use the copy function to copy the original files to a new directory with the prefix, and then use the rename function to rename the files with the specified prefix.

use strict;

use warnings;

use File::Copy;

my $prefix = 'new_';

my $new_dir = 'new_files';

mkdir $new_dir unless -d $new_dir;

opendir(my $dh, '.') or die "Can't open current directory: $!";

while (my $file = readdir($dh)) {

    next unless $file =~ /^.*\.txt$/;

    my $new_file = $prefix . $file;

    copy($file, "$new_dir/$new_file") or warn "Can't copy $file: $!";

}

closedir $dh;

opendir($dh, $new_dir) or die "Can't open $new_dir: $!";

while (my $file = readdir($dh)) {

    next unless $file =~ /^.*\.txt$/;

    my $new_file = $prefix . $file;

    rename("$new_dir/$file", "$new_dir/$new_file") or warn "Can't rename $file: $!";

}

closedir $dh;

Save the script as rename_files_method6.pl and run it with perl rename_files_method6.pl. This will copy all .txt files in the current directory to a new directory named new_files with the prefix new_, and then rename the files with the specified prefix.


Method 7: Using File::Find::Rule

This method uses the File::Find::Rule module to find and rename files with a prefix. The File::Find::Rule module provides a simple interface for finding files that match specified criteria. We can use the name method to match filenames and then use the rename function to rename each file with the specified prefix.

use strict;

use warnings;

use File::Find::Rule;

my $prefix = 'new_';

my @files = File::Find::Rule->file()->name('*.txt')->in('.');

foreach my $file (@files) {

    my $new_file = $prefix . $file;

    rename $file, $new_file or warn "Can't rename $file to $new_file: $!";

}

Save the script as rename_files_method7.pl and run it with perl rename_files_method7.pl. This will find and rename all .txt files in the current directory with the prefix new_.

Labels:

Wednesday, 31 July 2024

Mastering Multi-File Workflow in Vim

Vim, a powerful text editor beloved by developers, offers a multitude of features to enhance your workflow. If you’re working with multiple files—whether you’re writing scripts, coding, or editing text—knowing how to navigate and manage those files efficiently in Vim can significantly improve your productivity. In this blog post, we’ll explore various methods to handle multiple files in Vim as of 2024.

Read more »

Labels:

Saturday, 13 July 2024

Handling Auto-Generated Django Files in Pre-Commit with Regex

When working with Django, certain files, especially within the migrations directory, are automatically generated. These files often fail to meet the stringent requirements of tools like pylint, causing pre-commit hooks to fail. This blog post will guide you through using Regex to exclude these auto-generated files in your pre-commit configuration, ensuring smoother commit processes.

Understanding the Problem

Auto-generated files by Django, particularly those in the migrations folder, typically do not conform to pylint standards, resulting in errors during pre-commit checks. These files generally follow a naming convention that makes them identifiable, which we can leverage to exclude them using Regex patterns.

Read more »

Labels:

Monday, 8 June 2020

perl program to copy files from one directory to another directory

Perl script copies all the files with a given extension from one directory to another. The source and destination directories are passed as command-line arguments, along with the file extension.

Method 1:

#!/usr/bin/perl

use File::Copy;

my ($src_dir, $dest_dir, $ext) = @ARGV;

opendir my $dh, $src_dir or die "Couldn't open directory $src_dir: $!";

while (my $filename = readdir $dh) {

    next unless $filename =~ /\.$ext$/; # Match extension

    my $src_path = "$src_dir/$filename";

    my $dest_path = "$dest_dir/$filename";

    copy $src_path, $dest_path or warn "Couldn't copy $filename: $!";

}

closedir $dh;

Save the script as copyfiles.pl and run it with perl copyfiles.pl sourcedir destdir txt. This will copy all the files with the extension "txt" from "sourcedir" to "destdir".

Method 2:Using File::Find

The File::Find module provides a way to recursively traverse a directory tree and perform actions on files that match certain criteria.

#!/usr/bin/perl

use File::Find;

use File::Copy;

my ($source_dir, $dest_dir, $file_ext) = @ARGV;

find(sub {

    return unless -f $_; # Skip directories

    return unless /\.$file_ext$/; # Match file extension

    my $source_file = $File::Find::name;

    $source_file =~ s/^\Q$source_dir\E//; # Remove source directory from path

    my $dest_file = "$dest_dir$source_file";

    mkpath($dest_file); # Create parent directories if necessary

    copy($File::Find::name, $dest_file) or warn "Couldn't copy $File::Find::name: $!";

}, $source_dir);

Save the script as copy_files_method2.pl and run it with perl copy_files_method2.pl /path/to/source/dir /path/to/dest/dir txt. This will copy all files with the extension "txt" from the source directory to the destination directory and preserve the directory structure.

Method 3: Using system()

The system() function in Perl allows you to execute shell commands. You can use the cp command to copy files from one directory to another.

#!/usr/bin/perl

my ($source_dir, $dest_dir, $file_ext) = @ARGV;

my $command = "cp $source_dir/*.$file_ext $dest_dir/";

system($command) == 0 or die "Couldn't execute command: $command";

Save the script as copy_files_method3.pl and run it with perl copy_files_method3.pl /path/to/source/dir /path/to/dest/dir txt. This will copy all files with the extension "txt" from the source directory to the destination directory using the cp command.

Method 4: Using opendir() and readdir()

This method uses the opendir() and readdir() functions to read the contents of a directory and then copy the files matching the specified file extension.

#!/usr/bin/perl

use File::Copy;

my ($source_dir, $dest_dir, $file_ext) = @ARGV;

opendir(DIR, $source_dir) or die "Can't open $source_dir: $!";

my @files = readdir(DIR);

closedir(DIR);

foreach my $file (@files) {

    next unless $file =~ /\.$file_ext$/; # Match file extension

    my $source_file = "$source_dir/$file";

    my $dest_file = "$dest_dir/$file";

    copy($source_file, $dest_file) or warn "Couldn't copy $file: $!";

}

Save the script as copy_files_method4.pl and run it with perl copy_files_method4.pl /path/to/source/dir /path/to/dest/dir txt. This will copy all files with the extension "txt" from the source directory to the destination directory.

Method 5: Using glob()

The glob() function returns a list of filenames that match a specified pattern. You can use this function to get a list of files matching the specified file extension and then copy them to the destination directory.

#!/usr/bin/perl

use File::Copy;

my ($source_dir, $dest_dir, $file_ext) = @ARGV;

foreach my $file (glob("$source_dir/*.$file_ext")) {

    my $dest_file = "$dest_dir/" . (split /\//, $file)[-1];

    copy($file, $dest_file) or warn "Couldn't copy $file: $!";

}

Save the script as copy_files_method5.pl and run it with perl copy_files_method5.pl /path/to/source/dir /path/to/dest/dir txt. This will copy all files with the extension "txt" from the source directory to the destination directory.

Method 6: Using Path::Tiny

The Path::Tiny module provides a simple interface to manipulate file and directory paths. It also provides a copy() function that can be used to copy files from one directory to another.

#!/usr/bin/perl

use Path::Tiny;

my ($source_dir, $dest_dir, $file_ext) = @ARGV;

foreach my $file (path($source_dir)->children(qr/\.${file_ext}$/)) {

    my $dest_file = path($dest_dir, $file->basename);

    $file->copy($dest_file) or warn "Couldn't copy $file: $!";

}

Save the script as copy_files_method6.pl and run it with perl copy_files_method6.pl /path/to/source/dir /path/to/dest/dir txt. This will copy all files with the extension "txt" from the source directory to the destination directory.

The Path::Tiny module also provides other useful functions for working with files and directories, such as move(), rename(), and unlink(). It's worth exploring if you need to do more complex file manipulation in your Perl scripts.

Labels: