Removing Duplicates from an Array in Perl

When working with arrays in Perl, you might encounter situations where you need to remove duplicate elements. Perl, with its versatile data structures, offers several ways to accomplish this task. Let’s explore different approaches to remove duplicates from an array, demonstrating the flexibility and power of Perl.

Approach 1: Using a Hash to Filter Unique Elements

One of the simplest and most efficient ways to remove duplicates from an array is by using a hash. Hashes in Perl inherently prevent duplicate keys, which makes them ideal for this task.

my @array = qw(one two three two three);

my %seen;
my @unique = map { $seen{$_}++ == 0 ? $_ : () } @array;

print "@unique\n";

Explanation:
Here, map iterates over each element in @array. The %seen hash keeps track of elements we’ve encountered. If an element has not been seen before, it’s added to the @unique array.

Output:

one two three

Approach 2: Using a grep Filter

Another way to achieve this is by leveraging Perl’s grep function, which filters elements based on a condition.

my @array = qw(one two three two three);

my %seen;
my @unique = grep { !$seen{$_}++ } @array;

print "@unique\n";

Explanation:
The grep function filters the elements of @array. For each element, the block checks if the element has been seen before using %seen. If it hasn’t, it gets added to @unique.

Output:

one two three

Approach 3: Preserving Order Using a Hash Slice

For cases where preserving the original order of the elements is essential, a hash slice can be employed.

my @array = qw(one two three two three);

my %seen;
@seen{@array} = ();
my @unique = keys %seen;

print "@unique\n";

Explanation:
This method uses hash slices to assign all elements of @array as keys in the %seen hash, effectively removing duplicates. Then, keys %seen gives the unique elements.

Output:

one two three

Approach 4: Using the List::MoreUtils Module

Perl modules like List::MoreUtils offer a built-in uniq function to simplify the task. This method is especially useful for maintaining readability and avoiding manual implementations.

use List::MoreUtils qw(uniq);

my @array = qw(one two three two three);

my @unique = uniq(@array);

print "@unique\n";

Explanation:
uniq from List::MoreUtils handles everything internally, returning the unique elements while preserving their order.

Output:

one two three

Removing duplicates from an array in Perl can be done in various ways, depending on your specific needs, such as preserving order or using built-in modules. Whether you opt for a simple hash-based approach or leverage a CPAN module, Perl’s flexibility ensures you can efficiently handle duplicates in your arrays.

These examples showcase how Perl allows you to write concise and powerful code, tailored to your specific needs.