Remove Duplicates From Array in Java

An array can contain duplicate elements, and the array may be sorted or unsorted. Let us see different ways to remove duplicates from a given array in Java programming language.

Example:-

Sorted array = {10, 10, 20, 30, 40, 40, 50};
After removing the duplicates from array = {10, 20, 30, 40, 50};
Unsorted Array = {30, 50, 20, 50, 10, 20, 30, 10, 10, 40};
After removing the duplicates from array = {30, 50, 20, 10, 40};

In all examples, we will use toString() method of java.util.Arrays class to display the array. The Arrays.toString() method is used to convert array to string.

Using Loops

Let us discuss how to remove duplicates from sorted and unsorted array using loops. Here, we won’t use any temporary array, we will use seperate index.

Java program to remove duplicates from sorted array

In sorted array, removing duplicate elements is easy compared to the unsorted array. In the sorted array, next element can be either equal or greater/lesser (greater if array is sorted in ascending order, else lesser). If both are equal then ignore it else perform the operation. Let us see the steps.

Procedure to develop a method to remove duplicates from sorted array
a) Take an sorted array.
b) Take a variable initializing with 0, it store count of unique elements.
c) Find index of last element, lastIndex = array-size – 1
d) Iterate array upto before the last element
e) Compare two concusetive array elements. If both are same then move on, else increase count of unique element and store that unique element at approriate index in the same array.
f) For the last element, compare it with last unique element. If both are then ignore else perfom previous operation.
g) Return the copy of unique elements.

import java.util.Arrays;
public class ArrayTest {

   public static void main(String[] args) {
      // sorted array
      int arr[] = {10, 10, 20, 30, 40, 40, 50};
      
      // remove duplicates
      int newArr[] = removeDuplicates(arr);
      
      // display both arrays
      System.out.println("Original array: "
                     + Arrays.toString(arr));
      System.out.println("After removing duplicates: " 
                     + Arrays.toString(newArr));
      
   }
   
   // method to remove duplicates from sorted array
   public static int[] removeDuplicates(int[] arr) {
      
      int j = 0; // index without duplicates
      int lastIndex = arr.length-1; 
      
      // loop to iterate the array (except last element)
      for (int i = 0; i < lastIndex; i++) {
         if(arr[i] != arr[i+1]) {
            arr[j++] = arr[i];
         }
      }
      
      // for the last element
      if(arr[j] != arr[lastIndex])
         arr[j++] = arr[lastIndex];
      
      // return new copied array
      return Arrays.copyOf(arr, j);
   }
}

Output:-

Original array: [10, 20, 30, 40, 50, 40, 50]
After removing duplicates: [10, 20, 30, 40, 50]

The time complexity of this method:- O(n)

Java program to remove duplicates from unsorted array

The above program can remove duplicates only from the sorted array, but the below program can remove duplicates from any array which can be sorted/unsorted.

import java.util.Arrays;
public class ArrayTest {

   public static void main(String[] args) {
      // sorted/unsorted array
      int arr[] = {30, 50, 20, 50, 10, 20, 30, 10, 10, 40}; 
      
      // remove duplicates
      int newArr[] = removeDuplicates(arr);
      
      // display both arrays
      System.out.println("Original array: "
                     + Arrays.toString(arr));
      System.out.println("After removing duplicates: " 
                     + Arrays.toString(newArr));
   }
   
   // method to remove duplicates from array
   public static int[] removeDuplicates(int[] arr) {
      int index = 1; // index without duplicates 
      
      // loop to iterate the array
      for (int i = 1; i < arr.length; i++) {
         
         // current element
         int element = arr[i];
         
         // check element is in unique
         for(int j=0; j<index; j++) {
            if(arr[j] == element) {
               break;
            }
            
            // when not in unique
            if(j == index-1) {
               arr[index++] = element;
               // index got increased therefore use break
               break;
            }
         }
         
      }
            
      // return new copied array
      return Arrays.copyOf(arr, index);
   }
}

Output:-

Original array: [30, 50, 20, 10, 40, 20, 30, 10, 10, 40]
After removing duplicates: [30, 50, 20, 10, 40]

If you are wondering that how it is performing the operations then you can use below method,

// method to remove duplicates from array
public static int[] removeDuplicates(int[] arr) {

   System.out.println(Arrays.toString(arr));
   int index = 1; // index without duplicates 
      
   // loop to iterate the array
   for (int i = 1; i < arr.length; i++) {
      System.out.println("Unique elements are: "
              + Arrays.toString(Arrays.copyOf(arr, index)));
         
      // current element
      int element = arr[i];
      System.out.print("Current element: " + element);
         
      // check element is in unique
      for(int j=0; j<index; j++) {
         if(arr[j] == element) {
            System.out.println(", Already exist");
            break;
         }
         // when not in unique
         if(j == index-1) {
            arr[index++] = element;
            System.out.println(", unique");
            System.out.println("Index of unique = "+index);
            break;
         }
      }
   }

   // return new copied array
   return Arrays.copyOf(arr, index);
}

Remove duplicates from array in Java using Set collection

Using Java collections also we can remove the duplicate from array. The set collection can store only unique values, therefore in this program we will iterate to the array and try to insert all elements in the set collection. After inserting set will be converted to the array and it is returned to the caller method.

import java.util.Arrays;
import java.util.LinkedHashSet;

public class ArrayTest {

   public static void main(String[] args) {
      // sorted/unsorted array
      int arr[] = { 30, 50, 20, 50, 10, 20, 30, 10, 10, 40 };

      // remove duplicates
      int newArr[] = removeDuplicates(arr);

      // display both arrays
      System.out.println("Original array: " + Arrays.toString(arr));
      System.out.println("After removing duplicates: " 
                        + Arrays.toString(newArr));

   }

   // method to remove duplicates from array
   public static int[] removeDuplicates(int[] arr) {
      LinkedHashSet<Integer> set = 
            new LinkedHashSet<Integer>();

      // add elements to LinkedHashSet
      for (int element : arr) {
         set.add(element);
      }
      
      // return array result by 
      // converting set to array
      return setToArray(set);
   }

   // method to convert set to int array
   public static int[] setToArray(LinkedHashSet<Integer> set) {
      int temp[] = new int[set.size()];
      int i = 0;
      for (Integer element : set) {
         temp[i++] = element;
      }
      return temp;
   }
}

Output:-

Original array: [30, 50, 20, 50, 10, 20, 30, 10, 10, 40]
After removing duplicates: [30, 50, 20, 10, 40]

Remove duplicates from an array using Java 8 stream

We can also use the stream to remove the duplicates from a given array.

import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

public class ArrayTest {
   public static void main(String[] args) {
      // sorted/unsorted array
      Integer arr[] = { 30, 50, 20, 50, 10, 20, 30, 10, 10, 40 };
      System.out.println("Original array: " 
                         + Arrays.toString(arr));

      // remove duplicates
      removeDuplicates(arr);

   }

   // method to remove duplicates from array
   public static void removeDuplicates(Integer[] arr) {
      List<Integer> list = Arrays.asList(arr);
      List<Integer> listWithoutDuplicates = 
            list.stream().distinct().collect(Collectors.toList());

      // display
      System.out.println("After removing duplicates: " 
                         + listWithoutDuplicates);
   }
}

Output:-

Original array: [30, 50, 20, 50, 10, 20, 30, 10, 10, 40]
After removing duplicates: [30, 50, 20, 10, 40]

If you want to get both unique elements along with the duplicate elements then we can do it as follows:-

import java.util.Arrays;
import java.util.LinkedHashSet;
import java.util.Set;
import java.util.stream.Collectors;

public class Test {
    public static void main(String[] args) {
        Integer[] numbers = { 30, 50, 20, 50, 10, 20, 30, 10, 10, 40 };
        Set<Integer> uniqueNumbers = new LinkedHashSet<>();
        Set<Integer> duplicateNumbers = Arrays.stream(numbers)
                .filter(n -> !uniqueNumbers.add(n))
                .collect(Collectors.toSet());
        System.out.println(uniqueNumbers); // [30, 50, 20, 10, 40]
        System.out.println(duplicateNumbers); // [50, 20, 10, 30]
    }
}

If you enjoyed this post, share it with your friends. Do you want to share more information about the topic discussed above or do you find anything incorrect? Let us know in the comments. Thank you!

Search