Remove Duplicate Words From String In Python

Remove Duplicate Words From String In Python | Here, we will discuss how to remove duplicate words from a string in the Python Programming language with the help of some examples. 

Firstly let us see what are duplicate words in a string. Duplicate words are the world that is exactly the same identity in a sentence or a string. Example:- “Java is a high-level language and C++ is also a high-level language”.

Here “is” and “a high-level language” are the duplicate words in the given sentence and we have to extract them from our string/sentence using Python. After extracting the duplicate words output:- “Java is a high-level language and C++ also”.

Example-2:-
String: Python is popular and Java is also popular.
Output: Python is popular and Java also.
Here “is” and “popular” words are duplicated.

There are several ways to find duplicate words in a string in Python, here we will discuss some of them:-

  1. Using Counter class
  2. Using For Loop
  3. Using a Set

Remove Duplicate Words From String In Python Using Counter Class

This is a very simple and easy approach to removing duplicate strings from a sentence. The Counter is a Dict subclass for counting hashable items. Sometimes called a bag or multiset. Elements are stored as dictionary keys and their counts are stored as dictionary values.

The program to remove duplicate words from String in Python using Counter class is given below:-

from collections import Counter


def remove_duplicates(input):
    # split input string separated by space
    input = input.split(" ")

    # now creating a dictionary using a Counter method 
    # which will have strings as keys and their frequencies as value
    UniqW = Counter(input)

    # joins two adjoining segments in iterable way
    s = " ".join(UniqW.keys())
    print(s)


# Driver program
if __name__ == "__main__":
    input = "Java is a high-level language and C++ is also a high-level language"
    remove_duplicates(input)

Output:-

Java is a high-level language and C++ also

Python Remove Duplicate Words From String Using For Loop

In this type, we will not use any external library to remove duplicate words from the string. Here, We will delete the duplicate words in a sentence using for loop.  For this, the program is written below.

Python remove duplicate words from string using For loop

# Program without using any external library
a = "Python is popular and Java is also popular"
l = a.split()
b = []
for i in l:
    # If the condition is used to store a unique string
    # in another list 'b'
    if (a.count(i) >= 1 and (i not in b)):
        b.append(i)
print(' '.join(b))

Output:-

Python is popular and Java also

Remove Duplicate Words From String In Python Using Set

Set is an in-built data type in python that does not allow any repetition value. So here, we will use this concept to remove duplicate words in a string. Since it doesn’t maintain insertion order therefore we will not get the result as the above program.

string = '''Java is a high-level language 
and C++ is also a high-level language'''

l = string.split()
a = set(l)
print(' '.join(a))

Output:-

high-level C++ is also Java and a language

If you enjoyed this post, share it with your friends. Do you want to share more information about the topic discussed above or do you find anything incorrect? Let us know in the comments. Thank you!

Leave a Comment

Your email address will not be published. Required fields are marked *