Remove duplicates from text file

Hi all,
I have a text file containing strings as shown above and I would like to remove the duplicates so it only gives me the following output:

What is the best way to do this?

You can use Distinct()

uniqueRows = fileText.Split(Environment.NewLine.ToCharArray()).Distinct().ToArray()

uniqueRows is an array of strings



Thanks but how do I get the output in a write text file after this? Since uniqueRows is an array of strings, it returns an error when I try to use this variable to create the write text file.

Hi, thanks for your reply - how do I get it to output a write text file after this step? I am using outstring as the variable to create the file but it returns an error saying 1-dimensional array of String cannot be converted to String.

use delete activity after assign to outstring array string variable like:


then Create file with same name then use

For Each loop with append line Activity.


definitely you will get your required output

Let me know if any further doubt

Hi @keshana_t try this workflow for reference
Main.xaml (7.8 KB)


Nived N
Happy Automation

You can join uniqueRows like this:

newText = String.Join(Environment.NewLine, uniqueRows)

You can use this


Instead use of delete activity and create file .

Thank you - I now get an output but there is a gap between them which I do not want. How can I remove the space?

Thanks - The duplicate rows have been removed but I have a space between the urls as seen below. How can I remove that?

Hi Nived,
Thanks for your working - however I am using v2018.4.5 and it in the for activity, it says missing or invalid activity?

Use StringSplitOptions.RemoveEmptyEntries:

newText = String.Join(Environment.NewLine, fileText.Split(Environment.NewLine.ToCharArray(), StringSplitOptions.RemoveEmptyEntries).Distinct().ToArray())

Hi @keshana_t can u share screenshot of workflow?

Thank you!

1 Like

Thanks for your help but I’ve managed to resolve the issue. Really appreciate it!

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.