Remove duplicates from text file

Hi all,
image
I have a text file containing strings as shown above and I would like to remove the duplicates so it only gives me the following output:

https://google.com/home
https://google.com/feedback

What is the best way to do this?

You can use Distinct()

uniqueRows = fileText.Split(Environment.NewLine.ToCharArray()).Distinct().ToArray()

uniqueRows is an array of strings

then

image

Thanks but how do I get the output in a write text file after this? Since uniqueRows is an array of strings, it returns an error when I try to use this variable to create the write text file.

Hi, thanks for your reply - how do I get it to output a write text file after this step? I am using outstring as the variable to create the file but it returns an error saying 1-dimensional array of String cannot be converted to String.

use delete activity after assign to outstring array string variable like:

image

then Create file with same name then use

For Each loop with append line Activity.

image

definitely you will get your required output

Let me know if any further doubt

Hi @keshana_t try this workflow for reference
Main.xaml (7.8 KB)

Regards

Nived N
Happy Automation

You can join uniqueRows like this:

newText = String.Join(Environment.NewLine, uniqueRows)

You can use this

image

Instead use of delete activity and create file .

Thank you - I now get an output but there is a gap between them which I do not want. How can I remove the space?
image

Thanks - The duplicate rows have been removed but I have a space between the urls as seen below. How can I remove that?
image

Hi Nived,
Thanks for your working - however I am using v2018.4.5 and it in the for activity, it says missing or invalid activity?

Use StringSplitOptions.RemoveEmptyEntries:

newText = String.Join(Environment.NewLine, fileText.Split(Environment.NewLine.ToCharArray(), StringSplitOptions.RemoveEmptyEntries).Distinct().ToArray())

Hi @keshana_t can u share screenshot of workflow?

Thank you!

1 Like

Thanks for your help but I’ve managed to resolve the issue. Really appreciate it!

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.