How to read single csv file by multiple automation processes parallel

Help Activities
1

Hello,
I have a csv file that has to be read by multiple automation processes. Or even is any way to read only the csv file when it is opened?
Looking forward to hearing your responses.

Thanks,
Manohar E

2

Have you experienced any issues when attempting to do this? Or do you want to validate if it is possible?

3

@Forum_Chatbot, do you see a problem if two processes read a CSV file simultaneously using Read CSV activity?

1 Like
4

Reading a CSV file simultaneously by multiple automation processes can pose certain challenges. Here is what I found:

  1. Read same excel at the same time by multiple robots:

    • Concerns about whether multiple robots can access the same configuration files stored in Excel simultaneously. It’s a common discussion among the community.[1]

  2. Parallel http request from datatable:

    • Insights into launching multiple background processes at the same time and considerations when your dispatcher reads the CSV file.[2]

  3. Unable to read same excel file twice in the same process:

    • Discusses problems faced when trying to read the same CSV file multiple times within the same automation process and potential solutions.[3]

  4. Parallel Activity in CSV file:

    • Discusses running parallel activities on a CSV file with 5 Users for scenarios where simultaneous processing is needed.[4]

To ensure seamless operation:

  • Concurrency Control: Handle file locks well, making sure to prevent collisions or corruptions.
  • Reading Mode: Set proper reading modes as ‘Read-only’ if applicable.
  • Logging and monitoring: Implement robust logs and monitoring in case of access conflicts.

Please refer to these discussions for more in-depth community-driven insights:

Depending on your specific process requirements and environment, further fine-tuning and testing might be necessary.

If you need more detailed guidance tailored specifically to your use case, please provide additional details about your setup and process requirements.

  1. Read same excel at the same time by multiple robots ↩︎

  2. Parallel http request from datatable ↩︎

  3. Unable to read same excel file twice in the same process ↩︎

  4. Parallel Activity in CSV file ↩︎

5

Considering this, and that CSV doesn’t have lock mechanisms I see no problem with reading the file simultaneously

6

Yes, I am getting error stating that the file has being used by some other process.

7

In that case, I think the best is to copy it to Data\Temp and then read it

1 Like
8

Please try that out. I would assume that copying might also cause the same error.

@manohar_e, could you elaborate a bit on the business process behind this? Where does the CSV come from? How often does it change? How many parallel processes would, in theory, need to interact with it?

9

@dokumentor - Sorry, I misread. You mean one file. First, copy it for each process, then use it. Yes, this should do the trick.

There could be an issue if more than one copy occurs at the same time (especially if the CSV file is large), but adding a retry with a small delay should help.

10

Other than copying, do we have any other solution?

11

@manohar_e , try this

and

1 Like