Invoke Power Shell returns wrong encoding value

Hi,
I’m using an Invoke Power Shell activity to run a Python script. This script outputs multiple values that I have to use in a For Each activity later on.

Everything works fine, except one detail : I get the wrong encoding from the PowerShell Output.

When I run my python script alone, I get the correct enconding input, but once it goes through UiPath, it doesn’t work properly anymore.

Example of outputs :
with Python = “Cédric”
with UiPath = “CÚdric”

It converts the accented characters to another type of encoding, and I don’t know how to correct that since it’s only on UiPath level. I already tried manipulating encoding variables in UiPath, but it didn’t fix anything. Here’s the manipulation I’ve done through different Assign activities

// Output from the Invoke PowerShell is named "pyOut". For simplicity, this is a code that can be ran out of a For Each activity
// I pick the 1th element from my "pyOut" var since it's a value that has encoding issue
Encoding iso = Encoding.Latin1
Encoding western = Encoding.GetEncoding("windows-1252")
Byte[] dstBytes = Encoding.Convert(iso, western, iso.GetBytes(pyOut[1])
string dstString = western.GetString(dstBytes)

This solution was adapted from solutions I saw on StackOverflow. I precise that I’m working on UiPath processes with C# and not .NET

I’m pretty sure I’m close to figuring out what’s the issue, but I would definitely like some help since it takes too much time for its little usefulness on this project.

Thanks for any help you can provide

1 Like

Hi @PythonDev85,

well, I would have a few questions here…but maybe another approach first:
What happens if you directly use the Invoke Python Script activity instead?
https://docs.uipath.com/activities/other/latest/user-guide/invoking-a-python-script

1 Like

Thanks for your answer @RoboHeart

I don’t use this activity as its usability isn’t the best. I tried using it in the past, but it never worked, and almost everywhere I tried to look for answers, people were telling that calling a .py script using Powershell was better.

If you’re curious about my call, it’s something like this

"python 'Path/To/The/Python/Script' $arg1 $arg2"

I obviously set the right parameters like “IsScript” or in the arguments selection panel if there are some required. And I’m already using this strategy in other RPAs that work fine.

As my username suggests, I’m more used to Python, so it’s easy for me to build scripts that can interact with anything else.

Also, I know I could just use workarounds like printing every line I need onto a .csv file for example and then reading it from UiPath, as there’s an encoding parameter in the Read CSV actiivity BUT this is not what I want to do, because I want to reduce the potential use of “transiting files”.

Hope this helps :slight_smile:

Well, I suspect it could be in relation with your environment.
E.g. default encoding in PS<=5.1 is different from newer PS versions. Furthermore, if you have several versions installed UiPath will use the profile default (as far as I know) what can differ from the one where you tested your script manually.
What is the result when you are invoking [System.Text.Encoding]::Default.EncodingName? Please, compare it with the PS output in the environment where you successfully run your script.

Maybe this link could help you:

1 Like

Ok, so I did it, and you’re right, there’s something different !
In Powershell, I get “Western Europe (Windows)”
In UiPath, I get “Unicode (UTF-8)”

So now I understand where is the problem. Still, how do I fix that ?

Well, there could be more dependencies. Python version, data source, PS command, .NET…
In the end, you have to understand and harmonize your encodings.

Just to show you the difference only coming from different PS commands:

Python test script (test.py):

import sys
print(sys.stdout.encoding)
print(b"\xe9".decode("cp1252"))

Different outputs in PS5.1:
image
Different outouts in PS7.3:
image

Changing the encoding output in the Python script didn’t change anything in UiPath.

I know my python script could be made in UiPath using other tools, but this isn’t what I want, as using a Python script gives me more flexibility.

I’m using Python 3.10.10 right now if you’re curious about that. What’s messing with my is that I’m so close to getting the result I need. I just need to be able to convert the output I get, which is in UTF-8, to turn it into Windows-1252. Either it’s at the Powershell reading, or it could be later on, when using the value in UiPath

If you still want to use this combination of PS and Python you should try something like this to change the encoding of the console output:

$env:PYTHONIOENCODING = "utf-8"

or

$OutputEncoding = [Console]::OutputEncoding = (new-object System.Text.UTF8Encoding $false)
$env:PYTHONIOENCODING = "utf-8"

(depends on your env)
image

The line $OutputEncoding = [Console]::OutputEncoding = (new-object System.Text.UTF8Encoding $false) doesn’t work in Invoke Power Shell activity

The line $env:PYTHONIOENCODING = "utf-8" made it worse. Here is the result :
C├®dric

Okay…frankly, I am not sure how the Invoke PS activity is working in the background…but another (last) setting I know what could help is under Windows’s Region > Administrative > Change System Locale…unfortunately it is in beta state but give it a try (sorry, I have a German system):

Coming back at the post after the week-end. Took tries and other ways and finally I got a workaround.
I had a thought the issue might be coming from the Excel file I was calling within my Python script. So instead of reading it and then transmitting the data to UiPath (yeah I know I could’ve just used Read Excel activities, but I’m insisting on using Python for flexibility reasons), I turned my .xlsx file into an SQL table using “Pandas” python library. Now instead of calling the values from the Excel sheet, I call them from the SQL table, and now, I don’t have an encoding issue anymore. So this means, even though I might still encounter an encoding issue in the future for some other obscure reasons, the source of the problem was from the Excel file itself, not Python nor UiPath

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.