Bulk Add Queue Items and Add Queue Item have different behavior regarding to String

Hi,
I’m having issue has I’m trying to pass path in Transaction item for Queues in a pipeline.

Path are usually containing "" character, which is not escaped using the AddQueueItem activity (which is fine), but those strings are escaped using the Bulk Add Queue Items. This also means that I can’t upload unescaped strings using the the bulk upload in Orchestrator.

Is there any reason for that, it feels not consistent ? Is there any workaround ? I don’t want to have to parse it when getting the queues, because it means that I have to implement it on every consumer.

E.g :


In order to prevent issues with escaped quotes, I’m assuming CHR(34) would work. In VB this character corresponds to the quotation marks, so you could try passing this instead of the quotes.
myString = "this\is\my\string";
toAddToQueue = CHR(34) + myString + CHR(34);

Using this they should be uniform. I also think that there is a weird interaction when manually entering text into the ItemInformation Fields, maybe try passing the strings in as variables and see if they line up? I would assume that as variables they would be the same, but I could be wrong.

1 Like

The issue is not with escaped quote, but escaped backslash :D. I tried different stuff. Using variable, using argument and giving arguments, using a Datatable …

The outcomes is the same : Bulk add escaped characters to my string, while Add Queue Item doesn’t

Hmm, that’s pretty strange. Try making the string "this\is\my\string\\"?

Other than that there are some hackish solution like the following:
Making the queue item have two inputs: File Name, and Directory Path? Then when the process that consumes this queue accesses the Transaction Items runs it can just store it as a variable like this:
filePath = TransactionItem.SpecificContent("Directory Path").ToString + "\\" + TransactionItem.SpecificContent("File Name").ToString;

It’s not as clean as I’d like, but it works for the time being at least… I think if you add two backslashes it will not delimit it though.

Using the AddQueue :


Using the Bulk :

I can’t use the method you describe without adding this to all process that consume the queue, and can’t to that in the Bulk Upload activity as I’m uploading all the transaction ^^'.

I’ll will just add a parser from now, just feel weird.

So I played around with it a bit, and I get the same errors. I don’t know why it’s delimiting by default the ", \, etc if it doesn’t get rid of the delimiters when returning the string when called by Get Transaction Item.

One thing I did test though is that since “/” isn’t an illegal character, you can input the file path with forward slashes and it will work fine. It doesn’t try to delimit it and as a result Windows has no issues browsing for files with forward slashes despite defaulting to backslashes. So if you bulk upload the paths with NO QUOTES (it will delimit these with a backslash obviously) around them it should work fine. Here is an image example showing my testing:

NOTE: Mixing and matching \ and / in file paths does NOT cause errors either, so this should be a potential resolution? Just replace any \ in the datatable with /.

Yep ! It s auto added with character that have to be escapped :'l. I’ll parse it one time or another, but I feel inconsistent about having the slash in one direction in part of my process and in the other direction afterward (I’m unusure all apps could use a path using ‘/’ and also, I get path using Ctrl+C as I’m in VDI. Is fine, as I’m not alone to have that issue.
Thanks !

Hello Again :smiley:. I just realized that there is a total issue with that activity. Using Bulk Upload, the String encoding is converted. For instance, using a field in Specific content with accent or some special character like é or è will just be converted into UTF8 code, and getting the values from a Get Transaction will keep it as a string, with UTF8 code inside … I feel it really painfull, so I’ll keep using the Add Queue Item.

Any news / bugtracker I can post in ? Beside performance, there is no actual way of handling those character using the upload method in Orchestrator (through CSV upload). Even more ankward, Specific content isn’t converted into code, but the value of the specific is.
Example : image using image

Also getting the same issue with ‘Bulk Add Queue Items’ activity.
a DataTable with String column containing string formatted currencies (i.e., $99.00) look fine when in the DataTable. Once this is added to Orchestrator, it pre-prends \t to $99.00, making it \t$99.00 in Orchestrator.

image

Hi all, trying to bump up this topic. Any solutions/help regarding importing .csv files with strings containing special characters when uploading queue items into orchestrator? Whatever characters imported will be converted to \ufff

Uploading csv manually, I have no workaround, but using a robot you can upload item one by one, but you lose the atomicty of bulk upload if one fails. @MichaelL

But this is no error, those have to be converted to be stored, but if you know you will have those, you need to convert back to char before using it…

How is this no error ? Why is adding the one transaction and bulk add having different behavior ? Yes we can do that, but it’s painfull. If it needs to be done to “store them”, why isn’t it done internally, or is there anywhere in the doc that it does say that ?

Some things are not the way we want it to be, but that wont mean it is a bug or an error, it is that way for a technical reason… Why you say it is a pain to simply do like Convert.ToChar(myvalue)? We usually will do something like this TransactionItem.SpecificContent(“Directory Path”) or Convert.ToInt32(TransactionItem.SpecificContent(“Directory Path”)) so doing Convert.ToChar(TransactionItem.SpecificContent(“Directory Path”)) would not be any more difficult to do…

→ We cannot easly see fields in Orchestrator Queues.
→ We need to convert it also anywhere else (for instance through API).
→ The difference between bulk and non-bulk upload differs for each fields, this behavior is strange and not mentionned anywhere.
→ I can use special characters to name the field, but not inside …
→ Converting is ok, but why do I need to convert when I’m using already TransactionItem.SpecificContent(“field”).ToString ?

Can you give me a workflow example to solve the issue ?
Here is a sample I’m doing :

Here is what Orchestrator shows :

Here is what I receive, using Get Queue Items back in studio :
image
image

Convert.ToString give me this, so not really usefull …

how are you going to handle a file when I name it \u00eé.ext ?

Its frustrating because when using path inside transaction, I’m forced to use only Add Queue Item rather than Bulk Upload

Bulk will use json to upload the data and that will require encoding, thats why it is different than using the single activity that will use object model, the documentation is not detailed enough, thats why we have support or forum so we can try to get extra help, as of the Orchestrator website it cant be set to accept every encoding in the world just so you can see the data inside the transactions… The important thing is that after the bulk or non bulk upload, nothing is lost… I understand the frustration when those thing happen but some things are just the way they are, i also use a non standard encoding here in Brazil and know the pain, but it is something we have to do…

 bad code deleted

Hi again,

Exactly, that’s why I’ve posted here :D.

Modifying the last line of your code so it uses the re-encoded string, and using it in Invoke Workflow, it still gives me same result :

Note: Is that really ASCII ? Beside, it still escape characters, and doesn’t actually convert everything, so it cant handle the string \u00eé without being destructive (meaning re-encoding will give back éé in the best case.

Sorry about that code, i over complicated things and messed up:
Regex.Replace("\u00e9", "\\u(?<Value>[a-zA-Z0-9]{4})", Function(m) ChrW(Integer.Parse(m.Groups("Value").Value, NumberStyles.HexNumber)).ToString())

I still think that’s getting overcomplicated as a user point of view :frowning: .
I mean, here we’re only focusing about the “é” characters, but what about every others … ?
I couldn’t predict every specific characters a user would write in a file.
Still, i’m going to use Add Queue in a Loop, as it’s just straight forward …