Picture in Picture (Experimental)

Hello Nicolas,

I’m very good.

I will give this a shot and keep you posted.

Thanks for the quick response.

Andy

2 Likes

Hmmm… I looked into it. I don’t have that variable in my Environment Variables - neither in System or local user scope.

Is this something we have to add manually?

I’m sure PiP worked on my erstwhile laptop. On this laptop it is failing. Here is the error.

image

thanks.

1 Like

@AndyMenon the environment variable does not exists, you need to create it. Sorry if I wasn’t clear with it from the start.

1 Like

Understood. But I don’t recall seeing this in the documentation either.

Thanks for the clarification.
I will give it another try.

2 Likes

Created the variable and set it to an exaggerated value of 300. Process starts in PiP then appears to run. But in the background there is this error:

image

In Studio I get this error:

System error at initialization: Cannot communicate with the browser, please check the UiPath extension. at Source: Invoke SF_Open_Chrome workflow: Open in Chrome

This wasn’t happening in 2019 versions or earlier. Historically, I never had an issue running PiP.

Let me read that documentation once more.

2 Likes

I hope others find the following helpful. I ran some tests on my work laptop running Windows 10 Pro.

The application tested in PiP has been proven to work without PiP and is already in active service. Therefore the argument that something might be wrong with the application lacks merit.

Test 1 - Change Open Browser Activity Setting UserDataFolderMode > to Default

Browser Window refuses to open as a result of the following Error Message:

Open in Chrome: The browser already runs in the main session. Please stop the browser or set another user data directory and restart the automation in PIP.

The workaround to this problem in Attended Automations might be that the human user has to work with a Browser that is different from Chrome. If that does not work , and if the human user has to stop working to make way for the Robot, then that defeats the very purpose of having PiP.

The other option is Test 2.

FYI: I enjoyed working with this feature in the 2019 version. The Robot ran alongside in PiP mode without interrupting my work. I miss that very much now.

Test 2 - Change Open Browser Activity Setting UserDataFolderMode > to Automatic

Browser Window Opens but subsequent steps fail as a result of this Error

System error at initialization: Cannot communicate with the browser, please check the UiPath extension. at Source: Invoke SF_Open_Chrome workflow: Open in Chrome

This is because the PiP Browser profile folder is being created for the first time and we have to enable the Chrome Browser Plugin for the first time inside of the PiP window.

Restarting the application will go past this error, but will fail on the Click Activity

Reason: SimulateClick in the application was set to True. After I switched it to Blank, the process worked inside of PiP.

Strangely, SimulateType works but not SimulateClick! :open_mouth:

The next options was to set SendWindowsMessages to True and setting SimulateClick to Blank - And this worked in PiP

Test 3 - Purely Accidental - VPN Connectivity Fails when PiP Window is Open

I lost Wi-Fi connection midway during the test and when I got it back, Cisco VPN failed to reconnect complaining that a connection from a Remote Desktop Window cannot be established.

I was able to connect successfully once I closed the PiP Desktop.

3 Likes

Hi @AndyMenon

Does the Test 2 scenario work for consecutive runs after you enabled the extension in the separate Chrome profile?

2 Likes

Yep. Here is a test from the last 10 minutes. I repeated the test without any changes to my environment since my last test and ran the Automation from Studio after putting my laptop to sleep overnight. Next step for me is to package/deploy this Automation with PiP-enabled switch and test it via UiPath Assistant.

Nevertheless, here is a redacted screen shot so that others may find it helpful.

Markers 2 & 3 are important observations as we anticipate that Attended Automation users may ask these questions during training. They would also want parallel access to Chrome and Excel as the Robot works alongside them which is must have!

We have to train end users to enable the Chrome/Edge plugins for the first time they launch their Automations in PiP. May run into some bumpy ride here :frowning:

  1. Latest Timestamp of the test and the screen capture
  2. UiPath Plugin is enabled without having to do anything after we enabled it the first time - another Google session is accessible to user on the normal desktop offscreen
  3. Excel Application is open - another Excel session is accessible simultaneously on a second screen not shown
  4. SimulateType works on and off PiP desktop, but only one link I cannot reveal here had to be switched to use SendWindowsMessage so that it would work effectively on and off PiP mode
1 Like

I have tested the Automation from UiPath Assistant. Two Major Issues!

SimulateClick is highly unreliable in PiP mode!

  1. Out of the 15 or so click actions, I had to prompt the Robot to click on at least 6 of them!
  2. Robot would find the link but refuse to click on it and there is no set pattern
  3. In non-PiP mode, the Robot works reliably for 1000s of such clicks with SimulateClick set to True (as proven in currently active service).

I would appreciate it if someone could test this feature in Salesforce.

SendWindowMessages setting does not persist in Studio on Click Activity

I don’t understand this. After my first test in Studio today, the settings for the SendWindowMessages was wiped out.

The settings highlighted on the right (below) do not hold. Each time SimulateClick is reset back to True and SendWindowMessages is reset to Blank! I’m sure I set it the way it looks in the image because I have documented the problem in the Annotation. I have to remember to look at this activity every time before I run a test, or deploy the Automation.

Studio version is 20.10.2.

1 Like

Yes. Only one PiP per machine is available. This is a technical limitation

1 Like

@AndyMenon we will retest simulate in PIP but have not seen this unreliability so far. Regarding the SendWindowMessages property resetting after the first run, I was not able to reproduce it testing with UIAutomation.Activities 20.10.2-preview nor 20.10.6. What UIAutomation package are u using exactly? Thank you.

1 Like

@AndyMenon regarding training the end users to enable the Chrome/Edge plugins for the first time they launch their Automations in PiP, the OpenBrowser activity has a feature which can be used to open the browser with the configured profile at design time and enable the plugins before actually running in PIP. image

2 Likes

Hi @gheorghestan,

SimulateClick does concern me. I ran the PiP test a second time, and I had to manually click through at least two times during this test. Better than 6 times from the first test.

I’m not sure if this is going to help at all, but please note the following operating circumstances of the SimulateClick failures in PiP:

The SimulateClick failing intermittently happens on a page that gets loaded into the Browser in response to a previous search activity . The link that shows up is dynamic and changes each time.

The other location where SimulateClick fails entirely is that this link opens a new tab in the Browser and initiates a PDF download. This is the only link that functions differently from all other clickable links in the Automation. It works reliably on the normal desktop, but fails 100% in PiP.

These are my package versions:

The SendWindowMessages reset is really an anomaly. This is the first time I have seen this. I’m thinking if I have to delete the entire Click activity and recreate it from scratch. Sounds silly, but I actually created this Sequence from an existing one, and then changed things inside the new copy.

Thanks

1 Like

Hello again @gheorghestan,

Sorry if I wasn’t clear on elaborating the problem. But how would an end user in a business team who just runs automations and does not have access to the code in UiPath Studio, enable Browser plugins at design time in Studio? I’m talking about training these business users as described in my earlier post .

2 Likes

I am getting this error while using PiP. I posted this earlier but official support was not available then as it was in experimental phase

Cannot start the process in Picture in Picture mode because the feature is not enabled on your computer.
To use this, you have to start the process again and accept to enable it.

at UiPath.ChildSession.Client.ChildSessionClient.<>c__DisplayClass6_0.<b__0>d.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at UiPath.Service.UserHost.AttendedServiceProxy.d__54.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at UiPath.CoreIpc.Server.d__22.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at UiPath.CoreIpc.Server.d__21.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at UiPath.CoreIpc.Server.<>c__DisplayClass4_2.<<-ctor>b__6>d.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at UiPath.CoreIpc.IOHelpers.d__12.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at UiPath.CoreIpc.IOHelpers.d__18`1.MoveNext()

The feature does not work on Win10 Home.

What is your OS version?

Its Win 10 Enterprise

Home Editions of Windows 7, 8 and 10 are not supported

Is Windows 10 Pro supported, and if so does it require multiple sign in to be enabled?

Hi @Brandon_Cunningham,

It works on Windows 10 Pro.
Regarding the multiple sign-in: depends a lot on the setup you have. For example, if you’re using a login mechanism different than username/password (PIN, face recognition, etc.), then you’ll be asked to login each time. Also, after you activate PiP, you’ll be asked to enter the password until your first re-login (you can solve this by logging out and in again)

1 Like