Running workflows using the Raspberrypi Pico

News Tutorials
, ,
1

Target audience: Developers

This approach of using a microcontroller for automation is a proof of concept. I had lots of fun developing. In this tutorial, I show you how the Raspberrypi Pico can be used as an USB trigger to run UiPath workflows or any other automation.

My aim was to use the Raspberrypi Pico as my daily productivity booster, wherein the script would open up and log me into my most used applications (about 8) and accounts. This would only trigger when I insert the Pico into the USB slot. Typically, this approach of automating my daily chore saves me about 3 minutes per day and a whole lot of clicks and navigation. Wake up, switch the pc on, let the Pico prepare my day!

Before we begin, here are the applications and software needed to move forward

  1. Thonny :white_check_mark:
  2. UiRobot.exe (If you have installed UiPath Studio you are :white_check_mark:)

Here is an official walkthrough to setup the Raspberrypi Pico. As you go through the walkthrough, instead of using MicroPython firmware, download the CircuitPython firmware from Pico Download and continue with the setup process. In my case, I use CircuitPython 7.2.5

image
image1390×640 236 KB

Note : Since the MicroPython firmware does not support Human Interface Device (HID), we need to flash the CircuitPython firmware to the Pico. Once this is flashed into your Pico, open up Thonny and choose → Run → Select interpreter and choose CircuitPython (generic).

image
image876×541 30.9 KB

After selecting CircuitPython (generic) as the interpreter, you will see the following shell output.

image
image832×160 4.04 KB

Good! However, when we check for the HID libraries with the help("modules") command, you will not find the required HID drivers in the default version of CircuitPython firmware. CircuitPython includes all MicroPython libraries of many custom libraries. However, we need to add the HID driver to the lib folder of the Pico ourselves.

Thankfully, Adafruit already has done the heavy-lifting by releasing a HID driver.
Releases · adafruit/Adafruit_CircuitPython_HID · GitHub.

Since we downloded CircuitPython 7.2.5 during our initial setup, lets download the corresponding version of adafruit_hid driver.

image
image1251×929 54.4 KB

Download the zip file → Unzip the file → Copy the file in the lib folder
image
image1071×111 5.74 KB

→ Copy adafruit_hid to the Pico (It will be visible as a drive in your PC) and done :white_check_mark:

image
image944×115 4.69 KB

Recap : We now have installed Thonny, CircuitPython and HID library from Adafruit.

After the above setps, your Raspberrypi Pico can send keys to your computer, just like any other HID (mouse, keyboard, joystick). Great, but wait, keyboards come in various layouts and languages, how do we tell Pico which keyboard language and layout to choose?

That is where another library comes in. Cheers to Neradoc for sharing these. https://github.com/Neradoc/Circuitpython_Keyboard_Layouts/blob/main/PICODUCKY.md

Similar to the adafruit_hid driver, Download Circuitpython_Keyboard_Layouts zip file → Unzip → Navigate to the lib folder → Copy all or specific languages and keyboard layouts → Paste them in the lib folder of your Pico.

image
image1309×1228 66.1 KB

To check if all the drivers and libraries are installed, we run the following command in the Thonny shell (ensure you stop the Pico once so that it reboots). You do this by clicking on the Stop button once in Thonny.

import usb_hid
from adafruit_hid.keyboard import Keyboard
from adafruit_hid.keycode import Keycode
import time
from keyboard_layout_win_da import KeyboardLayout
from keycode_win_da import Keycode
kbd = Keyboard(usb_hid.devices)
layout = KeyboardLayout(kbd)
help(Keycode)

image
The output shows us all the mapped keycodes available for the Danish/Norwegian language and keyboard layout. In this tutorial, I will use the Danish :denmark: keyboard layout, which is the closest I can get to a Norwegian (NOB) layout of windows :norway:

image

Let the scripting begin!

import usb_hid
import time
from adafruit_hid.keyboard import Keyboard
from adafruit_hid.keycode import Keycode
from keyboard_layout_win_da import KeyboardLayout
from keycode_win_da import Keycode

# Keyboardlayouts : https://github.com/Neradoc/Circuitpython_Keyboard_Layouts/blob/main/PICODUCKY.md
# Set up a keyboard device and layout
kbd = Keyboard(usb_hid.devices)
layout = KeyboardLayout(kbd)

# Lets populate program dictionary to run
programs = {"UiRobot" : "C:\\Users\\%username%\\AppData\\Local\\Programs\\UiPath\\Studio\\UiRobot.exe execute --file  C:\\Users\\%username%\\Documents\\UiPath\\ForumAnswers\\BrowserTabManpulation.xaml" }

# Some keys need to be specified within a keycode dict. I do this to make the next steps easier. 
keycodeDict = {"\\":46,
               "-": 56,
               "ø": 51,
               "æ": 52,
               "å": 47,
               ":": Keycode.SEMICOLON,
               ".": Keycode.PERIOD,
               " ": Keycode.SPACEBAR}

The first helper function maps keycodes to letters and numbers.

def letter_to_int(index):
    """
    Returns the letter from the given index
    """
    alphabet = list('    abcdefghijklmnopqrstuvwxyz1234567890')
    return alphabet[index]

Why does the alphabet list have empty spaces in the first 4 items? That is because the keycode we want to use start from an index 4. Key A is found at keycode number 4. When in doubt, help(Keycode) command and Danish - Virtual Keys - Keyboard Layout Info can save you a lot of time.

image

In a for loop, we add all required alphabets and digits to the keycodeDict dictionary.

for i in range(4,40):
    keycodeDict[letter_to_int(i)] = i  # helper to map keycodeDict for all items in alphabet

The main helper function, which sends keys to the host machine.

def WriteText(text):
    """
    Send keys depending on the text input. 
    Special keys need SHIFT, rest of the keycodes found in keycodeDict
    """
    for i in text:
        if i==":":
           kbd.send(Keycode.SHIFT, 55)
        elif i=="(":
           kbd.send(Keycode.SHIFT, 37)
        elif i==")":
           kbd.send(Keycode.SHIFT, 38)
        elif i=="/":
            kbd.send(Keycode.SHIFT, 36)
        elif i=="_":
            kbd.send(Keycode.SHIFT, 56)
        elif i=="%":
            kbd.send(Keycode.SHIFT, 34)
        else:
            key = keycodeDict[i.lower()]
            kbd.send(key)

The code which performs the automation

def RunCommand(CommandText):
    """
    Run the automation when the trigger is active. 
    """
    kbd.send(Keycode.GUI, Keycode.R)  # Send windows+r
    time.sleep(0.2)  # wait for RUN window
   
    WriteText(CommandText)  # write the command 
    time.sleep(2)  # wait for type into to finish
    kbd.send(Keycode.TAB)  # Send Tab
    kbd.send(Keycode.RETURN)  # Send enter

Remember we are running this on a microcontroller. It does not understand that we need to run the above code only once when it is connected to the host machine. There is no pre and post event checks as the controller / firmware in its current form has no way to parse the GUI applications on screen. All we can exploit is static delays in-between sending keys. This is also the biggest drawback of this whole approach.

We need a while loop to ensure that the automation starts only when the microcontroller is activated (using a boolean check) and once done, it stops the loop i.e., run only once.

started = False
while started==False:
    time.sleep(10)  # I have some other automation. This delay can be lower. 
    RunCommand(programs["UiRobot"])
    started = True

When we insert the Pico in the USB slot, the 10 second countdown kicks in and the chosen automation / UiPath workflow runs.

image
image1920×2560 321 KB

A short video showing the behaviour when the Pico is connected to the host machine.

5 Likes