Here’s a new video that takes a quick swing at the Preview version of the UiPath OpenAI Integration Service connector.
I’ve also shared some of work in the downloads that’s linked in the video description below including the use of core OpenAI APIs and the tiktoken APIs using Python.
The video track covers the following:
00:00 - Intro
00:17 - PROLOGUE
03:52 - Connector Setup in Integration Service
05:09 - Using the OpenAI Connector in UiPath Studio
Thanks so much for the thorough video and rundown of the new OpenAI Connector. I think you brought up some very salient points around prompt-engineering/being aware of token consumption that we’re actively considering, as well.
As a note to the folks who may watch the video and/or review this thread, the ‘Instruction’ field in additional properties of the ‘Chat Completion’ activity currently has a default value of role: system content: you are a helpful assistant. Strings used in that field count towards your total token consumption. We are going to remove that default value so that it isn’t sent by accident, but it is something that users should be wary of.
Also, regarding the multiple ‘Instructions’ (or role/content) outside of the prompt, do you feel like that would be a valuable addition to the curated UiPath activity? We could always include the ability to add multiple ‘Instructions’ to capture context to be sent to the model along with the prompt. Let me know.
I’m very open to any other suggestions you or folks may have. Thanks again!
Hi @AndyMenon,
Thanks for sharing this content. It will definitely benefit users of this integration especially around practicing proper prompt engineering to conserve token usage. Although, looking at it in business automation angle leveraging UiPath, I believe single turn instructions with text completions (instructgpt/gpt3 models), that yields definite action paths from the workflow, would be most fitting for RPA use cases, as opposed to building conversational/chat based workflows. But still, I definitely agree that having the full context/history of the conversation would be useful in most a number of RPA use cases.
I’m planning to create similar content around using embeddings API as a way to build a searchable and custom knowledge base without having to include full data source/context (which is also impossible due to token limitation) into every single prompt, thereby conserving tokens and making it much more scalable.
Hi @Scott_Schoenberger,
Few shot prompting with multiple instructions, would definitely help in priming the prompt which can yield better output. Adding to this, I think i’ve also read somewhere from the documentations or cookbook that although gpt-3.5-turbo-0301 does not generally pay as much attention to the system message, while the latest versions as of current e.g. gpt-4-0314 (which API is not released yet to the public) will be paying attention on the system message much more. Therefore, for gpt-3.5 it would be good to place important instructions in the user message, but in the future versions system instructions will be more relevant.
Hi @zell12 , I agree with you . It’s definitely a challenge to create context based applications. But they will become necessary. For example in a CRM pipeline, if we have multiple updates along the way from the point of origination of an Opportunity, all the way to a Win, and AI is leveraged to interact with the customers, then it must be capable of understanding context and communicating to the customers.
Another use case I can theorize is health care where a patient caregivers go back into the history of the patient over a period of time. Context in such situations definitely matter as the duration of the care progresses and AI is tapped in to contribute in some way to the entire caregiving life cycle.
Next thing I would look at is the new GPT-4 models. They support 8K and 32K context tiers. Which means it opens up opportunities for us to tap into context-based capabilities of the LLM.
Besides, a context can also help get answers that we (may be able to) trust. As is today, absolute completions from the AI tend to have errors. But I’m hoping a contextual backing to the prompt will help elevate trust in situations where human-AI interactions must depend on trust.
How would AI help us if it cannot do more than just give us absolute answers, or multiple choice answers for us to pick from. Or if it cannot follow cues from their human overseers and respond accordingly? After all that is the “Intelligence” part of it that we are most interested in!
Thanks for your inputs. Many interesting conversations lie ahead i guess?
Thanks for taking the time to have this discussion with me and also for watching the video. I truly appreciate the fact that we are heading in a positive direction.
I will definitely follow your recommendations that you have published here. It’s really great to have received an answer on how we can reduce token expenditure when using the UiPath IS Connector for OpenAI.
I’ve chimed my thoughts on context based conversations elsewhere on this discussion thread.
Conversional AI is hard ask if the token number will keep pilling up, so newer models should have more tokens (but it will be expensive).
Also I noticed that when I downloaded the package the first time, and I drag and drop the activity “generate chat completion” the process crashed and I had to reopen it, I created a new process and the same thing happened again, but since then I couldn’t reproduce the error.
The studio crash happened to me as well and I made a mention of it in my video.
My theory is that when dragging the activity on to the designer surface it connects to Integration Service and renders the default connection in the connections dropdown. Maybe something is going on there. Also if we were to repeat adding the activity by disconnecting UiPath assistant from the Automation Cloud , then it might cause this failure as the Studio would be disconnected from the Orchestrator .
I ran another test with the changes and I see that the token count returned by the UiPath OpenAI connector matches with the core request that returned 22 tokens.
I was playing around with the connector and I tried to construct prompts that are structured based on OpenAI documentation and then send the prompt requests using the connector.
I’ve attached the file with the screen shots of the different tests as there are too many to list.
As seen from the test, the prompt is not a simple string, but a string form of a more elaborate prompt.
Again, this is purely experimental, but I’m sending it your way.
Thanks, Andy. That’s because the ‘Prompt’ field is curated in such a way as to only be inclusive of the ‘content’ part of the request (not role). The role will always be ‘user’. You can use the instruction field to add a ‘system’ role. Currently, you can’t add an ‘assistant’ role in this activity pack.
The reason for this is, again, to offer a more curated experience for less technical users.
What may be something we consider is the ability to add multiple instruction fields and to choose the role type for each instruction. That way you could add a ‘system’+‘content’ role and ‘assistant’+‘content’ along with the prompt that will continue to be ‘user’+‘content’. I think that’s what you’re asking for. Let me know.
Right. One reason why I put the complex instruction in the prompt text field of the connector because there isn’t an assistant role. I was curious to see what I get back.
We are thinking along the same line I guess. In the least, the Connector must resemble the use of the Core API usage whilst making it easy to use for less technical users . Talking about technical vs. less-technical use, would it be possible to switch the connector to an advanced mode in which we can directly post pre-formatted prompts with multiple system, assistant and user roles?
In the end all of this should make it for a seamless transition to use the connector with the versatility to address all range of users.
Thank you for taking my inputs into consideration.
Hey @AndyMenon, The distinction that you’re referring to ie. technical v. less technical users is something that we’ve baked into the product via curated activities and generic activities. You’ll note that for some connectors, we’ve also added generic CRUD activities like List All Records, Create Record, Delete Record, Update Record, etc. Those offer users unmodified access to any objects/methods that the vendor supports. Curated activities, offer things like method-chaining and simplified data types to abstract complexity and make it easier to focus on automating versus understanding underlying vendor APIs.
The intent with the OpenAI Connectors in preview was to offer simple curated activities to start out. We will definitely consider generic activities for the future, but I believe that the ability to configure ‘role’ would be a valuable part of the curated activity, as well.
Thank you so much for your inputs. They’re highly appreciated. Keep them coming.
Absolutely! I should have been much more clearer. I truly meant to state that the advanced mode can come in later iterations. But the roles are definitely the much valued priority.
Thanks for this, Russel! Would love to hear more about your approach to embeddings and how UiPath might be able to assist/make that process seamless. Admittedly, we aren’t thinking about adding embeddings as another activity in the connector, but likely abstracting to a degree to make it easy for users to create an embedding/index of their data. The resulting embedding/model could be referenced in a Studio process. Let me know if you’d be up for a chat sometime.
Roger that on the multi-instruction piece. I’ll see if that can be prioritized for a future version.
We do have access to GPT-4 and plan on releasing the multimodal modal as a new activity in future versions.
Anil_G
ashokkarale
Ajay_Mishra
Gautham_Pattabiraman
BHUSHAN_NAGAONKAR1
vrdabberu
ABHIMANYU_THITE1
lrtetala
samantha_shah
shyamala_shyamu