Finding empty spot on PDF to place stamp

Help Studio
,
1

Hi!
Im working on a project where I have to search trough a drawing containing lines and text.
I need to find an empty spot in the PDF.
My issue is that none of the drawings are the same so I have to search the pdf for a white area, as wide as the stamp, to make sure that the stamp does not cross any text or lines. Is this possible? I have tried to achieve this with a find image activity with a white image but i cannot get it to work properly. Most of the pdf files are created digitally but a few are also scanned.

2

Hi @johan.karlsson

It seems like a pretty challenging thing to do but here is an answer from Chat GPT to maybe give you some hints/tip:

Here’s a suggested approach using UiPath:

  1. Convert PDF to Image (if necessary): Since you’re dealing with scanned PDFs, it might be helpful to first convert the PDF to images. UiPath has activities for this.
  2. Process Images:(Check next blockquote for more information)
  • Loop through the images, if there are multiple pages.
  • For each image, use image processing techniques to identify white areas.
  • You can use a combination of techniques like color detection, thresholding, and contour detection to find areas that are predominantly white.
  1. Check for Stamp Placement:(Check next blockquote for more information)
  • Once you’ve identified potential empty spots, you can then check if they are suitable for stamp placement.
  • This might involve checking if there are any nearby lines or text that could be affected by the stamp.
  1. Stamp the PDF:(Check next blockquote for more information)
  • Once you’ve identified a suitable spot, you can then overlay the stamp onto the image.
  1. Reassemble PDF (if necessary):(Check next blockquote for more information) If you converted the PDF to images earlier, you’ll need to reassemble the stamped images back into a PDF.
  2. Save and Manage Files:
  • Save the final PDF with the stamped image.

Regarding your attempt with the “Find Image” activity, it’s important to note that it might not work as expected, especially if the drawings are not standardized or if there’s any variation in the white areas. Image matching can be sensitive to slight changes.

Remember that this process might require some experimentation and tuning to work effectively for your specific set of drawings. Also, consider implementing error handling and validation steps to ensure that the process works robustly.

If you’re facing specific issues with any of the steps or if you need more detailed guidance, feel free to let me know!

To process images in UiPath and identify white areas, you can follow these steps:

  1. Loop Through Images (If Multiple Pages):
  • Use the Directory.GetFiles activity to get a list of image files from a specified directory.
  • Use a For Each loop to process each image one by one.
  1. **Image Processing Techniques:**For this step, you’ll need to use activities from the UiPath.UIAutomation.Activities package. Specifically, you’ll need the “Invoke Method” activity to call methods of the System.Drawing.Bitmap class.Here’s an example of how you can perform basic image processing techniques to identify white areas:
  • **Load Image:**Use the Bitmap class to load the image:
    • Drag an “Invoke Method” activity into your workflow.
    • Set the TargetObject to new System.Drawing.Bitmap("path_to_image").
  • **Convert to Grayscale:**Convert the image to grayscale for easier processing:
    • Add another “Invoke Method” activity.
    • Set the TargetObject to the loaded image.
    • Set MethodName to ConvertToGrayscale.Note: You’ll need to create a custom VB.NET code for this method, as it’s not a standard method. You can do this by clicking on “Import” in the “Invoke Method” activity, and then “Browse for Types”.
  • **Thresholding:**Apply a threshold to segment the image into black and white regions:
    • Add another “Invoke Method” activity.
    • Set the TargetObject to the grayscale image.
    • Set MethodName to ApplyThreshold.
  • **Contour Detection:**Use contour detection to identify white areas:
    • Add another “Invoke Method” activity.
    • Set the TargetObject to the thresholded image.
    • Set MethodName to FindContours.
  • **Iterate Over Contours:**After finding contours, you can iterate over them and check for white areas based on their properties (e.g., area, color).
  1. Filtering White Areas:
  • Based on the properties of the contours, you can identify areas that are predominantly white.
  1. **Further Processing (Stamp Placement, etc.):**Once you’ve identified suitable spots, you can perform further actions like stamp placement.

Remember to experiment with parameters and thresholds to get the best results for your specific images. This process may require some trial and error to achieve the desired outcome.

Please note that implementing image processing in UiPath might require some familiarity with VB.NET or C# and understanding of basic image processing techniques. If you’re not comfortable with this, you might consider seeking help from someone with programming experience.

Checking for stamp placement and overlaying the stamp onto the image can be achieved using image processing techniques in UiPath. Here’s a step-by-step guide:

Check for Stamp Placement:

  1. Identify Suitable Spot:
  • After processing the image and identifying potential empty spots, you can use techniques like bounding box detection to get the coordinates of these spots.
  1. Check for Nearby Lines or Text:
  • For each potential empty spot, you can use a similar process as above to identify nearby lines or text. This may involve contour detection and analysis.
  • You’ll need to establish criteria for what constitutes an acceptable proximity to lines or text. For example, you might check if any contour intersects with the bounding box of the stamp.
  1. Stamp Placement Decision:
  • Based on the results of the previous steps, you can make a decision on whether the spot is suitable for stamp placement. If it passes your criteria, proceed with stamping.

Stamp the PDF:

  1. Load the Stamp Image:
  • Use the Bitmap class to load the stamp image:
    • Add an “Invoke Method” activity.
    • Set the TargetObject to new System.Drawing.Bitmap("path_to_stamp_image").
  1. Overlay the Stamp:
  • For each suitable spot, you’ll overlay the stamp onto the image:
    • Use the Graphics class to draw the stamp onto the image.
      • Add an “Invoke Method” activity.
      • Set the TargetObject to the image.
      • Set MethodName to DrawImage.
      • You’ll need to create a custom VB.NET code for this method. You can do this by clicking on “Import” in the “Invoke Method” activity, and then “Browse for Types”.
  • Ensure that the stamp is appropriately positioned within the bounding box of the empty spot.

Reassemble PDF (if necessary):

  1. If Images were Converted:
  • If you initially converted the PDF to images, you’ll now need to reassemble them back into a PDF.
  • Use the PdfSharp or iTextSharp library, which are .NET libraries that allow you to create and manipulate PDF files.
  • Follow the respective library’s documentation to merge the stamped images into a PDF document.
  • Save the final PDF with the stamped images.

Remember to handle exceptions and add validation steps to ensure that the process runs robustly. Additionally, testing with various types of drawings will help fine-tune the process for different scenarios.

There are alternative approaches to achieve your goal. Here are a couple of alternative methods:

Optical Character Recognition (OCR):

You can use OCR to extract text and analyze the layout of the document. This can be particularly useful when dealing with scanned PDFs. UiPath has built-in OCR activities that you can use for this purpose.

  1. Extract Text:
  • Use the OCR activities to extract text from the PDF.
  1. Analyze Layout:
  • Use string manipulation and regular expressions to analyze the text and identify areas where the stamp can be placed without overlapping with lines or text.
  1. Stamp Placement:
  • Once you’ve identified suitable spots, use image processing techniques to overlay the stamp onto the PDF.

Computer Vision Activities:

UiPath’s Computer Vision activities can be helpful in scenarios where you need to interact with elements within an application or document.

  1. Use CV Screen Scope:
  • Use the CV Screen Scope to interact with the document. This might involve zooming in to specific areas, scrolling, or navigating.
  1. CV Click and Type Into:
  • Use CV Click to position the cursor where you want to place the stamp, and CV Type Into to input the stamp’s text or image.
  1. Validation:
  • After placing the stamp, use CV activities to validate that it’s in the correct position.

Custom Activities or Plugins:

If the built-in activities in UiPath don’t provide the specific functionality you need, consider creating custom activities or using existing plugins that may offer more advanced image processing capabilities.

Python Integration:

You can integrate Python scripts into your UiPath workflow using the Python Activities package. Python offers powerful image processing libraries like OpenCV and PIL (Python Imaging Library) that can be leveraged for complex image analysis tasks.

Remember to evaluate these options based on your specific requirements, the complexity of the drawings, and the availability of resources and expertise within your team. Each approach has its strengths and might be more suitable for different scenarios.

3

I have tried a couple of these solutions as well as paired with some other potential solutions given by chatGPT but I do not get any of them to work properly. It seems like this is really hard to acheive in UiPath, i might have to give up on this project.