Columns are empty using data scraping

[sergio85 (Sergio Moreno)]

Hi everybody,

I am creating a script using the function Data Scraping. I created two columns which are filled with the same selector of the website as follow:

This is a part of the complete selector:

 <column exact='1' name='variable3' attr='text'>
  <webctrl tag='div' class='immobiliListaAnnuncioDettagli' idx='1'/>
 </column>
 <column exact='1' name='variable4' attr='text'>
  <webctrl tag='div' class='immobiliListaAnnuncioDettagli' idx='1'/>
 </column>

However, for some reason the column called variable4 is empty after scraping and the column called variable3 is filled. How does it happen?

Thanks

Sergio

[loginerror (Maciej Kuźmicz)]

Hi @sergio85

This depends on the structure of your website. The snippet from your Data Scraping activity follows the structure of the webpage. If you inspect the webpage, you will be able to find out the issue.

You can also search here on the forum for the data scraping xml or go directly to this thread for more guidance

(Tutorial) How to customize data scraping by editing XML

When we scrape data, we find the wizard will give us a choice of two columns for each field we scrape: Text URL We don’t always get what we want from the wizard. Consider this example: a) Go to http://www.amazon.com/. b) Type in looper pedals and hit Enter. (This will get us to our scraping point in our workflow.) [pedals] We’ll use these images for our preliminary scrape. Set up a workflow as shown in the image. [workflow] Double-click the “Attach Browser” activity called “Attach lo…

[sergio85 (Sergio Moreno)]

Hi @loginerror

Tks for your help. However, my question is if I have the same selector for both variables or columns, why is the variable4 empty? after executing the scraping routine. I think the selector is correct because it retrieved me data in the variable3. Even I want to create three columns with the same selector. Is it possible since it did not work as you see in the example posted.

Tks for your help

[loginerror (Maciej Kuźmicz)]

It can be that the attribute you want to get is not properly marked in the xml. You can change the attribute name by changing
text
to somethng different.

For example, if this is the html:

To extract the value of ‘topic-link’ you would use this:
attr='class'

If the value of the fourth column is in a different attribute than “text”, it will simply return blank.

[sergio85 (Sergio Moreno)]

Hi @loginerror

Tks for your answer. If I want to retrieve in both columns the same attribute text… Is it not possible?. If so could you let me know another approach that I can use please?.

Regards

Sergio

[loginerror (Maciej Kuźmicz)]

It is also possible that the path to the element is not correct. You can still use the hints from the topics linked above to edit the xml path for the fourth element. It is super hard to tell exactly without seeing the structure of the page you are working with.

Would it be possible for you to paste in the xml values from your Data Scraping activity together with a screenshot of the html structure of that fourth column elements (as shown via inspect element in your browser)?

[sergio85 (Sergio Moreno)]

Hi @loginerror

This is the html structure where you will find the selector:

And this is part of the xml that is missing from the first picture that I put in this message:

Tks for your help

Sergio

[loginerror (Maciej Kuźmicz)]

Is there a particular reason you want to extract the same value twice? It seems a bit redundant.

Either way, it should work, see here:

Could you maybe share the page url and values you want to extract?

[sergio85 (Sergio Moreno)]

Hi @loginerror

Tks for your prompt answer and help me. Apparently the Extract wizard show us that it is possible to extract the same value in both columns. However, when I paste the table in a excel sheet for any reason the second column is blank. Could you try the same to paste the data extacted in a excel sheet please?.