Data scraping from a single page

Random and other categories
21

thank you very much for this information, and also I have a doubt with Get attribute and that if condition. It only check “Opening and Closing dates” so how another field work.

22

The If condition on ‘Opening and Closing Dates’ was just a way of removing empty space from the data grabbed.

23

NewJobs.zip (23.7 KB)

I tried this code. but i can’t find where is the error.

24

First of all, I developed using IE so I don’t know if using Firefox causes an issue.

Second, you have not initialised your collection variable ‘JobList’. Please do that in the variables panel as below:

image
image1217×253 18.5 KB

Third, you have removed the filter from your FindChidren activity and now have a target selector instead. This means that too many elements are grabbed by the activity and cannot fit them onto the datatable in the last step. Why have you removed it?

image
image400×501 14.8 KB

2 Likes
25

Hi, Sorry I didn’t notice that JobList variable’s defalt value. I removed the filter from this FindChidren activity because just run a test for get any one of data from site.

I re-arrange all as above mentioned path.

image
image1366×768 83.5 KB

Still I got error.

26

Untitled8
Untitled81366×768 89.3 KB

Again I create same project from the beginning, it show an error in AddDataRow section.

27

NewJobs.zip (23.6 KB)

This is my new project.

28

Untitled9
Untitled91366×768 91.1 KB

all data are showed here but can’t add to data rows.

29

You have changed the target datatable it seems so that there are less columns than the array you are attempting to post to it. Therefore an error is returned.

Datatable has 3 columns:

image

JobList Array has 8 elements

**

image
image1259×675 32.8 KB
**

The same number of columns must equal the number of elements in your array

1 Like
30

Ok. thank you it’s seems ok…

31

Hi, How to get all data from a single div, that div contain all details of a job
(for example)
https://careers-sbec.icims.com/jobs/6389/manager-of-projects/job
I want to get data inside the above page from the header " OVERVIEW" to " WORK ENVIRONMENT". .

32

This will append the data to a separate text file per webpage that you feed it.

Obviously its not the full datatable you are looking for but you can easily adapt this then to feed the various items to a datatable knowing what you now know about the Find Children activity I would expect.

I hope this gets you most of the way there.

Jobs3.zip (6.9 KB)

image
image1890×884 50.6 KB

1 Like
33

Thank you for this code. I can save data using this code, But I want it to run in a loop. I attache my code here. Jobs3.zip (25.8 KB) .All urls are given in a loop and this got an error in find children section.

34

I change “” to “”, now I got some data. but still got errors

35

I change webctrl tag=‘p’ to webctrl tag=‘Div’ , now I got some data. but still got errors

36

Jobs3.zip (23.9 KB)
This code works fine and got 4 files. but after that it shows some error.