millions of data to be processed

Help
1

I would like to process millions of millions data fetched from DB and compare it with onscreen application. eg: From DB am fetching details of millions of users and comparing it with onscreen data table, what is the best way to do? by moving all the db details to excel sheet or is there any efficient/faster way to process.

1 Like
2

Hi @monish06

Since you about to deal with millions of data we seriously need to think about a way to efficiently process them. However, I really dont think comparing such a large number of data on system screen is a good option. Because it will take time to load the page, navigations and comparison. I’m thinking whether there is any possibility to reduce the time spent on screen loading and navigations.

How complex is your comparison of database data with application? Usually how many fields do you have to compare?

Is it just a value match or are you to perform some sort of calculations to do the comparisons?

All these has to be taken into consideration to think of a efficient way.

And also, is there any way which we can gain access to the database of the application as well? Rather than navigating through screens, it would be better to do a database to database comparison…

Check these options and let us know

2 Likes
3

If you still don’t have access to the database, I’ll suggest screen scrapping the table for whole and then compare

1 Like