3 Replies - 332 Views - Last Post: 05 September 2017 - 03:00 PM

#1 whitaker144  Icon User is offline

  • D.I.C Head

Reputation: 0
  • View blog
  • Posts: 50
  • Joined: 20-September 13

Loading large quantities of data onto a page & session handling

Posted 02 September 2017 - 12:14 PM

Hi,

I'm in very early stages of creating a web interface that would potentially load large quantities of data.

The jist of it is I'm making two API calls: The first authenticates a user and returns an object so I know whether the login was successful. The second returns an array of objects that can range anywhere from hundreds to tens of thousands (or even more). The second API call is really simple, it's a get method which returns an array of objects. One thing to note is the results are dependent on the user i.e results for one user won't be the same for another user.

Currently, my idea for implementing this is having a login that calls a servlet. The Servlet uses the login information, authenticates and makes the API call. Then using EL or JSTL I could do a foreach loop to start loading the data into a table on the page.

Is there a more elegant way to do this, or am I barking up the wrong tree entirely in trying to do this via. web interface?

Is This A Good Question/Topic? 0
  • +

Replies To: Loading large quantities of data onto a page & session handling

#2 Dankwansere  Icon User is offline

  • D.I.C Regular

Reputation: 82
  • View blog
  • Posts: 255
  • Joined: 09-November 09

Re: Loading large quantities of data onto a page & session handling

Posted 02 September 2017 - 04:46 PM

Well first you have to ask yourself, is speed a requirement for your project? Because in a worst case scenario analysis a user making an API request that will fetch "tens of thousands" of array object, and you using EL or JSTL to iterate through that array will take a some time to complete. If you are using 1 loop to then it will basically take O(N) for your loop traversal to finish and screen to render. Using nested loops to traverse through that array will take an even longer time O(N)^2.

I would recommend using Ajax for your second API call process. This will make the call to the 2nd API asynchronous and run in the background. Also with Ajax, while the operation is performing you can display operation status to user while he's doing something else, then when the operation is done you display the results.

Ajax Tutorial

This post has been edited by Dankwansere: 02 September 2017 - 04:47 PM

Was This Post Helpful? 1
  • +
  • -

#3 whitaker144  Icon User is offline

  • D.I.C Head

Reputation: 0
  • View blog
  • Posts: 50
  • Joined: 20-September 13

Re: Loading large quantities of data onto a page & session handling

Posted 02 September 2017 - 06:01 PM

Thanks for the reply. To me, speed doesn't matter too much up to a certain point and since it's just a personal project there's always time to change and optimize. I'd say if I started hitting 2 minutes or more in load time that's when I'd look at other options.

I'll definitely take a look at using Ajax for my call to get the objects. My other idea/thought with Ajax would be going to the database directly like I mentioned in my original post (tedious but much more efficient?). I could do a select count(*) on login, paginate based on the count, then make Ajax calls to a servlet to select 50-100 results based on the page number passed as a parameter. That way I'm only ever dealing with 50-100 objects rather than potentially thousands. I'll give the API a shot first and see how it goes. Thanks again!
Was This Post Helpful? 0
  • +
  • -

#4 CasiOo  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1567
  • View blog
  • Posts: 3,520
  • Joined: 05-April 11

Re: Loading large quantities of data onto a page & session handling

Posted 05 September 2017 - 03:00 PM

You should definitely go with a pagination approach. The user probably can't read/have room for all thousand records on their screen anyway
Was This Post Helpful? 1
  • +
  • -

Page 1 of 1