10 Replies - 578 Views - Last Post: 26 July 2014 - 05:16 AM

#1 nullReference  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 12
  • Joined: 23-July 14

Using individual ajax requests for each file upload

Posted 24 July 2014 - 06:52 AM

First off I want to say that I have this working in a development environment. My main reason for this post is to get opinions on how others might handle this situation and if multiple ajax requests would be the best way to go.

What the process does is allow users to upload multiple images, then for each selected image i create an ajax request that processes that particular image on the backend and sends it off to s3. Before I was doing this asynchronously I was running into timeout errors in nginx which is what prompted the idea. Now since each image uploads individually users don't have to wait for the page to reload and they can start filling out other form information while processing is completing. Yay, all is well in the world problem solved everyone is happy!!! Right??

My only concern with this process is when say 1000 or more users start uploading images asynchronously creating (1000 x (number of pics)) requests to the server. Which shouldn't really be a massive deal I don't think, should just be like having that many users hitting the site, as long as the server can handle the requests. Am I wrong to handle this the way I am or is my logic sound?

I have been debating setting up resque and developing a backend queuing system, but if it's not needed right from the get go I'd like to see my current process run in the wild :)

Is This A Good Question/Topic? 0
  • +

Replies To: Using individual ajax requests for each file upload

#2 astonecipher  Icon User is offline

  • D.I.C.
  • member icon

Reputation: 801
  • View blog
  • Posts: 3,501
  • Joined: 03-December 12

Re: Using individual ajax requests for each file upload

Posted 24 July 2014 - 09:41 AM

I don't have a link offhand, but run a server load test. The bulk of the time it is going to take is the upload speed in which the user can upload. The time it takes to process the requests is going to be minimal. But, that depends on the backend code that is doing the real work. After all, AJax is just a gateway.
Was This Post Helpful? 0
  • +
  • -

#3 nutandevjoshi  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 1
  • Joined: 24-July 14

Re: Using individual ajax requests for each file upload

Posted 24 July 2014 - 09:52 AM

U give ur code.
Was This Post Helpful? 0
  • +
  • -

#4 modi123_1  Icon User is online

  • Suitor #2
  • member icon



Reputation: 9579
  • View blog
  • Posts: 36,289
  • Joined: 12-June 08

Re: Using individual ajax requests for each file upload

Posted 24 July 2014 - 09:54 AM

Ah... what nutand? That didn't quite make sense.
Was This Post Helpful? 0
  • +
  • -

#5 astonecipher  Icon User is offline

  • D.I.C.
  • member icon

Reputation: 801
  • View blog
  • Posts: 3,501
  • Joined: 03-December 12

Re: Using individual ajax requests for each file upload

Posted 24 July 2014 - 09:55 AM

Quote

My main reason for this post is to get opinions on how others might handle this situation and if multiple ajax requests would be the best way to go.


Or are you looking to copy it?
Was This Post Helpful? 0
  • +
  • -

#6 modi123_1  Icon User is online

  • Suitor #2
  • member icon



Reputation: 9579
  • View blog
  • Posts: 36,289
  • Joined: 12-June 08

Re: Using individual ajax requests for each file upload

Posted 24 July 2014 - 10:00 AM

Copying opinions? Ha!

Asking for opinions is an okay thing..
Was This Post Helpful? 0
  • +
  • -

#7 astonecipher  Icon User is offline

  • D.I.C.
  • member icon

Reputation: 801
  • View blog
  • Posts: 3,501
  • Joined: 03-December 12

Re: Using individual ajax requests for each file upload

Posted 24 July 2014 - 10:02 AM

Only reason I can see why you would want code from an opinion poll...although that data does get copied and modified all the time.
Was This Post Helpful? 0
  • +
  • -

#8 ArtificialSoldier  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 486
  • View blog
  • Posts: 1,816
  • Joined: 15-January 14

Re: Using individual ajax requests for each file upload

Posted 24 July 2014 - 11:05 AM

Quote

My only concern with this process is when say 1000 or more users start uploading images asynchronously creating (1000 x (number of pics)) requests to the server. Which shouldn't really be a massive deal I don't think, should just be like having that many users hitting the site, as long as the server can handle the requests. Am I wrong to handle this the way I am or is my logic sound?

A request is a request, it doesn't really matter if they're submitting a form or using ajax. It's possible that you can use ajax to clobber the server with requests, but if you're not going overboard with requests then there's really not that much difference in the traffic pattern or bandwidth required.
Was This Post Helpful? 0
  • +
  • -

#9 nullReference  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 12
  • Joined: 23-July 14

Re: Using individual ajax requests for each file upload

Posted 24 July 2014 - 11:17 AM

View Postastonecipher, on 24 July 2014 - 09:55 AM, said:

Quote

My main reason for this post is to get opinions on how others might handle this situation and if multiple ajax requests would be the best way to go.


Or are you looking to copy it?


It's possible I didn't do a very good job of explaining my reason for posting. Basically, I'm one of those anal retentive types that obsesses over things that I'm not 100% certain about trying to find a reason for it not to work. So far not receiving any replies such as "OH MY GOD DON'T DO THAT!", or "WHAT ARE YOU THINKING!!!" is comforting.

I removed some of the code that was specific for my case and have the process broken out into a function that accepts the file array, a div element ID, and a url for which to make to post the request to. It should work, but haven't tested since I stripped the other code out:

/* Upload the given files using the provided url and append the output html
 * to the given div element
------------------------------------------------------------------------------*/
function imageUpload(files, divElement, url){
    var len = files.length;

    for(var i = 0; i < len; i++){

        var file = files[i];

        var formData = new FormData();
        formData.append("theFile" + i, file);

        (function(params){
            var file = params.file;
            var formData = params.formData;

			$.ajax({
				url: url,
				type: 'POST',
				data: formData,
				processData: false,
				contentType: false,
				success: function(returnData){
				
					$('#' + divElement).append(returnData);
					
				}
			});

        })({
            'formData' : formData,
            'file' : file
        });

    }
}


Was This Post Helpful? 0
  • +
  • -

#10 nullReference  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 12
  • Joined: 23-July 14

Re: Using individual ajax requests for each file upload

Posted 24 July 2014 - 12:06 PM

View PostArtificialSoldier, on 24 July 2014 - 11:05 AM, said:

A request is a request, it doesn't really matter if they're submitting a form or using ajax. It's possible that you can use ajax to clobber the server with requests, but if you're not going overboard with requests then there's really not that much difference in the traffic pattern or bandwidth required.


That had crossed my mind as well. Since I'm allowing multiple files to be selected if a user were to attempt uploading say 10k files this could cause some issues. I suppose the best way around this would be to limit the total number of files uploadable at any one time by a given user and monitor this activity.
Was This Post Helpful? 0
  • +
  • -

#11 Blindman67  Icon User is offline

  • D.I.C Addict
  • member icon

Reputation: 134
  • View blog
  • Posts: 592
  • Joined: 15-March 14

Re: Using individual ajax requests for each file upload

Posted 26 July 2014 - 05:16 AM

1000 user at the same time. Oh I wish I had that problem... LOL

Don't even consider that at this stage of development, there are plenty of ways to deal with that when the time comes. Your first issue will be getting a few hundred users, and of that those that will use your site daily or even weekly.

If you are at the level of 1000's of uploads on a consistent load you would be looking at needing over a million users using your site a week. That's money in the bank and server loads would be the least of your problems. How to spend your money, now that could be a problem...
:)
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1