4 Replies - 375 Views - Last Post: 04 June 2014 - 06:29 PM Rate Topic: -----

#1 xclite  Icon User is offline

  • LIKE A BOSS
  • member icon


Reputation: 906
  • View blog
  • Posts: 3,171
  • Joined: 12-May 09

Throttling submission of tasks to a ThreadPoolExecutor

Posted 04 June 2014 - 11:11 AM

Been a while since I've been in this forum.

I have a somewhat... advanced? Problem. I have a library that uses a ThreadPoolExecutor, backed by a bounded queue.

We submit a large number of tasks to this executor, constantly polling an external dependency for new work. However, this executor has a fixed size. When a ThreadPoolExecutor is busy, it queues up to the blocking queue's capacity tasks in its blocking queue, at which point it starts rejecting tasks.

What I'd rather do, instead of requesting more work, and then throwing it out, is to detect that we're totally busy and stop requesting work until a thread frees up.

Now, I know that one approach is to rewrite the workers such that they are infinite loops, who poll for work when they're finished. Due to the large number of consumers of this library, this is not an option.

I have come up with two other solutions.

1) This is one that I know will work, and has been suggested by <books I've read over the years>, which is to use a semaphore or other concurrent counter to track how many tasks I have submitted, and stop requesting more work when that semaphore is out of permits. The processors release a permit when they complete (whether due to exception or success).

2) The other option, since I know this is a ThreadPoolExecutor, is way simpler, but also is something I'm not sure is reliable.
I can compare executor.getActiveThreads() + executor.getQueue.size() to the maximum number we can handle before rejecting (maxThreads + queueSize). This change is far simpler, but I'm not sure if those calls are a reliable way of counting the current work.

I know that the state of the pool could change even between asking for active threads and the size of the queue. I'm curious if anybody has any alternative strategies for dealing with this.

Is This A Good Question/Topic? 0
  • +

Replies To: Throttling submission of tasks to a ThreadPoolExecutor

#2 CasiOo  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1401
  • View blog
  • Posts: 3,109
  • Joined: 05-April 11

Re: Throttling submission of tasks to a ThreadPoolExecutor

Posted 04 June 2014 - 11:50 AM

The system's capacity is reached, and I do not believe you should start using other queues or locks to execute the task later. This is what the ThreadPoolExecutor's BlockingQueue is for.
Can other machines come to help? Sharing the workload between several machines would be a good option.

I don't see a problem requesting more work to the ThreadPoolExecutor even when it has reached max capacity.
Supply your own RejectedExecutionHandler or catch the exception thrown when adding another task.
I would recommend the above, and not use remainingCapacity() of the BlockingQueue, since a worker might be available/busy right after the call
Was This Post Helpful? 0
  • +
  • -

#3 xclite  Icon User is offline

  • LIKE A BOSS
  • member icon


Reputation: 906
  • View blog
  • Posts: 3,171
  • Joined: 12-May 09

Re: Throttling submission of tasks to a ThreadPoolExecutor

Posted 04 June 2014 - 01:37 PM

We can add more threads/machines with no problem.

It's not that we're losing work, it's that the request for more work isn't free.

It could be possible to catch the exception and then backoff for a while or something, but that isn't actually that clean, and we still waste some requests.

Additionally, these work items have a max number of receipts before they go to a debugging queue, so I don't want to artificially bump them.
Was This Post Helpful? 0
  • +
  • -

#4 CasiOo  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 1401
  • View blog
  • Posts: 3,109
  • Joined: 05-April 11

Re: Throttling submission of tasks to a ThreadPoolExecutor

Posted 04 June 2014 - 02:32 PM

I'm thinking about extending the ThreadPoolExecutor with some event based functionality (Or maybe the queue). Of course notifying about the current workload.
I don't fully understand the problem, so I'll either need more details or another explanation, if that is not what you might be looking for ^^

Override beforeExecute and afterExcute, and notify about the workload in here.
It then gets pretty easy to know if you should stop sending tasks to the executor for a bit
protected void beforeExecute(Thread thread, Runnable runnable) {
	super.beforeExecute(thread, runnable);
	notifyCurrentWorkload();
}

protected void	afterExecute(Runnable runnable, Throwable throwable) {
	super.afterExecute(runnable, throwable);
	notifyCurrentWorkload();
}

private void notifyCurrentWorkload() {
	Workload workload = getWorkload();
	notifyWorkload(new WorkloadEvent(workload));
}

private void notifyWorkload(Workload workload) {
	//Fire to listeners
}

private Workload getWorkload() {
	//TODO
}


This post has been edited by CasiOo: 04 June 2014 - 02:33 PM

Was This Post Helpful? 0
  • +
  • -

#5 farrell2k  Icon User is offline

  • D.I.C Lover
  • member icon

Reputation: 849
  • View blog
  • Posts: 2,585
  • Joined: 29-July 11

Re: Throttling submission of tasks to a ThreadPoolExecutor

Posted 04 June 2014 - 06:29 PM

So you have an Executor with x number of threads allocated for work that constantly polls an external resource for work. It seems trivial to me to maintain a counter that tells the executor when it can poll for and execute a runnable. As each runnable starts, it increments the counter, and finishes, it decrements the counter. I guess if it were that easy, you would have done it already. :)
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1