1 Replies - 328 Views - Last Post: 12 October 2018 - 06:59 AM Rate Topic: -----

#1 bobsmith76   User is offline

  • D.I.C Regular

Reputation: 11
  • View blog
  • Posts: 323
  • Joined: 14-February 17

How do you kill a forked process?

Posted 12 October 2018 - 06:55 AM

This algorithm is literally the most amazing algorithm I've ever found. It will speed up any algorithm from as high as 3000 times to as low as 300. I have a program which now takes 3 hours whereas formally it would have taken 38 days and the program could easily get much bigger in the future. It could be the case that this algorithm is specific to Macs since I read somewhere that Macs are good at forking but I'm not certain. In any case, my problem is that I cannot stop this program once it starts. I have to kill the process from the activity monitor and since there are five processes running it takes quite a while to do this and is somewhat tedious. Plus I can only check that the process is done by putting pickles in a folder and then periodically checking the size of the folder, then killing the process once all of the files I want are created.


import os
p = print
def main_fork(func, total, *args):
    for i in range(5):
        p(i)
        start1, stop1 = 0, 0
        if total != -1:
            start1, stop1 = divide_range(5, total, i)
        newpid = os.fork()

        if newpid == 0:
            child(func, *args, start = start1, stop = stop1, fork_num = i, debug = False)


def child(func, *args, **kwargs):
    func(*args, **kwargs)
    os._exit(0)

def divide_range(divisions: int, total: int, idx: int):
    sec = total // divisions
    start = idx * sec
    if total % divisions != 0 and idx + 1 == divisions:
        stop = total
    else:
        stop = start + sec
    return start, stop




Here's the algorithm I use to determine that the job is complete. I don't have any questions about this one. I just post it here so that you understand what I'm talking about.

def fold_len(fold, limit):
    last_num = 0
    while True:
        fold2 = os.listdir(fold)
        b = len(fold2)
        p (B)/>/>
        if b > limit:
            p ('kill now')
            return
        time.sleep(5)
        total = b - last_num
        if total != 0:
            minutes = (limit / (total/5)) / 60
            p (f"minutes {minutes}")
        last_num = b



I would think that the way to kill the process would be to use the kill method:

os.kill



But I can't figure out how to adjust my syntax in order to get that to work.

I have some other questions. When I use 5 forks it revs the CPU up to 60% provided I close all other applications. When I use 8 forks I get no additional speed. Why? My next question is that by using multiprocess you can speed up an algorithm by 3 times, at least on my computer you can. Here again, the more multiprocesses you use results in no added speed. Why? 3 multiprocesses are just as fast as 8.

I'm also thinking what if for each fork I use 3 multiprocesses? Would this speed things up even more? Here is my code:

from multiprocessing import Process as process

def use_mult_process(func, *args, **kwargs):
    # 5 multiprocesses speed up by factor of 3 for count 2 million
    process_list = []
    fstart = kwargs['start']
    fstop = kwargs['stop']
    num = 5
    for i in range(num):
        p (f"mproces {i}")
        start, stop = divide_range_nonzero(num, fstart, fstop, i)
        kwargs['start'] = start
        kwargs['stop'] = stop
        q = process(target=func, args=args, kwargs=kwargs)
        q.start()
        process_list.append(q)

    for q in process_list:
        q.join()



def fork_n_mprocess(func, total, *args):
    for i in range(5):
        p(i)
        start1, stop1 = 0, 0
        if total != -1:
            start1, stop1 = vgf.divide_range(5, total, i)
        newpid = os.fork()

        if newpid == 0:
            child_process(func, *args, start = start1, stop = stop1, debug = False)


def child_process(func, *args, **kwargs):
    use_mult_process(func, *args, **kwargs)
    os._exit(0)

def divide_range_nonzero(divisions: int, start: int, stop: int, idx: int):
    total = stop - start
    fstart, fstop = divide_range(divisions, total, idx)
    return fstart + start, fstop + stop


def divide_range(divisions: int, total: int, idx: int):
    sec = total // divisions
    start = idx * sec
    if total % divisions != 0 and idx + 1 == divisions:
        stop = total
    else:
        stop = start + sec
    return start, stop




Although this code now revs the CPU up to 100% it results in no additional speed. It is just as fast without the multiprocess. Why?

This post has been edited by bobsmith76: 12 October 2018 - 06:56 AM


Is This A Good Question/Topic? 0
  • +

Replies To: How do you kill a forked process?

#2 modi123_1   User is online

  • Suitor #2
  • member icon



Reputation: 14502
  • View blog
  • Posts: 58,138
  • Joined: 12-June 08

Re: How do you kill a forked process?

Posted 12 October 2018 - 06:59 AM

Quote

I have a program which now takes 3 hours whereas formally it would have taken 38 days

Curious - what task is taking 38 days, and what app is cool running it for that long?
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1