Page 1 of 1

Parallel Processing

#1 DK3250   User is offline

  • Pythonian
  • member icon

Reputation: 556
  • View blog
  • Posts: 1,737
  • Joined: 27-December 13

Posted 17 December 2018 - 05:11 AM

Disclaimer
The subject of multiprocessing is huge. There is no way that I can cover all aspects in this tutorial. I’ll focus on a few basic situations; hopefully, this will provide enough introduction for yourself to find solutions to your particular problem/situation.
I’ve tried to make the tutorial as accurate as possible; if any details are not sufficiently covered, the error is mine, - please bear with me.

Parallel Processing
With this tutorial I’ll introduce some basic elements for Parallel Processing, utilizing the two standard libraries ‘threading’ and ‘multiprocessing’.

Threading

The ‘threading’ module allows you to run two or more code blocks, sharing the CPU. ‘threading’ is particularly suited if one program part is slow (say, writing a large file to desk) and another part (say, the GUI) should still be operational.
As default such two code blocks will share the CPU time.
‘threading’ is also suitable if one part of the code have a sleep period in the loop, - the other code block(s) will then have access to the CPU while the first is dormant.
The basic ‘threading’ module do not utilize additional cores in a multi-core CPU.
Let us jump to an example:
 from threading import Thread

def n_prime(num, primes):
    candidate = 5
    while len(primes) < num:
        for prim in primes[1:]:
            if prim * prim > candidate:
                primes.append(candidate)
                out = 'primes: ' + str(candidate) + '\n'
                print(out)
                break
            if candidate % prim == 0:
                break
        candidate += 2

 
def square(num):
    for n in range(num):
        out = 'Squares: ' + str(n) + ', ' + str(n*n)
        print(out)
        n += 1


def main():
    primes=[2, 3]
    t1 = Thread(target=n_prime, args=(100, primes))
    t2 = Thread(target=square, args=(250,))
    
    t1.start()
    t2.start()
    
    t1.join()
    t2.join()
    
if __name__ == '__main__':
    main()

First, two functions are defined, ‘n_prime’ and ‘square’. Those two functions we want to run in separate threads. To see the progress in the threads, a print statement is included in each function. ‘n_prime’ calculates the first 100 primes; ‘square’ simply calculates the first 250 square numbers.

In line 1 the ‘Thread’ class is imported from the threading module.

In line 26 and 27 the two Threads instances are created; note how function arguments are included as a tuple.

Line 29/30 starts the two threads.

The join() functions in line 33/35 makes the main program wait for the two threads to finalize before continuing.
If you run this code and check the output, it is immediately clear that the two functions run simultaneously; or rather, in parallel; until the first function is exhausted.
Without the join() functions (try comment them out), the main code will simply continue in parallel with the two threads. In our example the main code soon terminates but note: The threads continue after the main has terminated.

Once a thread is started, you cannot terminate it. If you start a thread with an infinite loop you can only stop it by terminating your python session. Not even ctrl-C will stop the infinite thread.

To stop any ‘wild’ threads upon ending of the main code, you can use the daemon flag, fx.:[ code ] t2 = Thread(target=square, args=(250,), daemon=True). The daemon flag will not work in IDLE and many other editors. Even running in a Windows command line did not work properly when I tested it (Python version 3.6.1), so be careful here. In general, it is better/safer not running indefinite loops in threads.

It is not possible to influence the thread much after it is started. One nice method, however, is by use of the Event() class of the threading module. The Event class host a single Boolean variable that can be set True or False using two methods: set() and clear(). To check for the event status use the is_set() method.

Let us run the example again, this time we want to stop the square() function when ‘n_prime’ is exhausted:

 from threading import Thread, Event  # import the Event class

def n_prime(num, primes, event):
    candidate = 5
    while len(primes) < num:
        for prim in primes[1:]:
            if prim * prim > candidate:
                primes.append(candidate)
                out = 'primes: ' + str(candidate) + '\n'
                print(out)
                break
            if candidate % prim == 0:
                break
        candidate += 2
    event.clear()  # set the event to False

 
def square(num, event):
    n = 0
    while event.is_set():  # check for the event status
        if n < num:    
            out = 'Squares: ' + str(n) + ', ' + str(n*n)
            print(out)
            n += 1


def main():
    primes=[2, 3]
    
    e = Event()  # create the Event instance
    t1 = Thread(target=n_prime, args=(100, primes, e))  # add the event as parameter
    t2 = Thread(target=square, args=(250, e))  # same
    
    e.set()  # set it True
    t1.start()
    t2.start()
    print("\nTwo threads has started\n")
    
    t1.join()
    print("\nFirst thread has ended\n")
    t2.join()
    print("Both threads has ended")
    
if __name__ == '__main__':
    main() 

Nothing prevents you from having two or more events for more complicated situations.
The Event class also has a wait() method. Link to the documentation: https://docs.python....threading.Event

My intention here was only to give an introduction to the Thread class.
I need, however, to briefly mention one more possibility for more advanced use: Sub-classing and method overriding.
You can sub-class the Thread class and override the __init__() and run() functions. In this way you can alter the inner workings of the Thread class. If you feel up to this, you are probably experienced enough to find the details in the documentation.

Ok; with Threads we share a single CPU core between two or more, well.., threads. Many modern computers have a CPU with more than one core. A CPU with four cores can perform (by and largely) as four separate CPUs. So, it is only naturel that we want to utilize this hardware for parallel processing.

The multiprocessing module
The multiprocessing module in python is huge and versatile. This intro will only scratch the surface of the module; but, hopefully, provide enough introduction for yourself to work out solutions to your particular needs.
I will demonstrate the Pool and Process classes only.

Calculating pi by the Monte Carlo method.
Imagine a unit circle centered at origo. We know the area of this circle is pi by definition (Area = pi * radius ** 2). If we only look at the first quarter (x and y in range 0 to 1), the area must be pi/4.

If we place a lot of random points (x, y) in the first quarter square, we can check whether they are inside the circle simply be checking if x**2 + y**2 < 1.

All points will be inside the unit square of area = 1. A fraction of the points will be inside the quarter circle with area = pi/4; this leads to:

Pi/4 = (number of points inside circle) / (total number of points)

The method converges quite slowly; so, to get a reasonable accuracy, we need many random points.

Why not distribute this simple calculation on more than one CPU core? Let us do just that, using the Pool class of the multiprocessing module:
 from multiprocessing import Pool
import random, datetime

def pi(i):
    # Calculation of pi by the Monte-Carlo Method
    hits = 0
    for _ in range(i):
        x = random.random()
        y = random.random()

        if x*x + y*y <= 1:
            hits += 1
        
    return hits

def myFunc(core):
    start = datetime.datetime.now()
    pool = Pool(core)
    size = 10000000
    inputs = [size] * 4
    outputs = pool.map(pi, inputs)
    print(outputs)
    print(sum(outputs)/size)
    end=datetime.datetime.now()
    print("Time used:", end-start, "Running", core, "cores")

if __name__ == '__main__':
    for core in range(1, 5):
        myFunc(core)


The pi() function is dead simple, simply returning the number of points inside the circle.

The myFunc() takes the number of ‘core’ as argument; this variable takes the value of 1-4 in a loop.

pool = Pool(core) defines how many cores we will use.

‘size’ (admittedly, the name is misleading) is not the number of points running in the pi() function – it is only one quarter of the numbers.

‘inputs’ which is 4 times ‘size’ is what we ultimately run through pi(). This argument must be an iterable (list, tuple, etc.), - cannot be a simple value.

We do this by mapping the function pi() to ‘inputs’ using the pool of CPU cores available, creating an ‘output’ list of length 4. The term ‘mapping’, in connection with multiprocessing, translates to ‘splitting the function and argument over the available cores’
Summing the 4 elements of this output list and dividing by ‘size’, establishes the desired result as pi/4*4 = pi.

If you run this code, you will get an output like:
[7856226, 7853522, 7853654, 7856208]
3.141961
Time used: 0:00:15.033418 Running 1 cores
[7855353, 7853998, 7854864, 7854217]
3.1418432
Time used: 0:00:11.614370 Running 2 cores
[7854937, 7854329, 7853437, 7852834]
3.1415537
Time used: 0:00:10.376172 Running 3 cores
[7854824, 7854571, 7852989, 7854690]
3.1417074
Time used: 0:00:10.307126 Running 4 cores


As you can see there is a time reduction when going from 1 to 2 or 3 cores, but not as much as you might expect. I’m not 100% sure why this is so, but naturally there is an overhead from the multiprocessing system. Moving to 4 cores has only a minor advantage over 3 cores; this is probably due to a base load from the OS and other systems running in the background. You might benefit from turning off as much as possible.
I even ran the code with ‘size’ = 100,000,000 to produce this output (modified):
Time used: 0:02:30.049028 Running 1 cores
Time used: 0:01:52.006341 Running 2 cores
Time used: 0:01:39.985899 Running 3 cores
Time used: 0:01:25.768918 Running 4 cores


Next, I want to introduce the Process class of multiprocessing.

Allow me to use a rather large example. The multiprocessing part is only the very last 10-12 lines.
I use this example, however, in order to demonstrate how multiprocessing can also be utilized in graphics/animations. And because games and graphics is what I do mostly. First a screen-dump:

Posted Image

You see four candles and a waving flag. Each are produced by a worker function. The worker functions themselves are not the subject of this tutorial; here I’ll only comment on what I directly relevant for the multiprocessing part. Otherwise, the worker functions are commented in the code.

We will see three examples, largely producing the same output – but with important differences.

In the first example worker1 produces a large display with four candles; worker2 is responsible for a small, borderless, display located on top of the first one.

Then the code:
 import multiprocessing, time

def worker1():
    def get_flame_color():
        """
        Generates colors linear distributed in the range
        (255, 255, 0) to (200, 100, 30)

        The colors will smooth and gradually
        turn from one shade to another.

        Red is 'master'-color, green and blue depends of red.
    """
        color = [255, 255, 0]              # start color
        colEnd = random.randint(200, 255)  # the red color end point

        while True:
            dif = colEnd - color[0]
            if abs(dif) >= 1:
                color[0] += dif/abs(dif)
                color[1] = 100 + 155/55*(color[0] - 200)
                color[2] = 30 - 30/55*(color[0] - 200)
                yield color
            else:
                colEnd = random.randint(200, 255)


    class Candle():
        def __init__(self, x):
            self.get_color = get_flame_color()
            self.x = x  # position
            pix = pygame.surface.Surface((4, 4))  #the individual 'pixels' in candle
            self.candle_surf = pygame.surface.Surface((20, 120))  # the candle surface
            for i in range(0, 20, 4):
                for j in range(0, 120, 4):
                    ran = random.randint(200, 255)
                    pix.fill([ran]*3)  # creation of random grey colors
                    self.candle_surf.blit(pix, (i, j))

            self.flame_surf = pygame.surface.Surface((64, 120))  # the flame surface

        def make_flame(self):
            """
            Draws the flame using simple geometric figures.
            'flame_surf' is size x4 allowing later antialiasing.
            """
            self.flame_surf.fill(TRANS)
            flick = random.randint(-6, 6)  # variation of flame top position

            for i in range(0, 14):
                color = next(self.get_color)  # get color from generator
                # the flame top
                polygon = ((1+2*i, 90), (32+flick, 2*i), (63-2*i, 90))
                pygame.draw.polygon(self.flame_surf, color, polygon, 0)
                # the flame body
                pygame.draw.circle(self.flame_surf, color, (32, 90), 32-2*i, 0)

            # mask to get flame shape
            pygame.draw.circle(self.flame_surf, TRANS, (-66+flick, 0), 100, 0)
            pygame.draw.circle(self.flame_surf, TRANS, (130+flick, 0), 100, 0)

            self.antialias()

        def draw(self):
            self.make_flame()
            screen.blit(self.candle_surf, (self.x, 160))
            screen.blit(self.flame, (self.x-6, 100))

        def antialias(self):
            self.flame = pygame.surface.Surface((32, 60))
            self.flame.set_colorkey(TRANS)
            for c in range(0, 64, 2):
                for r in range(0, 120, 2):
                    c1 = self.flame_surf.get_at((c, r))
                    c2 = self.flame_surf.get_at((c+1, r))
                    c3 = self.flame_surf.get_at((c, r+1))
                    c4 = self.flame_surf.get_at((c+1, r+1))
                    c5 = [sum(x)//4 for x in zip(c1, c2, c3, c4)]
                    self.flame.set_at((c//2, r//2), c5)


    import pygame, os, sys, random
    os.environ['SDL_VIDEO_WINDOW_POS'] = "200, 200"
    pygame.init()

    X = 900
    Y = 600

    screen = pygame.display.set_mode((X, Y))
    pygame.display.set_caption("Flickering Candle")

    BLACK = (0, 0, 0)
    BG = (126, 126, 255)
    TRANS = (1, 2, 3)
    
    candles = []
    for x_pos in [50, 100, 780, 830]:
        candles.append(Candle(x_pos))

    background = pygame.Surface((X, Y))
    for y in range(Y):
        color = [int(col * (1 + y / Y) * 0.5) for col in BG]
        pygame.draw.line(background, color, (0, y), (X, y), 1)    
        
    while True:
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                return
        screen.blit(background, (0, 0))
        for c in candles:
            c.draw()
        pygame.display.flip()
        pygame.time.wait(150)


def worker2(): 
    class Flag():
        def __init__(self, size, topleft):
            self.surf = pygame.Surface((size))
            pix = pygame.surface.Surface((4, 4))
            self.size = size
            self.topleft = topleft
            self.x_wave = 0
            self.y_wave = 0
            W, H = size
            h = int(H / 13)

            # sky
            self.background = pygame.Surface((X, Y))
            for y in range(Y):
                color = [int(col * (1 + (y+250) / 600) * 0.5) for col in BG]
                pygame.draw.line(self.background, color, (0, y), (X, y), 1)

            # stripes
            for i in range(13):
                if i%2 == 0:
                    color = RED
                else:
                    color = WHITE
                pygame.draw.rect(self.surf, color, (0, h*i, W, h), 0)

            # union
            pygame.draw.rect(self.surf, BLUE, (0, 0, int(0.7*H), 7*h), 0)

            # stars
            st_size = 7
            st_surf = pygame.surface.Surface((st_size*2, st_size*2))
            st_surf.fill(TRANS)
            st_surf.set_colorkey(TRANS)
            
            lst = []
            for i in range(5):
                lst.append((st_size//2 * math.cos(i * 4 * PI / 5 - PI / 10) + st_size,
                              st_size//2 * math.sin(i * 4 * PI / 5 - PI / 10) + st_size))
            pygame.draw.polygon(st_surf, WHITE, lst, 0)
            pygame.draw.circle(st_surf, WHITE, (st_size, st_size), st_size//5, 0)
            
            for i in range(11):
                for j in range(9):
                    if (i+j) % 2 == 0:
                        self.surf.blit(st_surf, (i*st_size, j*st_size))

        def draw(self):
            screen.blit(self.background, (0, 0))
            W, H = self.size
            px, py = self.topleft
            x_scale = W / 200
            y_scale = H / 100
            for i in range(W):
                for j in range(H):
                    color = self.surf.get_at((i, j))
                    cos = math.cos((i / W + self.x_wave) * 5 * x_scale)
                    sin = math.sin((j / H + self.y_wave) * 3 * y_scale)
                    jj = int(j + 10 * cos) + py
                    ii = int(i + 7 * sin) + px
                    screen.set_at((ii, jj-1), color) # avoid graphical imperfections
                    screen.set_at((ii, jj), color)
                    
        def wave(self):
            self.x_wave += 0.1
            self.y_wave += 0.07
            self.draw()
            pygame.display.flip()
            pygame.time.wait(40)


    import pygame, math, random, os
    os.environ['SDL_VIDEO_WINDOW_POS'] = "400, 450"
    pygame.init()

    X = 400
    Y = 300
    PI = math.pi

    WHITE = (255, 255, 255)
    RED = (191, 18, 56)
    BLUE = (0, 36, 105)
    TRANS = (1, 2, 3)
    BLACK = (0, 0, 0)
    BG = (126, 126, 255)

    screen = pygame.display.set_mode((X, Y), pygame.NOFRAME)

    size = (int(10*13*1.9), 10*13)
    flag = Flag(size, (100, 100))

    while True:
        flag.wave()
        pygame.event.pump()
        
        
if __name__ == '__main__':

    p = multiprocessing.Process(target=worker1)
    q = multiprocessing.Process(target=worker2)

    p.start()
    time.sleep(1)
    q.start()

    p.join()
    q.terminate()
    q.join()



Start focusing on the last 10 lines.

Two instances of the Process class are made. Each instance will call a worker function, here simply named ‘worker1’ and ‘worker2’
The instances are started, one after a delay of 1 sec. – This is to ensure that the larger screen is established before the smaller one.

When p joins (The p-Process stops, here by clicking in the top-right corner) we terminate the q-Process and let it join; the execution of the join() methods will give control back to the main code.

The view is a product of two pygame screens, one shows four flickering candles, the other a waving American flag. Please note in line 202 how the flag screen is made without border in order to show it inside the larger candle screen.

This is, graphically, a very fragile construction; if you move or click in the candle window, the flag window will be hidden. You can, however, bring it to the front again by activating it in the Windows process line.
A more robust approach would be placing the windows side by side, running different processes. Let me include an example (same code, slightly modified):

Posted Image

import multiprocessing, time

def worker1(pos):
    def get_flame_color():
        """
        Generates colors linear distributed in the range
        (255, 255, 0) to (200, 100, 30)

        The colors will smooth and gradually
        turn from one shade to another.

        Red is 'master'-color, green and blue depends of red.
    """
        color = [255, 255, 0]              # start color
        colEnd = random.randint(200, 255)  # the red color end point

        while True:
            dif = colEnd - color[0]
            if abs(dif) >= 1:
                color[0] += dif/abs(dif)
                color[1] = 100 + 155/55*(color[0] - 200)
                color[2] = 30 - 30/55*(color[0] - 200)
                yield color
            else:
                colEnd = random.randint(200, 255)


    class Candle():
        def __init__(self, x):
            self.get_color = get_flame_color()
            self.x = x  # position
            pix = pygame.surface.Surface((4, 4))  #the individual 'pixels' in candle
            self.candle_surf = pygame.surface.Surface((20, 120))  # the candle surface
            for i in range(0, 20, 4):
                for j in range(0, 120, 4):
                    ran = random.randint(200, 255)
                    pix.fill([ran]*3)  # creation of random grey colors
                    self.candle_surf.blit(pix, (i, j))

            self.flame_surf = pygame.surface.Surface((64, 120))  # the flame surface

        def make_flame(self):
            """
            Draws the flame using simple geometric figures.
            'flame_surf' is size x4 allowing later antialiasing.
            """
            self.flame_surf.fill(TRANS)
            flick = random.randint(-6, 6)  # variation of flame top position

            for i in range(0, 14):
                color = next(self.get_color)  # get color from generator
                # the flame top
                polygon = ((1+2*i, 90), (32+flick, 2*i), (63-2*i, 90))
                pygame.draw.polygon(self.flame_surf, color, polygon, 0)
                # the flame body
                pygame.draw.circle(self.flame_surf, color, (32, 90), 32-2*i, 0)

            # mask to get flame shape
            pygame.draw.circle(self.flame_surf, TRANS, (-66+flick, 0), 100, 0)
            pygame.draw.circle(self.flame_surf, TRANS, (130+flick, 0), 100, 0)

            self.antialias()

        def draw(self):
            self.make_flame()
            screen.blit(self.candle_surf, (self.x, 160))
            screen.blit(self.flame, (self.x-6, 100))

        def antialias(self):
            self.flame = pygame.surface.Surface((32, 60))
            self.flame.set_colorkey(TRANS)
            for c in range(0, 64, 2):
                for r in range(0, 120, 2):
                    c1 = self.flame_surf.get_at((c, r))
                    c2 = self.flame_surf.get_at((c+1, r))
                    c3 = self.flame_surf.get_at((c, r+1))
                    c4 = self.flame_surf.get_at((c+1, r+1))
                    c5 = [sum(x)//4 for x in zip(c1, c2, c3, c4)]
                    self.flame.set_at((c//2, r//2), c5)


    import pygame, os, sys, random
    os.environ['SDL_VIDEO_WINDOW_POS'] = pos
    pygame.init()

    X = 300
    Y = 400

    screen = pygame.display.set_mode((X, Y))
    pygame.display.set_caption("Flickering Candle")

    BLACK = (0, 0, 0)
    BG = (126, 126, 255)
    TRANS = (1, 2, 3)
    
    candles = []
    for x_pos in [100, 150]:
        candles.append(Candle(x_pos))

    background = pygame.Surface((X, Y))
    for y in range(Y):
        color = [int(col * (1 + y / Y) * 0.5) for col in BG]
        pygame.draw.line(background, color, (0, y), (X, y), 1)    
        
    while True:
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                return
        screen.blit(background, (0, 0))
        for c in candles:
            c.draw()
        pygame.display.flip()
        pygame.time.wait(150)


def worker2(): 
    class Flag():
        def __init__(self, size, topleft):
            self.surf = pygame.Surface((size))
            pix = pygame.surface.Surface((4, 4))
            self.size = size
            self.topleft = topleft
            self.x_wave = 0
            self.y_wave = 0
            W, H = size
            h = int(H / 13)

            # sky
            self.background = pygame.Surface((X, Y))
            for y in range(Y):
                color = [int(col * (1 + y / Y) * 0.5) for col in BG]
                pygame.draw.line(self.background, color, (0, y), (X, y), 1)

            # stripes
            for i in range(13):
                if i%2 == 0:
                    color = RED
                else:
                    color = WHITE
                pygame.draw.rect(self.surf, color, (0, h*i, W, h), 0)

            # union
            pygame.draw.rect(self.surf, BLUE, (0, 0, int(0.7*H), 7*h), 0)

            # stars
            st_size = 7
            st_surf = pygame.surface.Surface((st_size*2, st_size*2))
            st_surf.fill(TRANS)
            st_surf.set_colorkey(TRANS)
            
            lst = []
            for i in range(5):
                lst.append((st_size//2 * math.cos(i * 4 * PI / 5 - PI / 10) + st_size,
                              st_size//2 * math.sin(i * 4 * PI / 5 - PI / 10) + st_size))
            pygame.draw.polygon(st_surf, WHITE, lst, 0)
            pygame.draw.circle(st_surf, WHITE, (st_size, st_size), st_size//5, 0)
            
            for i in range(11):
                for j in range(9):
                    if (i+j) % 2 == 0:
                        self.surf.blit(st_surf, (i*st_size, j*st_size))

        def draw(self):
            screen.blit(self.background, (0, 0))
            W, H = self.size
            px, py = self.topleft
            x_scale = W / 200
            y_scale = H / 100
            for i in range(W):
                for j in range(H):
                    color = self.surf.get_at((i, j))
                    cos = math.cos((i / W + self.x_wave) * 5 * x_scale)
                    sin = math.sin((j / H + self.y_wave) * 3 * y_scale)
                    jj = int(j + 10 * cos) + py
                    ii = int(i + 7 * sin) + px
                    screen.set_at((ii, jj-1), color) # avoid graphical imperfections
                    screen.set_at((ii, jj), color)
                    
        def wave(self):
            self.x_wave += 0.1
            self.y_wave += 0.07
            self.draw()
            pygame.display.flip()
            pygame.time.wait(40)


    import pygame, math, random, os
    os.environ['SDL_VIDEO_WINDOW_POS'] = "500, 200"
    pygame.init()

    X = 300
    Y = 400
    PI = math.pi

    WHITE = (255, 255, 255)
    RED = (191, 18, 56)
    BLUE = (0, 36, 105)
    TRANS = (1, 2, 3)
    BLACK = (0, 0, 0)
    BG = (126, 126, 255)

    screen = pygame.display.set_mode((X, Y))
    pygame.display.set_caption("Waving Flag")

    size = (int(10*13*1.9), 10*13)
    flag = Flag(size, (30, 150))

    while True:
        flag.wave()
        pygame.event.pump()
        
        
if __name__ == '__main__':

    p = multiprocessing.Process(target=worker1, args=("200, 200",))
    q = multiprocessing.Process(target=worker2)
    r = multiprocessing.Process(target=worker1, args=("800, 200",))

    p.start()
    q.start()
    r.start()

    p.join()
    q.terminate()
    r.terminate()
    q.join()
    r.join()


Here I start three processes, two of them utilizing the same worker function.

You see how placing three windows side by side yields a graphical imperfect expression compared to the first version. I have made the leftmost window a ‘master’; when it is terminated, the two others will terminate automatically.

Allow me to show a final version, which both gives a nice graphical result and is robust. Introducing a forth Process only for handling the top bar with program termination:

Posted Image

import multiprocessing, time

def worker1(pos):
    def get_flame_color():
        """
        Generates colors linear distributed in the range
        (255, 255, 0) to (200, 100, 30)

        The colors will smooth and gradually
        turn from one shade to another.

        Red is 'master'-color, green and blue depends of red.
    """
        color = [255, 255, 0]              # start color
        colEnd = random.randint(200, 255)  # the red color end point

        while True:
            dif = colEnd - color[0]
            if abs(dif) >= 1:
                color[0] += dif/abs(dif)
                color[1] = 100 + 155/55*(color[0] - 200)
                color[2] = 30 - 30/55*(color[0] - 200)
                yield color
            else:
                colEnd = random.randint(200, 255)


    class Candle():
        def __init__(self, x):
            self.get_color = get_flame_color()
            self.x = x  # position
            pix = pygame.surface.Surface((4, 4))  #the individual 'pixels' in candle
            self.candle_surf = pygame.surface.Surface((20, 120))  # the candle surface
            for i in range(0, 20, 4):
                for j in range(0, 120, 4):
                    ran = random.randint(200, 255)
                    pix.fill([ran]*3)  # creation of random grey colors
                    self.candle_surf.blit(pix, (i, j))

            self.flame_surf = pygame.surface.Surface((64, 120))  # the flame surface

        def make_flame(self):
            """
            Draws the flame using simple geometric figures.
            'flame_surf' is size x4 allowing later antialiasing.
            """
            self.flame_surf.fill(TRANS)
            flick = random.randint(-6, 6)  # variation of flame top position

            for i in range(0, 14):
                color = next(self.get_color)  # get color from generator
                # the flame top
                polygon = ((1+2*i, 90), (32+flick, 2*i), (63-2*i, 90))
                pygame.draw.polygon(self.flame_surf, color, polygon, 0)
                # the flame body
                pygame.draw.circle(self.flame_surf, color, (32, 90), 32-2*i, 0)

            # mask to get flame shape
            pygame.draw.circle(self.flame_surf, TRANS, (-66+flick, 0), 100, 0)
            pygame.draw.circle(self.flame_surf, TRANS, (130+flick, 0), 100, 0)

            self.antialias()

        def draw(self):
            self.make_flame()
            screen.blit(self.candle_surf, (self.x, 160))
            screen.blit(self.flame, (self.x-6, 100))

        def antialias(self):
            self.flame = pygame.surface.Surface((32, 60))
            self.flame.set_colorkey(TRANS)
            for c in range(0, 64, 2):
                for r in range(0, 120, 2):
                    c1 = self.flame_surf.get_at((c, r))
                    c2 = self.flame_surf.get_at((c+1, r))
                    c3 = self.flame_surf.get_at((c, r+1))
                    c4 = self.flame_surf.get_at((c+1, r+1))
                    c5 = [sum(x)//4 for x in zip(c1, c2, c3, c4)]
                    self.flame.set_at((c//2, r//2), c5)


    import pygame, os, sys, random
    os.environ['SDL_VIDEO_WINDOW_POS'] = pos
    pygame.init()

    X = 300
    Y = 400

    screen = pygame.display.set_mode((X, Y), pygame.NOFRAME)

    BLACK = (0, 0, 0)
    BG = (126, 126, 255)
    TRANS = (1, 2, 3)
    
    candles = []
    for x_pos in [100, 150]:
        candles.append(Candle(x_pos))

    background = pygame.Surface((X, Y))
    for y in range(Y):
        color = [int(col * (1 + y / Y) * 0.5) for col in BG]
        pygame.draw.line(background, color, (0, y), (X, y), 1)    
        
    while True:
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                return
        screen.blit(background, (0, 0))
        for c in candles:
            c.draw()
        pygame.display.flip()
        pygame.time.wait(150)


def worker2(): 
    class Flag():
        """
        The flag surface is regtangular (self.surf)

        In the draw() method each pixel of the flag surface is pertubed in
        both x- and y-direction; the resulting pixel position is drawn to
        the screen.
        """
        def __init__(self, size, topleft):
            self.surf = pygame.Surface((size))
            pix = pygame.surface.Surface((4, 4))
            self.size = size
            self.topleft = topleft
            self.x_wave = 0
            self.y_wave = 0
            W, H = size
            h = int(H / 13)

            # sky
            self.background = pygame.Surface((X, Y))
            for y in range(Y):
                color = [int(col * (1 + y / Y) * 0.5) for col in BG]
                pygame.draw.line(self.background, color, (0, y), (X, y), 1)

            # stripes
            for i in range(13):
                if i%2 == 0:
                    color = RED
                else:
                    color = WHITE
                pygame.draw.rect(self.surf, color, (0, h*i, W, h), 0)

            # union
            pygame.draw.rect(self.surf, BLUE, (0, 0, int(0.65*H), 7*h), 0)

            # stars
            st_size = 7
            st_surf = pygame.surface.Surface((st_size*2, st_size*2))
            st_surf.fill(TRANS)
            st_surf.set_colorkey(TRANS)
            
            lst = []
            for i in range(5):
                lst.append((st_size//2 * math.cos(i * 4 * PI / 5 - PI / 10) + st_size,
                              st_size//2 * math.sin(i * 4 * PI / 5 - PI / 10) + st_size))
            pygame.draw.polygon(st_surf, WHITE, lst, 0)
            pygame.draw.circle(st_surf, WHITE, (st_size, st_size), st_size//5, 0)
            
            for i in range(11):
                for j in range(9):
                    if (i+j) % 2 == 0:
                        self.surf.blit(st_surf, (i*st_size, j*st_size))

        def draw(self):
            """
            Each pixel of the flag surface is pertubed in both x- and
            y-direction; the resulting pixel position is drawn to the screen.

            As the resulting image can be slightly larger tha the original,
            small imperfection may occour.
            To counteract for this, two neighboring pixels are colored for each
            pixel in the flat surface.
            """
            screen.blit(self.background, (0, 0))
            W, H = self.size
            px, py = self.topleft
            x_scale = W / 200
            y_scale = H / 100
            for i in range(W):
                for j in range(H):
                    color = self.surf.get_at((i, j))
                    cos = math.cos((i / W + self.x_wave) * 5 * x_scale)
                    sin = math.sin((j / H + self.y_wave) * 3 * y_scale)
                    jj = int(j + 10 * cos) + py
                    ii = int(i + 7 * sin) + px
                    screen.set_at((ii, jj-1), color) # avoid graphical imperfections
                    screen.set_at((ii, jj), color)
                    
        def wave(self):
            self.x_wave += 0.1
            self.y_wave += 0.07
            self.draw()
            pygame.display.flip()
            pygame.time.wait(40)


    import pygame, math, random, os
    os.environ['SDL_VIDEO_WINDOW_POS'] = "500, 200"
    pygame.init()

    X = 300
    Y = 400
    PI = math.pi

    WHITE = (255, 255, 255)
    RED = (191, 18, 56)
    BLUE = (0, 36, 105)
    TRANS = (1, 2, 3)
    BLACK = (0, 0, 0)
    BG = (126, 126, 255)

    screen = pygame.display.set_mode((X, Y), pygame.NOFRAME)

    size = (int(10*13*1.9), 10*13)
    flag = Flag(size, (30, 150))

    while True:
        flag.wave()
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                return


def worker3():
    import pygame, os
    os.environ['SDL_VIDEO_WINDOW_POS'] = "200, 170"
    pygame.init()

    X = 900
    Y = 30

    GREY = (245, 245, 245)
    RED = (222, 50, 50)
    DARK = (130, 130, 130)

    screen = pygame.display.set_mode((X, Y), pygame.NOFRAME)
    stop = pygame.Rect(854, 0, 46, 30)

    while True:
        mouse_pos = (0, 0)
        if pygame.mouse.get_focused():
            mouse_pos = pygame.mouse.get_pos()

        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                return
            elif event.type == pygame.MOUSEBUTTONDOWN:
                if stop.collidepoint(mouse_pos):
                    return

        screen.fill(GREY)
        if stop.collidepoint(mouse_pos):
            pygame.draw.rect(screen, RED, stop, 0)
            color = GREY
        else:
            color = DARK
        pygame.draw.line(screen, color, (871, 11), (879, 19), 2)
        pygame.draw.line(screen, color, (871, 19), (879, 11), 2)

        pygame.display.flip()
        pygame.time.wait(50)
        
    
        
if __name__ == '__main__':

    s = multiprocessing.Process(target=worker3)
    p = multiprocessing.Process(target=worker1, args=("200, 200",))
    q = multiprocessing.Process(target=worker2)
    r = multiprocessing.Process(target=worker1, args=("800, 200",))

    s.start()
    p.start()
    q.start()
    r.start()

    s.join()
    p.terminate()
    q.terminate()
    r.terminate()
    p.join()
    q.join()
    r.join()



The animations.
I have commented directly in the code of the worker functions. The features of the animations are not covered by this tutorial. However, feel free to ask if you need some details explained.

Closing remarks
As demonstrated, the Process class can be used to run two or more processes in parallel. If you need to transfer data between the processes, you’ll need some of the more advanced possibilities of the multiprocessing module, such as the Queue class, the Pipe class or the Manage class.

This are topics outside scope for this small tutorial. Link to the documentation: https://docs.python....processing.html

If you monitor the job-list while running these two multiprocessing examples, you will see that the Pool class example only starts one process, but it is allowed to use all the CPU resources up to 100%. The Process class examples will show two/three/four independent processes each drawing modestly on the CPU.

I find this very instructive; the Process class instances are, really, very much independent and can be completely unrelated.
The Pool class is more a distribution of a single process on several CPU cores.

Is This A Good Question/Topic? 0
  • +

Page 1 of 1