Page 1 of 1

Multithreading in Perl Rate Topic: -----

#1 Cbeppe  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 31
  • View blog
  • Posts: 216
  • Joined: 16-September 09

Posted 13 November 2011 - 11:55 AM

Multithreading in Perl

Introduction

Threading allows a program to do two or more things at once. They are very frequently used in GUI applications to allow the user to click on buttons while the program is performing another operation (like clicking 'cancel' on a download window).
To make this tutorial much easier to grasp, we'll only use a simple counting example to demonstrate how your computer could do two things at once. At the end, we will compare it to how it would be done without threading, and we will see that a lot of time can be saved by using threading in this case.

Let's get going...

We will need to place the following at the top of our script:
#!/usr/bin/Perl

# Always good practice
use strict;
# The threads module allows us to implement threading in our script
use threads;

# The number of threads used in the script
my $num_of_threads = 2;



Creating the Subroutines:

The next thing we will do is to write a subroutine that initializes an array that will contain all our threads:
sub initThreads{
        # An array to place our threads in
	my @initThreads;
	for(my $i = 1;$i<=$num_of_threads;$i++){
		push(@initThreads,$i);
	}
	return @initThreads;
}


Pretty simple stuff really.

The next subroutine is the one that will be run by each thread. In our example, it will make the thread count up to 100,000,000.
sub doOperation{
	# Get the thread id. Allows each thread to be identified.
	my $id = threads->tid();
	my $i = 0;
	while($i < 100000000){
			$i++
	}
        # Inform us that the thread is done and exit the thread.
	print "Thread $id done!\n";
	threads->exit();
}



Putting it Together

So far, we have created two subroutines. 'initThreads()' defines an array that will contain our threads, while 'doOperation()' provides the counting loop. Now we're going to put them together in the main part of the script.

#!/usr/bin/Perl

use strict;
use threads;

# Define the number of threads
my $num_of_threads = 2;

# use the initThreads subroutine to create an array of threads.
my @threads = initThreads();

# Loop through the array:
foreach(@threads){
                # Tell each thread to perform our 'doOperation()' subroutine.
		$_ = threads->create(\&doOperation);
}

# This tells the main program to keep running until all threads have finished.
foreach(@threads){
	$_->join();
}

print "\nProgram Done!\nPress Enter to exit";
$a = <>;

####################### SUBROUTINES ############################
sub initThreads{
	my @initThreads;
	for(my $i = 1;$i<=$num_of_threads;$i++){
		push(@initThreads,$i);
	}
	return @initThreads;
}
sub doOperation{
	# Get the thread id. Allows each thread to be identified.
	my $id = threads->tid();
	my $i = 0;
	while($i < 100000000){
			$i++
	}
	print "Thread $id done!\n";
	# Exit the thread
	threads->exit();
}


This may seem like a lot, but the subroutines are the same that we made before. The only new part is this:
# use the initThreads subroutine to create an array of threads.
my @threads = initThreads();

# Loop through the array:
foreach(@threads){
                # Tell each thread to perform our 'doOperation()' subroutine.
		$_ = threads->create(\&doOperation);
}

# This tells the main program to keep running until all threads have finished.
foreach(@threads){
	$_->join();
}


By calling the initThreads() method, we are given an array that contains an element for each of our threads. Next, we cycle through that array, creating a thread for each element. Those threads are then tasked with performing the "doOperation()" subroutine.

After assigning the task to each thread, we join each thread together. This is to keep the program running until all threads are finished. If we neglected to do this, the program would quit immediately after assigning the task. This would kill all threads, and no work would have been done.

So What's the Point?

My computer currently runs on a dual core CPU. This means that two different tasks can be performed simultaneously. If we were to not use threading, our program would be written as follows:
#!/usr/bin/Perl

use strict;

my $c = 0;
for(my $i=0;$i<2;$i++){
	while($c < 100000000){
		$c++;
	}
	$c=0;
	print "Count $i done!\n";
}
$a = <>;


This way, the program will first count to 100,000,000 and then do it again. When splitting the task into two different threads, the time taken to complete the two counts is effectively half. If you would like to test this claim, the following script can be used:
#!/usr/bin/Perl

use strict;
use threads;
use Benchmark qw(:hireswallclock);

my $starttime = Benchmark->new;
my $finishtime;
my $timespent;
my $num_of_threads = 2;

my @threads = initThreads();
foreach(@threads){
		$_ = threads->create(\&doOperation);
	}
foreach(@threads){
	$_->join();
}
$finishtime = Benchmark->new;
$timespent = timediff($finishtime,$starttime);
print "\nDone!\nSpent ". timestr($timespent);

print "\n\nNow trying without threading:\n\n";

my $starttime = Benchmark->new;

doWithoutThread();

$finishtime = Benchmark->new;
$timespent = timediff($finishtime,$starttime);
select(STDOUT);
print "\nDone!\nSpent ". timestr($timespent);

print "\nProgram Done!\nPress Enter to exit";
$a = <>;

sub initThreads{
	my @initThreads;
	for(my $i = 1;$i<=$num_of_threads;$i++){
		push(@initThreads,$i);
	}
	return @initThreads;
}
sub doOperation{
	# Get the thread id. Allows each thread to be identified.
	my $id = threads->tid();
	my $i = 0;
	while($i < 100000000){
			$i++
	}
	print "Thread $id done!\n";
	# Exit the thread
	threads->exit();
}
sub doWithoutThread{
	my $c = 0;
	for(my $i=0;$i<$num_of_threads;$i++){
		while($c < 100000000){
			$c++;
		}
		$c=0;
		print "Count $i done!\n";
	}
}


You can change the number of threads to be used to anything > 1. However, if you're on a dual core processor and choose to use 4 threads (for example), then you will notice that the time taken with multithreading will remain one half and not become one quarter of the time used without threading. This is because your hardware is limited. However, if you are on a quad core processor, you can change the number of threads to be used to 4 and you will see that only a quarter of the time will be used compared to when not using any threads at all.

---------

I hope this helps explain how to use threading in Perl. Be aware that not all kinds of tasks are suited for multithreading. As an example, file reading will not be done faster by using more threads as your Hard Drive can only write one file at the time.

I have attached both the thread demonstration script that we made and the script that will show the time difference when using multithreading compared to when not using it. I couldn't upload .pl files, so they are shown as .txt files. Simply change the extension and you can execute them ;)

If you have any questions, feel free to PM me or comment below :)

Cbeppe.

Attached File(s)



Is This A Good Question/Topic? 0
  • +

Replies To: Multithreading in Perl

#2 "Anuja"  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 2
  • Joined: 28-February 16

Posted 28 February 2016 - 11:42 AM

Hello Cbeppe

your earlier post is very easy to understand the multi threading in perl but i need to know, is there any another way to do multi threading for reading, code execution and printing of files in perl ?

I mean, I want to make my file I/O related perl script faster through multi threading. I have 8 core processor's IBM server in which i want to utilized all the processor to generate a single file that is appended by several small files through several threads. Is it possible by multi threading ? If yes then please tell me how ?

I would be very great-full to you , if you answered me as soon as possible.

Thank you

Anuja
Was This Post Helpful? 0
  • +
  • -

#3 Cbeppe  Icon User is offline

  • D.I.C Head
  • member icon

Reputation: 31
  • View blog
  • Posts: 216
  • Joined: 16-September 09

Posted 26 February 2017 - 05:29 AM

Hi Anuja.

I'm sorry to only reply to your question a year (almost to the day) later, but I haven't really coded anything in the last 5 years.

However, to answer your question: Multi-threading will most likely not help speed up writing to a single file. Firstly, in most cases I know of, one file can only be written to by one thread at a time. Secondly, your HDD or even SDD is the speed-limiting factor when it comes to writing to a disk. It doesn't matter if you have the world's fastest 84 core CPU, since the disk itself will be the bottleneck.

With that said, the best way to speed up a lengthy write to disk is to minimize the amount of write instructions.

Take a look:

Slow:
#!/usr/bin/Perl

use strict;
use Benchmark qw(:hireswallclock);

my $starttime = Benchmark->new;
my $finishtime;
my $timespent;
my $file;
my $filename = "count.txt";

# Count to 1,000 and add a line to the file each time.
for(my $i=1;$i<=1000;$i++){
	open $file,">>", ($filename);
	select($file);
	printf($i."\n");
	close $file;
}
# Benchmark timing
$finishtime = Benchmark->new;
$timespent = timediff($finishtime,$starttime);
select(STDOUT);
print "Done!\nSpent ". timestr($timespent);
my $a = <>;



Fast
#!/usr/bin/Perl

use strict;
use Benchmark qw(:hireswallclock);

my $starttime 	= Benchmark->new;
my $finishtime;
my $timespent;
my $file;
my $filename 	= "count.txt";
my $countstring	= "";

# Count to 1,000 and add the counted number to a string.
for(my $i=1;$i<=1000;$i++){
	$countstring .= $i."\n";
}

# Write our string to the file once
open $file,">>", ($filename);
select($file);
printf($countstring);
close $file;

# Benchmark timing
$finishtime = Benchmark->new;
$timespent = timediff($finishtime,$starttime);
select(STDOUT);
print "Done!\nSpent ". timestr($timespent);
my $a = <>;



The first example takes about 2.75 seconds on my laptop, while the second example takes 0.0013 seconds. They both produce the same result, but the faster example uses only one write to disk instead of 1000. I know this is an extreme example, but if you don't want to get into multi-threaded file processing (which I can't help you with), then I suggest minimizing the amount of writes to speed up your code. Furthermore, you could speed it up further by multi-threading the work and then having a single thread write to the file.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1