Page 1 of 1

BackgroundWorker Example (iterating files) Rate Topic: -----

#1 andrewsw  Icon User is online

  • But the opposite, you said.
  • member icon

Reputation: 5534
  • View blog
  • Posts: 21,842
  • Joined: 12-December 12

Posted 02 November 2014 - 05:37 AM

The main purpose of this tutorial is to provide a third example of the use of a BackgroundWorker in addition to the two examples provided by Microsoft here:

BackgroundWorker Class :MSDN

I don't have a problem with those two examples, and I highly recommend studying that page. However, most BGW examples follow the same pattern:

  • They use Thread.Sleep as a pretence for doing some actual work. This is fine, and sensible, for a first example, but it leaves people wondering how, and when, a BGW would be used in a real application.
  • A Fibonacci calculator. This is the classic example.

AdamSpeight2008 also has a tutorial here that uses Thread.Sleep, but also goes on to describe the use of a delegate. Take a look.

Typically, in real applications, BGWs are used when downloading, or streaming, large files, when performing bulk database operations, or performing complex scientific calculations. Of course, it is difficult to provide a simple example using one of these processes. (There is also the Task Parallel Library to consider.)



I'll admit that my example also doesn't do anything particularly useful. It simply iterates all the files and folders within a location that you specify, reporting back how long it took to complete the process. It also periodically updates the UI with the name of the file that it has currently reached. Nevertheless, it is a third example, and is more realistic than just using Thread.Sleep.

Edited: See the comments below, my process skips files from the current folder, so needs to run with a folder that contains subfolders.

Worker Summary

The process is quite logical really:

  • Start the Worker
  • The Worker can report its progress, so that the UI can be updated
  • The work being done will either complete or be cancelled (or error)
  • The RunWorkerCompleted event can distinguish between the work being completed or cancelled
  • The UI can then be updated accordingly, depending on whether the work completed or was cancelled.

The work being done could be anything EXCEPT anything that involves the UI.

It's up-to you to decide when, and how often, the Worker reports its progress.

Attached Image

  • The process can be cancelled
  • For every 100 files iterated the name of the current file will be added to the ListBox
  • The status-label will indicate that the process has been started, or cancelled, or, if the process completes successfully, will state how long the process took.

Notes:

Most often a ProgressBar is used to indicate progress of the work. In fact, the first argument to the ReportProgress() method is percentProgress. To use this with our example we would first have to work out how many files/folders need to be iterated (or, at least, a reasonable estimate of this number).

Do not attempt to update the UI with the name of every file/folder iterated. The files will be located far faster than the UI could possibly keep up with. I selected an interval of every 100 files but, for a fast computer, this number should probably be increased.

The Application

Create a new WinForm application. I named mine BackgroundExample.

Add a TextBox, two Buttons, a ListBox and three Labels. Also add a BackgroundWorker. Names:

txtLocation, btnStart, btnCancel, lstFound, lblLocation, lblStatus, lblSamples, bgWorker

Here is the full code for reference, discussed below (in code-order):
Option Strict On

Imports System.ComponentModel   'BackgroundWorker
Imports System.IO

Public Class frmBackground

    Private Sub frmBackground_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        bgWorker.WorkerReportsProgress = True
        bgWorker.WorkerSupportsCancellation = True
    End Sub

    Private Sub btnStart_Click(sender As Object, e As EventArgs) Handles btnStart.Click
        If Not bgWorker.IsBusy Then
            If String.IsNullOrWhiteSpace(txtLocation.Text) Then
                MessageBox.Show("Provide a location.", "No input", MessageBoxButtons.OK, MessageBoxIcon.Error)
                Exit Sub
            End If
            lstFound.Items.Clear()
            lblStatus.Text = "Starting.."
            bgWorker.RunWorkerAsync(txtLocation.Text)
        End If
    End Sub

    Private Sub btnCancel_Click(sender As Object, e As EventArgs) Handles btnCancel.Click
        If bgWorker.WorkerSupportsCancellation Then
            bgWorker.CancelAsync()
        End If
    End Sub

    Private Sub worker_DoWork(sender As Object, e As DoWorkEventArgs) Handles bgWorker.DoWork
        Dim location As String = CType(e.Argument, String)
        Dim worker As BackgroundWorker = CType(sender, BackgroundWorker)
        Dim watch As New Stopwatch
        watch.Start()
        IterateLocation(location, worker, e)
        watch.Stop()
        e.Result = watch.Elapsed.TotalSeconds
    End Sub

    Sub IterateLocation(ByVal location As String, ByVal worker As BackgroundWorker, ByVal e As DoWorkEventArgs)
        Dim sDir As String      'can't use Dir, it's a function
        Dim sFile As String
        Dim counter As Integer = 0

        If worker.CancellationPending Then
            e.Cancel = True
            Exit Sub
        End If

        Try
            For Each sDir In Directory.GetDirectories(location)
                For Each sFile In Directory.GetFiles(sDir)
                    counter += 1
                    If counter Mod 100 = 0 Then
                        If worker.CancellationPending Then
                            e.Cancel = True
                            Exit Sub
                        End If
                        worker.ReportProgress(0, sFile)     '0 is percentage complete, N/A
                    End If
                Next
                IterateLocation(sDir, worker, e)
            Next
        Catch ex As System.Exception
            'Debug.WriteLine(ex.Message)
            'we are only iterating, ignore exceptions
        End Try
    End Sub

    Private Sub worker_ProgressChanged(sender As Object, e As ProgressChangedEventArgs) Handles bgWorker.ProgressChanged
        lstFound.Items.Add(CType(e.UserState, String))
    End Sub

    Private Sub worker_RunWorkerCompleted(sender As Object, e As RunWorkerCompletedEventArgs) Handles bgWorker.RunWorkerCompleted
        If e.Error IsNot Nothing Then
            MessageBox.Show(e.Error.Message)
        ElseIf e.Cancelled Then
            lblStatus.Text = "Cancelled!"
        Else
            lblStatus.Text = String.Format("{0} seconds", e.Result)
        End If
    End Sub

End Class




Option Strict On

Imports System.ComponentModel   'BackgroundWorker
Imports System.IO

Public Class frmBackground

    Private Sub frmBackground_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        bgWorker.WorkerReportsProgress = True
        bgWorker.WorkerSupportsCancellation = True
    End Sub


These two properties could be set in the Form Designer if you prefer.
    Private Sub btnStart_Click(sender As Object, e As EventArgs) Handles btnStart.Click
        If Not bgWorker.IsBusy Then
            If String.IsNullOrWhiteSpace(txtLocation.Text) Then
                MessageBox.Show("Provide a location.", "No input", MessageBoxButtons.OK, MessageBoxIcon.Error)
                Exit Sub
            End If
            lstFound.Items.Clear()
            lblStatus.Text = "Starting.."
            bgWorker.RunWorkerAsync(txtLocation.Text)
        End If
    End Sub


This is fairly straight-forward. If the BackgroundWorker isn't already occupied (hasn't been started), check if there is a location to search. If there is a location, then clear the ListBox and start the Worker, providing the location as argument.

Note that the (optional) argument to RunWorkerAsync() is an object, we are just providing a simple string.
    Private Sub btnCancel_Click(sender As Object, e As EventArgs) Handles btnCancel.Click
        If bgWorker.WorkerSupportsCancellation Then
            bgWorker.CancelAsync()
        End If
    End Sub


Simples! If we can cancel the worker, cancel it. That is, cancel its Work.
    Private Sub worker_DoWork(sender As Object, e As DoWorkEventArgs) Handles bgWorker.DoWork
        Dim location As String = CType(e.Argument, String)
        Dim worker As BackgroundWorker = CType(sender, BackgroundWorker)
        Dim watch As New Stopwatch
        watch.Start()
        IterateLocation(location, worker, e)
        watch.Stop()
        e.Result = watch.Elapsed.TotalSeconds
    End Sub


To create this procedure-stub you can double-click the DoWork event in the Properties Window of the BGW.

Dim location As String = CType(e.Argument, String) This is how we read the argument supplied with RunWorkerAsync().

Dim worker As BackgroundWorker = CType(sender, BackgroundWorker) The BGW is already available within the Form-Class as bgWorker. However, it is preferable to take this approach, gaining a reference to the current worker via sender, because you may decide to add other workers in the future.

We are using a StopWatch to time the iterations. The helper-method IterateLocation() will do the iterating, but this method needs access to the worker and its EventArgs.

e.Result = watch.Elapsed.TotalSeconds This (optional) Result is also an object. For our example the elapsed-time in seconds (a double) to complete the iterations is our result. (This Result will be read, and displayed, in the RunWorkerCompleted event.)

    Sub IterateLocation(ByVal location As String, ByVal worker As BackgroundWorker, ByVal e As DoWorkEventArgs)
        Dim sDir As String      'can't use Dir, it's a function
        Dim sFile As String
        Dim counter As Integer = 0

        If worker.CancellationPending Then
            e.Cancel = True
            Exit Sub
        End If

        Try
            For Each sDir In Directory.GetDirectories(location)
                For Each sFile In Directory.GetFiles(sDir)
                    counter += 1
                    If counter Mod 100 = 0 Then
                        If worker.CancellationPending Then
                            e.Cancel = True
                            Exit Sub
                        End If
                        worker.ReportProgress(0, sFile)     '0 is percentage complete, N/A
                    End If
                Next
                IterateLocation(sDir, worker, e)
            Next
        Catch ex As System.Exception
            'Debug.WriteLine(ex.Message)
            'we are only iterating, ignore exceptions
        End Try
    End Sub


        If worker.CancellationPending Then
            e.Cancel = True
            Exit Sub
        End If


There is a complication with our code. This method will be called recursively to iterate different folders. We need to ensure that the method call(s) can be cancelled, so it is sensible to check for this condition each time the method is called. (In simpler, non-recursive, examples there may not be a need to call Exit Sub.)

If counter Mod 100 = 0 Then We are using a simple counter to update the UI after every 100 files. However, this value will be reset with each method-call. So, if the current folder doesn't have 100 files, then the UI won't be updated. This doesn't concern me because displaying filenames is really just to indicate that work is in progress. Besides, if the current folder doesn't have 100 files then one soon after probably will, or the whole process will have finished anyway.

If worker.CancellationPending Then We need to take this opportunity to check for cancellation again. If the current folder has thousands of files then they will continue to be iterated, because we don't otherwise check for cancellation until the method is called again.

worker.ReportProgress(0, sFile) This raises the worker's ProgressChanged event. The first argument is percentProgress, which we aren't using; the second is userState which is, again, an object, but we are just passing a string: the current filename.

IterateLocation(sDir, worker, e) It is a little clumsy to keep forwarding the worker-details with each recursive-call. Other approaches (Static or class-level variables) would also be clumsy to implement but, more importantly, would be error prone. It is the combination of asynchronous operation, and recursive calls, that makes it safest to relay the worker-details as arguments.

There are non-recursive ways to iterate files and folders. I have stuck with the recursive approach for my example. (This is also the approach used with the Fibonacci example.)

The Try..Catch block is necessary to ignore files or folders to which we do not have access. Attempting to read these raises an UnauthorizedAccessException. However, there are a couple more possible Exceptions that could occur. Although it is a very bad idea to ignore Exceptions, especially System.Exception, it is reasonable in this case. We are only iterating files, not reading or writing anything.
    Private Sub worker_ProgressChanged(sender As Object, e As ProgressChangedEventArgs) Handles bgWorker.ProgressChanged
        lstFound.Items.Add(CType(e.UserState, String))
    End Sub


It is in this ProgressChanged event that we can safely update the UI. We cannot, and must not, attempt to do anything with the UI in the DoWork event.

Recall that the UserState is the current filename that we passed in the ReportProgress() call. Add this to the ListBox.
    Private Sub worker_RunWorkerCompleted(sender As Object, e As RunWorkerCompletedEventArgs) Handles bgWorker.RunWorkerCompleted
        If e.Error IsNot Nothing Then
            MessageBox.Show(e.Error.Message)
        ElseIf e.Cancelled Then
            lblStatus.Text = "Cancelled!"
        Else
            lblStatus.Text = String.Format("{0} seconds", e.Result)
        End If
    End Sub


RunWorkerCompleted is the other event of a BackgroundWorker in which we have access to the UI. (The only other event is Dispose.)

"Completed" does not mean that the work was successfully completed; we have to first check that the work wasn't cancelled and that there wasn't an error.

Attached File  BackgroundExample.zip (13.09K)
Number of downloads: 504

Homework: Add a check that the location exists.

This post has been edited by andrewsw: 13 February 2016 - 11:49 AM


Is This A Good Question/Topic? 1
  • +

Replies To: BackgroundWorker Example (iterating files)

#2 andrewsw  Icon User is online

  • But the opposite, you said.
  • member icon

Reputation: 5534
  • View blog
  • Posts: 21,842
  • Joined: 12-December 12

Posted 02 November 2014 - 10:04 AM

Here is a variation that uses a Timer to add the current filename to the ListBox every 200ms, rather than after every 100 files.

Option Strict On

Imports System.ComponentModel   'BackgroundWorker
Imports System.IO

Public Class frmBackground
    Private _fileName As String = String.Empty

    Private Sub frmBackground_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        bgWorker.WorkerReportsProgress = True
        bgWorker.WorkerSupportsCancellation = True
    End Sub

    Private Sub btnStart_Click(sender As Object, e As EventArgs) Handles btnStart.Click
        If Not bgWorker.IsBusy Then
            If String.IsNullOrWhiteSpace(txtLocation.Text) Then
                MessageBox.Show("Provide a location.", "No input", MessageBoxButtons.OK, MessageBoxIcon.Error)
                Exit Sub
            End If
            If Not Directory.Exists(txtLocation.Text) Then
                MessageBox.Show("That folder doesn't exist.", "Invalid", MessageBoxButtons.OK, MessageBoxIcon.Error)
                Exit Sub
            End If
            lstFound.Items.Clear()
            lblStatus.Text = "Starting.."
            tmrUpdate.Interval = 200
            tmrUpdate.Start()
            bgWorker.RunWorkerAsync(txtLocation.Text)
        End If
    End Sub

    Private Sub btnCancel_Click(sender As Object, e As EventArgs) Handles btnCancel.Click
        If bgWorker.WorkerSupportsCancellation Then
            bgWorker.CancelAsync()
        End If
    End Sub

    Private Sub worker_DoWork(sender As Object, e As DoWorkEventArgs) Handles bgWorker.DoWork
        Dim location As String = CType(e.Argument, String)
        Dim worker As BackgroundWorker = CType(sender, BackgroundWorker)
        Dim watch As New Stopwatch
        watch.Start()
        IterateLocation(location, worker, e)
        watch.Stop()
        e.Result = watch.Elapsed.TotalSeconds
    End Sub

    Sub IterateLocation(ByVal location As String, ByVal worker As BackgroundWorker, ByVal e As DoWorkEventArgs)
        Dim sDir As String      'can't use Dir, it's a function
        Dim sFile As String

        If worker.CancellationPending Then
            e.Cancel = True
            Exit Sub
        End If

        Try
            For Each sDir In Directory.GetDirectories(location)
                For Each sFile In Directory.GetFiles(sDir)
                    SyncLock _fileName
                        _fileName = sFile
                    End SyncLock
                Next
                IterateLocation(sDir, worker, e)
            Next
        Catch ex As System.Exception
            'Debug.WriteLine(ex.Message)
            'we are only iterating, ignore exceptions
        End Try
    End Sub

    Private Sub worker_ProgressChanged(sender As Object, e As ProgressChangedEventArgs) Handles bgWorker.ProgressChanged
        'lstFound.Items.Add(CType(e.UserState, String))
    End Sub

    Private Sub worker_RunWorkerCompleted(sender As Object, e As RunWorkerCompletedEventArgs) Handles bgWorker.RunWorkerCompleted
        tmrUpdate.Stop()

        If e.Error IsNot Nothing Then
            MessageBox.Show(e.Error.Message)
        ElseIf e.Cancelled Then
            lblStatus.Text = "Cancelled!"
        Else
            lblStatus.Text = String.Format("{0} seconds", e.Result)
        End If
    End Sub

    Private Sub tmrUpdate_Tick(sender As Object, e As EventArgs) Handles tmrUpdate.Tick
        lstFound.Items.Add(_fileName)
    End Sub
End Class


It uses a field to store the current filename. I was reading this SO topic that suggests that it is safe to do this with a simple variable. The SyncLock is added to prevent more than one thread executing the same code-block.

This post has been edited by andrewsw: 02 November 2014 - 10:11 AM

Was This Post Helpful? 1
  • +
  • -

#3 robbie1973  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 6
  • Joined: 28-May 07

Posted 03 November 2014 - 07:32 AM

Just want to thank you for these Andrew.

Thank you very much indded. :)
Was This Post Helpful? 0
  • +
  • -

#4 SlowArrow  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 2
  • Joined: 13-February 16

Posted 13 February 2016 - 06:27 AM

View Postandrewsw, on 02 November 2014 - 05:37 AM, said:

It simply iterates all the files and folders within a location that you specify...


Should the code go on those files as well, which are in a directory with no directories, or this was not among the aims of the example?

To achieve this, in IterateLocation we should do the 2 loops "in sequence" of each other, I think.
Was This Post Helpful? 0
  • +
  • -

#5 andrewsw  Icon User is online

  • But the opposite, you said.
  • member icon

Reputation: 5534
  • View blog
  • Posts: 21,842
  • Joined: 12-December 12

Posted 13 February 2016 - 07:45 AM

You are right, I don't iterate the files of the folder, just within its subfolders. I should have explained this.

I wasn't really concerned about the precise routine, I just wanted something long running with some output (the file list) that I could update on the UI. However, I won't be adding anything to the tutorial as there are plenty of examples that can be found to iterate all files and subfolders.

If you modify the code yourself you could add it as a further reply ;)
Was This Post Helpful? 0
  • +
  • -

#6 andrewsw  Icon User is online

  • But the opposite, you said.
  • member icon

Reputation: 5534
  • View blog
  • Posts: 21,842
  • Joined: 12-December 12

Posted 13 February 2016 - 09:01 AM

For anyone interested though, here is my attempt to iterate ALL files and folders, ignoring any unauthorized access to files/folders:
Imports System.IO

Public Class Form1

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        ListFilesAndFolders(TextBox1.Text)
    End Sub

    Private Sub ListFilesAndFolders(location As String)

        For Each sPath In Directory.GetDirectories(location)
            ListBox1.Items.Add(sPath)
            Try
                ListFilesAndFolders(sPath)
            Catch ex As Exception
                Debug.Print(ex.Message)
                'ignore, only iterating
            End Try

        Next
        Try
            For Each sFile In Directory.GetFiles(location)
                ListBox1.Items.Add(sFile)
            Next
        Catch ex As UnauthorizedAccessException
            Debug.Print("Unauthorized file access")
        Catch ex As Exception
            'ignore
        End Try

    End Sub
End Class

Was This Post Helpful? 0
  • +
  • -

#7 SlowArrow  Icon User is offline

  • New D.I.C Head

Reputation: 1
  • View blog
  • Posts: 2
  • Joined: 13-February 16

Posted 13 February 2016 - 01:08 PM

View Postandrewsw, on 13 February 2016 - 07:45 AM, said:

I wasn't really concerned about the precise routine

Indeed, your explanations are more important, and I really like the way you do that :)/>

However, if one wants to list all of the files (but not the directory paths), besides keeping oneself to your initial code and to that reporting facility you had coded in, then just a very little modification is needed in the IterateLocation's logic (if I am not wrong ;)/> ):
    Sub IterateLocation(ByVal location As String, ByVal worker As BackgroundWorker, ByVal e As DoWorkEventArgs)
        Dim sDir As String      'can't use Dir, it's a function
        Dim sFile As String
        Dim counter As Integer = 0

        If worker.CancellationPending Then
            e.Cancel = True
            Exit Sub
        End If

        Try
            For Each sDir In Directory.GetDirectories(location)
                IterateLocation(sDir, worker, e)
            Next
            For Each sFile In Directory.GetFiles(location)
                counter += 1
                'If counter Mod 100 = 0 Then
                If worker.CancellationPending Then
                    e.Cancel = True
                    Exit Sub
                End If
                worker.ReportProgress(0, sFile)     '0 is percentage complete, N/A
                'End If
            Next
        Catch ex As System.Exception
            'Debug.WriteLine(ex.Message)
            'we are only iterating, ignore exceptions
        End Try
    End Sub


The commented "If counter Mod 100 = 0 Then" was needed to report each of the files (instead of showing the progression).
Was This Post Helpful? 1
  • +
  • -

#8 andrewsw  Icon User is online

  • But the opposite, you said.
  • member icon

Reputation: 5534
  • View blog
  • Posts: 21,842
  • Joined: 12-December 12

Posted 13 February 2016 - 01:42 PM

Thank you for this.

I should point out for others though ;) that this creates a slightly different example to mine. Mine specifically only updates the ListBox (and checks for cancellation) with a selection of the files that are being iterated. That is, mine only lists (reports) every 100th file that has been iterated, or the file that is encountered after each Timer interval. I didn't want to burden the UI with listing every file.

Of course, in reality, with a procedure like this then you're likely to want to list all the files. But, on the other hand, we shouldn't often have the need to (and should avoid) list hundreds of thousands of files ;).
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1