Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel.ForEach should not be used for IO bound tasks #22957

Merged
merged 2 commits into from
Mar 3, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Write a simple parallel program using Parallel.ForEach
description: In this article, learn how to enable data parallelism in .NET. Write a Parallel.ForEach loop over any IEnumerable or IEnumerable<T> data source.
ms.date: 02/14/2019
ms.date: 02/23/2021
dev_langs:
- "csharp"
- "vb"
Expand All @@ -19,7 +19,7 @@ This example shows how to use a <xref:System.Threading.Tasks.Parallel.ForEach%2A

## Example

This example assumes you have several .jpg files in a *C:\Users\Public\Pictures\Sample Pictures* folder and creates a new sub-folder named *Modified*. When you run the example, it rotates each .jpg image in *Sample Pictures* and saves it to *Modified*. You can modify the two paths as necessary.
This example demonstrates <xref:System.Threading.Tasks.Parallel.ForEach%2A?displayProperty=nameWithType> for CPU intensive operations. When you run the example, it randomly generates 2 million numbers and tries to filter to prime numbers. The first case iterates over the collection via a `for` loop. The second case iterates over the collection via <xref:System.Threading.Tasks.Parallel.ForEach%2A?displayProperty=nameWithType>. The resulting time taken by each iteration is displayed when the application is finished.

[!code-csharp[TPL_Parallel#03](../../../samples/snippets/csharp/VS_Snippets_Misc/tpl_parallel/cs/simpleforeach.cs#03)]
[!code-vb[TPL_Parallel#03](../../../samples/snippets/visualbasic/VS_Snippets_Misc/tpl_parallel/vb/simpleforeach.vb#03)]
Expand All @@ -43,14 +43,6 @@ In Visual Studio, there are Visual Basic and C# console application templates fo

From the command line, you can use either the .NET Core CLI commands (for example, `dotnet new console` or `dotnet new console -lang vb`), or you can create the file and use the command-line compiler for a .NET Framework application.

For a .NET Core project, you must reference the **System.Drawing.Common** NuGet package. In Visual Studio, use the NuGet Package Manager to install the package. Alternatively, you can add a reference to the package in your \*.csproj or \*.vbproj file:

```xml
<ItemGroup>
<PackageReference Include="System.Drawing.Common" Version="4.5.1" />
</ItemGroup>
```

To run a .NET Core console application from the command line, use `dotnet run` from the folder that contains your application.

To run your console application from Visual Studio, press **F5**.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,40 +1,84 @@
// <snippet03>
using System;
using System.IO;
using System.Threading;
using System.Collections.Concurrent;
surenderssm marked this conversation as resolved.
Show resolved Hide resolved
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
using System.Drawing;

public class Example
namespace ParallelExample
{
public static void Main()
class Program
{
// A simple source for demonstration purposes. Modify this path as necessary.
string[] files = Directory.GetFiles(@"C:\Users\Public\Pictures\Sample Pictures", "*.jpg");
string newDir = @"C:\Users\Public\Pictures\Sample Pictures\Modified";
Directory.CreateDirectory(newDir);

// Method signature: Parallel.ForEach(IEnumerable<TSource> source, Action<TSource> body)
Parallel.ForEach(files, (currentFile) =>
{
// The more computational work you do here, the greater
// the speedup compared to a sequential foreach loop.
string filename = Path.GetFileName(currentFile);
var bitmap = new Bitmap(currentFile);

bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone);
bitmap.Save(Path.Combine(newDir, filename));

// Peek behind the scenes to see how work is parallelized.
// But be aware: Thread contention for the Console slows down parallel loops!!!

Console.WriteLine($"Processing {filename} on thread {Thread.CurrentThread.ManagedThreadId}");
//close lambda expression and method invocation
});

// Keep the console window open in debug mode.
Console.WriteLine("Processing complete. Press any key to exit.");
Console.ReadKey();
static void Main()
{
// 2 million
var limit = 2_000_000;
var numbers = Enumerable.Range(0, limit).ToList();

var watch = Stopwatch.StartNew();
var primeNumbersFromForeach = GetPrimeList(numbers);
watch.Stop();

var watchForParallel = Stopwatch.StartNew();
var primeNumbersFromParallelForeach = GetPrimeListWithParallel(numbers);
watchForParallel.Stop();

Console.WriteLine($"Classical foreach loop | Total prime numbers : {primeNumbersFromForeach.Count} | Time Taken : {watch.ElapsedMilliseconds} ms.");
Console.WriteLine($"Parallel.ForEach loop | Total prime numbers : {primeNumbersFromParallelForeach.Count} | Time Taken : {watchForParallel.ElapsedMilliseconds} ms.");

Console.WriteLine("Press any key to exit.");
Console.ReadLine();
}

/// <summary>
/// GetPrimeList returns Prime numbers by using sequential ForEach
/// </summary>
/// <param name="inputs"></param>
/// <returns></returns>
private static IList<int> GetPrimeList(IList<int> numbers) => numbers.Where(IsPrime).ToList();

/// <summary>
/// GetPrimeListWithParallel returns Prime numbers by using Parallel.ForEach
/// </summary>
/// <param name="numbers"></param>
/// <returns></returns>
private static IList<int> GetPrimeListWithParallel(IList<int> numbers)
{
var primeNumbers = new ConcurrentBag<int>();

Parallel.ForEach(numbers, number =>
{
if (IsPrime(number))
{
primeNumbers.Add(number);
}
});

return primeNumbers.ToList();
}

/// <summary>
/// IsPrime returns true if number is Prime, else false.(https://en.wikipedia.org/wiki/Prime_number)
/// </summary>
/// <param name="number"></param>
/// <returns></returns>
private static bool IsPrime(int number)
{
if (number < 2)
{
return false;
}

for (var divisor = 2; divisor <= Math.Sqrt(number); divisor++)
{
if (number % divisor == 0)
{
return false;
}
}
return true;
}
}
}
// </snippet03>
// </snippet03>
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net5.0</TargetFramework>
</PropertyGroup>

</Project>
Original file line number Diff line number Diff line change
@@ -1,37 +1,60 @@
'<snippet03>
Imports System.IO
Imports System.Threading
Imports System.Threading.Tasks
Imports System.Drawing

Module ForEachDemo

Sub Main()
' A simple source for demonstration purposes. Modify this path as necessary.
Dim files As String() = Directory.GetFiles("C:\Users\Public\Pictures\Sample Pictures", "*.jpg")
Dim newDir As String = "C:\Users\Public\Pictures\Sample Pictures\Modified"
Directory.CreateDirectory(newDir)

Parallel.ForEach(files, Sub(currentFile)
' The more computational work you do here, the greater
' the speedup compared to a sequential foreach loop.
Dim filename As String = Path.GetFileName(currentFile)
Dim bitmap As New Bitmap(currentFile)

bitmap.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipNone)
bitmap.Save(Path.Combine(newDir, filename))

' Peek behind the scenes to see how work is parallelized.
' But be aware: Thread contention for the Console slows down parallel loops!!!

Console.WriteLine($"Processing {filename} on thread {Thread.CurrentThread.ManagedThreadId}")
'close lambda expression and method invocation
End Sub)


' Keep the console window open in debug mode.
Console.WriteLine("Processing complete. Press any key to exit.")
Console.ReadKey()
End Sub
End Module
'</snippet03>
Imports System.Collections.Concurrent
surenderssm marked this conversation as resolved.
Show resolved Hide resolved

Namespace ParallelExample
Class Program
Shared Sub Main()
' 2 million
Dim limit = 2_000_000
Dim numbers = Enumerable.Range(0, limit).ToList()

Dim watch = Stopwatch.StartNew()
Dim primeNumbersFromForeach = GetPrimeList(numbers)
watch.Stop()

Dim watchForParallel = Stopwatch.StartNew()
Dim primeNumbersFromParallelForeach = GetPrimeListWithParallel(numbers)
watchForParallel.Stop()

Console.WriteLine($"Classical foreach loop | Total prime numbers : {primeNumbersFromForeach.Count} | Time Taken : {watch.ElapsedMilliseconds} ms.")
Console.WriteLine($"Parallel.ForEach loop | Total prime numbers : {primeNumbersFromParallelForeach.Count} | Time Taken : {watchForParallel.ElapsedMilliseconds} ms.")

Console.WriteLine("Press any key to exit.")
Console.ReadLine()
End Sub

' GetPrimeList returns Prime numbers by using sequential ForEach
Private Shared Function GetPrimeList(numbers As IList(Of Integer)) As IList(Of Integer)
Return numbers.Where(AddressOf IsPrime).ToList()
End Function

' GetPrimeListWithParallel returns Prime numbers by using Parallel.ForEach
Private Shared Function GetPrimeListWithParallel(numbers As IList(Of Integer)) As IList(Of Integer)
Dim primeNumbers = New ConcurrentBag(Of Integer)()
Parallel.ForEach(numbers, Sub(number)

If IsPrime(number) Then
primeNumbers.Add(number)
End If
End Sub)
Return primeNumbers.ToList()
End Function

' IsPrime returns true if number is Prime, else false.(https://en.wikipedia.org/wiki/Prime_number)
Private Shared Function IsPrime(number As Integer) As Boolean
If number < 2 Then
Return False
End If

For divisor = 2 To Math.Sqrt(number)

If number Mod divisor = 0 Then
Return False
End If
Next

Return True
End Function
End Class
End Namespace
'</snippet03>
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net5.0</TargetFramework>
</PropertyGroup>

</Project>