Task granularity is where many Fork/Join workloads succeed or fail.
You can have:
- a correct algorithm
- enough CPU cores
- a good pool
and still get disappointing performance because tasks are the wrong size.
This is one of the most important tuning topics in the framework.
Problem Statement
Every recursive workload needs a threshold:
- below this size, do the work directly
- above this size, split again
That threshold controls:
- how many tasks are created
- how much scheduling overhead exists
- how much parallelism the pool can exploit
Too small and the runtime drowns in task overhead. Too large and there is not enough parallel work to keep workers busy.
Mental Model
Granularity is a balance between:
- scheduling cost
- useful work per task
Each task should be large enough to justify its existence, but small enough that multiple workers can participate.
There is no universal threshold such as:
- 100 items
- 1,000 items
The right answer depends on:
- cost per element
- combine cost
- CPU count
- data distribution
Symptoms of Too-Fine Granularity
Common signs:
- huge task counts
- lower throughput than sequential code
- high framework overhead
- lots of joining for little useful work
This often happens when developers recursively split until tasks are extremely tiny because “more parallelism sounds better.”
In Fork/Join, that instinct is usually wrong.
Symptoms of Too-Coarse Granularity
Common signs:
- some workers stay idle
- total runtime barely improves over sequential code
- one branch dominates while others finish early
This happens when the threshold is so large that the workload does not create enough parallel pieces.
Runnable Example
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveTask;
public class GranularityDemo {
public static void main(String[] args) {
int[] numbers = new int[200_000];
for (int i = 0; i < numbers.length; i++) {
numbers[i] = 1;
}
ForkJoinPool pool = new ForkJoinPool();
long result = pool.invoke(new SumTask(numbers, 0, numbers.length, 5_000));
pool.shutdown();
System.out.println("Result = " + result);
}
static final class SumTask extends RecursiveTask<Long> {
private final int[] numbers;
private final int start;
private final int end;
private final int threshold;
SumTask(int[] numbers, int start, int end, int threshold) {
this.numbers = numbers;
this.start = start;
this.end = end;
this.threshold = threshold;
}
@Override
protected Long compute() {
if (end - start <= threshold) {
long total = 0;
for (int i = start; i < end; i++) {
total += numbers[i];
}
return total;
}
int mid = (start + end) / 2;
SumTask left = new SumTask(numbers, start, mid, threshold);
SumTask right = new SumTask(numbers, mid, end, threshold);
left.fork();
long rightValue = right.compute();
long leftValue = left.join();
return leftValue + rightValue;
}
}
}
Now rerun the workload with several threshold values. That experiment teaches more than any rule of thumb.
Practical Tuning Strategy
Start with a sensible coarse threshold. Then measure several runs with:
- smaller thresholds
- larger thresholds
Track:
- total runtime
- CPU utilization
- task count if available
You are looking for the zone where:
- parallelism is enough
- management overhead is still small
Do not tune based on a single lucky run. Use repeated measurements.
Common Mistakes
Copying a threshold from another codebase
Different workloads have different costs per unit of work.
Ignoring combine cost
If the combine step is expensive, smaller tasks may amplify that cost.
Forgetting input skew
If some partitions are more expensive than others, task sizing may need to compensate.
Comparing only against other Fork/Join variants
The real question is whether Fork/Join beats the best simple alternative.
Decision Guide
A good threshold:
- creates enough work for all cores
- does not explode task count
- leaves each leaf task doing meaningful computation
If tuning still looks fragile, consider whether the workload is a weak fit for Fork/Join in the first place.
Key Takeaways
- Task granularity is one of the biggest determinants of Fork/Join performance.
- Too-fine granularity wastes time on scheduling and joining; too-coarse granularity leaves parallelism unused.
- Thresholds should be measured against a sequential baseline, not chosen by folklore.
- A practical threshold is workload-specific and worth tuning deliberately.
Next post: Performance Traps in Fork Join Code
Comments