Mastering TaskCompletionSource in C#
Most of the time async/await just works. You call a method that returns a Task, you await it, the runtime does the rest. The trouble starts when the thing you’re waiting on doesn’t return a Task at all: a callback-based API, an event that fires later, a response that arrives on a socket tagged with a correlation ID. Something has to turn “this will happen eventually” into a Task the caller can await.
That something is TaskCompletionSource<TResult>, usually shortened to TCS. It’s the primitive underneath a surprising amount of the framework, and once you know its three or four rules it’s hard to misuse. Without those rules it’s a reliable source of deadlocks and tasks that never complete.
Before going further, a quick filter, because TCS gets used in places it doesn’t belong. If the operation is already a Task, you don’t need it. If the callback API is the old Begin/End style with IAsyncResult, Task.Factory.FromAsync already does the adaptation. If you’re waiting for many events rather than the next one, you want a queue, and System.Threading.Channels is that queue. If the allocation shows up in a profiler, the pooled version of this idea is IValueTaskSource. TCS is for the gap between “an event will happen once” and “something wants to await it”. Everything below assumes you’re in that gap.
Anatomy of a TaskCompletionSource
A TCS owns a Task<TResult> whose completion you control by hand. Nothing runs on your behalf. You create it, hand out tcs.Task to whoever wants to await, and at some later point you decide how that task ends:
SetResult(value)completes it successfully.SetException(ex)faults it; awaiting code sees the exception rethrown.SetCanceled()moves it to the cancelled state, so awaiters get anOperationCanceledException.
Each has a Try… twin that returns false instead of throwing if the task has already finished. You’ll use the Try… versions far more often than the plain ones.
var tcs = new TaskCompletionSource<int>();
// Consumer: suspends here until someone completes the task
var consumer = Task.Run(async () =>
{
int result = await tcs.Task;
Console.WriteLine($"Result: {result}");
});
// Producer: somewhere else entirely, half a second later
_ = Task.Delay(500).ContinueWith(_ => tcs.TrySetResult(42));If the task is already complete when you await it, the await returns synchronously. If not, the awaiter registers a continuation and the Set… call later is what schedules it. That last detail matters more than it looks, and we’ll come back to it.
Since .NET 5 there’s also a non-generic TaskCompletionSource for operations with no result. Before that, the convention was TaskCompletionSource<object> or <bool> with a throwaway value, and you’ll still see plenty of that in older code.
The one constructor argument you should always pass
var tcs = new TaskCompletionSource<string>(TaskCreationOptions.RunContinuationsAsynchronously);By default, when you call SetResult, any continuations attached to the task run synchronously on the thread that called SetResult, before SetResult returns. So if a producer completes a TCS while holding a lock, and the awaiting code’s continuation tries to take that same lock, you’ve deadlocked yourself. Or less dramatically: a network read loop calls SetResult, the continuation does a few hundred milliseconds of work, and your read loop is stalled for that whole time.
RunContinuationsAsynchronously forces continuations onto the thread pool. The producer calls SetResult, it returns immediately, the awaiter wakes up on a pool thread. There are narrow cases where inline continuations are what you want for latency reasons, but they’re rare enough that the flag should be your default and its absence should be a deliberate choice with a comment next to it.
Completion, faulting and cancellation
SetResult vs TrySetResult
SetResult throws InvalidOperationException if the task is already complete. TrySetResult returns false. In real code there’s almost always more than one path that can finish an operation: a success callback, an error callback, a timeout, a cancellation token. Any two of them can race. Use the Try… variants and let the loser of the race fail quietly.
if (!tcs.TrySetResult(responseCode))
{
// Something else already completed it (timeout, cancellation, duplicate callback)
}Faulting
tcs.TrySetException(new InvalidOperationException("Gateway returned malformed frame"));The awaiter sees the exception rethrown from await tcs.Task. There’s an overload that takes IEnumerable<Exception> for the rare case where several things went wrong at once; they’re wrapped in an AggregateException on the task but await unwraps the first one.
Cancellation
Two things people get wrong here. First, a CancellationToken doesn’t cancel anything on its own. You have to wire it to the TCS:
Task<T> WithCancellation<T>(TaskCompletionSource<T> tcs, CancellationToken token)
{
token.Register(
static state => ((TaskCompletionSource<T>)state!).TrySetCanceled(),
tcs,
useSynchronizationContext: false);
return tcs.Task;
}Second, pass the token into TrySetCanceled(token) when you have it. That way the resulting OperationCanceledException carries the token, which is what Task.WhenAny and most framework code use to tell “cancelled because the caller asked” apart from “cancelled for some other reason”.
The registration returned by token.Register is disposable. Dispose it when the operation completes normally, otherwise a long-lived token (say, an application-lifetime token) accumulates one dead registration per operation. That’s a slow leak that only shows up in a service that’s been running for a week.
If all you want is a timeout, you don’t need any of this. Since .NET 6, Task has WaitAsync:
var result = await tcs.Task.WaitAsync(TimeSpan.FromSeconds(5), cancellationToken);The TCS itself stays incomplete after a timeout, so you still need to clean it up on the producer side, but the caller is released.
Adapting callbacks and events
Callback to Task
The classic case. You have a library with Begin…/End… pairs or a plain callback parameter, and you want callers to await it.
Task<string> ReadDataAsync(ILegacyApi api, CancellationToken token = default)
{
var tcs = new TaskCompletionSource<string>(TaskCreationOptions.RunContinuationsAsynchronously);
var registration = token.Register(() => tcs.TrySetCanceled(token));
api.BeginRead(result =>
{
try { tcs.TrySetResult(api.EndRead(result)); }
catch (Exception ex) { tcs.TrySetException(ex); }
finally { registration.Dispose(); }
}, state: null);
return tcs.Task;
}Every branch of the callback completes the TCS. That’s the discipline: if there’s any code path through the callback that doesn’t call a Try… method, you’ll eventually ship a task that never finishes.
For real APM-style Begin/End pairs that take an AsyncCallback and return IAsyncResult, you don’t need to write this yourself. Task.Factory.FromAsync does exactly this adaptation and has been in the framework since .NET 4.
Event to Task
Suppose a device driver raises SignalReceived and you want to await the next occurrence:
Task WaitForSignalAsync(IEventSource source, CancellationToken token = default)
{
var tcs = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
EventHandler? handler = null;
handler = (_, _) =>
{
source.SignalReceived -= handler;
tcs.TrySetResult();
};
source.SignalReceived += handler;
token.Register(() =>
{
source.SignalReceived -= handler;
tcs.TrySetCanceled(token);
});
return tcs.Task;
}The unsubscribe on both paths is the whole point. An event subscription is a strong reference from the publisher to your handler, and the handler closes over the TCS. Forget to unsubscribe on cancellation and every cancelled wait keeps a TCS alive for as long as the event source lives.
There’s a subtle race in the version above: if the event fires between += and the cancellation registration, both run and one Try… loses, which is fine. If the event fires before +=, you miss it. Whether that matters depends on your event source; if it does, you need a queue, not a TCS.
Request and response correlation
This is the pattern where TCS stops looking like a curiosity and starts looking essential. Imagine a protocol where you write a request onto a socket with a correlation ID, and some time later a response arrives on a separate read loop carrying the same ID. Nothing about that is naturally awaitable. With a dictionary of pending TCS instances, it becomes one method call:
public sealed class RequestClient
{
private readonly ConcurrentDictionary<Guid, TaskCompletionSource<Response>> _pending = new();
private readonly IConnection _conn;
public RequestClient(IConnection conn)
{
_conn = conn;
_conn.FrameReceived += OnFrame;
}
public async Task<Response> SendAsync(Request req, CancellationToken token)
{
var id = Guid.NewGuid();
var tcs = new TaskCompletionSource<Response>(TaskCreationOptions.RunContinuationsAsynchronously);
_pending[id] = tcs;
using var reg = token.Register(() =>
{
if (_pending.TryRemove(id, out var t)) t.TrySetCanceled(token);
});
try
{
await _conn.WriteAsync(req.WithCorrelationId(id), token);
return await tcs.Task.WaitAsync(TimeSpan.FromSeconds(30), token);
}
finally
{
_pending.TryRemove(id, out _);
}
}
private void OnFrame(Response resp)
{
if (_pending.TryRemove(resp.CorrelationId, out var tcs))
tcs.TrySetResult(resp);
// else: late response for a request that timed out or was cancelled; drop it
}
public void FailAll(Exception ex)
{
foreach (var (id, tcs) in _pending)
if (_pending.TryRemove(id, out _)) tcs.TrySetException(ex);
}
}Notice FailAll. When the connection drops, every outstanding request has to be faulted or the callers hang forever. Any time you hold a collection of pending TCS objects, you need a “fail everything” method and you need to call it from the place that detects disconnection.
The TryRemove in OnFrame is also what makes this safe against duplicate frames: the second copy finds nothing in the dictionary and is dropped. The same shape, a map of pending completions keyed by ID, turns up again when the thing you’re waiting on lives in another process; see coordinating distributed operations with Hazelcast.
Building synchronization primitives
Combine a TCS with a lock or an atomic and you can build async-friendly versions of most classic primitives. An async mutual-exclusion lock is the usual demonstration:
public sealed class AsyncLock
{
private readonly Queue<TaskCompletionSource<IDisposable>> _waiters = new();
private bool _isLocked;
public Task<IDisposable> LockAsync()
{
lock (_waiters)
{
if (!_isLocked)
{
_isLocked = true;
return Task.FromResult<IDisposable>(new Releaser(this));
}
var tcs = new TaskCompletionSource<IDisposable>(TaskCreationOptions.RunContinuationsAsynchronously);
_waiters.Enqueue(tcs);
return tcs.Task;
}
}
private void Release()
{
TaskCompletionSource<IDisposable>? next = null;
lock (_waiters)
{
if (_waiters.Count > 0) next = _waiters.Dequeue();
else _isLocked = false;
}
// Completed outside the lock. With RunContinuationsAsynchronously this
// is belt and braces, but it keeps the primitive correct even if someone
// removes the flag later.
next?.TrySetResult(new Releaser(this));
}
private sealed class Releaser(AsyncLock owner) : IDisposable
{
public void Dispose() => owner.Release();
}
}Usage is using (await myLock.LockAsync()) { ... }. The two things worth studying are that the TCS is completed outside the lock block, and that the fast path doesn’t allocate a TCS at all.
Should you ship this? Probably not. SemaphoreSlim(1, 1) with WaitAsync does the same job and has had a decade of bug fixes. The value of writing an AsyncLock is in understanding how the pieces fit, so that when you need something SemaphoreSlim doesn’t do (priority ordering, fairness guarantees, an AsyncManualResetEvent), you know how to build it.
Troubleshooting
InvalidOperationException: An attempt was made to transition a task to a final state when it had already completed.
You called SetResult/SetException/SetCanceled on a task that was already done. Two paths raced. Switch to the Try… variant; if you genuinely expected exactly one completion, log when Try… returns false so you can find the duplicate.
The await resumes on the wrong thread, or your read loop stalls.
Missing RunContinuationsAsynchronously. Add it to the constructor.
Memory grows steadily under load.
Look for a ConcurrentDictionary of pending TCS instances that only removes on success. Add removal in a finally, on cancellation, and in the disconnection handler.
OperationCanceledException but ex.CancellationToken doesn’t match the one I passed.
You called TrySetCanceled() without the token argument. Pass it through.
Unit tests hang instead of failing.
Wrap awaited TCS tasks in WaitAsync(TimeSpan) in tests. A hang tells you nothing; a TimeoutException with a stack trace tells you which completion path you missed.
Common pitfalls
A task that never completes. The number one TCS bug. Some code path through the producer doesn’t call a Try… method: an exception thrown before SetResult, an early return in a callback, a disconnection nobody handled. The caller sits on await forever and there’s no exception to find in your logs. Defend against it with WaitAsync timeouts on the consumer side and a FailAll-style sweep on the producer side.
Blocking on the result. .Result or .Wait() on a TCS task inside code that also needs to run the producer is a deadlock waiting to happen, especially on a UI thread or in older ASP.NET with a synchronization context. Await all the way down.
Synchronous continuations. Covered above. If you didn’t pass RunContinuationsAsynchronously, your producer is running the consumer’s code on its own thread, inside whatever locks it happens to hold.
Leaked registrations and subscriptions. Every token.Register and every += needs a matching dispose or -= on every exit path, including the cancelled and faulted ones. These leaks are invisible under test and obvious after a week in production.