CCRNo More Asynchronous Spaghetti

An Introduction

Presented by Igor Moochnick

Concurrency: is it necessary?

Concurrent programming is how to make your application to scale by doing more things at once.

If you have one CPU – you can do only one thing at a time, but it doesn't mean it is actually DOING something most of the time - in fact, 99% of the time, your CPU is literally turned off.

Nowadays more and more machines have more than one core, these machines CAN do more than one thing at once.

Don’t overdesign! Not every problem can or should be solved with a concurrent solution.

What is CCR?

Concurrency and Coordination Runtime CCR is a lightweight distributed services-

oriented architecture and a common language runtime (CLR)-based library.

The CCR makes programming asynchronous behavior much simpler than the typical challenge of writing threaded code.

CCR expresses concurrent programs in explicit continuation passing (CPS) style which is often quite an arduous way for a human to express concurrent calculations.

What is CCR?

Has nothing to do with a runtime, though. It’s a managed library that can be used from any managed language: C#, VB.NET, etc...

Released (December 12, 2006) as part of the Microsoft Robotics Studio, but can be used as a standalone library (check the license!)

Here is a short explanation …

A lot of asynchronous events?

Tired of asynchronous code coordination? Do you see a resemblance?

Glimpse into your future

He definitely used the CCR !

Example (compare APM & CCR)

Need to schedule 2 (or more) asynchronous operations and, as soon as all of them complete, execute some action

How to start with CCR? Add a reference to the Ccr.Core.dll using Microsoft.Ccr.Core; Create Dispatcher Create DispatcherQueue Create ports Declare actions (handlers)

Dispatcher class

Configurable number of threads – on construction. Default: 1 thread for every CPU (if 1 CPU – 2 threads).

No dynamically creating or destroying threads. Possible to set threads' scheduling priority.

Default: normal priority.

Unlike CLR: No dynamical thread to change number of thread pool

threads based on the workload – streamlined, simple and high-performance code.

No limitation on number of thread pools (dispatcher instances) with different thread priorities for different kind of tasks.

For debugging – gives a name for each thread.

Dispatcher Queue

Maintains a queue of delegates that identify methods ready to execute.

Dispatcher's threads wait for entries to appear in the DispatcherQueue.

Order is not important (whatever comes first: events or entries).

Use as many queues as you need. Unlike CLR: Dispatcher dequeue equally

(round-robin) from all the associated queues. Useful for different priority tasks.

Default constructor – CLR thread pool

Ports and PortSets

First-In-First-Out (FIFO) Possible to instantiate a port with up to 16 different

types. Port (that carries multiple values) works by maintaining unrelated sub-ports for each type. PortSet<int,string> pIS = new PortSet<int,string>();

Reading an int value from pIS = reading from the sub-port that contains ints. This read operation is unaffected by the state of the string sub-port.

Test() to atomically remove a value from a port (if the port is not empty).

If no ReceiverTask objects are registered or if none of the registered ReceiverTask objects want the item, the item is added to the Port's internal queue.

CCR Architecture

Items are posted into a Port object. Registered ReceiverTasks send the items to their arbiters,

which decide what callback method should ultimately execute to process the posted item.

The Arbiter then queues the work item task into a DispatcherQueue, and a Dispatcher (or CLR thread pool) thread will execute the method processing the posted item.

Arbiter is a static class that defines factories that create other Arbiters

In fact each constructed Arbiter contains a reference to one or more constructed ReceiverTask objects

You must activate the arbiters by calling Arbiter's Activate method. This will register all of the arbiter's ReceiverTask objects with the Ports and it tells the arbiter which DispatcherQueue to post work item tasks to as Port items are processed.

Arbiter

Single Item Receiver

Port<int> p = new Port<int>();Arbiter.Activate (dispatcherQueue, Arbiter.Receive(false, p, delegate (int i){ // … do something … Console.WriteLine(i) ;}) );p.Post(10);

Output:10

Handler

Persisted Arbiters

Some Arbiters can be persisted or not Choice and Interleave's

TeardownReceiverGroup require that their arbiters always be non-persistent

If you forget to activate your arbiters, items will continue to queue up in Port objects but they will NEVER get processed!

Multiple Item Receiver

Will be called only when the configured number of items has accumulated on the associated port or one item across multiple ports.

An array of items will be passed atomically to the handler.

Arbiter.MultipleItemReceive<T0>(bool persist, Port<T0> port, int messageCount, VariableArgumentHandler<T0> handler);

Arbiter.MultipleItemReceive<T0,T1>(PortSet<T0,T1> port, Handler<ICollection<T0>, ICollection<T1>> handler);

Arbiter.MultiplePortReceive<T0>(bool persist, Port<T0> [] ports, VariableArgumentHandler<T0> handler);

Handler

Example: Hello World! in Math(Matrix Multiplication)

From site: Scalar and Matrix Multiplication

Data Flow Diagram

Simplified graphical representation of the data flow in the Matrix multiplication example

Result

*+

portMatrix1 portMatrix2

Multiply handler

Sum handler

portSum portResult

Example: racing conditions

STOP! There is a catch! Change the number of threads for the Dispatcher to 0

(it’ll use a default number of worker threads) Result is changing all the time

Dispatcher is providing a real scheduling of work items across multiple CPUs and Threads

Order is unpredictable !!!Never count on it !!!

Common Arbiters

single item receivers for specifying handlers on a single port

a choice arbiter which will chose which of two receivers to fire based on the availability of messages

an interleave arbiter (explained later) a join arbiter for static join expressions MultipleItemReceive arbiters for specifying

dynamic joins

Predefined Arbiters (partial list)

FromHandler

FromIteratorHandler

Receive MultipleItemReceive MultiplePortReceive

JoinedReceive

Choice

Interleave

Just executes a method (not associated with a port)

Execute until enumerator is done (not associated with a port)

Process a single item from a port Process a bunch of items at once Process a single item from multiple ports

(one item per port) Process multiple items from 2 ports (one

item per port) Executes one & only one method from a

set of methods Calls back a set of methods to process

items from various ports. Arbiter is similar to a reader/writer thread synchronization lock

Choice Arbiter

Arbiter.Choice(PortSet<T0, T1> resultPort, Handler<T0> h0, Handler<T1> h1);

Arbiter.Choice(params ReceiverTask[] receivers);

The choice arbiter allows one to specify that one of two (or more) branches should fire and the others should be discarded. For example using the first method:

Example:Arbiter.Activate(taskQueue, Arbiter.Choice( Arbiter.Receive(false, port1, MyIntHandler), Arbiter.Receive(false, port2, MyStringHandler) ) );

Will run either MyIntHandler or MyStringHandler but not both. The choice will be determined by the availability of int or string message on the ports and if both types of messages are available then a non-deterministic choice is made.

Handler

Join Arbiter Arbiter.Activate(dispatcherQueue,

Arbiter.JoinedReceive<int,string>(true, p1,p2, IntAndStringJoinedHandler) );

The above will schedule the execution of the IntAndStringJoinedHandler when values can be atomically read on ports p1 AND p2 and will continue doing so as long as messages are available (in this example its a persisted receiver). The handler will be passed the values that were read. One can join over several ports of different types.

MultiportReceive is a form of a Join Arbiter:

Arbiter.MultiplePortReceive<T0>(bool persist, Port<T0> [] ports, VariableArgumentHandler<T0> handler);

Handler

Interleave Arbiter Arbiter.Interleave(

TeardownReceiverGroup teardownGroup, ExclusiveReceiverGroup exclusiveGroup, ConcurrentReceiverGroup concurrentGroup );

Expresses concurrent calculations Corresponds to multiple reads but single writers to a shared

state. Its a more high level, far more concise way to capture advanced protection schemes.

The one time (non-persistent) shutdown handler (any one from the TeardownReceiverGroup) will cause the entire interleave to stop, guaranteeing no delegates are running or will ever run again. Makes clean up easy!

Why not to use join? Interleave is error prone for this kind of operations.

HandlerHandler

Handler

Interleave Arbiter: continues

Teardown will occur when all handlers have stopped executing (it has strong guarantees). It will not wait until all queued messages have executed, only just when all handlers currently running are done. At least that is the intent.

The interleave arbiter is biased towards exclusive handlers and the teardown group, so any more queued messages will not be processed the moment an exclusive or teardown message is received.

Iterators: no more CPS Arbiter.Activate(dq, Arbiter.FromIteratorHandler(SaveWebSiteToFile));

IEnumerator<ITask> SaveWebSiteToFile() { … }

Arbiter.Activate(dq,Arbiter.ReceiveWithIterator(false,port,DoWorkHandler));

IEnumerator<ITask> DoWorkHandler(DoWork w) { … }

Inheriting from the IEnumerator interface allows one to specify "yield" points in a method at which control flow returns to the caller. When a MoveNext() method is called on such a method control flow resumes from the “yield” point, not from the start of the method.

The coordination primitives in CCR all inherit from the ITask interface and this is used to help identify the iteration variable for use with the IEnumerator interface.

Iterator: examplepublic void Start(){ Arbiter.Activate(taskQueue, Arbiter.ReceiveWithIterator(false, pMain, DoWorkHandler));}

IEnumerator<ITask> DoWorkHandler(DoWork w){ DoWork workA = new DoWork(); Result r = null;

// send message to ServiceA pServiceA.Post(workA);

// yield for result on response port attached to outbound message yield return Arbiter.Choice(workA.pResponse, delegate (Result Res) { r = Res;}, delegate (Failure F) { Trace.WriteLine("Failed");} );

// when either branch above happens, we will continue executing // here without ever having blocked a thread! if (r == null) yield break; // error occurred, break iteration

// send next message, depending on the result from A DoWork workB = new DoWork(r); pServiceB.Post(workB); // ... handle result form B etc. }

More on iterators

Schedule a task that will iterate over some message coordination

dispatcherQueue.Enqueue(new IterativeTask<int>(message, MyIntIterator));IEnumerator<ITask> MyIntIterator(int message){ Console.WriteLine(“got: {0}”, message); yield break;}

You can also associate a Reissue receiver with a port and an iterator Handler

Arbiter.Activate(dispatcherQueue, Arbiter.ReceiveWithIterator(true, port, DoWorkHandler));

Enumerator Handler limitations static IEnumerator<ITask> Handler()

{ yield return Arbiter.FromHandler(delegate() { Console.WriteLine("I got here..."); }); yield break;} BAD!

You can’t yield to any handler. You must yield to a receive, a choice or a join (or multiple item receive).

This is a limitation of the current version of the CCR. To get around this you can of course create a simple port, post and then yield return:

Port<bool> port = new Port<bool>();port.Post(true);yield return Arbiter.Receive(false,port,delegate(bool b){ … });

Don’t starve

Very hard to locate a blocked or misbehaving thread in the pool (very easy in GUI – only one thread)

System.Threading.Timer is a convenient way to shoot yourself in the foot – all instances are using the same thread pool (size: 50)

Few simple guidelines to avoid starvation: Avoid using blocking I/O in an event handler. Split individual, long-running computations into several

shorter ones. Choose useful intervals for timer-driven behavior. Serialize timer events of the same activity.

Timeouts – don’t hang forever

Never forget about Timeouts

Tell the DispatcherQueue to wait and then to post the current date and time into the timeoutPort:

Port<DateTime> timeoutPort = new Port<DateTime>();dq.EnqueueTimer(TimeSpan.FromMinutes(2), timeoutPort);Arbiter.Receive(false, timeoutPort,delegate(DateTime dt){ // … do stuff …});

Predicates = filters

A sample equal thousand words:

Port<int> Port = new Port<int>();

Arbiter.Activate(taskQueue,Arbiter.Receive(true, Port, HandlerThatRunsOnlyWithLargeInts, MyLargeIntegerFilter),Arbiter.Receive(true, Port, HandlerThatRunsOnlyWithSmallInts, MySmallIntegerFilter),Arbiter.Receive(true, Port, MyCatchAllIntHandler));

bool MyLargeIntegerFilter(int i){ if (i> 1000000) return true; else return false;}

Web Crawler – problem definition

Create a smart web crawler that will download images and the pages themselves recursively from the web.

Flush the downloaded content to your hard drive.

Download requests should timeout if hanged. Code should be 100% asynchronous.

Web Crawler – non-goals

Limit the depth of the recursion with a configurable parameter.

Download only pages descendants of the root site.

Save each page (and all its images) into it’s own folder.

Preserve the relationship of the pages, while writing to the disk, as nested folders.

Show the progress on the Windows form without affecting the user’s experience (non-blocking display).

Web Crawler – the strategy

Provide a continuous stream of URLs Interleave? FromIteratorHandler? Better: persistent Receive handler

Each URL request will either succeed, fail or timeout Definitely the Choice arbiter with timeout port

As soon as download completed – schedule the asynchronous write Choice arbiter – success or failure

Web Crawler – code & demo

YES !!!

Unbelievable !!!

IT WORKS!!!

IT’S EASY!!!

What about WinForms?

Use Microsoft.Ccr.Adapters.WinForms

ManualResetEvent MainThreadEvent = new ManualResetEvent(false);

// Bind dispatcher queue to WinformsAdaptorWinFormsServicePort port = WinFormsAdapter.Create(dispatcherQueue);port.Post(new RunForm(delegate { form = new WebCrawlerForm(port); form.HandleDestroyed += delegate(object sender, EventArgs e) { MainThreadEvent.Set(); }; return form; }));

// Wait for a “shutdown” signalMainThreadEvent.WaitOne();

// We need to inform the WinFormsAdaptor that it is about to quit.port.Post(new Shutdown());

// Let the system to finalize everythingSystem.Threading.Thread.Sleep(500);

Causality ~ <try/catch>

Causalities are a generalization of try/catch, across threads and processors, of the nested exception handling mechanism. Much more powerful however since it can deal with Joins (two different causalities coming together in a common handler).

They are NOT transactions When an exception is thrown, within any logically

related descendant of a Post operation, that exception can be cleanly caught and dealt within one place.

The CoordinationPort allows anyone within a causality to post a message to the "owner" of the causality, allowing you to implement all kinds of internal communication protocols.

Causality: example Port<Exception> pException = new Port<Exception>();

Arbiter.Activate(dispatcherQueue, Arbiter.Receive(false, pException,/* ExceptionHandler */ delegate(Exception ex){ System.Diagnostics.Trace.WriteLine(ex.ToString());}));

Arbiter.Activate(dispatcherQueue, Arbiter.FromHandler(delegate{// Define Causality scope (owner) Dispatcher.AddCausality(new Causality("test", pEx));// … do some stuff … that spawn AsynchronousHandler …}

void AsynchronousHandler() {

// … do some more stuff that throws exception throw new Exception(“bad things happen …”);}

Advanced CCR: the next steps

The observer pattern (what is called pub/sub or publication/subscription) can be implemented using the Subscription service. The service (implemented using the CCR) keeps a list of ports, and then posts the message N items (one for each subscriber). Service’s can be observed, used across nodes/machines. This keeps the CCR simple.

See Service Tutorials 4,5,6 for more info.

If a service is overkill in your scenario, you can write a simple class that does this: keeps a list of ports and replicates messages on them. Or you can write a simple arbiter (derive from Receiver) that does this.

Service orchestration …

Meet BOB (version 1)

Robot (self programmable), controlled by the CCR (running on Mobile device - PDA)

Communicates with a main computer to upload gathered information and to get new instructions

MS Robotics Studio (MSRS)

Writing an application using Microsoft Robotics Studio is a simple matter of orchestrating input and output between a set of services. Services represent the interface to software or hardware and allow you to communicate between processes that perform specific functions.

Simulation Runtime

Simulation runtime can be used in a variety of advanced scenarios with high demands for fidelity, visualization and scaling. At the same time a novice user can use simulation with little to no coding experience and develop interesting applications.

VPL – Visual Programming Language

Application development environment designed on a graphical dataflow-based programming model

Conclusion

The CCR is a CLR library that provides a consistent and scalable way to program asynchronous operations and coordinate among multiple responses.

Framework Class Library (FCL) classes, that already support the CLR's asynchronous programming model (such as Stream's BeginRead and BeginWrite methods) can be easily wrapped, allowing existing types to integrate with the CCR so that complex failure handling and coordination patterns can be coded in a robust, reliable and concise way.

The use of C# iterators for scheduling operations allows sequential programming without blocking OS threads, thus enabling scaling without sacrificing the simplicity of sequential code.

Acronyms

CCR – Concurrency and Coordination Runtime

APM – Asynchronous Programming Model CPS – Continuous Passing Style CLR – Common Language Runtime FCL – Framework Class Library MSRS – Microsoft Robotics Studio

Links

CCR Wikihttp://channel9.msdn.com/wiki/default.aspx/Channel9.ConcurrencyRuntime

CCR Patternshttp://channel9.msdn.com/wiki/default.aspx/Channel9.CcrPatterns

MSRS Homehttp://msdn.microsoft.com/robotics/

My e-mail: igormoochnick@yahoo.com My new Blog: http://igorshare.blogspot.com/ My new Web Site:

http://igor.moochnick.googlepages.com/