It’s amazing to me how seemingly complex programming techniques are really not that complex at all if you break them down into their basic Unix CLI equivalence.
Take, for example, Polyphonic C# where they introduce the concept of the asynchronous join pattern,
Continuations, futures, and whatnot - Thoughts on some asynchronous patterns - B# .NET Blog
Join patterns
If you’ve had a rendez-vous with Ada in a previous life, you’ll be familiar with the concept of join patterns (or shorthand joins). Our Microsoft Research department has done quite some work on this field already, as you can read here: http://research.microsoft.com/~crusso/joins/. It’s part of Polyphonic C#, which is by itself part of Cw and based on join calculus. We already got the LINQ inspiration from it, joins might be lurking around the corner too. There actually just two basic concepts to understand here:
* Asynchronous methods: imagine a keyword async as a modifier on methods (which would imply the method is a procedure, hence the void return type becoming redundant for those type of methods). No longer do you need to create separate versions of a method with or without the Async suffix to work around return type overloading limitations imposed by the runtime. You can think of the method as a wrapper around a task scheduling call, wrapping the entire body. Whether or not this spawns a new thread is another matter.
* Join patterns: also known as chords, is a declarative way of a WaitAll style of guard mechanism to enter a method.Our monk would look like:
class Monk { public async RequestCard(string to) { ... } public string GetCard() & public async RequestCard(string to) { ... } }Basically requests for cards queue up on the monk’s GetCard operation, ready to get processed when the monk is awake. Actually this sample is convoluted and our monk suffers from some concurrency diseases (think of the cards it would hand out if there are multiple outstanding requests - any ordering guarantees without “ticketing”?) but I won’t get in details for now - it’s just (yet) another way of thinking about asynchronous processing with today’s materials in the room. If you’re inspired by join patterns, check out the Polyphonic C# paper.
Sounds interesting, and extremely cool. And I can’t wait to start playing with this feature if and when it becomes a part of the core C# language specification. But I’m not sure it’s something we haven’t been able to do for *years* already In the mean time, for those that want to begin sinking their teeth into async/concurrent programming techniques, it seems to me this same general idea can be simulated using nothing more than nohup and Unix pipes. For example, take the following function which just does some random piping operation enclosed in a loop, taking an argument passed to it as an identifier as to which operation is currently taking place, prepending a formatted date in front of it, and then piping the result to sed to perform a search and replace operation on the current day of the week,
#!/bin/sh
for number in `seq 1 10`
do
echo `date "+%a %b %d %Y %T:%N"`_$1_$number | sed "s/`date '+%a'`/Fri/" -
done
Save the above into a file called “do_loop”. Now make that file executable,
chmod +x do_loop
Now create another file called “run_async_join_test” and fill it with,
#!/bin/sh
nohup ./do_loop 1 &
nohup ./do_loop 2 &
nohup ./do_loop 3 &
Now make that same file executable,
chmod +x run_async_join_test
… and then run that script,
./run_async_join_test
So how do we know what happened, and whether or not each call to the do_loop script ran asynchronously?
less nohup.out
… which will produce something similar to,
Fri Jun 03 2008 12:06:34:093829000_3_1
Fri Jun 03 2008 12:06:34:101356000_2_1
Fri Jun 03 2008 12:06:34:108742000_3_2
Fri Jun 03 2008 12:06:34:112851000_2_2
Fri Jun 03 2008 12:06:34:119344000_3_3
Fri Jun 03 2008 12:06:34:099333000_1_1
Fri Jun 03 2008 12:06:34:123482000_2_3
Fri Jun 03 2008 12:06:34:136901000_1_2
Fri Jun 03 2008 12:06:34:140938000_2_4
Fri Jun 03 2008 12:06:34:146709000_1_3
Fri Jun 03 2008 12:06:34:152805000_2_5
Fri Jun 03 2008 12:06:34:132047000_3_4
Fri Jun 03 2008 12:06:34:166683000_2_6
Fri Jun 03 2008 12:06:34:170302000_3_5
Fri Jun 03 2008 12:06:34:174037000_2_7
Fri Jun 03 2008 12:06:34:180902000_3_6
Fri Jun 03 2008 12:06:34:185055000_2_8
Fri Jun 03 2008 12:06:34:165604000_1_4
Fri Jun 03 2008 12:06:34:191628000_3_7
Fri Jun 03 2008 12:06:34:195973000_2_9
Fri Jun 03 2008 12:06:34:199629000_1_5
Fri Jun 03 2008 12:06:34:211291000_2_10
Fri Jun 03 2008 12:06:34:210545000_3_8
Fri Jun 03 2008 12:06:34:220584000_1_6
Fri Jun 03 2008 12:06:34:222941000_3_9
Fri Jun 03 2008 12:06:34:231110000_1_7
Fri Jun 03 2008 12:06:34:235546000_3_10
Fri Jun 03 2008 12:06:34:241966000_1_8
Fri Jun 03 2008 12:06:34:251812000_1_9
Fri Jun 03 2008 12:06:34:261000000_1_10
*SWEET*! The Polymorphic C# Join Pattern in a (Unix) Nutshell! :D (I smell a best seller! ;-)
Update: Before anyone else is left with the impression that I am downplaying the value of Polyphonic C# I should point out that the above is nothing more than a conceptual exercise intended to convey the general ideas behind the call/wait/continue asynchronous/concurrency programming models. In other words, this is intended as an oversimplified way to think about a much more complex topic. In other, other words, it’s a lot easier to learn complex coding techniques when we can break these techniques into byte-sized pieces using concepts we already understand (and more than likely use on a fairly regular basis) as our building blocks to extend from.
Oh, and BTW… If you want a *really* killer tutorial and C# async library to begin playing with immediately, take a look at http://tomasp.net/blog/csharp-async.aspx, a resource I’ve learned a tremendous amount of valuable information from. In fact, in case you find it useful, I extended TomasP’s library slightly, adding the ability to specify the return type of a given async method call by passing in the desired return type as a parameter.
Extended library: http://nuxleus.googlecode.com/svn/trunk/nuxleus/Source/Nuxleus.Extension.Linq/Async.cs
(the extended portions: The ReturnType enum. The additional ReadToEndAsync extension method which takes a ReturnType enum value as a parameter.)
Example usage: http://nuxleus.googlecode.com/svn/trunk/nuxleus/Source/CodeSamples/HttpGetRequestAsyncWorkflow_Test/Program.cs
What are you implying about MS research?
@dk,
>> What are you implying about MS research?
Nothing. MS Research is one of finest research establishments on the planet. I wasn't attempting to imply that Polyphonic C# joins weren't anything less than *fantastic*. Just that conceptually speaking this same general area of computer science can be thought of in terms that are easy to understand and put into use via tools we all use on a fairly regular basis. As far as I know there isn't an existing compiler that supports this (though come to think of it, maybe I better verify that for sure. If yes, I know what I'll be getting my hands dirty with this afternoon :D) so to highlight the potential future direction of one of my most favorite languages, this seemed like an appropriate real-world analogy to use.
Now to check and see if I missed the compiler download on the MSR site! :D
@dk,
Hmmm... Weird. I hadn't realized that Polyphonic C# was absorbed by the COmega project. In fact, I was under the impression the COmega was no longer an active MSResearch project. But from the looks of it -- http://research.microsoft.com/comega/ -- I guess that isn't the case.
Nice!
/me is digging deeper.
Sorry for the cheap shot -- please feel free to throw a chair across the room.
@dk,
Done and done. ;-)
No, I realized after your comment that the portion of the original post that now has a line through it definitely sent the wrong message. No chairs will be tossed (at least on your behalf ;-))
Thanks for bringing out the need to better communicate my overall point!
Hi folks,
Thanks for reading my post and absorbing it. Indeed, lots of those seemingly rocket science research concepts are not that difficult indeed. Think of lambda calculus, which is no more scary than delegates (and agreed, expression trees by extension) that are well-known for ages; or think of monads, maybe the name of it sounds scary but LINQ just is one of these and apparently people can deal with those just fine.
Back to the comparison between join patterns and equivalent concepts in shells. I agree we can learn a lot of things from the shell world, and indeed we do. PowerShell 2.0 has jobs, and there's this great cross fertilization between IT tool concepts and framework concepts. But we should even go that far in the (very near) future, think of the Extended Type System in PowerShell 1.0 versus concepts in the DLR on extending types (and by "extension", C# 3.0's and VB 9.0's extension methods).
The reason I blogged about this topic is the Web 2.0 era where concepts like these become increasingly important in order to deal with distributed systems programming, not surprisingly something which IT admins have dealt with (at least on the scripting level) for ages hence the parallels between these concepts.
For research projects like Comega and Polyphonic C# it's always hard to judge their "dead or alive" status. Things get absorbed by other projects, spin-offs are created (like the Singularity project has its take on language specific details as well, in this case Spec#) and ultimately things get productized in product or incubation project groups (think of LINQ, the Parallel Extensions, work done in Volta, etc).
Thanks,
-Bart
Hey Bart,
Thanks for taking notice of this post! I'm a *BIG* fan of your blog, finding tons and tons of insightful and meaningful info on a regular basis. For example, the post referenced in this entry. :)
>> Thanks for reading my post and absorbing it.
Thanks for writing it!
>> Indeed, lots of those seemingly rocket science research concepts are not that difficult indeed. Think of lambda calculus, which is no more scary than delegates (and agreed, expression trees by extension) that are well-known for ages;
Agreed. Given that we hacker types tend to be founded upon layers and layers of original creative thinking, yet this same creative thinking is founded upon the slightly more structured field of mathematics, the overlap of our creations is inevitable.
>> or think of monads, maybe the name of it sounds scary but LINQ just is one of these and apparently people can deal with those just fine.
It's funny to me how that works: It seems that familiarity of syntax, coding concepts, and usage patterns can easily represent the difference between high productivity and rapid hair loss for new concepts that -- at their very core -- are similar in function to an idea or process we are already familiar with.
>> I agree we can learn a lot of things from the shell world, and indeed we do.
I think the reason it's easier to grok something when thought of in terms of the shell is this is something that we interact with via individual steps rather that scripted tasks and functions which -- by their very nature -- tend to hide the more complicated functions and tasks behind generalized program names. Very declarative -- please take these values and give me something in return -- which ensures we can focus on the overlying macro process rather than the underlying micro processes. Just such an abstraction -- to me, anyway -- provides the perfect foundation for learning about higher level concepts rather than focusing on the lower level implementation details.
>> For research projects like Comega and Polyphonic C# it's always hard to judge their "dead or alive" status.
Oh, absolutely!
>> (think of LINQ, the Parallel Extensions, work done in Volta, etc).
All great and wonderful projects! I'm especially intrigued by Volta. *SO* much potential!
Thanks again for taking the time to follow-up with a response!