All I needed to know about Whidbey and Yukon I learned from Dune
It is a little after 01:00 and I’ve just spent the better part of my
emotional energy writing tonight’s Take Outs. Despite the late hour for this old
man and the lack of any creative fire in my soul at the moment, there’s just no
way I will be able to sleep until I get this off my mind. I’m on a mission now.
Let’s start with a little inspiration, shall we? I hope you’ve read
Frank Herbert’s Dune series -- or at
least watched the movie and or TV series.
"The worms are the spice the spice is the worms."
One reoccurring thread I’ve read is that since Tom Rizzo announce that both
Whidbey and Yukon are being "slipped" that Yukon must be the cause and that
separating the two is a sure remedy to getting Whidbey out the door faster.
Sorry, that’s wrong for two reasons.
- Larry O’Brien notes
here that we
really don’t know which of the two is to "blame" for the release.
- It would set the effort back more do undo this now that it reduce the
release cycle. This effect was well discussed by Mike Gunderloy
here.
But there’s more to it. Assume, for a moment, that the .NET is the spice
Mélange, and that both Whidbey and Yukon are Worms. If you know your Dune
trivia, you immediately get this next point: Without the spice, there can be no
worms; without the worms, there can be no spice.
A more conventional explanation is appropriate. At the center of all things
.NET is, of course, the .NET runtime (the CLR). The CLR is composed of a number
of things, but a principal element is the .NET Framework Base Class Library (FX,
as it sometimes called). The CLR also manages the thread pool and the memory
allocation used by all .NET applications that it runs. In versions 1.0 and 1.1
of .NET, there really wasn’t a way that some other process could leverage the
CLR while actually letting some other service control its resources.
Why would that matter? Good question. The parallel I would draw is VB, VBA
and VBScript. In VB, the application is responsible for managing its own
resources. In VBA, however, the application hosting the VBA objects has to least
help with work to sure that resources are well managed. In VBScript, it’s the
responsibility of the script engine to execute the code on it is thread and with
its memory manager. In VBA and VBScript, the resource managers provided by VB
had to be tweaked or replaced to work with the different environments hosting
the VBA or VBScript application.
It is obvious to most developers that without the CLR, Visual Studio.NET
would be sort of pointless. What isn’t so obvious is that Yukon would be almost
equally as pointless without it. What do I mean? We all know that, with Yukon,
an instance of the CLR runs inside of the SQL Server process somewhat like VBA
does. SQL Server has a more appropriate threading (or fiber) and memory
allocation management system for the type of work it does compared to what a
normally application might do. That’s groovy right; because it lets you write
assemblies and use them as types, functions and stored procedures. I hate to
break this to you, but that’s merely a beneficially (and highly marketable) side
effect. The real reason that the CLR is in Yukon in the first place may not be
so altruistic.
Consider SQLXML for a moment. This wonderful little bugger gave us the
ability to open up stored procedures on SQL Server 2000 as Web Services. It also
gave us the ability work natively with For XML result sets in Everett. The only
problem with SQLXML is that it was a bolt to SQL Server 2000, and as such, it
shows some seams in terms of security, technology, development and deployment.
This is partially due to the how the product is distributed (as an add-on), how
it gets implemented (partial as an set of ISAPI DLLs that run on IIS host acting
a proxy for the SQL Server) and that it is based on COM.
Now suppose you wanted to make SQLXML more seamless. How could you do that?
Turns out that Microsoft already had a great XML technology stack with the CLR,
so why not host the CLR inside of SQL Server?
Brilliant!
Problem.
The way the then current versions of the CLR worked, it would wind up
fighting with SQL Server for the in-process resources. That’s a train wreck just
waiting to happen. Worse, the then current version of the framework weren’t
modeled or designed to support letting another application do that work.
Solution? Change the Framework so that an application can overload the
"management" parts of the CLR with its own.
And so they did. Thus CLR version 1.2 "Spice" became the Yukon "Worm."
Of course, we’re getting a lot more than just SQLXML functionality out of
this effort. We’re also getting of the benefits of the CLR’s XML functionality
with the XML data type (as well as few other new types.)
Note that it may not have happened exactly that away. It may have been some
other feature of something else that was the catalyst. I can personally speak to
what Scoble
said. Microsoft is not one big collective where all the drones of one mind.
There are hundreds (if not thousands) of different teams working on different
parts of what we see was just one product.
So, that’s great for Yukon, but why does that impact Whidbey? Well,
obviously, there’s the immediate reason: a new CLR is a new CLR and if anybody
is going to program against it, they will need a new version of Visual
Studio.NET. I can hear people, though, asking, "but why does that mean I we
can’t have a version of the CLR that supports all of the good stuff in Whidbey
and another one latter that we use with Yukon?"
The answer that Microsoft will give is that "we want to give you the single
best application development platform" that will "change the world." Again,
that’s very good, but the real reason may not be so altruistic. Think about what
Microsoft could really do it any application could host an instance of the CLR.
Start with how we use Office VBA today. Then keep unfolding. How might Exchange,
BizTalk or SharePoint evolve?
Imagine Great Plains -- err -- Microsoft Business Solutions with those legs.
To accomplish that, though, Microsoft had a tough choice to make. Do we start
building lots of smaller releases of the CLR or do we have fewer bigger ones?
They have to make that decision based on what resources they have in terms of
developers, testers, managers and support staff. They also have to keep in mind
since each version of Visual Studio.NET works with a specific version (and
previous versions) only; they could easily be introducing a new version of
Visual Studio with each new product release. Down that path would be chaos and
madness for any corporation trying to manage a product portfolio the size of
Microsoft. They needed to get a CLR version they could standardize a generation
of products on.
And so they did. The CLR 1.2 "spice" became the Whidbey "Worm."
I’m just speculating on this next part. How do the "worms" become the
"spice?" Embedding the CLR in a product like SQL is huge gamble. It might never
have really worked very well. Or it might cause delays like the ones were
currently all bemoaning. I suspect that what we will find with each effort
though are further evolutions of the CLR driven not by features that we as
developers will use, but that are driven by strengths and abilities of the
features of the products that the CLR is being inserted into. Indigo is great,
but what might an MTA mean for it?
The key points so far are:
- What we are experiencing here is the result of Microsoft’s belief that
.NET can not only be a great run time for applications, but it can also be a
great engine within an application.
- To preserve sanity and resources, each generation of products will, in
my way of thinking, have to have a shared common version of the CLR.
- By embedding the CLR as an engine a product, non-developer features need
in the CLR are manifested. As these features are added, the CLR evolves.
"He who controls the Spice, controls the universe!"
In the early parts Dune series, the harvesting and use of the Spice Mélange
was the basis of virtually everything: interstellar transportation, the economy
and the politics of the empire. However, that Spice was only available on one
planet: Arakis, AKA Dune. The same thing is potentially happening on a much
smaller basis with not only the CLR, but also the Visual Studio IDE. We’ve
already established that there is version dependence on the CLR by the IDE, but
the IDE doesn’t belong to either the Whidbey or Yukon teams. Rather, it appears
that they share it. And that’s another reason Microsoft can’t just split Whidbey
from Yukon. The same economics that require one CLR per product generation
demands that there can be only one IDE.
There’s also that little problem of getting people to upgrade. How frequently
do you want to have potentially re-do each application you’ve worked on? And
retest. And potentially redeploy because some features that you don’t even use
-- but something else does -- has to change? Once a month? I doubt it? Once a
quarter? I doubt your manager would like that. Once a year? I doubt your
Customer or CxO would appreciate that very much at all. And that’s the dirty
bottom line here. Our economics aren’t really all that much different from
Microsoft’s, are they?
"The spice must flow"
While all of this unification stuff in interesting, Microsoft understands the
need to product into our hands. At the same time, because so much depends on the
CLR, they don’t have the luxury of being as agile as we’d like. Especially where
they’ve taken a beating in the past: Security, Scalability, Reliability and
Interoperability. They know they at least have to keep making incremental
improves in all of the regards on every single product or pretty no body is
going to use anything they have.
As an ASP.NET developer, I’m with you when you tell Microsoft "We want
Personalization! We want MasterPages! We want Generics! And we want it now!"
As a Database Developer, I’m with you when you say "We want XML storage! We
want easier user-defined functions! We want better Web Services Integration! And
we want it now!"
Since Microsoft has chosen to use the CLR as the vehicle to delivery all of
this, I have ask you to see the bigger picture and really understand why
Microsoft has to do it right rather than soon. Too much is resting on consistent
quality of these products not for them and for us. None of us can really afford
to be upgrading very six months. Impatience here in one quarter makes it a
lose-lose for all of us. Patience makes a win-win for all of us.
"We Fremen have a saying: ‘God created Arakis to train the faithful.’"
Hopefully, I’ve helped you understand why I feel that asking Microsoft to
hurry up Whidbey troubles me so much. We’ve all got a lot lose by not seeing
this strategy through. A few cheap gains today aren’t worth risk the potential
in what we have here, I believe. Of course, you are free to feel differently
that me. However, please do me one favor. If you see any value in this big
picture I’ve tried to paint, redirect that energy you have geared up to demand a
quick release of Whidbey into learning more about Yukon. ASP.NET developers, I
challenge you to really dig into Yukon and learn more about Reporting Services,
XQuery and using assemblies in Yukon. See what caching will do for you. Windows
Form developers, I’d love to see you dive head first into Notification Services
and Multiple Active Result sets. Office Developers, spend quality time with
Yukon’s Analytics and its ability to store and generate XML for you. Granted,
none of this will solve the day to day problems you have, it will certainly help
you -- I believe -- see the real value in what’s going on here. If we divide
ourselves now, we’re guaranteed a lose-lose. If we can be patient and
take advantage of the learning opportunities we have now, we all help to
guarantee a win-win.
I’m ready to help anybody that’s willing to do that as best I can.