May 2005 - Posts

Wooohoooo!

It's signed. CTP 15 is very close!

1187.07

This was a tough one folks.

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

Technet events now available...

The Business Intelligence Technet sessions are now available for viewing. This is good stuff.

http://www.microsoft.com/events/series/sqlserverbi.mspx

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

SQLIS.com

Allan and Darren are continuing to build an impressive resource in SQLIS.com. If you haven't visited there lately, take a few moments and go visit. Lately they've been archiving articles from various sources, including blogs. For a list of all the articles you can go here.

Thanks,

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

Next CTP...

The next CTP is close and I wanted to give you all a preview of some improvements that will be available there.

I'm happy to write that the new transfer tasks will be available. Through a series of planning changes and priority shifts, these tasks were pushed off until the very last minute, but they will be available in the next CTP. They will be backward compatible and integrated with the migration wizard to support migrating the DTS 2000 transfer task incarnations.

Another exciting development in the next CTP will be property expressions in the dataflow task. I'll blog more about this later, it deserves it's own dedicated blog. But, in a limited manner, some components will support property expressions on certain key properties. This will vastly simplify the solution to some difficult dataflow task configuration issues.

Look for these and other improvements in the next CTP.

Thanks,

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

Custom component UI

We've been seeing a heightened interest among SSIS customers in writing custom components as they move from understanding the basic IS  concepts to more advanced ones. The question has come up more than once, “Why is the UI code separated from the component code?” Not everyone notices this because you have to really dig deep into your system to notice it. To see what I'm talking about, take a look at a machine where SSIS is installed. Look in the %WINDIR%\Assembly folder. Under Microsoft.SQLServer..., there are a number of assemblies with identical names, one that has “UI” appended.

So, why do we do this?

Integration Services makes a pronounced distinction between design time and execution time. You see it in the dataflow/pipeline where there are design time and runtime interfaces and design time is when the components get built up, columns added and paths between them created.. In the runtime, certain operations are possible during design time, like changing the value of a read only variable or adding objects to collections that aren't available during execution time. We do this because of a few basic assumptions:

  • You want your execution time behavior to be as fast as possible.
  • You want to use as little memory as possible for non-essential activities during execution.
  • You want your packages to be determinant, or in other words, execute consistently.
  • You would rather not have non-essential assemblies on server systems.

The UI assemblies house the “UI“ portion of a component that is only visible when you're building the package in the designer. If the code for the UI were in the same assembly, everytime you ran a package with that component, the UI assembly would also load up with the component. Memory consumption would go up and the working set would increase dramatically. We assume that you'd rather use every spare megabyte of memory for operations.

This separation also makes it easier to enable writing UI for components since some of our components are written in native code (for speed), but the UI is all written in managed code.

Aside from the advantages noted above, there is a psychological element to this pattern. Because the design time elements are separated from the execution time elements, it's logically easier to see the separation as you develop the component. It's also nice to know that it's not actually necessary to write custom UI for components. Tasks and other runtime components without UI may be edited via the property grid and pipeline components without UI may use the advanced editor. In fact, even some of the stock components that will ship with IS don't have a custom UI.

Finally, at some point in time, it's possible that Microsoft may provide a small footprint install or other similar setup distribution that will not install UI which will make it easier to distribute and have a smaller footprint, essentially by eliminating design time elements. There is _no_ official plan for this by the way. Right now it's just an idea getting tossed around. Any interest?

So, when writing your own custom components, you should consider separating your UI code into a separate assembly to get the benefits described here.

Hope this helps. Let me know.

Thanks,

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

 

Save to SQL or File

We get this question a lot actually. “Is it better to save to SQL Server or to files”. The seemingly obligatory answer is “It depends”. This is not a new question either. The options to save packages to SQL Server and to files were available in DTS as well as the option to save to Metadata Services, which is no longer available in SQL Server 2005. It's a good question. It means people are trying to understand the tradeoffs. So here are some of them:

Advantages of Saving to Files

  • Easier to do shared source control
  • Ultra secure when using the Encrypt with User Key encryption option
  • Not subject to network downtime problems (saved locally)
  • May escrow deployment bundles including miscellaneous files
  • Less steps to load into the designer
  • Easier direct access for viewing
  • May store packages hierarchically in file system
  • Projects in Visual Studio are disk based and require the package to be in the file system
  • Generally, a better experience during development

Advantages of Saving to SQL Server

  • Easier access by multiple individuals
  • Benefits of database security, DTS roles and Agent interaction
  • Packages get backed up with normal DB backup processes
  • Able to filter packages via queries
  • May store packages hierarchically via new package folders
  • Generally, a better in-production experience

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

New SQL Server Magazine article

Douglass McDowell and Jay Hackney have teamed up to write a great article for SQL Server Magazine on Integration Services. They write a custom data source adapter for reading IIS log files. Very cool. Check it out here: http://www.windowsitpro.com/SQLServer/Article/ArticleID/45801/45801.html

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden