Kirk Haselden

What's new with Kirk?

Well, it's been a while since I last blogged... A lot has changed in terms of my role at Microsoft and the technologies on which I work. Integration Services is still a passion for me, and occasionally I try to help out on the forum and the various emails that I get from folks. I'm even starting to think about the second edition of my SSIS book. More on that later... However, since my last blog I've changed groups and am now working in the Microsoft Business Division in Office Business Platform under Bill Baker in the Office Business Applications group. My focus has been on more broad reaching technologies, not just BI per se, but things like EAI, EII, SCM, BPM, CRM and MDM. Roger Wolter and I recently published a whitepaper on MDM and it can be found here: http://msdn2.microsoft.com/en-us/library/bb190163.aspx I'm also moving this blog. I've always struggled with SQL Junkies. There are a number of technical challenges here and there seems to be no momentum or desire to address the problems. The new blog will be focused on SQL Server topics for sure, but will also cover more general topics such as Office BI, performance management, master data management, data quality, and of course, integration, especially Integration Services. My new blog will be on MSDN here: http://blogs.msdn.com/knight_reign/default.aspx Hope to see you there...

What's happened to this blog...

Well, it's been a while since I've blogged. And for good reason. The last six months, and in particular, the last three have been extremely busy and difficult. For the most part, if I was doing any writing, it was on my forthcoming book. Yes, it's done. Now it's simply a matter of letting the publisher machinery churn. I suppose, like anyone who has ever written a book, especially a technical reference, I've had some real disappointments, some trying times and some highs. I won't bore you with them all, but just to say that writing a technical reference is extremely difficult. The writing isn't so hard, in fact, it was pretty easy. I just had to poor out my brain into the chapters. The difficult part is the time and the things you have to give up to make it happen. In the time I took to write the book I missed essentially 2.5 child years out of my children's lives. That's the tough part.

On the bright side, I truly believe that I've been able to provide some real value to readers. Time and time again I've seen questions about the product that are answered by the book. There are roughly 40 sample packages, 7 custom tasks in various degrees of completeness including the expression task that let's you modify the value of a variable with an expression and an encryption task with a stock task UI that let's you encrypt/decrypt text files, 4 custom components including a JPEG metadata extractor source adapter, data profile transform and two ADO.NET destination adapters. There are two configuration utilities. One for editing configurations in a package directly and one for bulk editing a configuration in multiple packages simultaneously. All have the source code available. There are some especially helpful chapters on configurations, tuning the dataflow task, SSIS security and writing custom components. Although the original concept and table of contents for the book was much more expansive, I believe that the final results address the lionshare of topics that people really want to know. I hope you enjoy the book and that it helps you get the most out of Integration Services. You can pre-order the book here:

http://www.amazon.com/gp/product/0672327813/ref=sr_11_1/002-5741340-2320813?%5Fencoding=UTF8

Another factor in my recent recording reticence is a role change. I've taken a position in the Office Business Applications group. This is a great opportunity to help continue the emphasis on Business Intelligence at Microsoft. While my primary focus has been Integration Services for the last 5 years, this new role will allow me to spread my wings a bit and impact a wider range of products. My current project is not announced and probably won't be for some time, however I will be focusing more broadly on the full Microsoft BI Suite of tools and future entries you find here will likely reflect that. I still have a number of SSIS blogs that I want to get around to, but I'll also be tackling other subjects such as BSM, BPM, UDM, AS, MDM, EIM and other such TLAs.

Well, that's enough blathering from me for one night.

Thanks,

Universe.Earth.Software.Microsoft.Office.OBA.KirkHaselden

 

 

August vacation

I've been neglecting this blog for a while. Mostly it's because we've been so busy putting the final touches on Yukon, getting it ready to go out the door. We're also starting futures planning and discussions for the next version. Nothing set in stone yet, just discussing ideas. If you have some ideas you'd like to see us incorporate into the product in the next version or just some things you'd like us to improve upon, let us know. Some of you have already started giving helpful suggestions. Thank you very much. We're listening.

I've also been busy writing and this week, spending some time with my son Tyson and his friend Eric, on a road trip through the great northwest. A couple of days back we spent the day in Yosemite, picking up the highlights. I would have liked to spend more time, maybe done some hiking and camping there. But, it's not that kind of trip. As it was, we camped one night there and I practically froze. "It's the middle of August" I thought as I tossed in my summer sleeping bag in the car. Funny, but I didn't think about the fact that Yosemite is ~7000 feet above sea level. As it turns out, the temperature approaches freezing at night, and so did mine. Oh well, all part of the "fun".

Here's a picture from the trip. Not the best, but certainly representative of the kinds of views we're taking in. I would have liked to have waited around for magic hour and more favorable light, but, it's not that kind of trip either.

Today, it's Redwood National Park. Then up the Oregon Coast for as long as we can stand the twisties.

See the final slide deck. :)

Thanks,

Universe.Earth.US.Northwest.Redwood.N.P.Kirk.Tyson.Eric

Trip report...

Back from Houston. The long way. I didn't take a lot of pictures for various reasons. Mostly cuz I was lazy or put another way, I knew it was going to be a very long ride and every minute I spent doing something other than riding made it even longer.

I picked up the bike, a brand new (sorta) Yamaha FJR 1300 in Houston on Friday. The very first time I took it out for a spin, it rained cats and dogs. Saturday morning, 6AM, I drove out of Houston and had a great ride into Wichita.

Then, Sunday, I rode from Wichita to Burlington, Colorada. That is possibly the most windy section of road I've ever experienced. No big surprise. I lived in Wichita for years, I've learned to admit that now. :) So, I should have expected it, but there's nothing like tooling down the freeway at ~80 MPH (130 KPH) leaning 20 degrees to compensate for the 30 MPH crosswinds. Notice which section of the trip was the shortest? It seemed the longest.

Monday, I got rained on in Denver. Lightening like I haven't heard for years. It just doesn't do that sort of thing in the Northwest. It only sprinkles, rarely do we get a serious thunder storm. And people talk about how much it rains in Seattle. Sheeesh. Ate lunch at a local Black Eye'd Pea. The waitress was very helpful and friendly, but reminded me of Ichabod Crane. The rain passed by the time I finished eating and I was back on the road again. Rode until late Monday night into Evanston, Wyoming. Wyoming had lots of curvies and of most relief, no winds. Odd for Wyoming, but I wasn't complaining.

Tuesday, I hauled my sorry, sore butt out of bed for what I knew would be a very long haul into Seattle. About 900 miles in one day, I made it home at 11pm. 2700 miles in 4 days. Not too shabby.

After just one day of riding, my bike looked pretty hammered. Warning, don't look at this while eating.

A passerby asked
“What'dya hit a seagull or somethin?“ 
“No, just Wyoming mosquitos.”
“How could you see through that?“
“I don't know, it was late and dark, so I couldn't see anything anyways.“

Bon Apetit

Universe.Earth.Travel.Motorcycles.Yamaha.FJR..KirkHaselden

Flying to Houston tomorrow...

I'm flying to Houston tomorrow. Look for pictures of the return trip. :)

Universe.Earth.Travel.Motorcycle.YamahaFJR..KirkHaselden

I guess I asked for it...

How ironic that the post announcing that I'm turning on comments gets spammed 100 times...

http://sqljunkies.com/WebLog/knight_reign/archive/2005/04/18/12055.aspx

This has got to get better...

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

Under a cloud for weeks...

For those who care, you might have noticed I've been pretty quiet for the last few weeks.

I had been pretty healthy for quite sometime, then boom! I got the stomach flu, regular flu, then a cold and then bronchitis all within the last four weeks.

All that hacking and weezing, being under a dark hazy cloud all day.

Man, I'll take two or three days with a bad flu over that anytime...

Thanks for caring. :)

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

 

Comments back on...

I've noticed that since comments are off, I've been receiving more mail directly. Don't mind the mail. But, likely some of the questions folks ask others have as well. So, I've bitten the bullet and turned comments back on. Oh look! There's some SPAM now inviting me to a nice online gambling casino...

Off to hit the delete button. . . :)

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

Turning comments off...

As this blog grows the surface area for spammers just gets bigger and bigger. I spend substantial time everyweek just culling out the spam from the legitimate feedback, comments and other valid discussions. I really look forward to the pending upgrade I've heard about that will eliminate this problem.

Thanks,

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

This post is not about Paris Hilton, Britney Spears, Beyonce, or Christina Aguilera

Donald Farmer and I were talking about blogs and hits etc. and noticed that posts with certain phrases in their title get a lot of hits. He suggested a little experiment that I post something with Paris or Britney in the title. Well, just for fun, let's see how many hits this one gets. :)

And, please, no complaints from across the pond because I left out the Spice Girls. :)

K

Back online

Last weekend really sucked for me.

I had my Saturday all planned out. A few blogs I wanted to do. Some writing. Some fun stuff.

Boom! Harddrive went out on my home machine. Spent the whole weekend rebuilding my machine. The only bright spot was, I use foldershare, and all but the most recent data and some that I hadn't shared was backed up.

Monday, I came in and found that my work machine was equally toasted.

It was just my time I suppose.

Today, I'm just about back to base.

Arg

 

New name shame...

Sean McCown doesn't like our new name. He said so in his blog.

<STRICTLYPERSONALOBSERVATIONS>

I wasn't involved in the process to arrive at the new name, so I can't give an informative answer to the claims Sean makes. But I do know that we spent an enormous amount of time, effort, frustration and money arriving at and changing the name. Maybe this begs the question “What's in a name?“. I happen to believe, for various and sundry reasons that names are extremely important. After all, that's what the whole post was about. Having the right name, goes a long ways toward describing what a product does. In code, you can have a name like n or a name like nFileCount. The latter is more acceptable and desirable because it accurately self documents the variable. Likewise, Integration Services is about more than just transforming data. In our industry, “data integration” is increasingly important. The name Integration Services implies that there's more to the product than simple data transformation. It implies pulling data from various heterogeneous sources and this is important, source types. We don't live in a strictly database world anymore. Data is stored in so many different formats that it's important to be able to “integrate“ those different sources as well as transform them.

Also, those who were involved say that we _DID_ in fact ask our customers. We did solicit feedback. And the feedback we got from the people we asked was. “Change the name!“. People wanted to move away from the DTS name because they felt that the product had moved beyond DTS.

Viscerally, and perhaps from an insider's point of view, it's nice to be one of the guys. In our group there's SSRS, SQL Server Reporting Services and SSAS, SQL Server Analysis Services. Now comes SSIS, SQL Server Integration Services. (And of course SSMS, SQL Server Management Studio and SSDS, SQL Server Database Server) That should say something to our customers and to us internally that we're moving towards being a better integrated set of tools.

Personally, I like the new name. I took me a while to get used to it. But, I think it better represents what our product is all about.

</STRICTLYPERSONALOBSERVATIONS>

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

Latest reads...

So while staying in the rustic cabin in the mountains, I picked up the intriguing and informative little booklet tuned to my resting grey matter called:

Around the World in 100 Years, by Jean Fritz

I don't think it's in print anymore but it showed how early western explorers slowly opened up the world through progressively brave and sometimes cataclismic exploration into unknown territory. Basically, each explorer opened the way and simultaneously laid down the challenge for further exploration. It did a great job showing how long standing beliefs and lack of knowledge of unknown territories caused explorers to make assumptions that were sometimes way off base. In fact, some explorers desperately stuck to previous assumptions in spite of all indications to the contrary. For example, Christopher Columbus believed he had found the short cut to China even when all indications were clear that he'd found the Americas. It's a book tuned to 4th and 5th graders, but I still enjoyed it.

There's a remarkable similarity between the progression of early explorers and the progression of a new complex piece of software. I'll tell you about it sometime when I feel like boring you to tears...

Joel on Software, by Joel Spolsky

A former Microsoft employee, some of what Joel writes is quite familiar, especially the chapters discussing best practices etc. What's interesting is that Joel seems to have a grasp of what works and doesn't work at Microsoft and other software companies and has some great insights into managing software teams. As we're in the middle of a review cycle here at Microsoft, I especially appreciated his thoughts on that aspect of Microsoft. :)

http://www.amazon.com/exec/obidos/tg/detail/-/1590593898/ref%3Dnosim/joelonsoftware/002-4709218-9192042

We feel your pain...

Friday, I worked for literally hours with a patient and at times humorous customer in the TAP program, Paul Pisarek. It was a bit hilarious on my side actually, as I had multiple meetings and continued to IM and work on the package while sitting in a meeting to discuss SSIS futures for SQL, a meeting to discuss a resolution to the duplicated package IDs when copy/pasting in the designer and multiple ad hoc meetings as people walked into my office. The whole time I was IMing with Paul, moving from office to conference room to conference room with my laptop, bag, mouse, notepad and kitchen sink in my hands.

To make a very long story short, I had written a package for Paul that worked great on my system, but failed miserably for Paul. It turns out that he was seeing a bug that existed in the IDW11 build and I was working on a recent build in which that bug had been fixed. The lesson learned is, do what you can to update to IDW13 when it comes out in a few weeks. We've already fixed many, many bugs that were stumbling blocks. And it will also include a later drop of the Whidbey tools which have fixed issues with the environment.

In the end, we were able to come up with an even better design that worked around the bug. But, I experienced the pain some of you have felt as beta customers and it made me appreciate all the more how incredibly dedicated our beta customers are. We especially appreciate our MVPs and TAP customers that have patiently worked with us to improve the product day in and day out. It's not ready to ship yet, but it's really come a long way in large part because of our beta customers' willingness to push through the issues and report them.

I don't always do this kind of support. In fact, it's quite rare. But I wanted to see the problem all the way through and I wanted to experience a little of what our customers are experiencing. Granted his scenario was a bit unusual and most aren't experiencing this level of pain. But, it was a great experience for me, because I was able to do a real true to life usability test while working through a problem. It was eye opening. Thanks for your patience Paul!

Thanks,

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

 

Well, I did it...

I signed the contract...

Comments : ON

Who gets the first spam?

It's a race Kent...

SPAM is not good...

I've had to turn off comments. The same guy keeps spamming comments, just deleted 10 posted within the last day or two. Vermin!

Universe.Earth.Software.Microsoft.SQLServer.SSIS.Kirk

We're still figuring this out...

Jamie asked a question on the beta newsgroup. He asked "Is this the 'proper' way to do this."

That got be to thinking. It's interesting because, while we at Microsoft have been working on this product for almost five years now, we're still learning about it ourselves. That probably sounds funny coming from a team member who helped write it. But, when you think about it a bit, you see that IS is a very flexible product. This is one aspect of DTS we've tried really hard to retain in Integration Services. In fact, we've tried to design it so that people can accomplish work with IS that we didn't foresee. Also, IS is a fairly complex and feature rich product.

So, it shouldn't be too surprising when scenarios surface that we haven't thought of before. This has happened to me a few times already and will likely only increase in frequency as more and more people adopt IS for their integration and ETL solutions.

So, the answer to Jamie's question is, "maybe". I've found that there's almost always more than one way to do any given task in IS. What's proper for one user may not be appropriate for another. So, another answer might be, "It depends".

Best practices will likely emerge. I already have some that I've adopted as have others. I'll share those as soon as they're well formed and cogent.

I guess the big takeaway is that while Microsoft folk have been working with the product for a little longer, we're still learning about the product too and we don't always have an answer. We really appreciate all the professionals out there that are using IS, giving us feedback and supporting the efforts to make it better. We're starting to see patterns in the way IS is used and appreciate the pain you go through as you learn about IS.

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

A little too much reminiscing...

Euan's post here brings back some of the memories I'd much rather forget...

The toenails oohoho, the horrible toenails.

Thanks a lot Euan! That was one part of DTS/SSIS history I was hoping to put behind us. :)

Universe.Earth.Software.Microsoft.SQLServer.IS.KirkHaselden

Yes...

Well, we did it. We ZBB'ed. The halls filled with energy as we realized we've turned the corner on this thing and from here on out, we're truly polishing the product, getting it ready to deliver...

Thanks for your crossed fingers, feedback, testing efforts and kind words of support.

I promise the next blog will have something of substance. But for now, I'm just savoring the moment.

K