(This is a mirror from my post on http://blogs.msdn.com/kamalh )
It is hard to believe that SQL Server 2005 (aka Yukon) has finally been released to manufacturing. That happened last week, but it is still sinking in. Real proof comes in the form of http://www.sqlserverdatamining.com (run by Jamie and the rest of the DM crew) now running on the RTM build of SQL Server 2005 ! Check out the "The Art of Clustering" (http://www.sqlserverdatamining.com/ArtOfClustering/).
I have been roped into writing the foreword for a couple of books and as I step back and reflect on the development cycle for Yukon, the magnitude of this product is finally hitting me. In the two areas that I have been most involved with - Integration Services and Analysis Services, the changes have been nothing short of revolutionary. Integration Services (SSIS) is a new product (evolved from DTS, but not DTS) and Analysis Services is pretty much a re-write and it too is in many ways a new product.
Next week I am off to Europe to participate in the SQL Server 2005 launch activities and I guess when I see the product "unveiled" on stage it will hit me some more ...
A few weeks ago I met up with an old class mate of mine from Engineering College. Vijay Sikka is a great guy, smart, hard working and sincere and so it was so nice to meet after almost 16 years. In the meantime he had worked at Intel for a long time (in the research labs) and then moved on to do his own startups. The current one is Sikkasoft (http://www.sikkasoft.com/) and develops software for managing medical practices. Now, Vijay has a ton of experience in developing software and also is an intellectual. Add this all up, and what do you get ? A book ofcourse ! Check out "Managing ROI on Software Development" (Amazon sells it). It is about the only book that I know, which attempts to tackle this important yet difficult to quantify topic. I think that Vijay has done a great job in writing this book and if you are a software engineer or project manager or an IT business manager struggling with the complexities of managing software projects and trying to quantify what ROI you are getting, then this book is a must read.
Now if I only can get him to write a book on SQL BI !
I have decided to move my blog from sqljunkies.com to msdn. The new URL is http://blogs.msdn.com/kamalh . For a while though I may cross-post on both sites.
I haven't had time in the past 3 months or so to post anything. In the meantime I have changed teams within SQL Server and am now part of the Analysis Services team. SSIS though will always remain a part of me, a product and a team that I am immensely proud of. Certainly one of the highlights of my career.
Analysis Services is also a big part of me, I worked on that team (the original Plato/OLAP Services) from 1997 to 2001 before I moved on to manage the DTS/SSIS team and so in a sense I have returned home. The AS team is bigger than the SSIS team and so this transition places more responsibilities on me and includes new and bigger challenges. I am very excited about this change, it reunites with some old friends but also lets me help drive Business Intelligence (BI) further into the mainstream. We (Microsoft) already have made huge strides in bringing what traditionally were exclusive and expensive BI technologies like OLAP and Data Mining to pretty much anyone who wants to work with these technologies. Now we have the challenge and opportunity to make these technologies even more approachable and allow them to work against the largest scale data sets and the most complex problems.
Besides, making our products more approachable and address more and varied business scenarios, I am also excited about making the product even faster and in general more reliable (even more so than what it already is today).
I don't have any insights (not even my usual little and obvious ones) to post about AS yet, but hopefully that will be changing soon.
I am writing this from Orlando airport waiting for my flight (and hoping that the effects of the approaching storm doesn't delay the flight).
Today we finished the last of our 3 sessions. We (Ashvini and I) delivered sessions on:
- Loading a Data Wareshouse using SSIS
- Deploying and Managing SSIS
- SSIS Scalability and Perfromance
All three sessions went off quite well with a lot of nice feedback (both verbally and written in the evaluations) from attendees. It was really nice to be able to show and talk about the product to such a large number of people and listen to their questions and feedback. As I look back at the past 5 days, it is pretty clear that there is a great deal of curiosity and excitment around SSIS. Every day many people came up to me and said that they were excited to see and in many cases use the product. As the conference went on, this number kept increasing. Our informal "cabana" talks were pretty much packed. I had not anticiated this sort of attendance in these talks and had only planned for a simle Q&A with a few customers. I loved the format though. A few chairs, a hand-held mike and a flip chart was all we had, instead of the huge rooms and bright lights of the formal sessions. We did have a large flat screen monitor to show demos (and show demos we did), but the setting was much more intimate. My favorite moment was when some one asked "How does SSIS compare to products like Informatica, Ascential etc. ?". I just could resist and launched into (in retrospect) a bit too forceful description of the advantages of SSIS, beyond the obvious cost effectiveness. I am not sure I could have done the same during the larger sessions.
In addition to our sessions, there was the "BI Power Hour" where Bill Baker was the master of ceremonies for an incredible array of demos from all parts of the SQL BI paltform (AS, IS, RS and NS). Donald did the SSIS demos with his usual wit and preence. This was quite a session. It went at a breakneck pace, with the rest of us (from the BI team) throwing t-shirts, frisbees and beach balls into the audience. The audience loved it and very vocally showed their appreciation.
It was very gratifying to talk to the attendees and answer their questions and hear about their experiences, scenarios, concerns, feedback etc. The energy from such a large number of people is palpable and the buzz around SSIS was unmistakable. To me this buzz made this TechEd one of the best conferences I have been too. Extremely tiring and demanding, but still worth it.
Today Paul Flessner, Sr. VP for SQL Server (and other server products) gave the keynote talk at TechEd. It was a good talk aimed squarely at the ITPro audience. There were a lot of nice demos, including a robot armed with a buzz saw carting around the stage.
Of all the demos the one that really stood out was done by Donald showing the end-to-end BI capabilites of SQL Server 2005 (Integrate-Analyze-Report). It was centered around SSIS and doing data mining in the SSIS pipeline. It was effective and to the point and Donald brough the house down with his off-the-cuff humour.
For me personally it was moving to see the product take center stage before (arguably) the largest IT audience in the world. After 5 long years of working on this one release this was a sweet reward.
Thank you Donald for the great demo, and thank you eveyone who has put in their sweat, blood and tears to build this truly amazing product.
Yesterday was the first complete day of TechEd and if you have never been to one of these shows it's quite an exerience. There is a huge mass of humanity flowing into the convention center and onto common areas like the meal area and pavillion (where all the vendors have booths). With around 11000 attendees it can get overwhelming.
Ashvini and I had our first session. It went OK, not spectacular in terms of delivery but quite good in terms of the content and getting the message across. We also had fun with giving away Donald's new book and some T-Shirts.
There were a lot of good questions and I had a few people come over and say that they really enjoyed the session and are already wotking with SSIS and are working on getting it into production. There were also the long time DTS users who were totally amazed at the evolution from DTS to SSIS. I guess it sort of the feeling I had when I first got a ride in a Porsche and heard that engine whine and felt slammed back into my seat !
Today we have no sessions but will be doing a Cabana talk (which is more informal and interactive) on "Data Quality and Compliance with SSIS". I still need to prep for that one. Complicance is a hot issue and we do have some features to support it, but there is not a specific shape to add or button to press and so the potential solutions require some explaining and that explaining needs some preparation !
I made into Orlando last evening after a longish flight from Seattle, via Atlanta (including a sprint between gates).
Registered, got the free bag, speaker shirts etc. met up with a few others on the team and am now getting ready for my first session today which is on "Building Data Warehouses using SSIS". I need to nail down one off the demos still and so am under a bit of stress, but otherwise am really looking forward to this and my other 2 sessions and the Cabana talks. It's been a long road to Yukon and now the finish line is in sight and this TechEd is in a sense the final validation that indeed we are wrapping up the release. I am totally psyched !
If you want to get a look at what is going on at TechEd from an informal perspective, checkout http://mssql.members.winisp.net/ which will try to chronicle via Video some of the experiences of the SQL team (thanks to Euan Garden for setting this up).
OK, now it is off to the convention center to meet up with Ash (Ashvini Sharma who is co-presenting with me, or better who I am co-presenting with).
More later ...
I am doing a TechNet webcast on “Overview of SQL Server 2005 Integration Services”. This is part of a larger series of SQL Server 2005 Business Intelligence Webcast Series. More details are at: http://www.microsoft.com/events/series/sqlserverbi.mspx .
I am planning on doing an introductory session with a couple of new demos. I enjoy doing web casts and am looking forward to this one as well. This is fast becoming the season for presentations. June 6 kicks off TechEd and I have 3 sessions there (along with Ashvinis), then one more web cast in June on “Beyond ETL”. After that there is PASS and then PDC. I am not sure to what level I will participate in those 2 conferences. I think that it will be much more usefull to have someone like Kirk (with the detailed nuts and bolts knowledge and dev oriented approach) present at those conferences..
Talking of presentations, check out the SSIS related “episodes” at http://www.msdntv.com . These are done by Donald and really nice to watch and learn from. I have recorded an “episode” (somehow makes me feel Star Warish) as well and that should be showing up in the coming weeks.
The Asia MVP Summit finished last week, but my remaining days in Singapore became so hectic that I just didn’t get any chance to add to my blog.
On Day 3 of the summit I had my session on “In-depth SQL Server Integration Services”. From conversations with MVPs and observing Mahesh’s session it was pretty clear to me that the audience would be a mix of individuals with deep SQL (and SSIS) knowledge and others who were there just to find out what this Integration stuff is about. So my talk was more of “steep dive” instead of the promised “deep dive”. I started off with the basics, introducing SSIS and then doing demos that progressively went from simple integration to my favorite “Beyond ETL” ones including one showing text mining on RSS and then the package becoming the data source for a Reporting Services report (a topic for a later posting). This last demo though didn’t fully work because no data was returned by the package when it was executed during the report preview. Not sure what happened here, because the same demo had worked all morning and then has worked since. Most likely I got too caught up in hyping up the demo and did something wrong (pilot error).
All-in-all I think that it was a good session with a full room and good participation. I also got good feedback afterwards. SSIS is such a joy to talk about and show.
I really enjoyed meeting the Asian MVPs (if I met you and you read this, drop me a line) and hearing their feedback. I look forward to more opportunities to interact with them.
After the MVP Summit finished, I still had one more (unbelievably hectic) day left in Singapore.
The next day started with an early morning visit (I did a few of these during my stay in Singapore) to a customer who wants to implement the sort of data integration and cleansing scenario, combined with data mining which is unique to SSIS. Next I did a webcast for Microsoft technical sales people across Asia. I have done similar webcasts from Redmond, but the time zone difference has made it difficult for people in Asia to attend.
In the evening I presented about SSIS to the Singapore SQL Users Group (http://sgsql.sqlservercentral.com/). Sukhdev Singh who is a SQL MVP is the founder of this user group. I had expected modest attendance given that this is a fairly new user group and the low attendance for a similar event for the MSDN User Group the previous evening and the fact that it was at 6:30 in the evening on a Thursday. Boy was I wrong. This was held at the Microsoft Offices (really nice views of downtown Singapore !) in their auditorium which is a fairly large room. As I was setting up I could see a long line forming outside the room for registration. By the time I started the room was pretty much packed with about 150 people attending. I had a great time presenting first about what is new in SQL Server 2005 and then launching into SSIS demos. Towards the end Maheshp joined me (he was doing double duty presenting almost at the same time to a Student User Group) for the final 30 minutes to talk about SQL Management Tools. We got some good questions and in general a lot of interest.
To wrap up the day, around 10 PM we headed off to grab a bite at Lau Pau Sat, which is an outdoor “Hawkers Center” where a huge variety of ethnic fast foods is served. The sights, smells and tastes were totally fascinating. Around 11:00 we made our way back to the Hotel (and had a really hard time finding a cab). I was leaving the next day and so had to pack my bags and generally wrap up. The flight left Singapore at 6:00 AM and so I had to leave the Hotel by 3:30 AM. I took a quick nap from midnight to 3 and then got ready and headed off to the Airport. What a day it had been, almost non-stop action from 8:30 in the morning till midnight and then a huge long trip home just a few hours later.
It is really good to be back home now and focusing on getting CTP 15 out. I am tremendously excited about wrapping up SQL 2005 and getting it released to the user community. We have a way to go still, but we are getting closer and better by the day.
Day 2 was the first full day of talks in the technical tracks. The track of interest to me is the server track. Mahesh Prakriya (Maheshp) from the tools team (the folks who develop SQL Server Management Studio) gave a talk on SQL Server 2005 Manageability. He did a good job of covering not only the features of SSMS, but also going over the manageability related features in the SQL Engine. Mahesh showed up some of the useability enhancements in SSMS and it was good to see that these enhancements were exactly what some of the MVPs had been looking for.
Even more interesting were discussions with some of the MVPs after the talk and some of their observations during the talk. They not only are aware of current bugs/limitations, but are actually in touch with specific developers within the SQL team about these issues. The depth of their knowledge and their general awareness once again blew me away.
Yesterday was the first full day of the Asia MVP Summit, which is the event I am here in Singapore for. The Asia MVP Summit is an event that brings together Microsoft MVPs from across Asia-Pacific (including Australia and New Zealand). The summit was kicked off by Sean O'Driscoll who amongst other things manages the MVP program for Microsoft. Sean gave a great talk (accompanied by some great drumming!) and spoke of the value of a strong and vibrant community. He talked about his own experiences with a newly acquired Barbeque and how he is using the community around this particular manufacturer to get more out of his Barbeque and how it makes him feel connected with other users as well as with the manufacturer. This got me thinking a bit. We (Microsoft) talk a lot about maximizing potential, and I believe that MVPs and other community leaders are key to unlocking a lot of the “hidden” potential in our products. Our products are complex and sophisticated software and sometimes it takes the efforts of community leaders like the MVPs to maximize what you can do with them.
In general, MVPs are super important members of our community, not just for SSIS (more about our MVPs later) but for all of Microsoft Products. MVPs are leaders in the community who not only provide great technical expertise to other members of the community, but also echo back to the product teams the feedback from the community at large. To learn more about MVPs go to: http://mvp.support.microsoft.com/
Talking of MVPs, for SSIS our MVPs are: Darren Green and Allan Mitchell. These are two of the best MVPs (if not the best) Microsoft has and we all are very lucky to have them focused on SSIS. They run the great http://www.sqlis.com site which is a wealth of information related to SSIS and DTS. They have been focused on DTS and SSIS for a long time now and are an invaluable resource to everyone related to SSIS. Thanks Darren and Allan for all that you do for us.
Here in Singapore, I have had the pleasure of meeting MVPs from many different countries focused on many different Microsoft technologies. It is very interesting to hear the sorts of things they are working on and their feedback on our (Microsoft) products. Their depth of experience is truly amazing.
More on this soon…
I am writing this from Singapore, where I am currently to present at a regional MVP event. In addition to presenting a session on SSIS, I hope to meet with customers and the local user group.
Singapore is a fascinating city. It is high tech, clean and has really friendly people. It's a long way from Redmond (pretty much a full days flight), but I do not get as much of a chance to meet with users in this part of the world and want to make sure that we are getting good input and feedback from users here.
I plan to write more during my stay here.
In case you didn't note from Donald's blog and from the front page of the SQL Server page on Microsoft.com, Forrester Research has written a very nice note on SSIS. It is very insightful and correctly points out the rich features in SSIS, including but not limited to Data Quality, professional collaborative development environment and reliability. Our scalabaility and performance, especially in context of the price/value proposition is called “seductive”, a term rarely associated with data integration and/or ETL tools. It is great to hear a respected industry analyst say what we are already hearing in many ways from our customers.
We still have to tighten up the product in some places and to continue to raise the quality bar, but it is getting better and better and will be something we all will be very proud of when we finally ship it in out to the world.
It is always nice to see articles about SSIS and it is even better to see something just before Tax day (April 15 is one of my (if not the) least favorite days). So it was great to see this: http://www.developer.com/db/article.php/3497511 . Thiru Thangarathinam has written a nice introductory article. Check it out.
Talking of writing, I am just finishing an introductory “whitepaper” for our marketing folks and am in the middle of the final editing pass. The amount of little changes (capitalizations, tense changes etc.) made by the editors in my little writeup are astonishing. It is hard enough for me to follow all the suggested changes by the editors, and so I am really impressed with the thoroughness with which they have gone over my writeup. I can't even begin to imagine what it would be to write something really big like a book.
I am convincing Eduardo our Test Lead to start blogging about his experiences testing SSIS. This would be a different perspective from what you all have been hearing from the likes of Kirk, Ash and Donald who are pretty much focused on usage scenarios.
Another interesting twist to this is that Eduardo may be having a bi-lingual Blog ! Both in English and Spanish.
Go Eduardo, go.
Check out our new presence on MSDN: http://msdn.microsoft.com/SQL/sqlwarehouse/SSIS/default.aspx .
This site will provide a lot of information directly from the SSIS dev team and will evolve quite quickly in the near future.
Donald has written a very nice entry in his blog on the acquisition of Ascential by IBM.
I personally am thrilled by this news. To my extremely biased eyes, this looks like IBM getting ready for the release of SQL Server 2005 and adjusting their approach to data integration and including a major non-federated component. I hope that this will further lead to making data integration tools accessible (affordable) to a wider audience. We (up here in Redmond) have been pushing for years to make the benefits of BI available to the masses and it is interesting to see the rest of the market following our lead. In the end the real winner in all of this is the customer, and that is what makes me happy !
The impact here on us is not much, except that it validates ($1.1 billion worth of validation !) the approach we have been taking and the investments we are making in developing SSIS (SQL Server Integration Services).
I (and others on my team, notably Donald Farmer) tend to do a demo showing data from RSS feeds being processed in the SSIS (SQL Server Integration Services) data flow pipeline. I have been asked by quite a few people to post a sample package showing how to incorporate RSS data.
Turns out that it is quite easy to include data from a RSS feed. Try the following:
1. Start a new SSIS project
2. Add a Data Flow Task
3. Switch to the Data Flow tab by double clicking the Data Flow task
4. Add a XML Source to the data flow
5. Double click the XML Source to edit it
6. In the XML Source Editor, in the “Connection Manager” portion, make sure that the “XML File Location” is selected in the Data access mode. In the XML location text box, type in the url for a RSS feed (e.g., http://news.com.com/2547-1_3-0-5.xml)
7. Click the “Generate XSD...” button, give the XSD file a name and save it. Make sure that it is correctly reflected in the XSD location text box.
8. Click “Columns” in the list on the left hand side. If all goes well you should see the available columns from various outputs from the RSS feed.
9. Select “item” from the Output Name dropdown box.
10. Click “OK” to close the XML Source Editor.
Now you are ready to consume the RSS data.
The simplest way to do this is to add a destination component (e.g., OLEDB destination to your SQL Server, for which you need to create a new OLDEDB connection in the connection tray) and connect the XML Source to the destination.
Right click on the connecting arrow and add a Grid Data Viewer. Now run the package and you should be able to see data from the RSS feed as it gets loaded into your destination.
Hope this helps.
An interesting scenario to try with data from RSS feeds is to use the Text Mining component to filter out news items that may be of interest to you. Donald's blog has some tips on using Text Mining. Other possible data integration scenarios might include updating customers information with relevant and current news, to help with a sales call.
Following up on my post made during the MSDN webcast, if you want some *real* blogs about SSIS (the artist formerly knowns as DTS) try the following:
Donald is the Group Program Manager for the SSIS team and provides some great insight into the product: http://sqljunkies.com/WebLog/donald%20farmer
Kirk is the Dev Manager for our team and is quite prolific in his blog: http://sqljunkies.com/WebLog/Knight_Reign/
Ash is our “chief customer officer“ and a Dev Lead who is very much in touch with customers: http://sqljunkies.com/WebLog/ashvinis
I am currently doing a MSDN webcast and was asked about blogs from my team. I have not been keeping this blog upto date, but will do so going forward.
The reason for this post is to get the “going forward” started.