Rants
To many developers this may seem odd. I talk with many staunch ALT.NET guys, and the DDD philosophy seems to be that data is a top-down or at worse, bottom up design problem. The issue here is that there is an assumption that just simply not true that data design is part of most software development projects. The reality based on my experience as well as the experience of talking with developers in the community is that many projects (though its hard to exactly quantify what percentage) begin with existing data. This is especially true in the enterprise where data exists in many forms from new databases, legacy servers (e.g. mainframes) or even flat files and XML. It is the rare project that is new code against all new data.
For new projects, creating a new database schema that at least in some part mimics the design of a project makes sense. The problem is that over time the data must evolve. I think the reality is that depending on the organization this can be very different. There are some opinions that I think impact the maturation of a data model:
- "Well if they want access to the data outside the application, I will expose that as a service interface." - I think this is coming from the world of a consultant not a enterprise developer. The reality is that the data maturation happens over time with the help of DBAs, developers and IT people. Its a complicated situation and isolating the database to the non-development stake holders is something that just doesn't happen in the long run.
- "Transactional schemas are not a good solution for reporting." - While I don't disagree with this idea, it doesn't happen in the real work either. In most large organizations, they have need for data coming out of systems and cannot wait for data warehouses, data marts or reporting servers to be created and maintained. If they are created as part of the application development, they usually get the short straw of maintenance. Reporting happens against transactional schemas...period...get over it.
- "If the application is designed right, the DBA should never need to get involved...we can solve perf problems with caching and smarter data access." - This assumes that the data is isolated from the rest of the organization. This doesn't happen either. In addition, data changes over time and that's where DBA's can be exceptionally helpful. In my opinion, the form of the data can be improved over time as real data is used in a system. What works for a 1GB database rarely works for a 1 TB database. Lastly, there are often many more readers of data than you might expect. This is related to the reporting story, but also ad hoc attaching of data (damn you Excel and Access) happens. You can't control the gateway to the data as the guys that sign the checks sometimes make dumb decisions that impact the organization...that won't change.
Now I am not suggesting that we need to dumb down designs to deal with the nature of data, but understand that the IT infrastructure of deployed products is a lot more fluid than developers would like. For me it is, "Hope for the best and plan for the worst." This means understand that pushing a design philosophy that can help developers get their job done in a more efficient way as well as create better software is what we all are trying to do, but there are no black and white lines drawn here. No *rule* applies in all situations. I laud the ALT.NET guys for trying to inject better skills and tools into the process, I just rail against the ferocity of zealotry that comes in some of the message. When enterprise developers hear this, they just tune it out instead of taking what helps them and leaving the rest.
That's just my opinion...
UPDATE: Silverlight is now required to view the video...it no longer allows a downgraded experience.
I've been poking around the NBC-Olympic site. There has been consternation that Flash is used on the site too. The deal has been that the video playing would be done using Silverlight (which is why I am linking to http://www.nbcolympics.com/video). The enhanced player is really a nice design and it lets you view four events at once and see real-time stats. Impressive.
To dig deeper like I like to do, I decided to see what the experience would be like for people who don't have Silverlight installed. This is the prompt I get from the site:
What's interesting to me is that you can view video without Silverlight, but you get a better experience with Silverlight. If you can proceed without it, I wonder how many installations this will actually create. I am sure still a big #, but it would be nice to have required it...oh well.
I've spent much of the last couple of weeks trying to strengthen my LINQ knowledge. A friend of mine is one of the authors of a LINQ book so I figured it was a good match to dig deeper.
The book covers a lot of topics that emcompass LINQ including LINQ basics, but also LINQ to SQL, LINQ to XML. I like that it starts out with a discussion of the problem and doesn't dive directly into the solution. In addition, I think it teaches the technology without resorting to starting with database applications as the example. Anyone who has heard me talk about LINQ knows that I can't stand that LINQ to SQL is the wrong way to teach it to new people...they didn't fall into that trap.
In addition, I really like that there are lots of good examples and a great index. There was never an example I was looking for that the index didn't help me find. That's becoming rarer in books. I really liked their coverage of LINQ from both the consumer of LINQ and the provider of LINQ. Their discussion of the LINQ to Amazon provider provided quite a lot of good insight into how the inner workings of LINQ are put together.
My only hesitation at completely loving this book is that all the examples are either in C# or VB. This lends the book to feel a bit schizophrenic. I would have preferred a more bloated book where all the example in print were in both languages. This is especially true of LINQ because the language integration of LINQ is very dissimilar between the languages.
Overall, I would recommend the book to anyone trying to learn LINQ as a technology.
Tomorrow the Olympics starts and for Silverlight 2 its a big deal. In case you've been under the rock, Microsoft announced at MIX '08 that it had an agreement with NBC to broadcast every minute of every event of the Olympics through a Silverlight 2 driven web site. Its big...really big.
As far as I am concerned, this is a big risk, big reward sort of scenario. Lets discuss reward first. This puts Silverlight 2 squarely in the eyes of lots of end-users. This will help produce ubiquity of the Silverlight 2 runtime. This is going to get a lot of users to install the bits. But what's the risk? There are two risks:
- Silverlight 2's Installation causes problems
- Video Streaming can't scale
Installation issues may cause some end-user headaches, but hopefully it should be pretty smooth. I don't think this will happen. Video streaming on such a large scale is a bigger risk I think. Microsoft has been in this business a long, long time so I doubt its going to be an issue...but...if it doesn't scale correctly it will be bad.
The problem is that the actually streaming scalability has zero to do with Silverlight 2. Let me say that again, Silverlight 2 can play streamed video but if it doesn't scale, its not Silverlight 2's fault. Unfortunately, the next day it will be the TechCrunch's, CNet's and even Mary Jo's lead story.
As Silverlight 2 developers, we are in with Microsoft on this one. If it succeeds, it will make the Silverlight 2 RTM that much bigger with more demand for the skills involved. If it fails, it will be a scar on the face of Silverlight that will be hard to overcome.
I'll be watching with bated breath and clenched cheeks...will you?
I am proud to announce that in partnership, with DevWorx of México, the Silverlight Tour will now be available in Spanish for the Latin American and Spanish markets. The first three classes are happening on:
- September 29th - October 1st, 2008 - Mexico City, México
- September 29th - October 1st, 2008 - Guadalajara, México
- October 8th - 10th, 2008 - Monterrey, México
I gladly welcome DevWorx to the family of Silverlight Tour partners. If you are want to learn Silverlight 2 and are looking for a class in Spanish, they can definitely help you out!
UPDATE: I know the title is wrong and it should be IUpdateable but I didn't want to break any links for any RSS feeds that already had it.
If you haven't read Part 1 or Part 2 first, you should start there.
In this final part of the IUpdateable implementation, we will discuss the rest of the methods which includes ClearChanges, ResolveResource, ReplaceResource, SetReference, AddReferenceToCollection, RemoveReferenceFromCollection and SaveChanges.
ClearChanges is a simple method that simple undo's any changes that have accumulated. In my implementation, I simply clear the UpdateCache that we discussed in Part 2 as well as clear the NHibernate Session to undo any changed that were previously saved (but not persisted to the database).
ResolveResource is a funny little method. In the implementation documentation for the IUpdateable interface, they specify that in most cases you can return a token or anything you want that uniquely identifies an item instead of an actual object reference. For example, when you implement CreateResource the return value can be the actual resource or can be some token that you know how to resolve to the resource. This method is the one that will turn that token into an object reference if you've used that functionality. In our case we're actually returning object references so this becomes a non-issue and we just return the object that is passed in.
Next up is ReplaceResource. Not surprising, this method is used to complete replace a resource with another copy (who may have the correct values already set). The first parameter is a query who must return a single object. For my implementation, I first retrieve the single instance from the query and then walk through the properties to set the new values. Simple really.
The next couple of methods have to do with relationships between objects. The SetReference method takes three parameters: the target resource, the property name and the property value. The property value is an object who needs to be assigned to a property of the target. The property name to be assigned in the 2nd parameter. The implementation is pretty simple: I used the existing SetValue implementation to set the correct property with the correct value.
The AddReferenceToCollection and RemoveReferenceFromCollection are related methods. One adds an object to a collection and one removes it from the collection. Both of these methods take the same signature that SetReference uses (target, property name, property value). The difference here is instead of setting the reference, you need to add or remove it from a property (which is the collection). For my AddReferenceToCollection implementation, I use reflection to find an "Add" method and invoke it with the new reference. For the RemoveReferenceFromCollection, I do the same but I am looking for a "Remove" method to invoke. Using reflection in this way does not require that the collections follow any specific pattern but do require an Add and Remove method. I chose this implementation because I was copying ADO.NET Data Services' implementation that they use for the Entity Framework. I could have expected IList since that is the standard way that NHibernate usually implements collections, but I decided to use this implementation first and if I had to refactor it I would.
The last method to implement is SaveChanges. This method is pretty straightforward as all pending changes that the IUpdateable interface has applied should be applied during this call. IUpdateable expects that SaveChanges will be atomic (e.g. pass/fail, no partial updates). To enforce that requirement, I used the NHibernate Session object's BeginTransaction method to start a transaction so that I could rollback any changes that failed. Because I had kept a cache of updates so I could clear them if necessary, I walked through them once the transaction had began and applied them to the Session object. Finally I flushed the session to force the persistence to happen and ended the transaction. I did this all in a try...catch block so I could rollback the changes if any exceptions were thrown. SaveChanges does not have a return value so if it fails you are expected to return a DataServiceException. So in my catch block, after rolling back the transaction, I throw a DataServiceException and pass along the caught exception as the inner exception.
Overall the implementation was pretty simple. I hope that these articles will help anyone who is trying to implement this for their own data that they want to use in ADO.NET Data Services.
Microsoft has released a new version of their DeepZoom Composer tool that adds a number of features including panoramic stiching and integration with PhotoZoom! Go Grab it!
Only another week before my Boston Silverlight Tour stop. We will be in Boston for the Silverlight Tour on August 11-13th, 2008.
If you are in the market for three days of intense Silverlight development instruction, take a gander at the site. You can view the full Outline for the course here:
http://www.silverlight-tour.com/outline.aspx
I usually prefer to avoid just link posts, but since I have gotten this question a lot lately in my class and at user group talks, I thought i'd share. A couple of months ago Scott Hanselman convinced Microsoft to allow him to release something called RockScroll:
This tool is an addin for Visual Studio to be a scrollbar replacement for most code windows (though it doesn't work for XML files for some reason). It does some nifty things for me:
- Gives me a visual layout of the file instead of hoping that I am scrolling to the right place. Just click on the preview and it moves there.
- Bookmarks and breakpoints are marked as red and blue dots on the preview so I can find that breakpoint I set really quickly.
- Edited lines are highlighted so I can find the last place I edited.
As Scott says in his blog, its strictly "It works on my machine" but i've been running with it since it's release with zero problems and no slow-downs. Its now a must on all my development machines!
You can download it from Scott's blog:
http://www.hanselman.com/blog/IntroducingRockScroll.aspx
NOTE: This blog post is an adjunct topic to the IUpdateable series of posts that I have here and here.
As part of my work on the ADO.NET Data Services support for NHibernate.LINQ I decided that we should support IExpandProvider as well.
For the uninitiated, IExpandProvider is another interface that ADO.NET Data Services support to allow the expansion feature of Data Services to work. Essentially, expansion is about eager loading of related entities as part of a Data Service Query. For example, if you had a Data Services query that said:
http://www.silverlightdata.com/simple/HibProducts.svc/Products(1)
This URI would go retrieve the Product, but if you wanted to retrieve the Category (a related entity) as part of the single call, you could annotate the URI with a query string parameter called $expand to indicate the paths to eager load:
http://www.silverlightdata.com/simple/HibProducts.svc/Products(1)?$expand=Category
In fact, the expansion support allows you to specify multiple paths (separated by commas) to eager load:
http://www.silverlightdata.com/simple/HibProducts.svc/Products(1)?$expand=Category,Supplier
The way it works inside the Data Service is that checks to see if the context object specified in the Data Service support IExpandProvider (the built-in Entity Framework provider does this automatically). If it supports the interface, it calls the ApplyExpansions method. At that point, the provider is supposed to take the IQueryable interface and change the query to support the expansions. Remember, the methodology of Data Services is to take the URI syntax and convert the query into an IQueryable instance. At the end of the process it executes this query so the IExpandProvider happens before the database is ever involved. All it needs to do is to expand the expression tree to add the expressions necessary to expand the nodes.
So how did I go about it? My first step was to add expansion support to the LINQ interface. Before I got started, eager loading was the responsibility of the user of the LINQ query. There was no way to make eager loading happen in the expression tree. I modeled the support after the way that Entity Framework supports eager loading and added a new expression (called Expand in the NHibernate.LINQ provider) that instructs the underlying provider to eager load a certain path. Interestingly the ICriteria support in NHibernate already had support for eager loading so it was more plumbing to the NHibernate functionality that any real work. The expand works during the data source of the LINQ query:
var query = from timesheet in session.Linq<Timesheet>().Expand("Entries")
where timesheet.Entries.Count > 0
select timesheet;
By adding the Expand method call, it adds a hint to the query to eager load the Entries on the Timesheet object. Now that expansion is supported in the LINQ provider, I can wire up the IExpandProvider to simply add that expression for every path.
To support it, I simple walk through each of the paths that are passed to the ApplyExpansions method, but its not quite that simple. The method signature is:
IEnumerable ApplyExpansions(IQueryable queryable,
ICollection<ExpandSegmentCollection> expandPaths);
The first property is the query that you are going to modify for the expansions. The second parameter is a list of ExpansionSegmentCollections. Yup, a collection of a collection. I am unclear why this is so, but nevertheless, I simply go through each collection of collections (I've never been passed more than one, but perhaps MS can explain why). The ExpandSegmentCollection is a collection of ExpandSegment objects. This object supports two pieces of information: Name and Filter. Name is the path that is requested in the URI syntax; and Filter which it is unclear the exact use case. In my implementation, I simply threw an error if I found a filter since I couldn't find any code path that caused them so I didn't know how to modify the query to support them. My guess is that its there for future use.
So there you go, its added. This new interface in coordination with the IUpdateable makes any NHibernate object supported in not only LINQ but also ADO.NET Data Services. What do you think?
