Rants Tagged with “.NET”
To many developers this may seem odd. I talk with many staunch ALT.NET guys, and the DDD philosophy seems to be that data is a top-down or at worse, bottom up design problem. The issue here is that there is an assumption that just simply not true that data design is part of most software development projects. The reality based on my experience as well as the experience of talking with developers in the community is that many projects (though its hard to exactly quantify what percentage) begin with existing data. This is especially true in the enterprise where data exists in many forms from new databases, legacy servers (e.g. mainframes) or even flat files and XML. It is the rare project that is new code against all new data.
For new projects, creating a new database schema that at least in some part mimics the design of a project makes sense. The problem is that over time the data must evolve. I think the reality is that depending on the organization this can be very different. There are some opinions that I think impact the maturation of a data model:
- "Well if they want access to the data outside the application, I will expose that as a service interface." - I think this is coming from the world of a consultant not a enterprise developer. The reality is that the data maturation happens over time with the help of DBAs, developers and IT people. Its a complicated situation and isolating the database to the non-development stake holders is something that just doesn't happen in the long run.
- "Transactional schemas are not a good solution for reporting." - While I don't disagree with this idea, it doesn't happen in the real work either. In most large organizations, they have need for data coming out of systems and cannot wait for data warehouses, data marts or reporting servers to be created and maintained. If they are created as part of the application development, they usually get the short straw of maintenance. Reporting happens against transactional schemas...period...get over it.
- "If the application is designed right, the DBA should never need to get involved...we can solve perf problems with caching and smarter data access." - This assumes that the data is isolated from the rest of the organization. This doesn't happen either. In addition, data changes over time and that's where DBA's can be exceptionally helpful. In my opinion, the form of the data can be improved over time as real data is used in a system. What works for a 1GB database rarely works for a 1 TB database. Lastly, there are often many more readers of data than you might expect. This is related to the reporting story, but also ad hoc attaching of data (damn you Excel and Access) happens. You can't control the gateway to the data as the guys that sign the checks sometimes make dumb decisions that impact the organization...that won't change.
Now I am not suggesting that we need to dumb down designs to deal with the nature of data, but understand that the IT infrastructure of deployed products is a lot more fluid than developers would like. For me it is, "Hope for the best and plan for the worst." This means understand that pushing a design philosophy that can help developers get their job done in a more efficient way as well as create better software is what we all are trying to do, but there are no black and white lines drawn here. No *rule* applies in all situations. I laud the ALT.NET guys for trying to inject better skills and tools into the process, I just rail against the ferocity of zealotry that comes in some of the message. When enterprise developers hear this, they just tune it out instead of taking what helps them and leaving the rest.
That's just my opinion...
I've spent much of the last couple of weeks trying to strengthen my LINQ knowledge. A friend of mine is one of the authors of a LINQ book so I figured it was a good match to dig deeper.
The book covers a lot of topics that emcompass LINQ including LINQ basics, but also LINQ to SQL, LINQ to XML. I like that it starts out with a discussion of the problem and doesn't dive directly into the solution. In addition, I think it teaches the technology without resorting to starting with database applications as the example. Anyone who has heard me talk about LINQ knows that I can't stand that LINQ to SQL is the wrong way to teach it to new people...they didn't fall into that trap.
In addition, I really like that there are lots of good examples and a great index. There was never an example I was looking for that the index didn't help me find. That's becoming rarer in books. I really liked their coverage of LINQ from both the consumer of LINQ and the provider of LINQ. Their discussion of the LINQ to Amazon provider provided quite a lot of good insight into how the inner workings of LINQ are put together.
My only hesitation at completely loving this book is that all the examples are either in C# or VB. This lends the book to feel a bit schizophrenic. I would have preferred a more bloated book where all the example in print were in both languages. This is especially true of LINQ because the language integration of LINQ is very dissimilar between the languages.
Overall, I would recommend the book to anyone trying to learn LINQ as a technology.
In case you didn't catch it, I participated in a webcast called geekSpeak. This webcast was hosted by Glen Gordon and Chad Brooks. The topic today was "Silverlight and Line of Business Applications". While geekSpeak's usually focus on hands-on examples of creating code, we took a different tact today and discussed the larger topic of where Silverlight fits in the development ecosystem (at least Microsoft's ecosystem).
For my money, the real benefit in Silverlight is for applications that cross the firewall. This means Line of Business applicaitons are really for B2B and B2C solutions. Unfortunately, what I hear from the community is that people see Silverlight as a solution for porting their desktop and traditional 3-tier applications to the web. Is this a good idea? I don't think so. The problem is that desktop development usually involves business objects that tend to have a direct connection to the database. Moving these sorts of applicaitons to the web means that you need to create an extra layer of communications and serialization. There is a cost both in development and performance for these extra layers.
It comes down to a key question...why are you moving to a web model for your application? If you want to expand the reach of your application to more users and clients (outside your organization), Silverlight is still a great story. Unfortunately many organizations see web applications as a deployment solution. No install, no framework, etc. While clearly this isn't true for Silverlight per se, its also a bad reason to go to a web application. Technlogies like Click-Once and XBAP are a great solution for a better deployment story than traditional desktop applications.
Since I brought up XBAP, let's plug it a bit. I notice that even amongst WPF guys, XBAP is a lost story. If you're not familiar with it, essentially its an in-the-browser WPF applicaitons that is deployed via manifest files (e.g. like ClickOnce). This means you can have the richness of UI, the better data binding story and interactivity that WPF/XAML affords you without having to deal with the limitations of Silverlight. I suggest that many organizations that want to use Silverlight for internal applications (inside the firewall) should be doing XBAP instead.
So what about ASP.NET/AJAX? The big story here is that HTML-based interfaces still have the longest reach of all the Internet enabled applications. HTML just works on many more platforms and browsers than Silverlight or Flash. Before you commit to moving away from HTML-based UI's, spend some time with your server logs. Understand who is really using your existing application before you leave anyone in the dust. A better strategy is often to include fall-back functionality. For example, in my Silverlight Tour website (http://www.silverlight-tour.com), I decided that developers may have Silverlight installed so I wanted to give them a better experience by showing an interactive map of tour stops. But their bosses and accounting departments were unlikely to have it installed. In that case I made a design decision to never prompt to install Silverlight, but instead if it wasn't installed to show a simple table of the classes instead of the Silverlight app. This is a great solution to moving forward without leaving old users in the dust.
Why is Silverlight not a good solution inside the firewall? The two issues are infrastructure and security. In order to build solid line-of-business solutions with Silverlight, you need to have a way to communicate data with the server. Building this infrastructure can be labor intensive, but more importantly adds complexity. More moving parts == more than can go wrong. Security is the second issue. Silverlight (for good reason) is pretty locked down. This means you will need to often learn to work in tighter confines (limited access to storage, no access to file system, registry, ports). If your application is meant to work in trusted scenarios (e.g. Integrated Windows Authentication), the limitations of the high-security environment will be a real limiter.
So what does this all mean? I still think Silverlight is still the solution when you need to *extend* the HTML model in the browser. Line of Business applications across the firewall still need to be web driven in my opinion, but enhancing that story with Silverlight for soutions like data visualization, user interactivity or media is a great solution. While I think that creating whole-browser solutions make sense for some applications, it doesn't for many many more. My fear is that we will move from monolithic desktop apps to monolithic Silverlight apps. The web is still a disconnected model and deciding on building a single huge Silverlight app (instead of page-based functionality) just doesn't make much sense. If you are planning on building one of these monstrocities, please also read my recent blog about linkability in Silverlight:
http://wildermuth.com/2008/06/25/Doesn_t_Anyone_Bookmark_Anymore.aspx
When I deployed the small test application (http://www.silverlightdata.com/simple), it was pretty apparent that it was performing really badly. Some of this is because my ISP upload speed isn't great but it was still taking far too long I thought. This was a simple app with not much data, so I knew I wanted to find out what was happening. If you haven't seen the site, its basically an editor for the Product table in the Northwind database. Nothing special really.
ADO.NET Data Services does a lot of work for you under the covers. This is good because we're not being asked to write a lot of serialization and transportation code, but it can be bad because it becomes more difficult to see why things are happening. Fiddler2 to the rescue!
If you're not familiar with Fiddler2, its basically a great tool for watching web traffic so you can see what is actually happening over the wire. For ADO.NET Data Services, we want to see the actual communication over the wire to see what was being sent to and from the Silverlight application. First, grab the Fiddler2 tool (http://fiddler2.com/) and come with me!
So I fire up Fiddler2 and run the example page. What do I see? Nothing. None of the traffic is showing up. A quick Google search reveals that Fiddler doesn't work on localhost or 127.0.0.1 (kinda the same thing). I found an odd workaround...add a period after the IP address. So my "http://localhost:8000/Default.aspx" test page became "http://127.0.0.1.:8000/Default.aspx". The period after the "1" is not a mistake. Once I did that, the requests were showing up in Fiddler2. Wahoo!
So what did I see? Here's a look at the Fiddler2 window (You can click it for a bigger view):
Looking at the request to Products.svc, I picked the "TextView" tab to see the actual body of the message. Looking at it I noticed this Picture property that was base-64 encoded. Uh oh...I didn't even notice that we are getting pictures...in fact we're getting duplicate data. My LINQ request was:
var qry = from p in TheContext.Products.Expand("Supplier")
.Expand("Category")
orderby p.ProductName
select p;
I knew that I would be getting duplicate data by using the Expand extension method. Expand says to embed the related entity (the Supplier and the Category) so that our object model doesn't have to lazy load them. Of course I wasn't paying attention. The Category has a picture stored in the database. I am not even using it (and couldn't since the image is a GIF and Silverlight can't display GIFs) so retrieving it is pretty useless. Here's a quick view of the underlying Entity model:
The fix for me was to simply remove it from the model, but I am looking at different ways of really solving it including delving into how the "select" statement might filter these out, though I expect that doing three queries (one for Products then just returning Suppliers and Categories to remove the duplicate data would be much more efficient yet.
One caveat, this is the first release of the ADO.NET Data Services' Silverlight Client Library so give it time to improve some of the performance issues. But like other data access libraries, sometimes small changes can make large improvements in performance.
Hope this helps you debug your own ADO.NET Data Services projects.
UPDATE: I had the PUT/POST reversed. It reads correctly now. (Thanks to commenter Rob for pointing it out).
Now that Silverlight 2 Beta 2 has launched, we have the ability to consume ADO.NET Data Services (formerly Astoria) from within Silverlight projects. ADO.NET Data Services are a perfect match for client-side technologies like Silverlight and ASP.NET AJAX.
For the uninitiated, ADO.NET Data Services is a new part of the .NET 3.5 framework that supports exposing a data model (e.g. LINQ for SQL, Entity Framework, etc.) as a set of queryable REST endpoints. ADO.NET Data Services maps the four data verbs into the four HTTP verbs:
- Create == POST
- Read == GET
- Update == PUT
- Delete == DELETE
Essentially it provides a way to use a data model across the firewall. It works by exposing IQueryable endpoints through a URI-based syntax allowing developers control over how the data is retrieved through:
- Filtering
- Sorting
- Paging
- Shaping
In addition ADO.NET Data Services utilizes JSON and Atom (though Plain Old XML may be supported eventually) as the serialization formats. These make it easy to consume in client-side interfaces like Silverlight and AJAX. For more information on ADO.NET Data Services, see there site:
http://msdn.microsoft.com/en-us/data/bb931106.aspx
Before you can consume a Data Service in Silverlight, you will need to create a model and service. See Guy Burstein's walkthrough of creating the service here:
http://blogs.microsoft.co.il/blogs/bursteg/archive/2008/05/12/visual-studio-2008-sp1-ado-net-data-service-walkthrough.aspx
You *can* use LINQ to SQL as your data model but currently it does not have support for updating via ADO.NET Data Services so if you need to read and write data, you should start with an Entity Framework data model.
In Silverlight 2 Beta 2, ADO.NET Data Services is composed of an in-memory library allows asynchronous LINQ queries that are translated into the URI syntax automatically. Before you can get started you will need a a client-side version of the Data Context object and data contract classes for the entities in your data model. To do this, simply use the DataSvcUtil.exe tool located in the C:\Windows\Microsoft.NET\Framework\v3.5\ directory. Typically you would call it by specifying the URI of the service, the name of the class file to create and what language to use:
DataSvcUtil.exe /uri:http://localhost:8888/YourService.svc
/out:dataclass.cs
/language:CSharp
Once that class is created, add it to your Silverlight project. You will also need to add a reference to the System.Data.Client.Services.dll assembly. Now we are ready to get some data.
First we need to create an instance of the context class. This class exposes each of the model types as queryable properties. When creating an instance of the class, you must specify a URI to the service itself:
NorthwindEntities ctx =
new NorthwindEntities(new Uri("/Products.svc", UriKind.Relative));
Querying data with Data Services looks much like any other LINQ-based query:
var qry = from p in ctx.Products
orderby p.ProductName
select p;
Normally, you could execute the query directly by calling something like ToList() against the query. Since that operation would cause a synchronous web request to happen across the network, that isn't supported in Silverlight 2 Beta 2. In fact, if you try you will simply get a NotImplemented exception. In order to execute these queries, you will need to cast the query into a DataServiceQuery. The DataServiceQuery allows you to call BeginExecute to start an asynchronous query as seen below:
// Cast the query to a DataServiceQuery
DataServiceQuery<Product> productQuery =
(DataServiceQuery<Product>)qry;
// Start the execution
productQuery.BeginExecute(new AsyncCallback(OnLoadComplete),
productQuery);
Once the query completes, it will call the OnLoadComplete method that was specified in the AsyncCallback. In this method, you first grab the DataServiceQuery that you sent with the request then end the execution to retrieve the results as seen below:
void OnLoadComplete(IAsyncResult result)
{
// Get a reference to the Query
DataServiceQuery<Product> productQuery =
(DataServiceQuery<Product>)result.AsyncState;
// Get ther esults and add them to the collection
List<Product> products = productQuery.EndExecute(result).ToList();
}
While its not as straightforward as synchronous execution, the new ADO.NET Data Services certainly works well in Silverlight 2 Beta 2 and provides a great way to use existing or planned data models over Internet applications. There are a couple of caveats:
- Error handling and communication is very confusing right now as most real errors are being swallowed by the server instead of communicating back to the client. To find out what is really happening, use of Fiddler and enabling breaking on the throwing of all .NET exceptions will help a lot.
- There are bugs on updating data that may get in your way. Using Batch Saves will solve most of these issues.
- Currently the Data Contract objects do not support INotifyPropertyChanged or INotifyCollectionChanged so data binding may be affected in some cases.There are partial methods for detecting changes to specific properties which makes implementing the INotifyPropertyChanged interface trivial, but not quick. Future versions (e.g. post-RTM) will give us better control over the generation of the data contract classes.
- There is no built-in Visual Studio support to build the data contract/context classes. This is also coming soon, though the exact schedule is unknown.
I am currently working on two examples for this (a simple one and a more complex one) as well as an article for MSDN, all of which will greatly expand the details of how to use ADO.NET Data Services (including how to save changes back to the server. Be sure to watch here for details of those samples and articles.
I've known Julie Lerman (or is it Julia these days ;) for a long time now. She's an excellent resource for everything data related. In particular she's been keeping up with the Entity Framework and ADO.NET Data Services (formerly Astoria) updates in .NET 3.5 and VS SP1 Beta that was just released this week. If you are upgrading projects (like I am), she has two excellent blog posts about how to upgrade your projects:
New Entity Framework Changes
New ADO.NET Data Services (Astoria) Changes
Quick fix for a problem that was haunting me today:
If you upgrade an ASP.NET 2.0 app to 3.5 and have .xaml files in your project that are part of a Silverlight 1.0 or 1.1 project, the conversion wizard converters them to Build-type: "Page" and adds a custom build for building the WPF files. If this happens you'll get a cryptic error:
"error MC6000: Project file must include the .NET Framework assembly 'WindowsBase, PresentationCore, PresentationFramework' in the reference list."
Go through your xaml files and mark them as "Content" and remove any custom tools and it will fix it.
I will be at the Alabama Code Camp this weekend to talk about Silverlight, the Entity Framework and Astoria. The talks I am doing are:
Feel free to stop by and harrass me with questions or disagreements. I'll be there all day.
I got to spend some time with Scott Hanselman on his Podcast (Hanselminutes) discussing Astoria and how it *not* just exposing your database to the Internet.We delved into REST and how Astoria and Silverlight are a good mix for the right application.
Check it out when you have a chance.
ADO.NET Data Services Silverlight Extensions now released. This library is used in Silverlight 2.0 projects to access Astoria without all the headaches of serialization. Check it out!
I will be updating my http://silverlightdata.com site soon with a new build including these bits.
