Thursday, November 29, 2007

ASP.NET .NET 3.5 Extensions Preview to Include Entity Framework Beta 3, ADO.NET Data Services

Scott Guthrie's .NET Web Product Roadmap (ASP.NET, Silverlight, IIS7) post of November 29, 2007 states that Entity Framework Beta 3 (presumably with EDM Designer CTP 2) and a VS 2008 RTM-compliant preview of ADO.NET Data Services (formerly Project Astoria) will debut next week in conjunction with the newly-named ASP.NET 3.5 Extensions Release. Scott says, in part:

  • ADO.NET Data Services: In parallel with the ASP.NET Extensions release we will also be releasing the ADO.NET Entity Framework.  This provides a new modeling framework that enables developers to define a conceptual model of a database schema that closely aligns to a real world view of the information.  We will also be shipping a new set of data services (codename "Astoria") that make it easy to expose REST based API endpoints from within your ASP.NET applications.

"In parallel with" and "also be shipping" don't necessarily mean "with the other pieces next week." Microsoft representatives wouldn't confirm to me that Entity Framework (EF) Beta 3, EDM Designer CTP 2, and the ADO.NET Data Services will release next week. I'll update this post when the actual release data is public.

Update 12/7/2007: Microsoft released EF Beta 3 and EDM Designer CTP 2 on 12/6/2007, as reported in Entity Framework Beta 3 Available for Download of that date. Still no word on the Astoria release date other than "almost here." See the "ADO.NET Data  Services CTP1 'Almost Here' with LINQ to Astoria, New Wire Formats, and Increased Security" topic in LINQ and Entity Framework Posts for 12/3/2007+.

Note: Joe Wilcox says "'Astoria' will be included in next week's ASP.NET 3.5 Extensions preview release. Astoria is Microsoft's incubation project for exposing data as a service" in his Microsoft Updates Web Developer Roadmap post of November 29, 2007.

As "Something to watch," Joe adds:

Microsoft's data model emphasis, which is more than just about rich Internet applications delivery. The emphasis directionally reveals something about Microsoft's forthcoming Web services platform. Data-driven may not be a new concept, but Microsoft is positioning for a leap frog over competitors. That's topic for another post. :)

Other Elements of the ASP.NET .NET 3.5 Extensions Preview

Scott's post also states that the extensions release will include:

  • ASP.NET MVC: This model view controller (MVC) framework for ASP.NET provides a structured model that enables a clear separation of concerns within web applications, and makes it easier to unit test your code and support a TDD workflow.  It also helps provide more control over the URLs you publish in your applications, and more control over the HTML that is emitted from them.  You can learn more about it from Part 1 of my ASP.NET MVC Tutorial series.  I'm hoping to find time this weekend to write and post Part 2 of the series. [Read more about the MVC Framework here.]
  • ASP.NET AJAX Improvements: New ASP.NET AJAX features in the ASP.NET 3.5 Extensions release will include better browser history support (back/forward button integration, and server-side history management support via a new <asp:history> server control), improved AJAX content linking support with permalinks, and additional JavaScript library improvements.
  • ASP.NET Dynamic Data Support: The ASP.NET 3.5 Extensions release will deliver new features that enable faster creation of data driven web sites.  It provides a rich scaffolding framework, and enables rapid data driven site development using both ASP.NET WebForms and ASP.NET MVC.
  • ASP.NET Silverlight Support: With the ASP.NET 3.5 Extensions release we'll deliver support for easily integrating Silverlight within your ASP.NET applications.  Included will be new controls that make it easy to integrate Silverlight video/media and interactive content within your sites.

Update 11/30/2007: Rick Strahl's What's Ailing ASP.NET Web Forms essay provides the view of the current state of ASP.NET and a look at the future of ASP.NET MVC. It's written by an independent Web developer with many years of ASP and ASP.NET experience.

ASP.NET Dynamic Data Support

"Dynamic Data Support" sounds to me like Rob Conery's open-source SubSonic data access layer and Web page generator that I described in my SubSonic will be the "Convention-Driven Toolset for Microsoft’s New MVC Framework" for ASP.NET post of November 11, 2007 (updated 11/13, 11/18, and 11/29/2007.)

Rob's working at Microsoft (but living on Kaua'i) and still maintaining his Weke Road personal blog. My post's "SubSonic's Continuing Open-Source Status" section described Rob's and Microsoft's commitment to continuing development of SubSonic as an open-source project.

The "Moving Forward with .NET 3.5" topic of Rob's State Of SubSonic, November 2007 post of November 28, 2007 says:

We’re shelving the query tool update that we’ve been planning. In fact we’re shelving any further feature change/innovation in favor of moving forward with .NET 3.5.

There are soooo many language enhancements to .NET 3.5 that we have a chance to refactor and trim our feature set really nicely. For instance, our entire Sugar library can be redone as extension methods - which is what they should be.

In addition, as many know, we have a lot of love coming for the new MVC framework.

It doesn’t make sense to rev two branched versions of SubSonic that run on a different platform, so the next major version will be a version rev to 3.0, and will work on .NET 3.5 only.

This doesn’t mean we won’t support/fix/tweak/patch 2.0 - we’ll always try to make it better. But in terms of features, it’s pretty much set.

With regard to incorporating LINQ in the query tool and meshing with the MVC Framework, the "New Features for [SubSonic] 3.0" topic says:

Eric and I have been talking a lot about the query tool and how to make it work with LINQ. I think we have a nice scheme planned, so expect to see a new query tool as we’ve discussed in the past.

We’re going to throw all kinds of love at the new MVC toolset. All Kinds. It’ll be magic. And we’ll cook up some nice reference apps for you to play with. More on that later (when I figure out just what features I’ll be working on in the MVC toolset.

Judging from this post, if Dynamic Data Support is based on SubSonic and the preview arrives next week, it would probably be an unmodified or slightly modified version of SubSonic 2.0. When the bits arrive next week, I'll update this and the 11/11/2007 post.

Silverlight 1.1 Becomes Silverlight 2.0

According to Scott, the next public Silverlight preview will include major upgrades to the WPF UI framework, add a "rich set of controls," including a DataGrid, provide REST, POX, RSS, and WS-* (plus, presumably ATOM Pub), and include LINQ to XML support. The number and significance of new features warrants a full version upgrade, so Silverlight 1.1 will become Silverlight 2.0.

Microsoft plans a Silverlight 2.0 beta version for 2008Q1 with a GoLive license. VS 2008 Express Edition users will be glad to hear that the Silverlight toolset for VS 2008 will work with the free editions. (VS 2008 Professional Edition is required for the just-released Silverlight 1.1 tools.)

Monday, November 26, 2007

LINQ and Entity Framework Posts for 11/26/2007+

Note: This compendium is updated daily or more often, depending on availability of articles.

JD Conley: Compile Your LINQ to SQL Queries for Performance

Rico Mariani's Linq to SQL Compiled Queries are thread-safe post of November 29, 2007 led me to JD Conley's Linq to Sql Surprise Performance Hit article of November 28, 2007. JD develops Facebook apps with VS 2008, .NET 3.5, and LINQ, and intends to release an open-source asynchronous Facebook library.

JD ran into a serious performance problem in his Photo Feeds Facebook app, which uses LINQ to SQL, that was causing his page to load slowly. It turned out that three page loads caused 1,176,879 calls to SqlVisitor.Visit and pinned the CPU. Compiling the query decreased load time by a factor of five and the app now "barely registers on the CPU when the page loads."

Rico says compiled queries are thread-safe, which is reassuring.

Daniel Moth Demonstratrates and Explains the Parallel Exensions

Microsoft UK's Daniel Moth posted the following short video clips and introductions to Channel9 on November 29, 2007:

Intro to Parallel Extensions to the .NET Framework

In this 22:27 tour of the Parallel Extensions sample code, you will learn where to focus your energy when you explore the samples. Watch this video in order to see how Parallel Extensions to the .NET Framework fit in the large picture and what goal it aims to solve.

Parallel LINQ (PLINQ)

Achieving "Declarative Data Parallelism" with the Parallel Extensions is achieved through Parallel LINQ (PLINQ). This 18:50 video explains how to parallelise your LINQ queries, how it works under the covers and how to configure it for advanced scenarios

How to use the static Parallel class

The .NET incarnation of the solution to "imperative data parallelism" lives in the static Parallel class of the Parallel Extensions assembly. If that sentence confuses you a bit, watch the 20:15 video for the full explanation.

His Parallel Extensions videos Errata post of the same day corrects minor gaffes, such as the "PFX" code name, pipelining description, and return sum instruction. He includes the first reference I've seen to the Parallel Extensions Team Blog, which went into business on November 29, 2007.

Thanks to Mike Taulty for the heads-up. (Mike: If you're reading this, Ralf Rottman has the answer for running Entity Framework Beta 2 on VS 2008 RTM. He doesn't know if the tools work.)

Added: 11/29/2007 

Parallel Extensions Team Blog Starts with Updates to the MSDN Magazine Articles

The new Parallel Extensions Team Blog posted the following five articles on November 29, 2007:

The "Luke Hoban Gets the Most Complex LINQ Query Award (and PLINQ Compatibility)" topic of my LINQ and Entity Framework Posts for 10/2/2007+ post has more details on the two MSDN Magazine articles.

Added: 11/29/2007

Parallel Extensions to .NET Framework 3.5 December CTP Available

As promised in Soma Somasegar's blog, you can download the Microsoft Parallel Extensions to .NET Framework 3.5, December 2007 Community Technology Preview starting today. ParallelFX includes Parallel LINQ (PLINQ), which Soma describes as follows:

Declarative data parallelism in the form of a data parallel implementation of LINQ-to-Objects.  This allows you to run LINQ queries on multiple processors.

PLINQ samples include the ray-tracing project that I described in the "Luke Hoban Gets the Most Complex LINQ Query Award (and PLINQ Compatibility)" topic of my LINQ and Entity Framework Posts for 10/2/2007+ post, which also describes the Parallel FX program and task parallel library. However, there's no sign of the project described in the "Joe Cheng Solves the Wide Finder Problem with LINQ and PLINQ" topic of my LINQ and Entity Framework Posts for 10/5/2007+ post.

So far, I haven't been able to find MSDN's "Concurrent Programming Developer Center" that Soma mentions in his post.

Update: 11/30/2007: Charlie Calvert provided this link to the Parallel Computing Developer Center.

Thanks to Mary Jo Foley and Fabrice Marguerie for the heads-ups.

Added: 11/29/2007

Entity Framework Beta 3 and ADO.NET Data Services (Astoria) Preview Due Next Week

According to Scott Guthrie's .NET Web Product Roadmap (ASP.NET, Silverlight, IIS7) post of November 29, 2007, Entity Framework (EF) Beta 3, presumably EDM Designer CTP 2, and a VS 2008 RTM-compliant preview of ADO.NET Data Services (formerly Project Astoria) will debut next week in conjunction with the newly-named ASP.NET 3.5 Extensions Release. Scott says, in part:

  • ADO.NET Data Services: In parallel with the ASP.NET Extensions release we will also be releasing the ADO.NET Entity Framework.  This provides a new modeling framework that enables developers to define a conceptual model of a database schema that closely aligns to a real world view of the information.  We will also be shipping a new set of data services (codename "Astoria") that make it easy to expose REST based API endpoints from within your ASP.NET applications.

For more details and commentary about the ASP.NET 3.5 Extensions Release and the version number change of Silverlight 1.1 to 2.0, see today's ASP.NET .NET 3.5 Extensions Preview to Include Entity Framework Beta 3, ADO.NET Data Services post.

Update 1200 PST: "In parallel with" and "also be shipping" don't necessarily mean "with the other pieces next week." Microsoft representatives wouldn't confirm to me that Entity Framework (EF) Beta 3, EDM Designer CTP 2, and the ADO.NET Data Services will release next week.

Added: 11/29/2007

Keyvan Nayyeri Syndicates Feeds with LINQ to SQL and WCF

Keyvan's Create RSS and Atom Feeds with LINQ and WCF Syndication in .NET 3.5 post of November 29, 2007 shows you how to take advantage of Windows Communication Framework's Syndication library and LINQ to SQL to generate Atom 1.0 or RSS 2.0 feeds from a SQL Server data source.

Added: 11/29/2007 

Diego Vega: Another New Entity Framework PM and Stealth Blogger

The Entity Framework (EF) team is staffing up program management in advance of the Beta 3 drop. The latest addition is Diego Vega whose been on the job but under the radar since mid-October. Diego's program area is LINQ to Entities.

Diego's latest post, Entity SQL Non-Quirkiness of November 27, 2007, describes the unwanted (but not unexpected) result of omitting the VALUE modifier in an aggregate (MAX()) subquery of an Entity SQL SELECT statement.

His first EF-related post, Choosing an Entity Framework API of November 7, 2007 contrasts the Entity Services (relational-to-entity mapper) and Object Services (object-to-entity mapper) layers, both of which accept Entity SQL Queries. Only the ObjectServices layer and its ObjectQuery type support Entity SQL and LINQ to Entities queries.

Like the other recent PM hire, Alex James (see the "Alex James Joins the ADO.NET Team as Program Manager for Metadata" topic of Link and Entity Framework Posts for 11/19/2007+), the ADO.NET team didn't announce that Diego had signed on and was blogging about EF. Thanks to Julie Lerman (again) for the heads up.

Added: 11/29/2007

Download Guy Burstein's LINQ to SQL Presentation and Sample Code

Advantech Senior Consultant and Microsoft MVP Guy Burstein presented a session on LINQ to SQL at Microsoft Israel's Developer Academy 2 conference on November 27, 2007. If you'd like a quick but comprehensive introduction to LINQ to SQL click here to download his slide decks and C# demo code.

Added: 11/28/2007 

Stefan Cruysberghs Queries Outlook and OneNote with LINQ

Add LINQ to Outlook and LINQ to OneNote as two new third-party LINQ implementations. Stefan's .NET - Querying Outlook and OneNote with LINQ post (in English) provides the following elements:

  • Wrapper class for Outlook 2007
  • Wrapper class OneNote 2007
  • Querying Outlook with LINQ
  • Querying OneNote with LINQ
  • Execute queries on Office data with LINQPad (additional article)
  • Source code for classes and console demo application

The Wrapper class for OneNote is quite complex because OneNote uses XML as its data store.

Added: 11/29/2007

Julie Lerman and Beth Massi Present Sessions at DevTeach 2007 Vancouver

DevTeach is conducting it's first West Coast conference in Vancouver BC starting Monday 11/26/2007 through Friday 11/30/2007.

Julie Lerman is delivering Introduction to ADO.NET Entity Framework on Wednesday, and Real World Entity Framework and Databinding in ASP.NET with LINQ on Thursday.

Beth Massi demoed her Visual Basic Tip of the Day Browser app and built a VB 2008 add-in for part of Ken Levy's Tuesday keynote address. She also presented Strategies for Moving VB6 Apps to .NET on Tuesday.

Beth's post mortem of her scheduled presentation and a description of her impromptu "Conquering XML with LINQ in Visual Basic 9" session on Thursday is in her DevTeach Part Deux post of 11/29/2007.

Note: Most Western Canadians I know are Anglophones and Francophobic.

Added: 11/28/2007 Updated 11/29/2007

Converting SQL to LINQ, Part 2: FROM and SELECT

The VB Team's Bill Horts continues the series about SQL to LINQ query conversion, which he started with Converting SQL to LINQ, Part 1: The Basics.

Added: 11/28/2007

Scott Guthrie: Silverlight 1.1 Tools for VS 2008 RTM Available

According to Scott's Silverlight 1.1 Tools Alpha for Visual Studio 2008 Available for Download post of November 26, 2007, you can download the updated Microsoft Silverlight 1.1 Tools alpha for Visual Studio 2008 RTM from here and the quickstart tutorial for the tools here.

Scott says:

The next public preview of Silverlight will include a ton of new runtime features, as well as a significantly enhanced VS 2008 tooling support.  I'll be blogging more details about this shortly.

Hopefully, the promised LINQ to XML addition will be in the next public preview.

Added: 11/27/2007 

Scott Wisniewski Generates a US Map from TIGER Files, LINQ to XML, and WPF

In what's probably the most complex jobs of data binding attempted so far with LINQ to XML and WPF, the VB Team's Scott Wisniewski describes how to draw a US map in his Map LINQ: Create Dynamic Maps with Visual Basic 9.0 and WPF article for MSDN Magazine's December 2007 issue. The map includes color-coded population data for all counties in each conterminous US state. Moving the mouse over a state emphasizes its outline.

The following screen capture has been widened to more closely represent the aspect ratio of a conventional US map. Click the image to display a full-size capture.

The sample code (30 MB, expands to about 150 MB) is available for download here. After you unzip the files to their designated folders, move the PolygonData and PopulationData folders from the ...\WPFApplication1 folder to the ...\WPFApplication1\bin folder and set WPFApplication 1 as the Startup project.

The XML files (133 MB) take a while to load and process with LINQ to XML queries, so be patient after you press F5.

Dietel Adds Microsoft LINQ Resource Center

Dietel & Associates, Inc. have added a Microsoft LINQ Resource Center to the 13 existing resource centers on their site. The LINQ center offers links to resources in the following categories:

"LINQ to Visual Basic" is an implementation that's new to me.

Bart De Smet Releases LINQ to Active Directory (LINQ to AD)

LINQ to LDAP has received an upgrade to Visual Studio 2008 and a name change to LINQ to Active Directory (a.k.a. LINQ to AD) according to Bart De Smet's November 25, 2007 LINQ to Active Directory (formerly known as LINQ to LDAP) is here post.

The new version offers the following features:

  • Translates into LDAP filters according to RFC 2254.
  • Simple and approachable entity model with support of propagating updates back.
  • Supports mappings to both the System.DirectoryServices (.NET) and ActiveDs (COM) APIs.
  • Ships with a set of samples.

You can download the new code from CodePlex.

Julie Lerman Unravels Entity Framework Namespace Ambiguities

Update 11/27/2007: Julie updated her Entity SQL difference when using ObjectQuery vs EntityClient post to indicate that ObjectQuery and Entity Client handle container name in entity SQL the same way. See the comment to this post by Danny Simmons.

Julie's Entity SQL difference when using ObjectQuery vs EntityClient post explains that EntityClient requires you to include the Entity Data Model's container name in the FROM clause of your Entity SQL statement, as in:

SELECT c FROM NorthwindEntities.Customer AS c

But you don't include the namespace in ObjectServices queries, such as:

SELECT c FROM Customer AS c

Because you've already assigned the namespace to an ObjectContext variable with a statement, such as:

Dim Nwind [As NorthwindEntities] = New NorthwindEntities

ObjectServices queries throw a runtime exception if you prefix the entity with its namespace.

Off-Topic: Pat Helland Recounts his Tech*Ed 2007 Developers Presentations

If you haven't heard, Pat Helland (of Fiefdom and Emissaries fame) has returned to Microsoft after a few years' exile at Amazon.com and now writes a blog. (I covered for Fawcette Technical Publications his participation with Paul Flessner in the Autonomous Computing keynote for Tech*Ed 2002.)

To mark his renewed presence on the conference circuit, Pat has posted backgrounders, abstracts, and links to slides for his two technical sessions and the closing general session at Tech*Ed 2007 Developers Barcelona:

My Very Large Databases: Bricks, BitVault and BigTable post of April 6, 2006 has some additional background on the topics Pat covered at Barcelona.

As a bonus, Pat posted a lavishly illustrated, running account of a one-week Mediterranean cruise with his wife to celebrate their second anniversary:

The series is definitely worth a read, especially if you've never ridden a cruise ship.

Bill McCarthy Uncovers XAttribute Namespace Issue in LINQ to XML

Bill warns in his Don't "p" on your XAttributes post of November 26, 2007 not to assign pn (where n is an integer) as a namespace prefixe when programmatically adding attributes in LINQ to XML. You're likely to incur an exception caused by using the same namespace prefix twice.

Ken Tucker Creates an Excel Spreadsheet with the Open XML SDK and LINQ to XML

Ken's Using LINQ to XML to create an Excel spreadsheet post of November 25, 2007 contains the VB code to use the Microsoft SDK for Open XML Formats to create an Excel 2007 spreadsheet document, workbook, and string table, and then uses VB's literal XML feature to add cells from the Northwind database.

His post includes the complete code for the process that you can cut and paste into a sample project.

Wednesday, November 21, 2007

Serializing Object Graphs Without and With References

Updated: 11/25/2007 and 12/12/2007 (Added C# class definitions, plus C# and VB object initialization code. See below.)

Windows Communication Foundation (WCF) defaults to the DataContractSerializer (DCS) for serializing services' object graph payloads to well-formed XML Infosets. DCS is capable of serializing 1:many associations of an object graph, such as Customers -> Orders -> OrderDetails, into a hierarchical XML Infoset, which has a structure that any WS-* SOAP client should be able to deserialize correctly.

An alternative to DCS is the NetDataContractSerializer (NDCS), which can serialize graphs with 1:many and many:1 (bidirectional) associations by preserving association references in ID/IDREF pairs of Id and Ref values. NDCS delivers graph fidelity at the expense of interoperability because it requires clients to support CLR types. A factor in lack of interoperability with other vendors' Web service toolkits is the complexity of schemas genrated by NDCS and Microsoft's patent that appears to cover NDCS's method of serializing/deserializing objects.

Note: The Entity Framework (EF) maps associations to Navigation Properties.

The online help topics for DCS and NDCS use trivial classes with two or three members; no help topics that I've seen demonstrate serializing generic lists. To verify that DCS and NDCS behave as expected with object graphs of moderate to high complexity generated from relational databases, I used my LINQ In-Memory Object Generation (LIMOG) utility. LIMOG autogenerates C# and VB classes designed for serialization with DCS and NDCS. LIMOG also creates object initializers to generate customized List<T> or List(Of T) instances for serialization and deserialization. The three default data sources for classes are Northwind, AdventureWorks, and AdventureWorksLT (Lite) sample databases. Click here to download the AdventureWorks (OLTP)'s entity-relationship diagram in HTML or Visio format; click here to open an HTML entity-relationship diagram of AdventureWorksLT.

Note: The original impetus for creating the LIMOG app was to generate mock objects that could substitute for entities generated by LINQ to SQL and the Entity Framework and speed unit testing. Using mock objects also frees the unit test from infrastructure dependency, which conforms to the separations of concerns principal of software design. Serializing files generated by LIMOG can provide provide payload size and complexity estimates for WS-* Web services.

Here's a screen capture of the LIMOG utility after generating the VB code for the Order class and a List(Of Order) instance with 100 members (click for full-size image):

Note: Marking the Cyclic References check box adds the [DataMember] attribute to members representing many:1 associations.

Serializing Partial Object Graphs with DCS

DCS uses opt-in serialization; DCS serializes classes decorated with a [DataContract] attribute and properties with a [DataMember] attribute only. These two attributes have properties that let you customize the resulting document, such as specifying object and member names and member sequences. (DCS sequences properties alphabetically by default.)

By default, DCS doesn't preserve references by default so it creates duplicates of objects that normally would share a common pointer to the same object in memory (i.e., ReferenceEquals = True). This means that the default DCS implementation can't handle cyclic references (cycles) created by bidirectional 1:many and many:1 relationships. I raised this issue in my LINQ to SQL and Entity Framework XML Serialization Issues with WCF - Part 1 post of October 9, 2007.

DCS offers a PreserveObjectReferences property to solve the preceding problem. However, there are no commonly observed standards for serializing and deserializing the ID/IDREF attributes in XML Infosets required to handle bidirectional references, so Microsoft's XML team purposely made it difficult to set DCS's PreserveObjectReferences property to true.

LINQ to SQL handles this problem by not applying the <DataMember()> attribute to many:1 properties, such as Order.Customer, Order.Employee, Order.Shipper, Order_Detail.Order or Order_Detail.Product. Thus the serializer doesn't follow the paths necessary to include the Products, Categories, Suppliers, Shippers, and Employees classes in the resulting document, which has this structure:

<ArrayOfCustomer xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <Customer>
    ...
    <Orders>
      <Order>
        ...
        <Order_Details>
          </Order_Detail>
          ...
          </Order_Detail>
          ...
        </Order_Details>
      </Order>
      ...
    </Orders>
  ...
  </Customer>
</ArrayOfCustomer>

VB code to serialize the CustomerList tree is:

Dim fs As FileStream = New FileStream("NwindGraphDCS.xml", FileMode.Create)
Dim dcs = New DataContractSerializer(GetType(List(Of Customer)))
dcs.WriteObject(fs, CustomerList)
fs.Close()

and to deserialize it is:

Dim fs As FileStream = New FileStream("NwindGraphDCS.xml", FileMode.Open, _
    FileAccess.Read)
Dim dcs = New DataContractSerializer(GetType(List(Of Customer)))
CustomerList = CType(dcs.ReadObject(fs), List(Of Customer))
fs.Close()

Click here to open a 39.1 kB text file with autogenerated VB code for a set of Northwind classes suitable for serialization with DCS. (The class code is autogenerated from database metadata by a custom VB project, not by LINQ to SQL.) Notice that the [DataMember] attribute is missing from the last one to three members of several classes.

Added 12/13/2007: The C# version of the DCS-formatted class is here and NCDS-formatted class is here.

Click here to open the 125-KB, 5,012-line XML file generated from 100 Order and related Customer and Order_Detail objects by a test harness that executes the preceding functions.

Added 12/13/2007: The C# version of the three functions that generate the three-tier object graph are here and the VB version is here.

Click here to open its 69-line, easily read XSD schema generated by Visual Studio 2008's schema inference engine. (Schema files are renamed with an xml extension so they open in IE.)

EF Beta 2 doesn't serialize many:1 or 1:many associations. It's up to the programmer to write code to download EntityCollections<T>, which correspond to LINQ to SQL's EntitySets. If the client application depends upon many:1 associations, for example to replace surrogate (usually numeric) foreign key values with readable names, it must request the missing EntityReference. EF hides foreign key values whether or not they are surrogate keys, which makes reconstituting many:1 associations unnecessarily difficult. 

In either case, you must regenerate the associations with code. The entity classes contain a single read-only property, MemberName As EntityCollection(Of EntityType) for 1:many associations. Many:1 associations generate a read-write MemberName As EntityType property and a MemberName As EntityReference(Of EntityType).

All three properties are decorated with <XmlIgnoreAttribute(),  SoapIgnoreAttribute()> attributes and lack <DataContract()> or <DataMember()> attributes, so the WCF doesn't searialize them. Therefore, SOAP clients don't receive the information required for a SOAP request to obtain the entities.

LINQ to Objects simplifies writing the code for client-side reconstructions of associations. However, the code is very "chatty" and thus isn't well suited to service-oriented architecture.

Serializing Complete Object Graphs with NDCS

XML Infosets are well-suited to representing acyclic tree data structures, such as the preceding Customer -> Order -> Order_Detail example, but have serious problems with representing edge-labeled directed graphs that result from the cyclic references (cycles) introduced by adding many:1 relationships. Cyclic references lead to endless loops and an XML document of infinite size. NDCS handles serialization of graphs with cycles by assigning an integer ID value (Id attribute) to each unique element when it's first encountered and then substituting a REF[erence] (Ref attribute) as a pointer to the element when it's encountered again.

Code to serialize the CustomerList graph with NDCS is similar to that for DCS except that DCS specifies the generic class name in the constructor while NDCS's XML document contains the CLR type declaration (System.Collections.Generic.List`1[[NwindObjects.MainForm+Customer, NwindObjects, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]). Here's the NDCS serialization code:

Dim fs As FileStream = New FileStream("NwindGraphDCS.xml", FileMode.Create)
Dim ndcs = New NetDataContractSerializer()
ndcs.Serialize(fs, CustomerList)
fs.Close()

Deserialization code also is similar to that for DCS:

Dim fs As FileStream = New FileStream("NwindGraphDCS.xml", FileMode.Open, _
    FileAccess.Read)
Dim ndcs = New NetDataContractSerializer()
CustomerList = CType(ndcs.Deserialize(fs), List(Of Customer))
fs.Close()

It's easier to view the serialized XML Infoset than to describe its structure, so click here to open the 410-kB, 15,299-line XML file with all 2,626 references preserved. The autogenerated 39.5 kB text file for classes designed for serialization with NDCS is here.

The primary and imported XML schemas inferred for the XML document are available here (203 KB, 7,136 lines) and here (600 bytes). Whitespace added by VS 2008 contributes 90% of the primary schema's original 2 MB file size as inferred; whitespace has been removed from the linked version.

Note: This is a very large schema for a SOAP WSDL file and it uses the xs:import feature that some Web service clients don't support in WSDL files.

Using NDCS for WCS Serialization

Unfortunately, it's not so simple to substitute NDCS for WCF's default DCS on either the service or client component. The Indigo team deliberately obfuscated the substitution process by requiring additional user-written code to enter the serialization and deserialization pipelines, remove the DataContractSerializerOperationBehavior objects from and create and add NetDataContractSerializerOperationBehavior objects to each Operation (see Configuring and Extending the Runtime With Behaviors and WCF's NetDataContractSerializer by Aaron Skonnard.) Autogenerated service proxies also have problems with the service's CLR type declaration for generics (shown above).

Microsoft's party line appears to be: If we can't serialize LINQ to SQL or EF objects in an interoperable manner we'll enforce unidirectional serialization by not serializing many:1 associations (LINQ to SQL) or not serializing any associations (EF). This policy is likely to result in EF sharing LINQ to SQL's failure to provide an "out-of-the-box multi-tier story."

Note: NCDS appears to serialize NHibernate's Bag objects, which provide custom collection types that implement IList<T>. Tim Scott and Greg Banister's Remoting Using WCF and NHibernate post of May 9, 2007 describes a unit test that serializes an NHibernate.Collection.Bag MemoryStream and deserializes it back to the same type. However, this test, which is the in-memory analog of the preceding code snippets, doesn't duplicate WCF serialization. There is nothing in the post that suggests that Bags contain cyclic references. (See Frans Bouma's comment of May 28, 2007 to this post.)

Note: The preceding section was added on November 26, 2007

Probable Patent Coverage of NDCS Serialization

The process by which the NDCS converts a cyclic edge-labeled directed graph to an acyclic edge-labeled directed tree appears to be covered by US Patent 20040239674 "Modeling graphs as XML information sets and describing graphs with XML schema" published on December 2, 2004 and assigned to Microsoft Corp. The inventors are luminaries of Microsoft's early XML Web services efforts: Tim Ewald, Don Box, Keith Ballinger, Martin Gudgin and Stefan Pharies. Here's the abstract of the patent:

Systems and methods for modeling graphs as XML information sets and describing them with XML schema. An edge labeled directed graph is converted to an edge labeled directed tree representing some of the edges directly and some of the edges indirectly. A graph is completely traversed such that all nodes are visited and all edges are traversed. Nodes are included by value initially and then by reference. A schema is provided that describes the structure of an XML tree tha[t] contains graph data.

The patent provides a detailed description of the serialization process with a simple XML Infoset example.

Obviously, intellectual property issues cloud interoperability of services that use NDCS serialization.

Additional References:

Sunday, November 18, 2007

Link and Entity Framework Posts for 11/19/2007+

Note: This compendium is updated daily or more often, depending on availability of articles. Updated November 20, 21, 23, 24 and 25, 2007.

Keith Farmer Takes on N-Tier Patterns with LINQ to SQL

From the "How Did I Miss This" department? (I missed it because Keith has been a infrequent poster—five posts from March to October 2007.)

Keith's LINQ and 3-Tier Dogma post of November 6, 2007 encourages developers not to be "pattern bigots" (my terminology) when it comes to multi-tier LINQ to SQL implementations. He says, undoubtedly in response to Dinesh Kulkarni's that LINQ to SQL Has No "Out-of-the-Box Multi-Tier Story:

The 3-Tier Model is just a pattern. And, like all patterns, It is meant to serve our needs, not the other way around.

He then goes on to examine the issue of the purity of separation of the data and logic layers:

    • Contractual purity (there is a set of well-known interfaces between the tiers)
    • Binary purity (the types are completely separate in each tier, *and* there is a contract between each)

and explains the advantage of abandoning binary purity in favor of contractual purity. He concludes:

The encapsulation of the data context is the creation of the logical layer.

His follow-on Encapsulation and LINQ to SQL post of November 13, 2007 provides a concrete implementation of encapsulating an IQueryable with a TableView class and an abstract class for wrapping a DataContext with a DataContextWrapper class. The article concludes with a sample implementation of DataContextWrapper.

Added: 11/25/2007

Bill McCarthy Excises LINQ to XML's Duplicate Namespace Declarations

Bill's Cleaning up your XML literal namespaces post of November 24, 2007 includes the code for a short utility program that removes redundant xmlns declarations from elements other than the root when you use the Element1.Add(Element2) method.

Added: 11/24/2007

CoDe Magazine Focuses on Data Access with LINQ and the Entity Framewok

The Data Programmability CoDe Magazine Focus Issue of December 2007 includes the following articles focusing on LINQ:

and these devoted to the Entity Framework:

All authors are from Microsoft's Data Programmability group.

Other articles cover SQL Server Compact Edition (SSCE) 3.5 and the new Sync Framework.

Added: 11/24/2007

LINQ to MPI.NET in Development at Phoster Incorporated

Adam Sobieski, a Phoster Incorporated developer, currently is completing development of System.Collection and System.Collection.Generic implementations for MPI.NET, which is a high-performance message-passing interface for .NET. According to his Update: LINQ to MPI.NET post of November 24, 2007, Adam plans to write a LINQ implementation for the collections.

Here's the University of Indiana's description of MPI.NET:

MPI.NET is a high-performance, easy-to-use implementation of the Message Passing Interface (MPI) for Microsoft's .NET environment. MPI is the de facto standard for writing parallel programs running on a distributed memory system, such as a compute cluster, and is widely implemented. Most MPI implementations provide support for writing MPI programs in C, C++, and Fortran. MPI.NET provides support for all of the .NET languages (especially C#), and includes significant extensions (such as automatic serialization of objects) that make it far easier to build parallel programs that run on clusters.

Adam's interest in LINQ to MIT.NET stems from his work on distributed AI applications using C# 3.0 and .NET 3.5. According to Adam's RETE, CEP, ESP post of the same day:

There are two types of machine reasoning popular today: CEP/ESP and RETE. There’s also combinations (Bruno Berstel, Alexandre Alves). I’m thinking P2P distributed applications with RETE/ESP using LINQ programming practices will allow for rapid-prototyping of some sophisticated AI applications. I’m presently brainstorming some developer libraries in this area (Semantic Grid, Knowledge Grid).

It's not yet clear to me how LINQ to MPI.NET would fit in with the ParallelFX program and PLINQ. PLINQ is directed to multi-core processors. My PLINQ Gathers Momentum post of December 8, 2006 (last updated 7/30/2007) includes many links to early PLINQ articles.

Added: 11/24/2007

Frans Bouma Reaches the Ninth Wave of LINQ to LLBLGen Pro Development

LINQ to LLBLGen Pro is coming along nicely using the LLBLGen Pro API to emulate LINQ to SQL but with the API's database-agnostic SQL intermediate language (IL). Frans is dropping the Skip() and Take() methods in favor of the API's pageNumber and pageSize values for paging, bypassing CROSS APPLY for SelectMany(), and implementing GroupJoin() and DefaultIfEmpty().

Frans' Developing Linq to LLBLGen Pro, part 9 post of November 23, 2007 provides the details of the trade-offs in implementing the preceding Standard Query Operators (SQOs) with the LLBLGen Pro API for multiple RDBMSs.

Note: The "Ninth Wave" refers obliquely to Eugene Burdick's prize-winning 1963 novel, which uses the Celtic ninth wave as a metaphor and Northern California as a backdrop.

Added: 11/23/2007

Bart De Smet Releases RTM Version of LINQ to SQOs

Back in the early days of the LINQ CTPs for VS 2005, Bart De Smet created a custom implemention of the LINQ Standard Query Operators (SQOs). His LINQSQO goes RTM post of November 22, 2007 announces that the RTM version of LINQ to SQOs is available on CodePlex.

The source code contains 148 unit tests that you can run with Visual Studio 2008 Pro or higher. This screen capture shows VS 2008 Pro's Test Results window displaying a few unit test results on the Query class library:

Added: 11/23/2007

Regenerating Original Entities from Entity Framework's ObjectStateManager

Neither the Entity Framework's ObjectContext or LINQ to SQL's DataContext is serializable, so you can't access the ObjectContext.ObjectStateManger's or the DataContext.Table<T>'s original values for an entity.

Danny Simmons' Creating an original value object from the separate values stored in the ObjectStateManager post of November 21, 2007 shows you how to create an an entity with all original property values populated from the partial (changed-value(s) only) data in the ObjectStateManager. Presumably, this entity would be serializable for cross-process transport.

Note: Don't confuse the term "value object" in Danny's post time with "complex objects" (a.k.a. Martin Fowler value objects.)

This post relates to Danny's earlier post about serialization depth (see below.)

Added: 11/23/2007

Alex James Joins the ADO.NET Team as Program Manager for Metadata

Alex James has moved from New Zealand to Seattle to join the ADO.NET Team as Program Manager for Entity Framework (EF) metadata. His initial Introductions blog post appeared on November 9, 2007 but the other ADO.NET EF folks didn't post an announcement, so I missed it. (Thanks to Julie Lerman for the heads-up).

Alex added the following EF-related posts after November 9:

Alex's posting frequency bodes well for his participation in the EF community.

Added: 11/23/2007

Danny Simmons on Serializing Entity Framework Object Graphs

Serializing Entity Framework (EF) object graphs to enable multi-tier data access scenarios incurs the same problems that caused Dinesh Kulkarni to admit that LINQ to SQL Has No "Out-of-the-Box Multi-Tier Story, as discussed in my post of October 18, 2007.

By default, the current EF CTP2 doesn't serialize associated entities. This omission requires that the client to make a method call for each set of instances required and then reconstruct the relationships locally. In this January 27, 2007 Entity Web Service: Returning Category thread Danny recommends creating a facade object to contain associated objects. The thread starter uploaded the alpha 0.1 version of a facade implementation to CodePlex.

Binary serialization for EF entities preserves associated entities but is limited to remoting or services on intranets because the payload won't pass through Internet firewalls. Binary serialization creates a deep clone of associated objects; that is, references to shared objects turn into individual copies of these objects. This results in what's called a "data explosion" and very large service/remoting payloads.

Danny's Question: Deep serialization of an object graph--how deep should it go? post of November 20, 2007 discusses three potential serialization options for future CTPs or versions:

  1. Return the entire object graph with 1:many and many:1 associations intact. Such graphs generate extremely large payloads and won't serialize with the default DataContractSerializer of Window Communication Foundation (WCF) because of cyclic references (also called cycles.)
  2. Remove the 1:many associations to prevent conflicts when multiple operations return graphs that include duplicate objects. LINQ to SQL implements this option by not serializing any 1:many associations. This is required to prevent cyclic references.
  3. Specify subgraphs on a method-by-method basis.

To serialize complete graphs with 1:many and many:1 (bidirectional) associations intact requires a rather complex workaround for the Windows Communication Framework's default DataContractSerializer or use of the NetDataContractSerializer.

Note that neither ObjectContext nor DataContext objects are serializable, which limits their identity managers and change trackers to in-process use.

Unless the ADO.NET team chooses one of the three options for EF v1, EF won't have an "out-of-the-box multi-tier story" either.

I'm working on a post with some examples and an analysis of a choice between options 2 and 3. Click here for a preview of the Serializing Object Graphs Without and With References post of November 21, 2007 (updated 11/23 and 11/25/2007) with downloadable files containing examples of serialized XML Infosets and schemas generated by the DataContractSerializer (DCS) and NetDataContractSerializer NDCS. Test with in-memory List<T> collections show that the NDCS serializes and deserializes bidirectional associations correctly.

Added: 11/20/2007 Updated: 11/23/2007 and 11/25/2007

Zlatko Michailov: Compiling LINQ to Entities Queries

Zlatko's previous post, Behavior of Variables in LINQ to Entities Queries of November 16, 2007, recommended compiling queries to remove LINQ to Entities' uncertain treatment of variables assigned literal values as constants or parameters.

Today's brief post, LINQ to Entities: Compiled Queries, explains the compilation process, which, like compiling LINQ to SQL queries, improves performance. Parameters are explicit for compiled queries.

Bill Horst of the VB Team: Converting SQL to LINQ Series

The VB Team posted the first of a series of articles, Converting SQL to LINQ, Part 1: The Basics (Bill Horst), on 11/19/2007. This episode covers basic LINQ query syntax. Future articles will cover LINQ equivalents to these T-SQL keywords:

  • FROM and SELECT
  • DISTINCT, WHERE, ORDER BY, Operators
  • Functions (Scalar and Aggregate)
  • GROUP BY and HAVING
  • Joins
  • UNION, TOP and Subqueries

InterLINQ Provider Alpha 0.2 Serializes Expression Trees

Pascal Schaefer and Manuel Bauer of the Swiss Software Solution Center have posted the Alpha 0.2 release of the InterLINQ library, including sample implementations for NHibernate, LINQ to SQL and LINQ to Objects, and a help file.

InterLINQ serializes the expression tree of a LINQ query so you can transport it across machine boundaries and execute it with a LINQ implementation on a server. It's up to you how to return the query result set to the client. LINQ to SQL lets you mark its partial classes as unidirectionally serializable, so classes with m:1 EntityRefs and 1:n EntitySets lose their EntityRefs because of the circular reference.

Tip: Right-click the InterLinq.chm file, choose properties, and click the Unblock button to read the help pages.

Danny Simmons: Entity Framework Beta 3 Due in a Few Weeks

From an answer to a question by Henrik Dahl in the ADO.NET (Pre-release) forum:

We're working hard to wrap up the last round of testing and bug fixes on the EF and the EF designer.  The plan is to release beta 3 of the EF/CTP 2 of the designer in a few weeks (think "early December").

Thanks to Julie Lerman for the heads-up.

Paul Stovell Explains SyncLINQ for Real and Derived Client State

In his Why SyncLINQ should matter to you post of November 15, 2007 and follow up Three categories of SyncLINQ queries article of November 16, Paul explains how derived state differs from real state in desktop client apps. He als demonstrates how SyncLINQ keeps derived state synchronized with real state as the data underlying real state changes.

Paul's in the process of coding the first SyncLINQ iteration. He writes:

In terms of right now, I have just implemented the first set of aggregate extensions, and I have a basic dependency system working. Over the coming week I’ll be working to implement an operator, and refactor much of the dependency system to support the overloads above.

His Introducing SyncLINQ post of October 16, 2007 includes a seven-minute video clip of SyncLINQ in action.

Note: There's still no library to download.

Mary Jo Foley: VS 2008 Rumored to be Available Monday 11/19/2007

Mary Jo writes "Sources say Microsoft will announce availability on Monday, November 19" in her "Visual Studio 2008 to go gold as soon as Monday" column of November 18, 2007.

eWeek's Daryl K. Taft quotes the MSDN Subscriptions WebLog post of November 16, 2007 in his Visual Studio 2008 Is Imminent post of November 17, 2007 as stating "Visual Studio 2008 is anticipated out early next week, with availability for subscribers." However, on November 18, 2007, the posts first sentence read "Visual Studio 2008 is anticipated out very soon, with availability for Subscribers." I assume the blog author edited the post without revision marks.

Be sure to review Dinesh Kulkarni's two LINQ Project General Forum posts about Beta 2 to RTM changes fo LINQ to SQL:

Jim Wooley suggests:

  • Open the DBML file as XML and change the UTF encoding from 16 to 8. Otherwise the compiler will complain about the DBML file's format.

Update 11/19/2007 1530 PST: LINQ to SQL Beta 2 to RTM Breaking Changes has been published as a downloadable .docx file. Charlie Calvert's Visual Studio 2008 Available for Download post has links to independently published files, such as the C# LINQ and language samples for Visual Studio 2008, which were updated 11/9/2007. The Visual Studio 2008 Samples page has a current list of samples for VB and C#.

Hopefully, the LINQ to Objects, LINQ to XML and LINQ to DataSets teams will post Beta 2 to RTM changes, if any, soon.

Update 11/19/2007 0700 PST: The rumors were correct. Microsoft made the download available but the MSDN servers were throwing  "Server Error in '/' Application. The file '/home.aspx' has not been pre-compiled, and cannot be requested." errors from the entry page for the download: https://msdn.one.microsoft.com/home.aspx?ApplicationID ..., which is not a good sign. Subsequently, the error messages changed to the more common "Server too busy."

Update 11/19/2007 0900 PST: Servers finally became operable.

Update 11/19/2007 1200 PST: Scott Guthrie has posted Visual Studio 2008 and .NET 3.5 Released with a summary of new features and links to related resources. Guy Burstein notes that you can download the Visual Studio 2008 and .NET Framework 3.5 Training Kit, too (120MB.)

Dinesh Kulkarni Writes a Common Base Class for LINQ to SQL Queries

If you want to "encapsulate a pattern and common behavior" across multiple business entities generated by SqlMetal.exe or the O/R Designer, Dinesh's "LINQ to SQL Tips 2: how to use common base class for all entities post of November 9, 2007 (released November 18, 2007) shows you how.

Danny Simmons: Handling Conditional Relationships in the Entity Framework

If your DBA has set up Relationships from Hell as described in Danny's An Interesting Relationship Mapping Scenario post of November 17, 2007 there might be a solution that works with Entity Framework v1, but it's more likely that you'll need to model a containment relationship with v2 (or later).

The many:one relationships Danny describes are defined by foreign key and type column values. The foreign key value points to the primary key of a table that's selected from a group of tables by a value in a type column. Only one many:one relationship occurs at any instance.

Danny describes a containment relationship thusly:

The basic idea of this construct is that "child" entities may be related to one or more "parent" entities but only one parent at a time.

The relationship Danny describes doesn't fit with my understanding of a containment relationship, which is more in tune with that described in the Object Containment topic of class notes for a Database Systems and Structures class at Simon Fraser University.

Tuesday, November 13, 2007

LINQ and Entity Framework Posts for 11/12/2007+

Note: This compendium is updated daily or more often, depending on availability of articles. Updated: November 14, 15, 16 and 17, 2007.

LINQ to Entities Variables May be More Variable Than Expected

Zlatko Michailov points out in his November 16, 2007 Behavior of Variables in LINQ to Entities Queries this behavior of local variables used as constants that's specific to LINQ to Entities:

When a local variable is used in a lambda expression it behaves as a parameter, i.e. it’s value is re-evaluated each time the query is executed; when a variable is used in a static expression, it’s value is picked at compile-time, and then it’s used as a constant.

Zlatko provides sample code that demonstrates this problem.

Entity Framework Beta 3 for VS 2008 RTM will provide a CompiledQuery.Compile((ObjectContext, variable1, variable2 ...) => (QueryExpression)) method that results in a query in which all local variables used as constants retain their compile-time values during multiple query executions.

Looks to me like the hard way to freeze variable values.

Added: 11/17/2007

Zlatko Michailov Explains How to Parse Entity Framework DataReaders

EntityDataReaders with DbDataRecords returned by the Entity Framework's Entity SQL require parsing to determine whether the DbDataRecord represents a CollectionType, EntityType, ComplexType (value object), RowType, or RefType.

Zlatko's November 16, 2007 How to Parse an EntityDataReader post describes a class and methods that "'visit' an arbitrary EntityDataReader to obtain the metadata" for determining the built-in type. ("Visit" is appropriate because Zlatko's code emulates a simple implementation of the visitor pattern.)

The complexity of this process might convince you to take advantage of the object layer and, potentially, LINQ to Entities with complex entity graphs.

Added: 11/16/2007 16:30 PST

Rick Strahl Does C# 3.0 Anonymous Types

Rick's Anonymous Types in C# 3.0 post of November 15, 2007 explains that the anonymous types you create on the fly with the var keyword are immutable and have a scope that's limited to the currently executed method.

Rick didn't mention that anonymous types will infer field names. Writing 
var anon = new {DateTime.Now, SomeField=12};
causes Intellisense to treat Now as the field name.

VB 9.0 also supports anonymous types using With syntax such as:
Dim anon = New With {DateTime.Now, .SomeField = 12}

Added: 11/16/2007

XML Team Publishes LINQ to XML RTM Documentation

The LINQ to XML page points to the following elements:

See Also: Concepts offers these links:

The team also published an interesting interview of Chris Lovett, who's an architect on the Data Programmability Tools team in SQL Server and works with Visual Studio's XML tools.

Added: 11/16/2007

Ben Hall Unit Tests LINQ to SQL with MbUnit and TypeMock

Ben Hall takes on unit testing LINQ to SQL, which RhinoMocks developer Oren Eini says is Awkward Testability for Linq for SQL.

Ben starts his How To Unit Test - Linq to SQL and Mocking post of November 15, 2007 by running tests on the DataContext with MbUnit from a ProductController, then adds a layer called LinqProductRepository between the the DataContext and ProductController, which he test with a live database connection.

He then sets up an interface and a MockRepository with RhinoMocks and runs tests against the ProductController with lists created by C# collection initializers.

Finally Ben tests TypeMock, which claims to be able to mock any object. The  Mocking Linq - Preview post of August 19, 2007 by Eli Lopian, TypeMock's developer, shows how TypeMock can Mock an entire LINQ to SQL statement and return a fake Customers collection.

As usual, Ben provides the source code for this three mocking projects.

Added: 11/15/2007 17:00 PST

Update: 11/16/2007: Eric Hauser's Building a Generic DAO / Repository with LINQ post of November 8, 2007 contains code for an IRepository interface and its unit-testable implementation for the DataContext. The Repository class is a bit more filled out than Ben's.

IBM Announces "Blue Cloud," a Large-Scale Astoria Clone

Not to be outdone by Microsoft's Astoria and its RESTful "data in the cloud" capabilities, IBM announced from Armonk (NY) and Shanghai (China) on November 15, 2007 its "Blue Cloud" technology "will allow corporate data centers to operate more like the Internet by enabling computing across a distributed, globally accessible fabric of resources, rather than on local machines or remote server farms."

The press release names a single customer, the Vietnamese Ministry of Science and Technology, as having "this week announced a cloud computing project with IBM." Reading below the fold shows that the project is a simple Internet portal.

The way I read the tea leaves, corporate data centers presently are allowed to do what Blue Cloud purports to enable. I also question whether science and technology in Vietnam has reached a scale that requires giving "citizens, communities and government organizations" access to this "rich content source" by "a distributed, globally accessible fabric of resources." A 1U Dell server with ASP.NET and Astoria would probably satisfy their needs.

According to the press release, "IBM’s first Blue Cloud offerings are expected to be available to customers in the spring of 2008." I doubt that IBM will need to hire extra security folks to quell the crowds surging to sign up when Blue Cloud RTMs.

Added: 11/15/2007

Architectural Point of View for LINQ-Enabled Data Access Video

Architectural evangelist Bill Zack delivers an 11:47 Architectural Point of View (arcPOV) presentation on LINQ to SQL, LINQ to DataSets, and LINQ to Entities. Here's the Channel9 description:

Bill Zack presents an Architect Point of View (arcPOV) presentation on selecting the right LINQ model for your application. Bill covers 3 different flavors of LINQ; LINQ to SQL, LINQ to Datasets, and LINQ to Entities. Bill discusses how these flavors of LINQ can be used to access data in relational databases. He then covers the differences and the trade-offs involved in choosing between them.

Added: 11/15/2007

NLinq for .NET 2.0 Beta 2 Released

NLinq is an open-source (MIT-licensed) implementation of LINQ to Objects that runs under VS 2005 and .NET 2.0. NLinq accepts LINQ queries in a literal string, so you can emulate dynamic queries. It's not clear who needs this functionality but if anyone does, the source code is available from CodePlex.

Added: 11/14/2007

LINQ to Lucene 2.1 Query Version Released

LINQ to Lucene, an open-source LINQ implementation for the open-source Lucene.Net 2.0 full-text search engine reached release 4, where it's capable of querying Lucene indexes, on November 13, 2007. The source code is based on Visual Studio 2008 Beta 2, licensed under the Microsoft Permissive License (Ms-PL, no version number) and contains Unit Tests for the MSTest implementation that's included with VS 2008 Professional Edition.

According to Locksley, the project's coordinator:

The goal of LINQ to Lucene is to provide developers with the ability to enjoy full-text searching using a fast-proven search-engine within the .Net managed CLR. The plan is to get the LINQ to Lucene code-base pushed up to CodePlex for additional developers to generate feed-back and improve and enhance.

Lucene.Net 2.0 is "a source code, class-per-class, API-per-API and algorithmatic port of the Java Lucene search engine to the C# and .NET platform utilizing Microsoft .NET Framework."

Note: Dare Obasanjo reported in his RSS Bandit v1.5.0.17 Released post of September 16, 2007 "a ton of stability issues related to our usage of Lucene for search engine for our full-text search" feature in the RSS Bandit reader. This is not to say that LINQ to Lucene is unstable but it's something to watch for.

Added: 11/14/2007

Ben Hall's Unit Tests for LINQ to XML RSS Reader

In his How To Unit Test - Linq To XML post of November 13, 2007, Ben Hall demonstrates unit testing of a simple LINQ to XML RSS Reader project with MbUnit, an open-source "generative test framework." Ben hall is an MBUnit project commit member.

Sample code is available for download.

Added: 11/14/2007

Optimize LINQ to SQL Performance

is the cover story for Visual Studio Magazine's November 2007 issue. Here's the deck:

Persisting business objects to SQL Server 2005 or later with object/relational mapping with LINQ to SQL can exact a performance penalty. Learn how to take maximum advantage of LINQ's strongly typed query capabilities without overtaxing the database server or losing front-end agility.

The story has tables of relative performance data for lazy versus eager loading of entities of varying sizes, with and without associated EntitySets. In all cases, DataSets beat LINQ to SQL times to load DataGridView controls via BindingSource components. The sample test harness code is available for downloading. 

You'll notice that 1105 Media's Redmond Media Group (RMG) has updated the magazine's site design, which looks 20 years younger than it's FTPOnline predecessor.

The article also is a cover story for RedmondMag.com (November 2007).

Added: 11/13/2007

Scott Guthrie's Model-View-Controller Framework Tutorial uses LINQ to SQL

Scott's detailed ASP.NET MVC Framework (Part 1) post of 11/13/2007 describes the ASP.NET team's forthcoming Model-View-Controller (MVC) Framework for Visual Studio 2008 and .NET 3.5. His sample project is the starting point for a simple e-commerce storefront site based on the Northwind Products table.

The post, which I estimate is at least 15 feet long, describes MVC architecture and its benefits, especially separation of concerns, and then continues with a detailed description of building the MVC Framework starting with the ASP.NET MVC Web Application template. The template adds Models, Views, and Controller folders to the main project and a ProjectNameTestProject folder for adding unit tests of the controller with MSTest. (VS 2008 Profession Edition includes the MSTest client).

Scott then derives a ProductsController class from the Framework's new System.Web.MVC.Controller class, which contains helper methods and added controller functionality. For this tutorial, Scott uses LINQ to SQL as the object/relational mapping (O/RM) tool and data access layer (DAL), although he says:

The ASP.NET MVC Framework enables you to use any data access pattern or framework you want in order to retrieve and manage your models.  If you want to use ADO.NET DataSets/DataReaders (or abstractions built on top of them) you can.  If you prefer to use an object relational mapper (ORM) like NHibernate, LLBLGen, WilsonORMapper, LINQ to SQL/LINQ to Entities you can absolutely use those as well.

It's interesting to note that Scott didn't mention the SubSonic DAL and persistence layer. Rob Conery, SubSonic's creator says SubSonic will be the "Convention-Driven Toolset for Microsoft’s New MVC Framework" for ASP.NET and Rob went to work for Microsoft yesterday.

Scott mentions that the new System.Web.IHttpRequest and System.Web.IHttpResponse interfaces will be easy to mock. It will be interesting to see how his team mocks LINQ to SQL's DataContext, especially for shopping cart updates.

Overall, it looks to me as if the ASP.NET team has come up with an excellent MVC framework starter edition, but it seems to me that it needs SubSonic's (or someone else's) scaffolding, migrations, and RDBMS-agnostic persistence layer to compete with Ruby on Rails.

Added: 11/13/2007

Update 11/14/2007: Mike Taulty's tempted to use the MVC Framework to create a RESTful Web service. In today's ScottGu on ASP.NET MVC Framework post, Mike says: 

What I see is a framework that might very nicely help me build a RESTful service. It does a nice job (and presumably a very flexible job) of routing URLs and query strings through to my classes and then we go get some data and invoke the view. In Scott's case, the view is an ASPX page producing HTML but I might equally want to invoke something to produce me some XML or JSON data.

I'm not sure whether the framework comes with a mechanism for routing through to ASMX or not but, either way, it looks like it would be easy to take this and make use of it in a service context.

Interesting idea.

Dot Net Rocks! Interviews Pablo Castro About Astoria

Show #289 pits the Technical Lead of the Entity Framework and Astoria against Carl Franklin and Richard Campbell in a 72-minute interview described as follows:

Pablo Castro from the SQL Server Product Group at Microsoft talks about his work with Astoria, an infrastructure for bringing web technologies and data sources together. Astoria uses the Entity Framework, which Pablo is also involved with.

The question of the week: Has Pablo had his picture taken since High School?

Added: 11/13/2007 17:00 PST

Surprise: Don Box Likes VB

Yes Steve, I've Tried (11/12/2007): Personally, my dream stack would be ubiquitous WS-Security/WS-Trust over HTTP GET and POST and tossing out WSDL in favor of doing direct XML programming against payloads from VB9 (or XQuery), but hey, I have unusual tastes.

Lambda in VB (11/11/2007): I'm starting to write more VB code lately. The XML support rocks.

More importantly, a huge amount of people who use the technology I work on see it through that lens, so I'd be an idiot not to walk in their shoes as much as possible.

Query syntax in VB (11/12/2007): "One other feature I'm liking in VB is that you can omit the trailing [S]elect clause in a query."

Thanks to Beth Massi for the heads-up in her Don Box on Visual Basic 9 post today.

Added: 11/13/2007

Bart De Smet Explains the Origins of LINQ to SharePoint

LINQ to SharePoint Implementation is a five-minute interview video from day 5 of Tech*Ed Developer 2007 (Barcelona) carries the following description:

Bart De Smet, Software Developer Engineer, explores LINQ (Language-Integrated Query) - which is one of the key features in .NET 3.5 Visual Studio 2008.  LINQ allows you to write queries inside your favourite language – C#3.0 or Visual Basic 9.0. Bart explains the reasons for a running a recent LINQ to SharePoint implementation project.

Another reason for Bart writing a LINQ to SharePoint implementation might be its current and future ubiquity. According to Joe Wilcox's report, Ballmer: Advertising Is the Future, on the Microsoft shareholders' meeting held today:

Gates spoke about future investments and made absolutely clear that SharePoint, with "over 100 million customers," would anchor Microsoft's server applications portfolio. He likened SharePoint to Office, asserting that SharePoint would similarly take a "central position as a key tool."

Bart previously joined Glen Gordon and Lynn Langit for a geekSpeak Webcast on Wednesday, October 31, 2007. Here's a link to the LINQ to SharePoint code.

Added: 11/13/2007, Updated: 11/13/2007 15:00 PST

Beth Massi Fixes Her LINQ to XML VS Tip of the Day Browser App

In this brief Merging XML using Visual Basic 9 post of November 13, 2007, Beth discovers that MSDN blogs' Rss.xml files only contain the last 15 posts, so she adapts the LINQ to XML code to merge the current content with that stored on disk to maintain a complete archive.

The original code is in her Visual Studio Tip of the Day Browser post of a week ago.

Added: 11/13/2007

Mike Taulty: LINQ's Not a Fit with SQL Server's SQLCLR

I'm not sure why one would want to write an SQLCLR stored proc, function, or whatever that uses LINQ to Something queries, but Mike says it can't be done in his LINQ and SQLCLR post of November 13, 2007.

Mike's attempt to load System.Core.dll in SQL Server failed SAFE verification and he knows of no plans to make LINQ run with SQL Server 2008's SQLCLR feature.

Added: 11/13/2007

Mike Taulty Draws a XAML Sine Wave with LINQ to XML

Mike was more successful in demonstrating that you can write XAML with LINQ to XML. 

The few lines of code in his November 13, 2007 Drawing a XAML graph from VB post draws a white sine wave on a black canvas.

I would have been more interested to see the code for the "thing" he did at Tech*Ed Developer in Barcelona last week.

Added: 11/13/2007

Updated 11/14/2007: As Mike notes in a comment, the source code for the "thing" (LINQ to XML demo) is available here. (Also, I corrected his misspelled name in these two posts. Sorry, Mike.)

Rick Strahl Explains Type Initializers in C# (in Detail)

Rick gave an attendee of his LINQ to SQL session at Dev Connections a bum steer on "what happens behind the scenes when you use type initializers in C# 3.0." 

To set the record straight, Rick posted Type Intializers in C# 3.0 on November 12, 2007 with examples of how the compiler translates a type initializer and what an anonymous type looks like in MSIL.

For more on his Dev Connections sessions, see his My DevConnection Fall '07 Session Slides and Sample Links post or the "Rick Strahl Posts Sample Code for His DevConnections LINQ Presentation" topic of my LINQ and Entity Framework Posts for 11/5/2007+.

Added: 11/13/2007

Charlie Calvert Describes Partial Methods (in less Detail)

LINQ to SQL's autogenerated code includes CRUD partial methods for each table so you can substitute stored procedures for the dynamic SQL generated from the LINQ query by the  expression tree.

In his Partial Methods post of November 11, 2007, Charlie provides an example of a partial method and lists the rules governing use of partial methods.

Added: 11/13/2007