There’s been a lot of talk, momentum and questioning for where and how occasionally connected designs fit into the application landscape.  There have been a lot of questions for “if and how” developers can take their existing apps “offline”.  There have been a number of efforts for enabling offline apps and some longer term plans.  In VS 2008 (Orcas) we introduced Sync Services for ADO.NET as a developer focused way to synchronize data.  The roadmap continues with the Microsoft Sync Framework releases.  We’ve started to show “Astoria Offline” as a means to enable sync over REST based HTTP protocols enabling more generic services.  But do any of these “solve the problem”.  It’s been a while since I’ve done a brain dump on some thoughts related to the evolution vs. revolution of app models, so what better way to dump thoughts than a blogicle. 

Can you take your current app and make it offline?
The reality is “it depends”.  There are so many app designs, it’s hard to say definitely what works and what doesn’t.  There are some well established things that have worked and haven’t, but it’s not fair to say these apply 100% of the time.  One size doesn’t fit all.

Intercept/fall back
Do you go with the intercept model?  When online and connected, the app communicates directly to the service.  (Note I’m using “service” here to represent any remote source.  It could be a database “service” a Web Service, a WCF service, or some other API that’s only available when your device is connected to “the network”).  When the network falls away, the app “falls back” to a local “cache”.

This is generally a bad (opinion), or complicated (fact) model.  This assumes one of two things.  Either your user knows they’re going offline and they synchronize before they yank the cable or they predict when the network fails.  Of course the application can constantly synchronizing data in the background.  Crystal ball? When do you really know or have time to sync beforehand?  You’re working in your office, and Outlook pops up reminder.  Or worse, you get an IM or Text message asking “where are you”.  You grab your laptop and run… oh, wait, you forgot to sync.  Do you have the info needed for the meeting? 

For the background sync, one of two things could be happening.  When you “fall offline”, your view of data suddenly changes because your local copy is so out of date compared to the data you were just looking at, or your synching often enough that there’s no difference, asking why you’re even trying to work online in the first place.  Or, you’ve effectively created a denial of service attack by asking, has anything changed, has anything changed, has anything changed?

Outlook 2003 (I think that was the version) tried this fallback model.  It was awful.  You’re sitting in your office, writing an email, looking at your inbox.  The network goes down, your inbox view completely changes, or worse Outlook froze, and you lost that message you were carefully wording.

 

Always work locally
In this model, you synchronize everything you need, and make all your interactions against the local store.  This is sort of the Outlook model today (Outlook 2007).  There are equal problems here.  Is it realistic to have all the data locally?  Even Outlook data can be significantly large, and that’s clearly your data.  What about public folders?  By default, these are only available online.  If you’re online, they work.  If you’re not, they don’t.  Fairly straight forward; but what about historical data for all the products the company has shipped?  What about your list of customers, or the current list of stocks and their prices? 

The reality is data volume is growing at an exponential rate.  Storage continues to get cheaper and faster.  1TB disks and Solid State storage are becoming mainstream.  The problem is while connectivity is getting better; the bandwidth isn’t increasing at the same rate.  And, even as it does, is it realistic to have everything local?  Do you keep one of everything that Home Depot or Safeway keeps in stock?  Would be really nice, but not all that practical.

One size fits nobody
The reality is there’s no easy answer.  This is really an architectural and user model change is required to make this happen.  We’ve been through these before.  In the “old times”, families ate what they grew.  Corn, milk, meat, etc.  Others only had Ice or Milk that was delivered to their house daily.  You could think of this as the “online model”.  Today, we go shopping to Safeway, D’Agostino, or your local grocery store.  The stores don’t grow anything locally.  They stock goods they get from somewhere else.  You take stuff home and stock it in your pantry, frig, freezer.  When you throw out the garbage it goes into the can.  The can gets picked up on Thursday.  It gets taken to the transfer station, and then off to the dump.  The point is we have lots of “store and forward”/sync models already in place.  What’s really interesting is the “online” things are actually more expensive.  It turns out it’s cheaper to stock goods that may be used, and accept a percentage of loss for things that don’t sell than it is to create on demand.  Why do we think electronic bits and bytes should be “online” only?  Why are 1’s and 0’s more expensive than physical goods?

Evolutionary approach: Mapping concepts
If we look at the concepts we have today in web/service/stateless models, they break down into two common, and one advanced models:

Get
Get maps very easily to HTTP:Get.  Essentially you make a request to a domain, application and method.  You likely pass parameters and of course all the authentication goo.  The result of the get is returned and you now have a copy of information relevant to that point in time, and the specific question you’ve asked.  Most likely, but no guarantee it’s accurate by the time you quickly display it on your 64bit, quad laptop with a gigabit connection to the internet.  Even historical data is subject to “correctness”- remember the Florida elections J  But, we live with “copies” all the time.  Copies are tangible, things that are “just there”.  There’s a separate question of how “out of date” the copy is.

Submit
You have some information you’re sending back.  You expect some action to be taken on the data you’re submitting.  That submit can be accepted, rejected or modified.  You get an immediate response, but no guarantee what you’ve submitted will be accepted.  That might happen later on, it might not.  You hope, or expect to be notified if something changes.  But is that concept built into our app models?  Or, do we just assume the world is globally connected, and everything is always running so things can be asked and answered immediately, with 100% accuracy, in one big transaction so no two people can allocate the same “goods”…. Whew, that’s as ridiculous to assume as it is to type.

Can Get and Submit evolve?
With Get, you can implement a “caching” model.  Rather than Get directly from the source, imagine you Get through an abstraction layer.  For each Get the abstraction layer keys off the domain/app/method and name/value pairs for parameters.  Let’s say I make a service call to: http://AcmeServices.com/Books/GetBookByName.asmx.  I pass in:  Title=SQL Server Compact 

In the GetCache, it looks for the service and name/value pairs that match.  If it finds a result, it may not have to actually make the network call.  It could just return the cached result.  But what about stale data?  You can put an expiration policy on the cache.  For a list of states, a day is likely more than generous.  For pricing, maybe an hour.  The point is you can customize this per source.  For many things, one could argue most queries, the data is fairly static and the app could benefit immensely by caching the “gets”.  If, however, you were to look online first, than only if the resource wasn’t available look in the local cache than what benefit are you getting?  Sure, the app would “work offline”, but you don’t see any benefit in the online mode.  Each request still takes the time to roundtrip.  If you implement a smart caching model you can provide dramatically faster user responses. …and in most cases, the data is perfectly accurate.  Now it would sure be nice to know when something changes, so if California does sink, Iraq or Canada become the 51^st state, you don’t have to watch CNN and ask all your clients to either wait for the cache to update, or “invalidate” the cache.  I talked about notifications a bit here, so I’ll keep going.

Submit
This one gets interesting as well.  You have an order to submit for some widgets.  Sure, you can submit directly to a “Web Service”, or web page.  But in many situations the order submission doesn’t actually mean the order will ship.  Accounting needs to verify your credit.  The products that were supposed to be in stock may have been routed to a bigger customer, or “fell off the truck”.  Again, the point is just because you may get an immediate acknowledgement the order was accepted, doesn’t mean it will be fulfilled.  So, why not accept this as the model, and when it works, it just works.  When a problem occurs, it’s not really a problem because it’s built into the system.  What if you “queued it up” for sending?  In this case we save the message locally, in a “logical queue”, and when the service is available, we send it on its way.  I used the term “logical queuing” as I’m not implying you must use something like SQL Server Broker, MSMQ or WCF Queues, etc.  These tend to be a bit over complicated for what you need on the client, and it makes it difficult to act, query or evaluate the data in its transient state.  A while back I wrote a blogicle on and some samples for Logical Queuing.

Sync
This is what some might assume is the answer to all of these problems.  If I can synchronize changes from other nodes (the server and other clients) with my local store, and I can make changes locally and have them synchronize back “up” to the other nodes, doesn’t that solve everything?  In the land of demos, sure.  But in the real world there are just too many complications.  The volume of data, requirements for business logic, and workflow make it difficult to assume Sync can solve all the problems.  Where sync does do really well is for synchronizing reference data.  That list of states, codes, or even larger volumes like the product catalog.  The Logical Queuing blogicle and sample demonstrate some thoughts here.  I recently saw some work the Microsoft Mobile Line of Business CRM team, and it also uses a similar model.

Revolution is on the horizon
With all the complications and half way solutions, you have to wonder what’s the real answer to all this mess.  Will this problem just go away because we’ll have high speed internet available everywhere?  Even if you could be guaranteed internet access everywhere, do we think that every service will be available all the time?  As the world gets more connected, will systems be more reliable and capable of enlisting in massive distributed transactions?  Or, will computing systems evolve to what every other large scale system as evolved to. Queuing, Caching, Reserving, Buffering, Acknowledgement, Conflict Detection & Resolution and Notifications. 

While I talked a bit about the food supply earlier, this is also a bit like the VB Classic to Web transition.  For those old enough to remember the great 90’s there was a big transition from “fat clients” to “thin clients”.  No subtle suggestion of which is better there J  When developers first started building ASP Classic (pre .net) apps they simply moved their session state to the ASP Session object.  Which meant web servers could serve about 5-10 people before falling over.  Not exactly the promise we had all hoped for.  It took a while to realize the app models had to change. But, moving from Client apps to Web apps was a bit more obvious.  Mostly because the UI changed (for the worse), but at least users could get their work done because we couldn’t get the VB apps deployed.  (yes, I’m over simplifying things, but run with me for a minute). 

Since developers had to re-write their UI anyway, it was an obvious change.  Making the transition from online to offline mode apps isn’t as obvious.  The only UI required is the UI really telling them when problems occur.  Who wants that? 

The reality is things are a changing.  Nothing to announce, nothing to promise, but there is a revolution on the way.  Revolutions take time for people to live through denial that it’s coming, but it comes as a wave of reality that just smacks you with “duhh, I guess that was a stupid assumption”.

While we all wait for the revolution, you can look at things like ADO.NET Sync Services, the Sync Framework, Astoria Offline, and lots of goodies the Patterns and Practices team has done.

Steve