Why Mashups = (REST + ‘Traditional SOA’) * Web 2.0

by Sherif

Mashups. It’s a term thats been heavily used in the Consumer and Web 2.0 space, however has been lightly thrown around in the enterprise space with claims that its lacking standards and maturity.

For a long time, I’ve been thinking where this is all headed. There are two main conclusions I would like to draw at the end of this:

1) In terms of how this is going to affect IT in the enterprise, I think whats going to happen is we are going to see two types of ‘developers’ emerge as a result of what is happening in the Web 2.0 space. The ‘Mashup Developer’ (which generally might be a business user, or a slightly tech-savvy person), and the ‘traditional developer’ (your true nerd). However, it is important to note that you will still need both!

2) In terms of how this is going to pan out and the Maturity of Mashups – I think ‘RESTful Web Services’ will play a critical part in boosting the state of Mashups, and this is yet to come a long way. As this happens, I think you would start to see a trend – in the increase of Enterprise Web 2.0 adoption. At present, I think we are a very long way off this.

Let met take a few steps back so we are all on the same page…

Mashups
What are Mashups?
I’d point you to Wikipedia for this, the source of all truth :-), but to put it in simple terms: A Mashup is a new service, that combines functionality or content from existing sources. These existing sources can be Web Services (through the use of API‘s), RSS feeds or even just other Websites (by screen-scraping).

What are some examples of Mashups?
Mashups on the haven’t really been around for too long. A quick trend analysis on Google, and you will see, they have only really come into the scene last year:

Google Trends - Mashups

However, there are some excellent examples Mashups today, I’ve listed some of my favorites below:

  • chicagocrime.org – This was one of the first to come out. It displays a website with crime stats in Chicago and maps then on a map. It’s very detailed and lets you even drill down to the crime type, and the exact street that crime was committed on.(Police Stats + Google Maps)
  • housingmaps.com – Displays hose prices for various states in the USA (Real Estate + Google Maps)
    feedmashr.com – The most popular content from various websites (Digg, del.icio.us, reddit…)
  • flickrvision.com – Google Maps + Yahoo! Flickr Photos to display live photos of peoples location around the world as they upload.

As you can see Mashups produce some amazing results, especially when it comes to trend analysis or getting the best content you can from various sources. What more examples? Checkout the Mashup section on Programmable Web Blog.

Some would argue the concept of Mashups has been around for years – and this ‘hype’ over Mashups is over the top. For years we have been getting data from other systems and combining data with the existing systems to produce a result. This is true, however, I think the hype over Mashups over the last year or so has clearly been on the ease of end-users producing applications themselves. This is generally done through modern-day Mashup editors (which I will talk about below), and not the olden way of SFTP/FTP-ing a file then doing some sort of transformation and loading of data through scripts and other mechanisms.

So yes, Mashups have been around for years. However, the concept of the end-user being able to easily ‘drag-drop’ and put together a hacked up application within minuites hasn’t – and this is what its all about.

Mashup Sources
1) API’s: REST vs SOA? or SOA + REST?
As we mentioned earlier, mashups is basically a mix of various data sources. Lets start off firstly with API’s. Because API’s expose system functionality to other systems and users, they play a critical part in the whole mashup space.

Service Oriented Architecture (SOA)
Lets start off discussing the Service Oriented Architecture (SOA). When I refer to ‘traditional SOA’ I mean (web services through such things as SOAP, and the older XML-RPC protocols). API’s generally require the user to make Web Service calls from their application passing through relevant parameters, and getting results based on what was passed through.

Almost every Web 2.0 startup wanting to expose its functionality to external users or other developers would have some form of API available. Examples include the Facebook API, Google Maps API (and the many other Google API’s) as well as the dozens of other API’s available for hundreds of sites.

programmableweb.com: Top API’s for Mashups

In general, API’s which work on what I called ‘Traditional SOA’ (IE SOAP/XML-RPC) provide a very powerful way to interact with other services programmatically. However, this does come with a down side – its generally quite technical and requires developers who have a good understanding of protocols and programming languages. How does this fit into Mashups? Well, whilst it can be quite technically complex to interact with other sites for the average business user. The introduction of Mashup editors try to do all this hard work for you and give you a drag-drop feel to make things a lot easier.

Therefore, Although traditional SOA does play an important part in the Mashup equation, it must work well with Mashup editors and other architectural styles which are simpler…. (In comes the RESTful Architecture style…)

RESTful Architecture
In comes Representational State Transfer (REST). REST in essence, is an an architectural style for implementing web services. This is done over the HTTP Protocol (without SOAP).

With a ‘RESTful Web Service, each service is easily identified by a unique URL (yes, the actual website address). Essentially, when the server receives the request, it knows what to do based on the URL path. Whats an example?

http://www.myonlineshop.com/product/12345 (the Restful way of doing things)

In essence the URL now is a noun not a traditional verb:

http://www.myonlineshop.com/getProduct?productID=12345 (this is the traditional way we did things)

All that would change now is the HTTP action parameter. This would basically equate to the database transaction or (CRUD Action - Create, Read, Update or Delete) that is wanted to be performed:

HTTP Method CRUD Action Description
POST CREATE Create new resource
GET RETRIEVE Retrieve the resource
PUT UPDATE Update resource
DELETE DELETE Delete resource

RESTful web services have open been mentioned as alternatives to 'Traditional SOA' (SOAP / XML-RPC), however, I believe they need to work hand-in-hand to provide a mix of sources for Mashups (and Mashup editors) .

With web services that are generally SOAP-based, the request and response are hidden. SOAP requests must be interpreted as they are received at the server to determine the operation to perform and the arguments required to perform that operation. They are generally passed through as a parameter, which is essentially a function/method call. SOAP does not currently provide a mechanism for caching results, where as REST can (through the standard HTTP caching).

Therefore, the REST service architecture shows itself as a much more simpler, more standardized in terms of CRUD-style applications and is very similar to the HTTP protocol. Something maybe an average End-user could understand?

Same result, different views
What I am trying to conclude at is that you will need both for effective mashups. Essentially, the Traditional SOA way of doing things and the RESTful way of doing things would both yield similar results. However you might find that the Traditional SOA would allow you to do more powerfull things (and as a result is more complex) and the RESTful oriented architecture would allow you to easily do simple things (and this being much easier to implement)

REST becomes an architecture style applicable to systems that would want operate with a Resource view.
SOA, on the flip-side, has a different view of the web. It uses the web as a application infrastructure between service providers and consumers otherwise known as the Service view.

SOA and REST are architecture styles applicable to different application domains. It is not useful to argue which one is better without knowing the target application. REST works well with distributed resources and SOA does better with distributed services. Resource interactions are different from, but in general simpler than, service interactions.

To get the best result - you need both. Some situations require a RESTful architecture, some require a 'Traditional SOA' architecture, and some might even have both!

Mashup Styles

2) So we've got REST + SOA, how does Web 2.0 fit into this?
When I reffer to Web 2.0 in this article, I am generally referring to the social networking space as well as the features and services provided by these sites. Two main things I want to draw out of the 2.0 space:

A) Social Context is the next big thing.
For mashups to really take off, we need to be able to capture the context of information. A lot of people in the enterprise would often say that the Enterprise is suffering from information overload. Information becomes relevant, and more useful when its placed in the right context. I strongly believe that if a Mashup can leverage of some form of context, it would then be able to provide the relevant information to the user.

There is a lot of hype around Facebook lately. If you ask people why they would usually say due to the amount of users its got - a hell of a lot!. However the power of Facebook really comes into play once they can tap into the social trending. How much more effective could advertising be if the advertiser knew who the 'trend setters' were, and targeted them directly? With Facebook, you can quickly see if a user adds an application, how the effect would filter down on his/her friends.

Now Facebooks insn't necessarily going to be the best example to draw information from for the Enterprise. However, what If you applied the same theory to something like LinkedIn. Lets use the example if you were a HR recruitment person, using Linked in to find people for a Job. Imagine the power you could have if you could create a mashup to find out who in the employment business could change companies, and draw two or three other people to come with them. From that social context you could quickly see which people in a company would be much more valuable in terms of their people 'pull' factor.

B) RSS Feeds
One other thing we can draw out of the Web 2.0 space is RSS feeds. RSS gives us the capability to extract content from a source, in a neat little way and allow us to also stay updated on a regular basis. Mashups will work well when mixed with both Web services (the two styles I mentioned above) as well as RSS feeds to obtain content that might not be as-dynamically updates as the type of content you would normally get form an API service.

The Mashup-world today
So where are we at today?
Mashup Tools & Editors
There are plenty of Mashup tools and editors that really allow users to start playing around with this stuff. It's all still very primitive at present, but will grow into something significant over the coming years. Below is a list of Mashup tools and editors I have played with on the net:

  • Microsoft Popfly: Microsoft Popfly is Microsoft's Mashup Editor.
    I've played around with it and I'm quite impressed. It pre-gives you a whole bunch of web services already available. And all you have to do is 'drag-drop' content from one source to another. Its very useful to an end-user and requires little technical understanding.
    I thought I'd give it a go. Within minutes I was able to pull my friends avatars from Facebook, and display them in a collage via a few clicks. It's pretty cool. Popfly is built on Microsoft Silverlight (it's try-hard Flash equivalent). Microsoft's Steve Ballmer describes it as: "(Popfly is) a (mashup) tool for end users, not necessarily codeheads."
  • Yahoo Pipes!: Yahoo Pipes is Yahoo's flash-based tool to
    Yahoo Pipes 'aggregate, manipulate, and mashup content from around the web'. Yahoo Pipes was one of the first mashup editing tools to come out. It appears to be targeted to the slightly more technical people. However it has a 'drag-drop interface'. Its quite easy to use, I've used it myself to pull content from two various sources and create a mashup - was pretty impressive. Checkout a simple video to get started:
  • Google Mashup Editor: The Google Mashup Editor (GME), is defiantly the most advanced out of all the ones I have seen. It's got a whole stash of documentation, and defiantly the 'least user friendly' - that generally means its the most powerful, and it is. Its basically got a tag based markup language, that lets you also embed HTML into your results.
    Its very 'componentized' too - Java developers would love this. For example, lets say I want to create a mashup to read content from a site:
    1) Firstly I would define my data source(s) via a ' 2) Then you would define your result set similar to a tag-based result set for a set of JSP's.
    3) Then you could mix and match data from other sites/sources/services.

There are many other tools out there, but those are the main ones I have come across.

Is this the end of Portals?
When the whole concept of Portals first come out, the whole idea was that you could draw content from various sources using little Portlets, and personalize it to what you want.

Is this the end of Portals? No, I don't believe it is. In fact, look at the success of iGoogle (aka Google Start Page) which is essentially a Portal platform that lets you add and configure your own widgets/gadgets (aka Portlets) to do what you want.

I see portals providing a platform to leverage off these services. The example above of iGoogle is one, another would be Facebook. If you think of Facebook as a platform, and all the widgets/gadgets as what makes the overall content, you have a mashup. Through a portal-like interface you have brought in content from many sources: Web Services (SOAP, XML-RPC), REST, RSS Feeds and even Screen Scraping.

So in summary, portals are staying, if anything they will be come richer with more composite content from many sources.

The end result
Mashups in the Enterprise
I think the consumer space has already embraced Mashups, and we have seen this through the large amount of mashups available today on the web (just Google it!). However when it comes to Mashups in the enterprise, it's still got a bit to go.

I often refer to Masups as the modern-day 'Macro'. Let me explain: Often in the enterprise you would find a lot of business decisions being made from a whole bunch of Excel spreadsheets, Access databases and macros (sad, but true). Most of these macro's would have been developed my business users themselves, rather than the full-on tech IT Programmer. Why? Because it was simple enough to get by, and gave them just the right amount of information they need. Not only that, but it was the cheaper solution than engaging IT - and why not if they can do it themselves? This is still the case today.

As business start to embrace mashups more, I believe mashups will play this similar role of a modern-day macro. Additionally, it will allow them it to obtain much more detailed information from their various sources (rather than a bunch of spreadsheets), in a much richer format (through combining such things as videos, images, spreadsheets, and other sources).

Will this give end-users critical data which they can make key decisions off? Probably not, I think it will just play a small factor in decision making. At the end of the day I believe mashups in the Enterprise, will just be another tool which business users can utilize that will help them make decisions by analyzing and comparing data from various sources.

The need for the mix: (REST + Traditional SOA) * Web 2.0
In summary, you need the mix before Mashups really take off. I believe both REST and 'Traditional SOA' will work together alongside all the other various ways of getting content on the web to achieve this.

Traditional SOA Web Services + RESTful oriented architecture Web Services + RSS Feeds + Social Context + Screen Scraping = Good Mashups!!

The results
Show easy its going to be? Well I showed you an example above -Flickr Vision. Look how easy it is to create something like this through a Mashup Editor. Microsoft Popfly gets photos from Flickr and hooks them to Microsoft Virtual Earth (the equivalent of Google/Yahoo maps but Microsoft):

This is done in 30 seconds! That my friends is my Mashup rant.