Thursday, May 22, 2008
Querying Google Base using GLinq in C#
The GLinq project is a beta project that provides strongly-typed access to Google Base. Check it out at: http://www.codeplex.com/glinq.
Once you download the beta, you can run the sample application, which consists of this code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace ConsoleTest
{
class Program
{
static void Main(string[] args)
{
//You need a google key. It's easy - http://code.google.com/apis/base/signup.html
Console.WriteLine("Enter your Google Base Key (http://code.google.com/apis/base/signup.html):");
string GoogleKey = Console.ReadLine();
GoogleItems.GoogleContext gc = new GoogleItems.GoogleContext(GoogleKey);
var r = from ipods in gc.products
where ipods.BaseQuery == "mp3 players" && ipods.Brand == "apple"
where ipods.Price > 200 && ipods.Price < 400
orderby ipods.Price descending
select ipods;
foreach (GoogleItems.Product product in r.Take(100))
{
Console.WriteLine("{0} for ${1}",product.Title, product.Price.ToString("#.##"));
}
Console.ReadKey(true);
}
}
}
This returns a big list of the first 100 items returned from Google Base:
16GB Apple iPod Touch for $399.99
IPOD for $399.99
Black 80GB Video Apple Ipod for $399.99
Apple 160GB iPod classic â?" Black for $399.99
Apple iPod Photo 60GB - 15000 songs & 25000 photos in Your Pocket for $399.99
Apple 160GB iPod classic â?" Black for $399.99
16GB Apple iPod Touch for $399.99
Apple 60 GB iPod with Video Playback Black for $399.99
Apple iPod Touch 16GB WiFi Digital Music/Photo/Video Player for $399.99
BOSE SoundDock Portable Black Digital Music System for the iPod for $399.99
Apple iPod touch 16GB* MP3 Player (with software upgrades) - Black for $399.99
Apple 8GB iPod touch for $399.99
APPLE IPOD 8GB TOUCH for $399.99
BOSE SoundDock Portable Black Digital Music System for the iPod for $399.99
Apple 16GB iPod touch for $399.99
Sunday, May 11, 2008
Web Application Architecture in 2008 and Beyond
This post is about the future of Web Application Architecture in 2008 and beyond. It details some trends in "returning to the basics" regarding the adoption of REST-based services that are happening right now that I believe will lead to companies being able to build solid long-term platforms for service integration and collaboration with external partners and end-users alike.
Back to the Basics, Son
A good architecture should minimize the degree of difficulty for end-users, software components, and external vendors and third-party software to use a system. It should maximize the opportunity for the same people and software to derive value from and contribute value to the system.
Prerequisite Reading
To fully appreciate this post, I recommend you read a little more about the history of REST and WWW architecture as well as how they compare and contrast to SOAP / RPC models for Web Services and Service Oriented Architecture.
To summarize what you will find in the background reading, here is what REST boils down. Note that I am applying the example HTTP here because HTTP is an example of a REST-based architectural style.
- HTTP focuses on finding named resources through the global addressing scheme afforded by the URI standard
- HTTP allows applications to GET, PUT, POST, and DELETE to those URIs.
- GET retrieves a representation of an resource located at a specific URI
- PUT attempts to replace the resource at the given URI with a new copy
- REST is not a "buzzword" or a "fad". It is a description of how the WWW actually operates and how it has scaled to the size and success that it has thus far.
As explained in the Wikipedia entry, and alluded to above, addressability and the uniform interface are two of the most crucial features for a REST-based architecture.
I want to compare HTTP's implementation of a REST based architecture with a similar example of SQL.
In HTTP, we have the standard verbs, or methods, GET, POST, PUT, and DELETE.
Similarly, in SQL, we have SELECT, INSERT, UPDATE, DELETE.
Imagine a web site with the following URL:
http://www.mysite.com/Items/ViewItem.aspx?id=123
Here, we have several parts:
http://www.mysite.com = Host address
/Items = Subdirectory off the root
/ViewItem.aspx?id=123 = This breaks down into several parts:
View = verb (action)
Item = noun (resource)
id = parameter name
123 = parameter value
The actual HTTP request for this would look something like:
GET /ViewItem.aspx?id=123
Now, consider an alternative URL:
http://www.mysite.com/Items/123
Here we still have the first two parts the same, but we roll up all the others into the /123. This makes sense because we already have a verb, it's GET. We already know we are looking at the "Items" subdirectory, so that identifies the type of noun that we want already.
Detailed Example of Exposing and Consuming Data
Let's take the example a little further with microformats.
http://www.mysite.com/Items/123
Background Reading
These are some links for understanding Web (REST) architecture.
Real World Business Point of View
- Mark Nottingham (principal architect at Yahoo) discussing how Yahoo uses REST-based principles for integrating all of their properties: http://www.infoq.com/presentations/services-without-soap-yahoo
- Overview of REST: http://en.wikipedia.org/wiki/Representational_State_Transfer
- Paul Prescod on the REST vs. SOAP debate: http://www.prescod.net/rest/rest_vs_soap_overview/
- Pete Lacey on InfoQ about REST and WS-*: http://www.infoq.com/interviews/pete-lacey-rest
- My recent book review of RESTful Web Services: http://joshuagough.blogspot.com/2008/02/book-review-restful-web-services.html
- Other recent interviews with architects on InfoQ: http://www.infoq.com/rest/
- Steve Jones' post "Want to be cool? Learn REST. Want a career? Learn WS": http://service-architecture.blogspot.com/2006/11/want-to-be-cool-learn-rest-want-career.html
- Roy Fielding's Ph.D. dissertation that introduced the term, "Architectural Styles and the Design of Network-Based Software Architectures": http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
- RESTful Web Services book from O'Reilly: http://www.oreilly.com/catalog/9780596529260/
These are just some miscellaneous notes.
The more things change the more they stay the same
During the early years of the web we had a few building blocks:
- HTTP (GET, POST, PUT, DELETE)
- URI (http://www.hostname.com/resource)
- HTML (and friends like the blink and marquee tags)
- User agents (Netscape, Mosaic, Lynx, MSIE)
- CGI and PERL
- Cascading Style Sheets
- XML
- Java
- Javascript
- PHP
- .NET
- SOAP based Web Services
- XmlHttpRequest
- Python popularity growth
- Ruby
- Ruby on Rails with ActiveRecord
- Asynchronous Javascript and XML (AJAX)
Things that will never happen
My grandfather Gene Gough worked at IBM for 35 years as a programmer and manager. He sent out a funny email the other day. I don't know if this list is accurate or not, but here is the list posted on line: http://www.rense.com/general81/dw.htm.
Here are three good ones from the list
I worked with the people at CDC's Epi-X (http://www.cdc.gov/epix) program in the past, and I think they'd strongly disagree not just the first one, but with all three."Louis Pasteur's theory of germs is ridiculous fiction." -- Pierre Pachet, Professor of Physiology at Toulouse, 1872
"I think there is a world market for maybe five computers." --Thomas Watson, chairman of IBM, 1943
"There is no reason anyone would want a computer in their home." -- Ken Olson, president, chairman, founder of Digital Equipment Corp. 1977.
We are all familiar with the now famous "Technology Adoption Curve", which looks like a standard bell curve. You can read more about this here: http://en.wikipedia.org/wiki/Diffusion_of_innovations
REST Notes
http://www.infoq.com/interviews/pete-lacey-rest#
Q: Can you elaborate a little on why you think these would be tightly coupled? I mean everybody is talking about loose coupling and SOAP
A: I miss that one. When I was selling Systinet's products I talked about loosely coupled, when I present information for Burton Group on SOA I certainly stress "Trying to do your best to loosely couple client to server", because obviously loose coupling is a very good idea. The issue and I have said this before in print, is that you can strive for the largest amount of loose coupling possible in the SOAP WS-* world, and when you do the best job possible and when you obey all the best practices, you still end up with a tightly coupled system.
And you know the answer, but for the audience out there the fact of the matter is when you are creating a SOAP web service, a client, you have to know a great deal about that service, whether you learnt it via WSDL or some other mechanism, you have to know a great deal about it, and generally if the service changes in any not even terribly significant way, your client has to change with it or it will simply stop working. And you can go through heroic efforts to keep the existing clients alive and you can try and do your best at the design and development phase to loosely couple them, but the fact that the client has to know all the operation names and all the message formats before it can get any value out of a web service, is tight coupling to me.
Where of course in contrast all I need to know about a RESTful web service is its URI. Now I might not be able to derive all of the business values that the service can offer, but I can "GET" it and maybe something interesting will come from that, and it may be all the information I need. So properly designed RESTful system is dramatically loosely coupled, whereas a properly designed SOAP WS-* based system is unfortunately tightly coupled and all you can do is your best effort to avoid more tightly coupling than necessary.
Saturday, May 10, 2008
RESTful Web Services: Notes
=Chapter 8: REST and ROA Best Practices=
*Expose all interesting nouns as Resources
*Perform all access to Resources through HTTP's uniform interface: GET, PUT, POST, DELETE
*Serve Representations of the Resource, not direct access to it
*Put complexity into Representations not into access methods!
=The Generic ROA Procedure=
*Figure out data set
*Split data set into resources
*Name the resources with URIs
*Expose a subset of the uniform interface
*Design the representation(s) accepted from the client
*Design the representation(s) served to the client
*Integrate the resource into existing resources, using hypermedia links and forms
*Consider the typical course of events: what's supposed to happen? Standard control flows like the Atom Publishing Protocol can help
*Consider error conditions: what might go wrong? Again, standard control flows can help
=Addressability=
*RESTful web services should be highly addressable through thousand or infinitely variable addresses
*By contrast: RPC / SOAP based web services typically expose just one or a few addresses
*URIs = Universal Resource Identifier
*Never make a URI represent more than one resource
*Ideally, each variation of representation should have its own URI (think of a URI with en-us for English, ko-kr for Korean, etc)
*Set Content-Location header to specify the canonical location of a resource
*URIs travel better if they specify both a resource and a representation
=State and Statelessness=
*Resource state: stays on server, sent to client as a representation
*Application state: stays on client, until passed to server to use for Create, Modify, or Delete operations
* Service is "stateless" if server never maintains in memory or disk any application state
** Each request is considered in isolation in terms of resource state
** Client sends all application state to the server with each request, including credentials
=Connectedness, aka Hypermedia as the Engine of Application State=
=Server guides client to paths for state transitions of resources using links and forms
=Links and forms are the "levers of state" transition
=XHTML and XML are good representational formats
=Uniform Interface=
*Resources expose one or more of HTTP's interface
*GET: requests info about a resource: returns headers and a representation
*HEAD : same as GET, but only headers are returned
*PUT: assertion about resource state. Usually a representation is sent to the server and the server tries to adjust the resource state to match the representation
*DELETE: assertion that the resource should be removed.
*POST: attempt to create a new resource from an existing one. Root resource may be a parent resource or a "factory" resource. POST can also be used to append state of an existing resource.
*OPTIONS: attempt to discover which other methods are supported (rarely used)
*You can overload POST if you need another method, but consider first whether you can implement your need by simply designing another resource
*For transactions, consider making them resources as well
=Safety and Idempotence=
*GET or HEAD should be safe: resource state on server should not change as a result
**Server can log, increase view count, but client is not at fault
*PUT or DELETE should be idempotent: making more than one of the same request should have the same effect as one single request.
**Avoid PUT requests that are actually instructions, like "Increment x by 5"
**Instead, PUT specific final values
*POST requests for resource creation are neither safe nor idempotent
** Consider: Post Once Exactly
=New Resources: Put Versus Post=
*PUT can only create new resources when it can calculate the actual URI
*POST can create new resources even when the server decides the new URI
**Ex: /{dbtable}/{itemid}, POST to /{dbtable} and the server returns the new URI
=Overloading POST=
*Can use POST to transform the resource into an RPC-style message processor (think: SOAP web services)
*Use of overloaded POST (for XML-RPC or SOAP) is strongly discouraged by the author.
**Using this breaks the Uniform Interface.
**No longer is the web a collection of well-defined URIs with a uniform interface, instead:
**It becomes a collection of known entry points into a universe of DIFFERING INTERFACES, few compatible with each other
*Legit overloaded POST:
**Work around lack of PUT and DELETE support
**Work around limitations on URI length
***POST http://name/resource?_method=GET and payload contains huge data set.
**Avoid methods in GET URIs: /blog/rebuild-index. This is not idempotent
=This Stuff Matters=
*Principles are not arbitrary
*Advantages: simpler, more interoperable, easier to combine than RPC
*JSG: They have, in fact, revolutionized the world by being so simple and constrained to allow loosely coupled "links" from all over the planet to anywhere else.
=Why Addressability Matters=
*Every interesting noun, or concept, is immediately accessible through one operation on its URI: GET
*URIs provide:
** Unique structured name for each item (you own your own domain name so name.com/item/12345 is always unique)
** Allows bookmarking
** Allows URIs to pass to other apps as input
** Allows for mashups you never imagined (pipes.yahoo.com)
* URIs are like:
** Cell addresses in Excel
** File paths on disk
** JSG: Longitude and Lattitude coordinates
** JSG: XPath queries against XML
** JSG: SQL SELECT statements against relational tables
=Why Statelessness Matters=
*The king of simplifying assumptions!
*Each request contains all application state needed for server to complete request
**No application state on server
**No application state implied by previous request
**Each request evaluated in isolation
*Makes it trivial to scale application up
**Add a load balancer and there is no need for server affinity
*Can scale up until resource (database) access becomes the bottleneck
*JSG This is where COM Interop introduces database latency by forcing connections open longer than they need to be because over marshalling data across process boundaries
*Increases reliability (requests that timeout can simply be requested again)
*Keeping session state can harm scalability and reliability, so use it wisely
** JSG if using a cookie, the cookie can be used to reinstantiate the srver side state for the user no matter what server handles the request
=Why the Uniform Interface Matters=
*Provides a standard way of interaction
*Given http://www.example.com/myresource, you know:
**GET retrieves it
**POST can attempt to append it or place a subordinate resource beneath it
**DELETE can assert that it should be removed
=Why Connectedness Matters=
* Provides a standard way to navigate from link to link and state to state
=Resource Design=
*Need a resource for each "thing" in your service
**Apply to any data object or algorithm
*Three types of resources:
**Predefined one-off resources: static list, object, db table
**Large (maybe infinite) number of resources of individual items: db row
**Large (usually infinite) number of resources representing ouputs of an algorithm: db query, search results
*For difficult situations, the solution is almost always to expose another resource.
**May be more abstract, but that is OK
=Relationships between Resourcs=
*Alice and Bob get married, do you:
**PUT update to Alice and to Bob, or:
**POST new resource to the "marriage" factory resource?
**Answer: you should create a third resource that links to both Alice and Bob
*JSG this leaves you wondering how you navigate the other direction, but this is not any different at all from a relational database table that has a linking table.
=Asynchronous Operations=
*A single HTTP request itself is synchronous
*Not all requests finish and many take a long time
*Use the 202 status code "Accepted"
*Example: ask server to calculate huge result, sever returns:
202 Accepted
Location: http://jobs.example.com/queue/job11a4f9
*This URI identifies the job uniquely for the client to come back to later
*Client can GET the URI for status updates and DELETE it to cancel it or DELETE the results later
*This overcomes the asynchronous "limitation" by using a new resource URI
*Caveat: use POST when you will spawn a new resource asynchronously to avoid breaking idempotency if you were to use GET or PUT
=Batch Operations=
*Factory resources can accept a collection of representations and creates many in response
*Create a "job" in response with a URI for the client to check status, or:
*Use WebDAV extension for 207: multi-status; client needs to look in entity body for a list of codes
=Transactions=
*You can implement them as resources, just like batch operations can be
*Example: financial transaction
1. POST to a transaction factory to get a URI for your transcation (201 Created response)
Request:
POST /transactions/account-transfer
Response:
201 Created
Location: /transactions/account-transfer/11a5
2. PUT new balance for checking account to this URI
PUT /transactions/account-transfer/11a5/accounts/checking
balance=150
3. Then PUT the new value for the savings account:
PUT /transactions/account-transfer/11a5/accounts/savings
balance=250
3. Commit the transaction
PUT /transactions/account-transfer/11a5
committed=true
*The server should make sure the representations make sense (no deleted money, no newly minted money, etc)
*RESTful transactions are more complex to implement, but they have advantages of being addressable, transparent, archived and linked
=When in Doubt, Make it a Resource=
*Anything can be a resource
*Strive to maintain the Uniform Interface
=URI Design=
*URIs should be well-designed and meaningful
*URIs should be "hackable" to increase the "surface area"
*Make it so clients can bookmark just about anything to get right to it
**Don't make clients have to repeat dozens of manual steps to get back to a view of a resource
*Go for general to specific
** Example: /weblogs/myweblog/entries/100
*Use punctuation to separate multiple data inputs at the same level
*Use commas when order matters (eg long and lat)
*Use semi-colons when order doesn't matter: /color-blends/red;blue
*Use query variables only for algorithm inputs
*URIs denote resource, not operations: almost never appropriate to put method names in them
**/object/do-operation is a bad style
=Outgoing Representations=
Bookmark: Page 254
Note to Self: Music and Salsa
I am taking piano/keyboard lessons at http://www.l5pmusiccenter.com
And, I'm continuing to take salsa lessons, usually at http://www.takeholdballroom.com and http://www.pasofinopro.com. Check out http://www.atlantagasalsa.com for a full list of salsa events in the area. I'll be taking a cruise to Alaska in August and it is a salsa cruise!
Thursday, April 24, 2008
Article Review: Managing an Agile Project by Jeff Pallermo
http://www.code-magazine.com
First, About Jeff
Jeff is the founder of the MVC Contrib project at http://MVCContrib.org and he blogs here: http://codebetter.com/blogs
He's interviewed on Polymorphic Podcast here about MVC and MVC Contrib: http://polymorphicpodcast.com
My favorite quote from this interview is "If you are writing new code without automated tests, you are writing instant legacy code".
My second favorite part is where he says that they use CruiseControl, NAnt and NCover for MVC Contrib and they do not accept any contributions that have less than 95% code coverage in their automated tests. NO EXCEPTIONS
Jeff leads a .NET Boot Camp for .NET Teams every six weeks: http://www.headspringsystems
This advanced agile curriculum will cover everything involved in developing software in .NET, from setting up a new project and defining the architecture to implementing functionality in a loosely-coupled and testable manner. We will immerse ourselves in domain-driven design, test-driven development, design patterns, object-relational mapping, inversion of control (IoC), pair programming, automated builds, and continuous integration (CI). Students will discover which practices cause projects to fail and which practices help projects succeed. The course will include a strong focus on solid principles and values that can be applied to any .NET project. With a solid understanding of Agile values and object-oriented programming, students will emerge from the training with a refocused view on software development and the tools to immediately bring value back to their companies. All developers will take back working code developed during the course using the techniques and practices taught.
Comments on Article
Pallermo makes some very good points about Communication and Expectations. In my experience with iterative development, these were absolutely critical.
One project I worked on for four years in which Scrum techniques were applied was CDCs Epi-X system (http://www.cdc.gov/epix), a peer-review and emergency notification system for alerting about emergency health threats. We held weekly planning and strategy meetings in which the business side, the scientists and epidemiologists, related plans and goals for upcoming activities. On the development side, we had "stand up" meetings that were crucial for the team to understand the goals and problems for each day. After each milestone and release, we held reviews in which we cleaned up our environments, tightened our procedures, and made other improvements.
During a four year period we made about 15 releases, each bounded, each small and highly testable. That is roughly one release every three months.By breaking down the dilverables into smaller milestones, we were able to deliver new value continuously while also maintaining the mission-critical operations of the system on a day to day basis. Communication and documentation between the users, the business staff, and the technical staff was what enabled our success as a team.
On With the Highlights from the Article
Aiming for Agile
- Combine XP, Lean, and Scrum practices
- Agile project management understands that scope is a moving target.
- The larger the scope, the more it will change.
- The software manager should foster an environment where questioning assumptions is welcome.
- The software team will apply a critical eye to every story on the table and evaluate whether that story will contribute to the end goal.
- Every story must stand up to criticism in order to prove worthy of the team's time.
- Working on the right thing is as important as making the thing worked on right.
- In an agile project, you tend to employ more generalists than specialists. With a team of generalists, you want folks who are skilled in a variety of areas.
- Defect-free stories delivered. Each story represents a unique behavior in the software from which the customer benefits. Stories delivered with defects aren't as valuable, so don't count those.
- Customer satisfaction. Ultimately, you are creating software for a customer. Without the customer, you wouldn't be creating the software. Keeping them happy is a great metric. It's simple. Just ask their satisfaction level on a scale from one to ten.
- Consistent velocity. Iteration over iteration, velocity should remain constant. If it continues to change, something is wrong. After about the sixth iteration, treat any significant change in velocity as a cause for alarm. If the team make-up is consistent, the velocity should remain constant. A slowing of velocity could be a sign of a codebase that is less than maintainable.
Manage the customer's expectations. Use daily "stand up" meetings to communicate the following quickly:
- What I accomplished yesterday.
- What I hope to accomplish today.
- What barriers are impeding progress.
- Armed with this daily knowledge, the customer will start to feel like part of the team instead of an outsider.
- If the customer isn't engaged properly, there will be an us vs. them environment, and that is dysfunctional.
- When your team is in competition with the customer, neither will win.
- Defects are gaps between what your team committed to and what they delivered.
- A bug is anything that bugs the user. I'm not sure we'll ever get away from bugs, but constant communication sure helps.
- A defect is a gap in the story contract and acceptance criteria.
- A bug is not a defect since it's an annoyance that has yet to be discussed.
- When someone finds a defect, the team should drop everything and fix the defect right away.
- A defect signifies that a story which the team has already labeled "done" is, in fact, not finished. Finish the story and move on.
- Institute a zero-defect policy.
After an iteration, have the team take about 30 minutes to document the following:
- Good (what you would like to continue).
- Bad (what you would like to stop).
- Ugly (a fun category if an issue was particularly messy).
Wednesday, February 27, 2008
Book Review: RESTful Web Services
I'm currently reading the book RESTFul Web Services, by Leonard Richardson and Sam Ruby. Find it at http://www.amazon.com/RESTful-Web-Services-Leonard-Richardson/dp/0596529260.
This post summarizes what I'm learning from the book. Here is the book's basic premise:
Give me Some of that Flux Capacitor: Back to the Future
Many of today's most modern and successful web applications are beginning to adopt architectures that look more and more like yesterday's World Wide Web architecture, circa 1994. This is a good thing. In the beginning, there was HTTP, URI, and HTML. This was good. This post will explain why based on the arguments of the authors of this book. The basic architecture of the World Wide Web has been term REpresentational State Transfer (REST) by one of its principal architects, Roy Fielding.
From the REST article in Wikipedia, here is a basic definition of REST:
REST strictly refers to a collection of network architecture principles that outline how resources are defined and addressed. The term is often used in a looser sense to describe any simple interface that transmits domain-specific data over HTTP without an additional messaging layer such as SOAP or session tracking via HTTP cookies.
REST is About Resources and Their Addresses
To understand more, just visit the REST article in Wikipedia and read it before reading the rest of this post. In summary, the takeaway from that definition are these two key concepts:
- Resources
- Addresses
Remember that when we discuss REST and the WWW, we're talking about creating, retrieving, updating, and deleting Resources and about specifying Addresses at which locate those resources.
How the WWWW Architecture Works via RESTful Principles
As explained above, I won't rehash details better explained already on the REST article in Wikipedia and in Roy Fielding's dissertation, but I will briefly summarize what REST is with a simple example.
All of us who use the web are already familiar with Resources and Addresses. The content of the article that describes REST on Wikipedia is in fact a Resource. it is a document resource full of HTML text and links to other documents. And, of course, this Resource has an Address, http://en.wikipedia.org/wiki/Representational_State_Transfer.
When you browse to the URL http://en.wikipedia.org/wiki/Representational_State_Transfer your browser issues a request with a command and an argument. If you have Live HTTP Headers installed for Firefox or a similar plugin for IE, you can spy on the details of this request. It looks like this:
GET /wiki/Representational_State_Transfer HTTP/1.1
Host: en.wikipedia.org
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
If-Modified-Since: Wed, 27 Feb 2008 16:18:00 GMT
As another example, when you are on the www.asp.net web site and you do an Advanced Search for forum posts on the keyword "Scottgu", your browser issues a POST command:
POST /search/default.aspx HTTP/1.1
Host: forums.asp.net
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://forums.asp.net/search
Cookie: __utma=57644090.77168603.1204177720.1204177720.1204177720.1; __utmb=57644090; __utmc=57644090; __utmz=57644090.1204177720.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none); CommunityServer-UserCookie605740=lv=Fri, 01 Jan 1999 00:00:00 GMT&mra=Thu, 28 Feb 2008 00:48:36 GMT; CommunityServer-LastVisitUpdated-605740=; CSAnonymous=1cf2e1d1-51fe-4609-be1d-b4618c323b27; __utma=185208894.1130547723.1204177819.1204177819.1204177819.1; __utmb=185208894; __utmc=185208894; __utmz=185208894.1204177819.1.1.utmccn=(referral)|utmcsr=wiki.asp.net|utmcct=/|utmcmd=referral
Content-Type: application/x-www-form-urlencoded
Content-Length: 9138
__EVENTTARGET=ctl00%24bcr%24ctl02%24ctl00%24SearchButton&__EVENTARGUMENT=&__VIEWSTATE=<deleted>&ctl00%24bhcr%24SearchForm1%24ctl00%24TitleBarSearchText=&ctl00%24bcr%24ctl02%24ctl00%24Keywords=Scottgu
An Embarrassment of Riches: This is Your Grandfather's HTTP, But He Kept His Treasures From You!
Both GET and POST are called HTTP Methods, or HTTP Verbs. These are the two most commonly used and known HTTP methods. However, there are others that almost nobody talks about, except for people who are writing applications using a REST-based approach. You can read more about these at http://rest.blueoxen.net/cgi-bin/wiki.pl?HttpMethods
The four most important HTTP methods are GET, PUT, POST, and DELETE. Summarizing what other tutorials will explain in more detail, it's safe to say you can see the correspondence between these methods and the SQL DML statements SELECT, INSERT, UPDATE, DELETE.
Note: The PUT method can actually serve as both INSERT and UPDATE, and POST's purpose is more amorphous. A great explanation of the comparison between these HTTP Methods and SQL CRUD operations is found here on Paul Downey's blog: http://blog.whatfettle.com/2006/08/14/so-which-crud-operation-is-http-post/
REST versus RPC
Below is a compete section from the Wikipedia entry about REST. This illustrates the difference between RESTful web services based on HTTP and RPC-based web services, based upon the SOAP protocol.
The statements below refer to REST in the context of Web Services, specifically as opposed to SOAP. Note that Fielding’s dissertation presents REST in the context of information and media access, not web services. It does not contrast REST to RPC (remote procedure call).
- REST
- Resources—Commands are defined in simple terms: resources to be retrieved, stored / get, set—difficult to do many joins
- RPC
- Commands—Commands are defined in methods with varying complexity: depending on “standard”—easier (?) to hide complex things behind a method
- REST
- Nouns—Exchanging resources and concepts
- RPC
- Verbs—Exchanging methods
A RESTful web application requires a different design approach from an RPC application. An RPC application is exposed as one or more network objects, each with an often unique set of functions that can be invoked. Before a client communicates with the application it must have knowledge of the object identity in order to locate it and must also have knowledge of the object type in order to communicate with it.
RESTful design constrains the aspects of a resource that define its interface (the verbs and content types). This leads to the definition of fewer types on the network than an RPC-based application but more resource identifiers (nouns). REST design seeks to define a set of resources that clients can interact with uniformly, and to provide hyperlinks between resources that clients can navigate without requiring knowledge of the whole resource set. Server-provided forms can also be used in a RESTful environment to describe how clients should construct a URL in order to navigate to a particular resource.
Example
An RPC application might define operations such as the following:
getUser()
addUser()
removeUser()
updateUser()
getLocation()
addLocation()
removeLocation()
updateLocation()
listUsers()
listLocations()
findLocation()
findUser()
Client code to access this application may look something like this:
exampleAppObject = new ExampleApp('example.com:1234')
exampleAppObject.removeUser('001')
With REST, on the other hand, the emphasis is on the diversity of resources, or nouns; for example, a REST application might define the following resources
http://example.com/users/
http://example.com/users/{user} (one for each user)
http://example.com/findUserForm
http://example.com/locations/
http://example.com/locations/{location} (one for each location)
http://example.com/findLocationForm
Client code to access this application may look something like this:
userResource = new Resource('http://example.com/users/001')
userResource.delete()
Each resource has its own identifier noun. Clients start at a single resource such as the user resource that represents themselves, and navigate to location resources and other user resources. Clients work with each resource through standard operations, such as GET to download a copy of the resource’s representation, PUT to paste a changed copy over the top of the original, or DELETE to remove the data or state associated with the resource. POST is sometimes used interchangeably with PUT, but can also be seen as a “paste after” rather than a “paste over” request. POST is generally used for actions with side-effects, such as requesting the creation of a purchase order, or adding some data to a collection. Note how each object has its own URL and can easily be cached, copied, and bookmarked.
Uniform interfaces in REST and RPC
The uniform interface allows clients to access data from a range of resources without special code to deal with each one, so long as it is actually uniform. The content returned from a user resource could be the globally standard and RESTful HTML, a less RESTful industry standard representation such as UserML, or an unRESTful application-specific data format. Which content is returned can be negotiated at request time. The content could even be a combination of these representations: HTML can be marked up with microformats that have general or industry-specific appeal, and these microformats can be extended with application-specific information.
Uniform interfaces reduce the cost of client software by ensuring it is only written once, rather than once per application it has to deal with. Both REST and RPC designs may try to maximise the uniformity of the interface they expose by conforming to industry or global standards. In the RPC model these standards are primarily in the form of standard type definitions and standard choreography. In REST it is primarily the choice of standard content types and verbs that controls uniformity.
Notice the decoupling of Noun and Verb in the REST sample code section. This is clean, simple, flexible, and feels very much like working with a relational database in which you use those four basic commands, SELECT, INSERT, UPDATE, DELETE to operate on various different types of data. In the RPC sample, the same verb gets repeated and paired with every single type of noun.
Just imagine how insane you would feel writing SQL DML statements if you had to write table-specific operator statements for each table in your database? Don't forget, because they are table specific, you'd actually have to be writing the implementation code for those, perhaps in C, C++, or maybe even C# if you're using SQL Server 2005 which has .NET built in.
SELECT_User(LastName), SELECT_Book(Title)
FROM_User JOIN_User(ID) = JOIN_Book(UserID)
WHERE_Book(PublishDate > '1/1/1990') AND WHERE_User(LastName = 'Box')
If I had to write this terrifying code, in which I have to manually write C# code to create verb + noun commands for each action and table, I would surely welcome the day someone decided to decouple the verbs and nouns, giving me the ability to write this:
SELECT LastName, Title
FROM Users a INNER JOIN Book b ON a.UserID = b.UserID
WHERE PublishDate > '1/1/1990' AND LastName = 'Box'
Washing out Your Mouth with SOAP after Swallowing a Bitter Blue Pill
So when it comes to XLM Web Services, why is it that most of us, at least in the Java and Microsoft .NET universe, have been brought up to believe that SOAP, Simple Object Access Protocol is the correct way to implement XML Web Services? As this book argues, this may not be the case, and increasing numbers of web developers and large companies like Yahoo and Amazon have already boarded the REST bus. In some cases they've been riding it comfortably for years by now.
Paul Prescod provides an excellent summary of differences between SOAP and REST in his article Roots of the REST/SOAP Debate. In his article he cites a interesting quote by Don Box, one of SOAP's principal authors, and also the author of the quintessential Windows programming book Essential COM. In the quote, Box states that had XML Schema been completed in 1998, they never would have created SOAP in the first place! Here is the complete context of the quote, which details.
http://www.infoworld.com/articles/hn/xml/02/06/06/020606hnbox.html?s=rss&t%3b=news&slot%3b=3
Prior to his calling on developers to build applications, Box detailed the birth of SOAP in 1998, and how it came about as a result of a failed effort to provide DCOM for Unix developers.
"We're entering year five of SOAP. It's kind of interesting that what started as a kind of a sabbatical, fun project ... turned into this kind of monster. But be careful what you wish for is the moral of the story," said Box.
SOAP was Microsoft's "second run up the hill," following the DCOM-for-Unix effort, he said. Box was not a Microsoft employee when he helped develop SOAP."The first run up the hill was DCOM on Unix. Had DCOM on Unix succeeded, we would not be sitting in this room," said Box. The effort failed because it resembled Windows programming too closely for Unix purists, Box said.
SOAP, he said, arose out of an HTTP foundation, with HTTP being a protocol accepted by multiple development camps. XML also was critical."Really, the novelty of SOAP was simply saying we're going to take XML and make it work for data," Box said.
An upcoming version of SOAP, Version 1.2, needs to be the final version, Box said.
Box also stressed that XML Schema is the dominant technology for Web services. "Had XML Schema been done in 1998, we would not have done SOAP," he said."The reality is, XML Schema is the foundation for the rest of XML," said Box. Technologies such as XML Query take XML Schema "for granted, as a given," he said.
SOAP messages need to function with XML Schema, he said. "XML Schema is an inevitability. Resistance is futile. There is no point in not embracing this thing and I strongly encourage those of you who work in Web services technologies [to] make sure your story is straight" with respect to XML Schema, Box said.
Box also addressed some maligning of the UDDI (Universal Description, Discovery, and Integration) specification at the conference, saying it has the most opportunity for growth and that the upcoming Version 1.3 of UDDI hopefully will fix problems with identifiers.
"It's hard to talk about UDDI without taking an apologetic tone," Box said.
Top Developers and Corporations Concur: REST is BEST, SOAP is DIRTY.
If you don't believe that SOAP is too complicated, or you believe that REST sounds too simple, don't worry. You don't have to use it. That won't stop Amazon from using it. Or Yahoo. Or Flickr. Or Microsoft. Yes, Microsoft, but more on that next.
Take a look at Amazon's S3 Simple Storage service: http://www.amazon.com/gp/browse.html?node=16427261
Or, read about Yahoo's REST-only web services: http://developer.yahoo.com/faq/#rest
Flickr's REST documentation: http://www.flickr.com/services/api/request.rest.html
The list just goes on and on and on. Just Google it for more examples. Better yet, just take a look at the kinds of applications people are building on top of Amazon's Web Services platforms:
Stop that REST Bus, Microsoft's Coming, and They're Bringing a Whole Bunch of Programmers
Now you will not hear much about it yet from their marketing engine since Visual Studio 2008, Windows Server 2008, and SQL Server 2008 are all the rage and focus at this time, but Microsoft's engineering team is playing catch up right now trying to capitalize these trends. Competent and competitive Microsoft shops should take heed and be prepared well in advance by studying and learning from the competitors and projects that the MS crew are embracing and extending at this time.
Microsoft's ASP.NET team has already released a preview of the ASP.NET 3.5 Extension, available at
Microsoft's ADO.NET Data Services: REST Based Web Services
To get a sneak peek at Microsoft's REST architecture for Web Services, take a look here: http://astoria.mslivelabs.com/
Here is the summary of the technology from the home page:
The new wave of web applications are built on technologies such as AJAX and Microsoft Silverlight that enable developers to build better, richer user experiences. These technologies bring a shift in how applications are organized, including a stronger separation of presentation from data.
ADO.NET Data Services (also known as Project code name “Astoria”) consists of a combination of patterns and libraries that enables any data store to be exposed as a flexible data service, naturally integrating with the Web, that can be consumed by Web clients within a corporate network or across the Internet. ADO.NET Data Services uses URIs to point to pieces of data and simple, well-known formats to represent that data, such as JSON and ATOM/APP. This results in data being exposed to Web clients as a REST-style resource collection, addressable with URIs that agents can interact with using standard HTTP verbs such as GET, POST, or DELETE.For the system to understand and leverage semantics over the data that is surfacing, ADO.NET Data Services models the data exposed through the data service using a model called the Entity Data Model (EDM), an Entity-Relationship derivative. This organizes the data in the form of instances of "entity types", or "entities", and the associations between them.
Microsoft's MVC Framework for ASP.NET
Related to the ADO.NET Data Services is the MVC Framework, also part of the ASP.NET 3.5 extensions. Learn more about that here: http://quickstarts.asp.net/3-5-extensions/mvc/MVCOverview.aspx
Here is the summary of the MVC framework from that page:
The Model View Controller (MVC) architectural pattern separates an application into three main components: the model, the view, and the controller. The ASP.NET MVC framework provides an alternative to the ASP.NET Web-forms pattern for creating MVC-based Web applications. The ASP.NET MVC framework is a lightweight, highly testable presentation framework that (as with Web-forms-based applications) is integrated with existing ASP.NET features, such as master pages and membership-based authentication. The MVC framework is defined in the System.Web.Mvc namespace and is a fundamental, supported part of the System.Web namespace.
MVC is a standard design pattern that many developers are familiar with. Some types of Web applications will benefit from the MVC framework. Others will continue to use the traditional ASP.NET application pattern that is based on Web forms and postbacks. Other types of Web applications will combine the two approaches; neither approach precludes the other.
The MVC framework includes the following components:
Models. Model objects are the parts of the application that implement the domain logic. Often, model objects also retrieve and store model state in a database. For example, a Product object might retrieve information from a database, operate on it, and then write updated information back to a Products table in SQL Server.
Views. Views are the components that display the application's user interface (UI). Typically, this UI is created from the model data. An example would be an edit view of a Products table that displays text boxes, drop-down lists, and check boxes based on the current state of a Product object.
Controllers. Controllers are the components that handle user interaction, manipulate the model, and ultimately select a view to render that displays UI. In an MVC application, the view only displays information; the controller handles and responds to user input and interaction. For example, the controller handles query-string values, and passes these values to the model, which in turn queries the database by using the values.
--To Be Completed--

























