Attention and Focus: Distracted, Loss of Critical Thinking

There is an interesting interview on Point of Inquiry by the author of a book called Distracted: The Erosion of Attention and the Coming Dark Age. Here is the link:

http://www.pointofinquiry.org/maggie_jackson_distracted_the_erosion_of_attention_and_the_coming_dark_age/

The book has only 3.5 stars on Amazon, and ironically one of the comments says the book is a prime example of what it condemns!

Anyway, the observations the author and host D.J. Grothe make are about the increasing number of "connections" that people have via the online, networked world, but the decreasing depth and fluency of those connections. They discuss how these days people are expected to know more about a larger number of things.

This reminds me of the concept of the long tail, described here: http://www.thelongtail.com/about.html.
The crux of the long tail is that people now buy fewer items of a larger number of items, such
that the total volume of sales across the larger number equals the total volume of sales for the most popular market leaders.



So, it seems as if people are stretching themselves to pay attention to more and more things across a long tail because they feel that knowing that little bit across the horizontal scope complements the larger amount they know about a few things in the vertical rise.

Scott Ambler has written about the "Generalizing specialist" in terms of Agile teams on his blog here: http://www.agilemodeling.com/essays/generalizingSpecialists.htm

Here is a brief excerpt:

A generalizing specialist is someone with a good grasp of how everything fits together. As a result they will typically have a greater understanding and appreciation of what their teammates are working on. They are willing to listen to and work with their teammates because they know that they’ll likely learn something new. Specialists, on the other hand, often don’t have the background to appreciate what other specialists are doing, often look down on that other work, and often aren’t as willing to cooperate. Specialists, by their very nature, can become a barrier to communication within your team.

So, are long tails and generalizing specialists intrinsically linked to the distraction topic or am I proving the author's point by just introducing tangents?

Well, I think they are related, and not necessarily bad. The decrease of attention and erosion of critical thinking is bad, no doubt, because it represents an inability for people to perform the top-to-bottom in-depth analysis needed to make wise decisions.

However, the increased side-to-side attentiveness is good in that it shows that people are becoming more aware of the relationships and connections of parts into a whole.

Understanding that whole, however, requires a thorough understanding of the top-to-bottom view of, at minimum, a complete vertical slice of a process.

In software engineering, this means that a developer can grasp the overall architecture of a system by having a thorough and deep knowledge of one particular module, top-to-bottom, if and only if that module is representative of the larger system as a whole.

That is to say that the module under examination follows the same basic design structure and coding standards, design principles, and design patterns that are adhered to by the rest of the system. This helps achieve something akin to the concept of economies of scale.

On the other hand, if an engineer must contend with dozens of differing formats, approaches, spaghetti code bases, incompatible languages and toolsets, then a lot of understanding is lost or can never be gained simply due to something like the effects of diseconomies of scale.

For a team to achieve a level of productivity and efficiency in software engineering, they have to apply design principles that are in accordance with shared purpose, communication, maintainability, and flexibility. Creativity is an important aspect, but creativity should be expressed in terms of solving the problem domain in terms of the most understandable and simple design, rather than in terms of the most terse, most elegant or innovative, or unique design.

This may at first sound rigid and unflexible, but the introduction of basic, rudimentary constraints, such as the generic interface seen in REST, provides a context within which creativity can be expressed, yet also affords communication and connectivity to external components without cumbersome setup time or negotiation.

So, to bring this back to the idea of distraction, I want to point out that it is important for people to use the tools they have to help them find the right path, and then focus intently on understanding that path and exploring that path to the fullest, to understand it from the top-to-bottom, and to build in the side-to-side extension mechanisms to enable future modification and expansion without impact to the core features and functionality.

These are core principles expressed in the literature of design patterns and agile design guidelines, but they must be carefully studied and learned. They cannot be gleaned by glancing over a few web sites or thumbing through a few books. They have to be explored, experimented with, and ultimately employed in real systems in order for engineers to comprehend them fully.

Review: Scrum and XP from the Trenches

I am reading the book Scrum and XP from the Trenches by Henrik Kniberg. It is about his personal experience successfully implementing a Scrum-based methodology in his development team of 40 people in Stockholm, Sweeden.

You can read the book as a PDF on InfoQ's web site here: http://www.infoq.com/minibooks/scrum-xp-from-the-trenches

Or, you can purchase the book at LuLu here:
http://www.lulu.com/content/899349

Review
Here are my comments on each section of the book:

Foreward by Jeff Sutherland

Jeff co-created Scrum. In his foreward he makes these points:

JSG paraphase:

• Scrum is becoming very important to venture capitalists (especially his own partners)
• Teams must know their project velocity -- This is how much of a backlog a team can accomplish in a given sprint. (See http://www.scrumalliance.org/articles/39-glossary-of-scrum-terms#1110 for details)
• Only teams that match all of the criteria below in the excerpts will be funded by his venture capitalist partners.
  

Direct excerpts:

• Iterations must have fixed time boxes and be less than six weeks
  long.
• Code at the end of the iteration must be tested by QA and be
  working properly.
• A Scrum team must have a Product Owner and know who that
  person is.
• The Product Owner must have a Product Backlog with estimates
  created by the team.
• The team must have a Burndown Chart and know their velocity.
• There must be no one outside a team interfering with the team
  during a Sprint.
  

Foreward by Mike Cohn

Mike is a founding member of the Scrum Alliance. He has written books, articles, and speaks regularly. Learn more at http://www.scrumalliance.org/profiles/8-mike-cohn.

Mike makes these points:

• Scrum and XP are both practical, results-oriented approaches. They are about Getting Things Done
  
• Prototype early and often rather than documenting requirements at an exhaustive level
  
• Avoid excess modeling, prefer prototyping instead
• Work on things that have potential to become part of the actual working solution
  
• Refer to other resources for the theory behind Scrum. It is out there, but this book focuses on implementing Scrum successfully

Preface

• He was skeptical going in, but was convinced shortly thereafter starting
• Scrum worked for the author's 40 person team
• Team's quality was way below acceptable, but implementing Scrum solved their problems
• He will use Scrum by default for new projects in the future unless there is a specific reason not to use it
  

1. Introduction

• Scrum is not a magic bullet
• You tailor it to your context
• His team was fighting fires all the time
  
• Quality was low
• Overtime was up
• Deadlines were missed
• "Scrum" was just a buzzword to most people
• Implemented across teams of 3 to 12 people
• Learned about different ways of managing backlog (Excel, Jira, index cards, etc)
  
• Experimented w/different sprint sizes (2 to 6 weeks)
• Used other XP practices like pair programming
• Used continuous integration
• Used TDD
• Took Ken's certification course
• Most useful info came from real "War Stories" and case studies of people actually solving problems with Scrum
  

2. How we do product backlogs

• Backlog is hear and soul of Scrum
• Backlog contains customer's desired features, prioritized by most critical
• Call the backlog items User Stories or just Stories
• ID, Name, Importance, Initial Estimate, How-To-Demo, Notes
• 6 fields were most often used
• Kept in Excel spreadsheet on shared drive with Multiple Editing allowed
• Additional: Track, Components, Requestor, Bug Tracking ID if needed
• Keep the product backlog at the Business Level not a technical level
• Let the team figure out the How-To Technical Level
• Ask Why as many teams as needed to get to the underlying intent if the Product Owner does state it in technical terms, and move technical language to Notes field.
  

3. How we prepare for Sprint planning

• Product Backlog MUST exist first in ship-shape form
• One Product Backlog and One Product Owner per Product
• All items should have a unique Importance level
• Leave large gaps. 100 does not mean 5 times more important than 20. Easier than making 20.5 if you did 20 and 21 instead when item C comes up in the middle
• Product Owner should understand the intent of each story
• Others can add stories, but only Product Owner can assign importance
• Only the team can add an estimate
• Tried using Jira for keeping the backlog, but too many clicks for Product Owner
• Have not yet tried VersionOne or other Scrum tools

4. How we do Sprint planning

TODO page 25

5. How we communicate Sprints

6. How we do Sprint backlogs

7. How we arrange the team room

8. How we do daily Scrum

9. How we do Spring demos

10.How we do Spring retrospectives

11.Slack time between Sprints

12.How we do release planning and fixed priced contracts

13.How we combine Scrum with XP

14.How we do testing

15.How we handle multiple Scrum teams

16.How we handle geographically distributed teams

17.Scrum master checklist

18.Parting words

Recommended reading

RESTful Web Services: Notes

This is continued from a prior post. These are my notes from the book RESTful Web Services that Mike (http://www.slipjig.org) and I have been reading. He is finished. He's already implementing a commercial application using RESTful design and the ASP.NET MVC framework at http://www.aps.net/mvc.


=Chapter 8: REST and ROA Best Practices=
*Expose all interesting nouns as Resources
*Perform all access to Resources through HTTP's uniform interface: GET, PUT, POST, DELETE
*Serve Representations of the Resource, not direct access to it
*Put complexity into Representations not into access methods!

=The Generic ROA Procedure=
*Figure out data set
*Split data set into resources
*Name the resources with URIs
*Expose a subset of the uniform interface
*Design the representation(s) accepted from the client
*Design the representation(s) served to the client
*Integrate the resource into existing resources, using hypermedia links and forms
*Consider the typical course of events: what's supposed to happen? Standard control flows like the Atom Publishing Protocol can help
*Consider error conditions: what might go wrong? Again, standard control flows can help

=Addressability=
*RESTful web services should be highly addressable through thousand or infinitely variable addresses
*By contrast: RPC / SOAP based web services typically expose just one or a few addresses
*URIs = Universal Resource Identifier
*Never make a URI represent more than one resource
*Ideally, each variation of representation should have its own URI (think of a URI with en-us for English, ko-kr for Korean, etc)
*Set Content-Location header to specify the canonical location of a resource
*URIs travel better if they specify both a resource and a representation

=State and Statelessness=
*Resource state: stays on server, sent to client as a representation
*Application state: stays on client, until passed to server to use for Create, Modify, or Delete operations
* Service is "stateless" if server never maintains in memory or disk any application state
** Each request is considered in isolation in terms of resource state
** Client sends all application state to the server with each request, including credentials

=Connectedness, aka Hypermedia as the Engine of Application State=
=Server guides client to paths for state transitions of resources using links and forms
=Links and forms are the "levers of state" transition
=XHTML and XML are good representational formats

=Uniform Interface=
*Resources expose one or more of HTTP's interface
*GET: requests info about a resource: returns headers and a representation
*HEAD : same as GET, but only headers are returned
*PUT: assertion about resource state. Usually a representation is sent to the server and the server tries to adjust the resource state to match the representation
*DELETE: assertion that the resource should be removed.
*POST: attempt to create a new resource from an existing one. Root resource may be a parent resource or a "factory" resource. POST can also be used to append state of an existing resource.
*OPTIONS: attempt to discover which other methods are supported (rarely used)
*You can overload POST if you need another method, but consider first whether you can implement your need by simply designing another resource
*For transactions, consider making them resources as well

=Safety and Idempotence=
*GET or HEAD should be safe: resource state on server should not change as a result
**Server can log, increase view count, but client is not at fault
*PUT or DELETE should be idempotent: making more than one of the same request should have the same effect as one single request.
**Avoid PUT requests that are actually instructions, like "Increment x by 5"
**Instead, PUT specific final values
*POST requests for resource creation are neither safe nor idempotent
** Consider: Post Once Exactly

=New Resources: Put Versus Post=
*PUT can only create new resources when it can calculate the actual URI
*POST can create new resources even when the server decides the new URI
**Ex: /{dbtable}/{itemid}, POST to /{dbtable} and the server returns the new URI

=Overloading POST=
*Can use POST to transform the resource into an RPC-style message processor (think: SOAP web services)
*Use of overloaded POST (for XML-RPC or SOAP) is strongly discouraged by the author.
**Using this breaks the Uniform Interface.
**No longer is the web a collection of well-defined URIs with a uniform interface, instead:
**It becomes a collection of known entry points into a universe of DIFFERING INTERFACES, few compatible with each other
*Legit overloaded POST:
**Work around lack of PUT and DELETE support
**Work around limitations on URI length
***POST http://name/resource?_method=GET and payload contains huge data set.
**Avoid methods in GET URIs: /blog/rebuild-index. This is not idempotent

=This Stuff Matters=
*Principles are not arbitrary
*Advantages: simpler, more interoperable, easier to combine than RPC
*JSG: They have, in fact, revolutionized the world by being so simple and constrained to allow loosely coupled "links" from all over the planet to anywhere else.

=Why Addressability Matters=
*Every interesting noun, or concept, is immediately accessible through one operation on its URI: GET
*URIs provide:
** Unique structured name for each item (you own your own domain name so name.com/item/12345 is always unique)
** Allows bookmarking
** Allows URIs to pass to other apps as input
** Allows for mashups you never imagined (pipes.yahoo.com)
* URIs are like:
** Cell addresses in Excel
** File paths on disk
** JSG: Longitude and Lattitude coordinates
** JSG: XPath queries against XML
** JSG: SQL SELECT statements against relational tables

=Why Statelessness Matters=
*The king of simplifying assumptions!
*Each request contains all application state needed for server to complete request
**No application state on server
**No application state implied by previous request
**Each request evaluated in isolation
*Makes it trivial to scale application up
**Add a load balancer and there is no need for server affinity
*Can scale up until resource (database) access becomes the bottleneck
*JSG This is where COM Interop introduces database latency by forcing connections open longer than they need to be because over marshalling data across process boundaries
*Increases reliability (requests that timeout can simply be requested again)
*Keeping session state can harm scalability and reliability, so use it wisely
** JSG if using a cookie, the cookie can be used to reinstantiate the srver side state for the user no matter what server handles the request

=Why the Uniform Interface Matters=
*Provides a standard way of interaction
*Given http://www.example.com/myresource, you know:
**GET retrieves it
**POST can attempt to append it or place a subordinate resource beneath it
**DELETE can assert that it should be removed

=Why Connectedness Matters=
* Provides a standard way to navigate from link to link and state to state

=Resource Design=
*Need a resource for each "thing" in your service
**Apply to any data object or algorithm
*Three types of resources:
**Predefined one-off resources: static list, object, db table
**Large (maybe infinite) number of resources of individual items: db row
**Large (usually infinite) number of resources representing ouputs of an algorithm: db query, search results
*For difficult situations, the solution is almost always to expose another resource.
**May be more abstract, but that is OK

=Relationships between Resourcs=
*Alice and Bob get married, do you:
**PUT update to Alice and to Bob, or:
**POST new resource to the "marriage" factory resource?
**Answer: you should create a third resource that links to both Alice and Bob
*JSG this leaves you wondering how you navigate the other direction, but this is not any different at all from a relational database table that has a linking table.

=Asynchronous Operations=
*A single HTTP request itself is synchronous
*Not all requests finish and many take a long time
*Use the 202 status code "Accepted"
*Example: ask server to calculate huge result, sever returns:

202 Accepted
Location: http://jobs.example.com/queue/job11a4f9

*This URI identifies the job uniquely for the client to come back to later
*Client can GET the URI for status updates and DELETE it to cancel it or DELETE the results later
*This overcomes the asynchronous "limitation" by using a new resource URI
*Caveat: use POST when you will spawn a new resource asynchronously to avoid breaking idempotency if you were to use GET or PUT

=Batch Operations=
*Factory resources can accept a collection of representations and creates many in response
*Create a "job" in response with a URI for the client to check status, or:
*Use WebDAV extension for 207: multi-status; client needs to look in entity body for a list of codes

=Transactions=
*You can implement them as resources, just like batch operations can be
*Example: financial transaction

1. POST to a transaction factory to get a URI for your transcation (201 Created response)

Request:
POST /transactions/account-transfer

Response:
201 Created

Location: /transactions/account-transfer/11a5

2. PUT new balance for checking account to this URI

PUT /transactions/account-transfer/11a5/accounts/checking

balance=150

3. Then PUT the new value for the savings account:

PUT /transactions/account-transfer/11a5/accounts/savings

balance=250

3. Commit the transaction

PUT /transactions/account-transfer/11a5

committed=true

*The server should make sure the representations make sense (no deleted money, no newly minted money, etc)
*RESTful transactions are more complex to implement, but they have advantages of being addressable, transparent, archived and linked

=When in Doubt, Make it a Resource=
*Anything can be a resource
*Strive to maintain the Uniform Interface

=URI Design=
*URIs should be well-designed and meaningful
*URIs should be "hackable" to increase the "surface area"
*Make it so clients can bookmark just about anything to get right to it
**Don't make clients have to repeat dozens of manual steps to get back to a view of a resource
*Go for general to specific
** Example: /weblogs/myweblog/entries/100
*Use punctuation to separate multiple data inputs at the same level
*Use commas when order matters (eg long and lat)
*Use semi-colons when order doesn't matter: /color-blends/red;blue
*Use query variables only for algorithm inputs
*URIs denote resource, not operations: almost never appropriate to put method names in them
**/object/do-operation is a bad style

=Outgoing Representations=
Bookmark: Page 254