The Antifragility of the Web

We’re used to taking the web for granted. We expect it to be there as substrate, with its addresses, declaratory documents, universally available programming language and the links between pages.

Ah, the links. There’s the rub. How many times have you followed a link and got a 404 or a different page than you were expecting? Links rot. As Tim Berners-Lee says, eventually every domain becomes a porn site.

So we want to do better. We want to build a non-web web. A special place for ourselves and our friends that is self-contained, and where all the pages and links are in the same database, and they can’t rot.

Instead of these messy links with protocols and domains in we just use @names or +names and #topics and tag. It’s easier for people to do, and self-consistent and grows explosively. Biz dev gets excited about the reciprocal deals we can do with other content owners.

If you’ve read Nasim Taleb’s Antifragile, you know what comes next. By shielding people from the complexities of the web, by removing the fragility of links, we’re actually making things worse. We’re creating a fragility debt. Suddenly, something changes - money runs out, a pivot is declared, an aquihire happens, and the pent-up fragility is resolved in a Black Swan moment.

The special place disappears entirely. Or, if we’re lucky, the Archive Team lights the cat signal and emergency archivists preserve it in formaldehyde somewhere else, the clock stopped, the links severed.

Meanwhile, out there on the web, people can still connect and discuss and say what went wrong, and do better next time. The web itself is antifragile. It interprets our business models as damage and routes around them. If we’ve learned, we’ll respect this next time we make something.

Keep ALL the versions

Back in the 1980s, storage was expensive and slow. You had a copy of your document in memory and you would be asked every time you wanted to save it out to disk, because you didn't want to fill the disk up. That paradigm is so out of date now it's embarrassing to try to explain to my sons what the little floppy disk icon is in Microsoft Word. "What's that?" "it's a floppy disk." "Oh yeah, I think I saw one in the garage once"

The world view of having to load and save is being gradually eroded. Apple has changed the operating system - Mountain Lion no longer has Save and Save As, but instead a model of going back through edit histories. Google Docs originally didn’t have a floppy disk icon, but put it back because people were looking for it in user testing. Now it has been removed.

We've had source code control as programmers for a long time. But the github world takes that further: the first thing you do is clone a project into your own repository, then start forking it. You can eventually merge stuff back later, but there is the assumption that things are happening in parallel. As James Governor put it:

Open Source used to count download numbers as a measure of developer success.

Today, we increasingly use forks as the metric of traction.

Wikipedia has put this into the public consciousness by having publicly visible edit histories, so you can go back and forwards in time over the history of the article. The paradigm of "storage is not a problem, we should keep every version of everything ever" is moving through culture to be a default assumption.

This will be something we want on mobile too. The issue of "which have I got on the phone and which have I got in the cloud?” is what makes it tricky. I think there will be a battle between Apple and Google about how you present that to the user in a coherent way - Google Drive and iCloud are taking different paths here, with DropBox actually working between both. Google Drive not storing copies of Google Documents locally is a mistake I expect them to fix.

Everything is moving in this direction, even low level system design. The growth of functional programming is all about not having contention over a single copy of things in memory but having paths through data that are modifying things in their own version of the world.

If you think about the difference between the way JavaScript handles stuff and the way Java does, Java still has a ‘data structure being passed around’ world view; JavaScript has closures that are passed around that contain the entire state of the current machine at the time of that event which are held as you go off and do something else and come back. One of the reasons that node.js feels so nice when you're writing web programming is that it has the feeling you’re used to on the client side that you do something and then call something, passing it a callback. You end up writing the server side stuff in the same way, and it's just naturally parallelizable. It's easy to spin up more machines for it because you've written stuff with the presumption that each instance of it is wholly independent.

Not everything can be written that way. The core of node is in C because it has to deal with the raw machine contention and deal with this routing, but the spread of the functional world view is a natural fit for web apps.

The other potential that node.js makes manifest is convergence of client and server code. If you're running and writing the same code in the same language, with the same libraries running on both the client and the server you can decide to migrate bits back and forth much more easily.

You don’t have to spend so much time deciding which is which and without worrying about the boundaries of the world and different shapes of the data structures. So if you're creating a JSON object on the server and passing it to the client you can decide whether you do that or not, at which point you do that, at which point you render it and which point you don't. That sort of fluidity is going to become more important over time. Pat Patterson showed how this can work for mobile apps too.

The programmers' world view has changed on this and it is permeates out to the world as programmers make those ideas available to the public in usable form. The invention of ‘Undo’ was a huge part of what made the Mac great - enabling users to experiment safely. Being able to retrospectively undo mistakes later, or learn from others’ public variations on a theme is going mainstream too.

This post is based on discussions I had on This Week In Google ep 143, with Gina Trapani, Jeff Jarvis and Tom Merritt which were transcribed by Michael Shook from this video of the show Updated: At the same time, JP Rangaswami wrote Warning: Contains Warnings which give more context about how 'Undo' helps protect innovation.

Draw Something CEO, grace and high school mathematics

Dan Porter, CEO of OMGPop, has had a good week. His game, Draw Something (it is an asynchronous Pictionary for cellphones, like Words With Friends is an asynchronous Scrabble) has taken off like mad, and Zynga bought his company for over $200 million. However, one employee didn't go along to Zynga, and Dan's been whining on twitter:

What's so interesting about success is the number of failures who try to ride on your back. Shay Pierce is just one of many...

— Dan Porter (@tfadp) March 31, 2012

The one omgpop employee who turned down joining Zynga was the weakest one on the whole team. Selfish people make bad games. Good riddance!

— Dan Porter (@tfadp) March 31, 2012

This has drawn some reactions from others, eg Notch, CEO of Minecaraft:

@tfadp You're an insane idiot.

— Markus Persson (@notch) April 1, 2012

and Dick Costolo, CEO of Twitter:

@tfadp I'm sure you realize by now what a nitwit comment that was, but wow, what a nitwit comment that was.

— dick costolo (@dickc) April 1, 2012

and the lovely Tom Coates:

I hope I never say anything this awful: @tfadp "What's so interesting about success is the number of failures who try to ride on your back."

— Tom Coates (@tomcoates) April 1, 2012

Now just before this crass public display of arrogance, he said something just as telling:

4 years of HS math I got every prob right. Teachers said was wrong way to the answer. Math, business, games, I'll stick with my wrong way.

— Dan Porter (@tfadp) March 30, 2012

The thing is, Draw Something has a maths problem. The so-called Birthday Paradox is kicking in. This is named for the unexpected result that if you have 23 people in a room, there's a 50:50 chance two of them have the same birthday.

There's a similar effect with games. If you keep randomly picking a word from a list, you'll see repeats quickly. Classic board games understand this - this is why Balderdash, Pictionary, Trivial Pursuit etc insist you use a discard pile after shuffling and picking a question, so you only pick a new card from those you haven't seen. Draw Something isn't doing this, so we're all seeing words repeat, which is discouraging play. This is all over twitter too:

Too many repeat words in DrawSomething. :|

— Shreya Ghoshal (@shreyaghoshal) March 30, 2012

I've played Draw Something too much so every word now is a repeat. #firstworldpain

— ∆li (@AleeZaidee) April 1, 2012

Just now coming off a week-long "Draw Something" binge. The fact that I keep getting the same words over & over is helping.

— Aimee Mann (@aimeemann) March 31, 2012

One way or another, I think Draw Something has peaked.

When you're the merchandise, not the customer

Jonathan Zittrain posted today that he is not the source of the quote widely attributed to him:

I participated in the Berkman Center’s fascinating HyperPublic symposium in the summer of 2011. When moderating a panel I invoked the aphorism that “When something online is free, you’re not the customer, you’re the product.” It’s a way of encapsulating the idea that online free services usually make money by extracting lots of data from users — and then selling that data, or using it for targeted availability of those users for advertising, to advertisers. In that sense, the advertisers are the clients, and the users enjoying free content are what’s being sold. (Of course, sometimes that happens even when the user pays.)

I didn’t coin the phrase, and since it was featured (and attributed to me!) in wordsmith.org’s wildly popular “word a day” as a thought for the day accompanying the word “enceinte” — I sought to nail down its provenance.

The first use of the quote that we can find is as a comment within the famed MetaFilter community in August 2010. The user’s name is blue_beetle, who might be someone named Andrew Lewis. It’s entirely possible I saw it there, as MeFi is one of my five favorite sites on the Web.

I was pretty sure this idea dates back further, so I went digging. First I found Josh Klein's 2009 blogpost, which cites Philip Broughton's 2008 book Ahead of the Curve: Two Years at Harvard Business School

"My favorite moment comes in an anecdote about an MBA candidate who, not getting his way, complains to an administrator, “I’m the customer! Why are you treating me so badly?”

To which the administrator responds, “you’re not the customer. You’re the product.”

But the sense is not quite the same there - an MBA is not a free web service after all. Going back a little further, this 2006 discussion at Joel on Software Is the Magical Fairy-tale For Google Engineers about to End? (nicely prefiguring James Whittaker's Why I left Google) includes this contribution from Drew K:

Like clam pointed out, Google's customers are the advertisers. "Skooter" is a user. Just like with ad-supported broadcast TV, you're not the customer, you're the product.

The idea is pretty well-expressed there, but I think we can go back further. In 2004, Coding Forums discussed the then-new Gmail, and liorian commented:

From a Google perspective, you're not the customer. The ad service buyer is the customer. You're the commodity. By making you a more attractive commodity, i.e. by making sure to only serve you an ad if you are in the target population for it, they are making the ads pay better for their customers, and they can reap a large part of the difference to their competitors, the other ad services.

This isn't a new idea then, as the analogy to television makes clear. The earliest, most thorough exegesis of this idea I have found is Claire Wolfe's 1999 article Little brother is watching you The Menace of Corporate America which opens with:

Perhaps because you're not the customer any more. You're simply a "resource" to be managed for profit. The customer is someone else now — and usually someone without your best interests at heart.

And has a continuing refrain of “Who is the Customer? Not you”, ending with

Who is the customer? Not you, whose life is reduced to someone else's salable, searchable, investigatable data. The customer is everyone who wishes to own a piece of your life.

The underlying warning is definitely worth thinking about — Maciej Ceglowski eloquently made the case for why you should pay Software Artisans on a recent TummelVision — but the deeper changes to what it means to be a customer matter too. There are other things we take part in without paying or being sold, because we find shared value in them, and the net enables those too.

QR Codes: bad idea or terrible idea?

People have a problem finding your URL. You post a QR Code. Now they have 2 problems. Or more:

1. They see a chunk of robot barf on your poster, and have to realise it isn't a crossword puzzle, but a QR code.
   
2. They need to take a digital photograph of it with their phone. If they have a laptop, even with a camera, this requires physical contortions
   
3. They need an application on their phone that can make sense of a QR code.
   
4. They need a lot of patience as they fiddle with it.
   
5. They need a working network connection to resolve it.

Conversely, with a URL they could type it in, take a photograph of it and type it in later, or if they have the right app, it will recognise the URL text from the image and make it clickable.

That is the irony of this. QR Codes ignore years of research and culture on how to communicate meaning in symbolic form designed to be captured by image processing tools behind a lens. We have this technology. It is called writing.

Written language has a set of symbols that are relatively unambiguous, that are formed of curves rather than hard edges making them resilient to noise, and have been market-tested for milennia. QR Codes don't just ignore this, they ignore the relative success of one dimensional barcodes. Notice something about a barcode? It has the number printed on it as well, so you can type it in if the scan fails. QR Codes don't do this, so it's far too easy to put the wrong one in, or fail to replace a mockup. Which is why so many QR codes link to Justin's site instead.

The only place you should use QR codes is if you have a dedicated reader for them, like a classic barcode scanner, and a workflow that is designed for this that actually saves time. If you do empirical research on using QR codes for the public, you'll likely see 80% worse performance than text like this museum did. By all means try the experiment and report your results. Put up a QR code and a printed URL and see which gets the most usage.

Or listen to others:

a majority of our respondents knew more or less what they were for, very few (n=2, or around 7%) were successfully able to use QR codes to resolve a URL, even when coached by a knowledgeable researcher.[..] A strong theme that emerged — which we certainly found entirely unsurprising, but which ought to give genuine pause to the cleverer sort of marketers — is that, even where respondents displayed sufficient awareness and understanding of QR codes to make use of them, virtually no one expressed any interest in actually doing so.

As Alexis Madrigal puts it:

Is it really faster and better to use a QR code that will direct you to part of a marketing campaign rather than getting a broader sweep of information by simply using the browser that you already use all the time on your phone? In the instant cost-benefit analysis I do every time I see a QR code, it has yet to make sense for me to fire up the decoder app I have installed on my phone.

QR code at the bus stop to get time of next bus. Really useful in the dark. Not. yfrog.com/mgicpqj

— Martin Geddes (@martingeddes) January 27, 2012

Google Plus admits they want fake names

Today, after 7 months, Bradley Horowitz announced that Google Plus will accept some pseudonyms. Kinda. If you can prove you're already famous. And can convince their robot it looks like a name. However, Google Engineer Yonatan Zunger spills the beans in a comment on that thread:

First of all, you might ask why we have a names policy at all. (i.e., why we don’t simply go with the JWZ proposal) One thing which we have discovered, while putting some miles on the system, is that it is indeed important to have a name-based service rather than a handle-based service. This isn’t a matter of functionality so much as of community: You get a different kind of community when people are known as Mary Smith than when they are known as captaincrunch42, and for a social product in particular we decided that the first kind of community is the one we want to build. In order to do that, we want to establish a general norm that the names you put in to the system should be names, not handles.

So one thing that our name checking flow tries to catch is handles, which should normally be nicknames, shown in addition to a name. The other important thing it’s trying to catch is people who are creating individual accounts, rather than +Pages, for non-human entities such as businesses or organizations. The behavior of +Pages is deliberately restricted in the system, and we don’t want people to be creating fake human accounts to circumvent that. The name check turns out to be a very powerful tool to catch these.

Our name check is therefore looking, not for things that don’t look like “your” name, but for things which don’t look like names, period. In fact, we do not give a damn whether the name posted is “your” name or not: we will not challenge you on this basis, nor is there any mechanism for other users to cause you to be challenged for this.

There are two main cases where the name check screws up. One is false positives: people (such as you) who have unusual names which get flagged because they looked like handles. Being able to appeal via things such as drivers’ licenses is useful for this case, since it’s a simple “oh, we got this wrong.” The other case is people such as +trench coat, who are so well-known under this handle that it would be bizarre not to let them onto the system under this name. For this case, we allow appeals based on being well-known under the name: thus the ability to prove the “established pseudonym.” We’ve deliberately set the threshold for that latter case fairly high for now, but we intend to continue to tune it; the objective is that the frequency of such names should basically be the same as their frequency in meatspace.

So to answer your questions one-by-one:

(2) “Meaningful following” only applies to cases of established pseudonyms which do not look like names. The definition of “meaningful” is deliberately vague so that we can tune it, so that it behaves in a natural fashion.

(3) That’s correct; drivers’ licenses are for false positives, not pseudonyms.

(4) Unusual names will indeed hit friction, because of false positives. We’re trying to minimize that, but it’s going to take some trial and error.

(5) Google+ can absolutely be your first identity online. No matter what your language, no matter where you come from. The “established pseudonym” logic should apply to a very small subset of people. If some groups are seeing a higher false positive rate than others, that’s a bug, not a feature, and we have the data available to spot this situation and remedy it.

(posted in full, in case of subsequent retraction, and because G+ doesn't have permalinks for comments)

Yonatan admits what Bradley obscures:that this is an Identity Theatre issue. They don't want your name, They don't care if you have a forename in one language and a surname in another. Let me quote this exactly:

Our name check is therefore looking, not for things that don’t look like “your” name, but for things which don’t look like names, period. In fact, we do not give a damn whether the name posted is “your” name or not: we will not challenge you on this basis, nor is there any mechanism for other users to cause you to be challenged for this.

This is what I suspected when I wrote Google Plus must stop this Identity Theatre

Google+ is letting an algorithm decide what is a name and what isn't. You will be forced into it's Procrustean idea of what names are, or be harassed for it. You have to pass as normal, like call centre workers forced to learn to sound American.

You can create disposable accounts with fake names, as long as they look plausible to Yonatan's bot.

This algorithm has allowed people called 'panel heater' 'The Phoenix Rising' 'tous les mais du monde' and Mehr Decent , a bot with a well-known actress's photo posting links to a single website to follow me (and that's just in the most recent 30 I checked).

So Google continues to encourage fakers and discourage those who need a pseudonym for good reasons.

Could Apple make premium devices in the USA?

After This American Life's disturbing episode on Apple's Chinese factories, the NYT wrote a defence of Apple, which said it was just too expensive to build their products in the USA:

Not long ago, Apple boasted that its products were made in America. Today, few are. Almost all of the 70 million iPhones, 30 million iPads and 59 million other products Apple sold last year were manufactured overseas.

Why can’t that work come home? Mr. Obama asked.

Mr. Jobs’s reply was unambiguous. “Those jobs aren’t coming back,” he said.

For computers, phones and tablets, it's hard to make a real premium product, as the economies of scale work so well - Tim Cook's Apple has closed in on PC prices by a focus on costs and suppliers, and by building fewer models and relying on Chinese flexibility to ramp them up.

The Gold iPad 2 had a huge premium price, but also weighed more the 3 times as much as a normal iPad.

Instead, what if Apple made premium USA iPads, MacBooks and iPhones? They could have a distinctive look, so people knew they were US made, focus on the higher-end models, and charge a premium markup for the warm glow of supporting US jobs.

How much more would it cost? Hard to say, according to the NYT:

It is hard to estimate how much more it would cost to build iPhones in the United States. However, various academics and manufacturing analysts estimate that because labor is such a small part of technology manufacturing, paying American wages would add up to $65 to each iPhone’s expense. Since Apple’s profits are often hundreds of dollars per phone, building domestically, in theory, would still give the company a healthy reward.
[...]
Another critical advantage for Apple was that China provided engineers at a scale the United States could not match. Apple’s executives had estimated that about 8,700 industrial engineers were needed to oversee and guide the 200,000 assembly-line workers eventually involved in manufacturing iPhones. The company’s analysts had forecast it would take as long as nine months to find that many qualified engineers in the United States.

In China, it took 15 days.
[...]
A few years after Mr. Saragoza started his job, his bosses explained how the California plant stacked up against overseas factories: the cost, excluding the materials, of building a $1,500 computer in Elk Grove was $22 a machine. In Singapore, it was $6. In Taiwan, $4.85. Wages weren’t the major reason for the disparities. Rather it was costs like inventory and how long it took workers to finish a task.

Compared the the huge price disparities for other goods, these seem modest; for example, Timoni found a nice carry-on bag recently:

Couldn't find carryon I wanted. Then found a nice one: "This is good, I could get this." Price? $8,000. *doh*

— timoni west (@timoni) January 9, 2012

So here's my proposition for Tim Cook:
Reopen the Elk Grove Apple factory to sell top-line Apple products, designed for those who want 'designer' luxury goods, and are willing to pay more for exclusivity. Make the 'made in USA' a key argument for a premium price. that way you need fewer staff than in China, and paying them well just adds to the cachet of the devices. You could cover them in Jasper Johns Flag, visibly number them as a limited edition, or come up with something more creative. As a way of extending the product line to a new, higher price point, while quieting those who wish Apple did more in the US, it seems an a obvious move.

Translation from sanctimonious bluster to English of Chris Dodd's statement on the internet blackout protests

WASHINGTON —The following is a statement by Senator Chris Dodd, Chairman and CEO of the Motion Picture Association of America, Inc. (MPAA) on the so-called “Blackout Day” protesting anti-piracy legislation:

Senator and CEO - let's lead with the revolving door promises to politicians

“Only days after the White House and chief sponsors of the legislation responded to the major concern expressed by opponents and then called for all parties to work cooperatively together,

Why are my former colleagues listening to their constituents about legislation? Don't they stay bought?

some technology business interests are resorting to stunts that punish their users or turn them into their corporate pawns, rather than coming to the table to find solutions to a problem that all now seem to agree is very real and damaging.

Maybe if we keep saying copyright infringement is a real problem without evidence, they'll believe it.

It is an irresponsible response and a disservice to people who rely on them for information and use their services.

How dare they edit their sites unless we force them to under penalty of perjury and felony convictions?

It is also an abuse of power given the freedoms these companies enjoy in the marketplace today.

Tomorrow was supposed to be different, that's why we bought this legislation.

It’s a dangerous and troubling development when the platforms that serve as gateways to information intentionally skew the facts to incite their users in order to further their corporate interests.

Being the gateways and skewing the facts is our job, dammit.

A so-called “blackout” is yet another gimmick, albeit a dangerous one, designed to punish elected and administration officials who are working diligently to protect American jobs from foreign criminals.

I am high as a kite

It is our hope that the White House and the Congress will call on those who intend to stage this “blackout” to stop the hyperbole and PR stunts and engage in meaningful efforts to combat piracy.”

What have the Romans done for us? Apart from instantaneous global communications, digital audio and video editing, the DVD, Blu-ray, Digital projection, movie playback devices in everyone's pockets and handbags...

How to fight this nonsense

(with apologies to John Gruber and Mark Pilgrim)

Facebook, Twitter and Google Plus shun HTML, causing the infographic plague.

By choosing images over links, and by restricting markup, Facebook, Twitter and Google+ are hostile to HTML. This is leading to the plague of infographics crowding out text, and of video used to convey minimal information.

The rise of so-called infographics has been out of control this year, though the term was unknown a couple of years ago. I attribute this to the favourable presentation that image links get within Facebook, followed by Twitter and Google plus, and of course though other referral sites like Reddit. By showing a preview of the image, the item is given extra weight over a textual link; indeed even for a url link, Facebook and G+ will show an image preview by default.

Consequently, the dominant form of expression has become the image. This was already happening with LOLcats and other meme generators like Rage Comics, where a trite observation can be dressed up with an image or series of images.

Before this, in the blogging age, there was a weight given to prose pieces, and Facebook and Google preserve some of this, but the expressiveness of HTML through linking, quoting, using images inline, changing font weight and so on, is filtered out by the crude editing tools they make available.

Feeds and feed readers started out this way too, but rapidly gained the ability to include HTML markup. Twitter went back to the beginning, and added the extra constrain of 140 characters because of it's initial SMS focus. Now it is painfully reinventing markup, though the gigantic envelope and wrapper of metadata that accompanies every tweet. This now has an edit list for entities pointing into it, and instructions for how to parse this to regain the author's intent is part of the overhead of working with their API.

Image links, however — at least those from recognised partners — are given privileged treatment. Facebook and Google have emulated this too, leading to the 'trite quote as image' trope. The spillover of this to news organisations became complete this year, with blogs and newspapers falling over themselves to link to often-tendentious information presented in all-caps and crude histogram form.

So here's my plea for 2012: Twitter, Facebook, Google+: please provide equal space for HTML. And for authors and designers everywhere, stop making giant bitmaps when well-written text with charts that are worth the bytes spent on them could convey your message better.

Update: My son made a Rage Comic version of this post (with an explanation) why.

Our brains make the social graph real

Brilliant web essayist Maciej Cegłowski of Two Steaks and Pinboard fame has focused his considerable insight on the area of web standards I've been involved with for the past few years. You should go and read his The Social Graph is Neither now.

Maciej is spot on in his criticisms:

This obsession with modeling has led us into a social version of the Uncanny Valley, that weird phenomenon from computer graphics where the more faithfully you try to represent something human, the creepier it becomes. As the model becomes more expressive, we really start to notice the places where it fails.

Personally, I think finding an adequate data model for the totality of interpersonal connections is an AI-hard problem. But even if you disagree, it's clear that a plain old graph is not going to cut it.

Clearly you can't model human relationships exactly in software. Keeping track of a few hundred of them in all their nuanced subtlety is why our brains are so huge compared to other animals. As Douglas Adams put it:

Of course you can’t ‘trust’ what people tell you on the web anymore than you can ‘trust’ what people tell you on megaphones, postcards or in restaurants. Working out the social politics of who you can trust and why is, quite literally, what a very large part of our brain has evolved to do.

It is an act of hubris to attempt to represent such vital things as human relationships in a database, and those who have done so often do resemble Maciej's Mormon bartender - Orkut Büyükkökten, Mark Zuckerberg and Jonathan Abrams do seem to have made what danah boyd has called Autistic Social Software.

The thing is, people seem to find these attempts helpful. As Maciej points out, we're good at forming subcultures and relationships even around the most primitive of tools. He pokes fun at opensocial.Enum.Drinker.HEAVILY, but when we were compiling the OpenSocial Person fields, we found a high degree of convergence between the 20 or so social network sites we reviewed. Despite their crudity, the billions of people using these sites do find something of interest in them.

People choose to model different relationships on different sites and applications, but being able to avoid re-entering them anew each time by importing some or all from another source makes this easier. The Social Graph API may return results that are a little frayed or out of date, but humans can cope with that and smart social sites will let them edit the lists and selectively connect the new account to the web. Having a common data representation doesn't mean that all data is revealed to all who ask; we have OAuth to reveal different subsets to different apps, if need be.

The real value comes from combining these imperfect, scrappy computerized representations of relationships with the rich, nuanced understandings we have stored away in our cerebella. With the face of your friend, acquaintance or crush next to what they are saying, your brain is instantly engaged and can decide whether they are joking, flirting or just being a grumpy poet again, and choose whether to signal that you have seen it or not.

As danah says:

While we want perfect reliability for our own needs, we also want there to be failures in the system so that we can blame technology when we don’t want to admit to our own weaknesses. In other words, we want plausible deniability. We want to be able to blame our spam filters when we failed to respond to an email that someone sent that we didn’t feel like answering. We want to blame cell phone reception when we’ve had enough of a conversation and “accidentally” hang up. The more reliable technology gets, the more we have to find new ways for blaming the technology so that we don’t have to do the socially rude thing.

So here's to approximate, incomplete social web standards.

'with Amazon' replacing 'with Google' on Android?

Amazon is set to launch an Android Tablet on Wednesday. What if they license their code too? Android as experienced on phones is actually two separate software bundles - the Open Source core of Android, and the proprietary 'with Google' applications, including the App Market, Maps, Gmail, Talk, Contacts, Listen and other apps bound to Google services, and requiring a business development deal to ship with a device. Eric Schmidt explicitly discussed this strategy at Dreamforce.

Now there are already more Android devices than I can count that don't follow the 'with Google' playbook, including the Barnes & Noble Nook that probably inspired this response from Amazon, but there are hints of a broader strategy here. What if Amazon offered an alternative to Google's top half of Android? I think Amazon does not really want to be in the hardware design business, but wants to be sure that they can't be locked out of it or forced to pay extra by Apple, Google or any other potential competitor. As well as releasing their own 7" tablet, they could offer an Open Source or lightly licensed version of their stack to other hardware developers.

Why would Amazon do this? Because they are primarily in the shopping and media business. Apple has stopped them selling eBooks and media inside their apps on iPad/iPhone; Google has banned their App Store from the Google Android Market. Amazon could even offer a referral fee for anything bought via their store as an incentive for device manufacturers to ship it.

An even bolder step wold be to actually fork Android. Google has a delayed-open model for Android source, where a new version is released in public after a closed development process, without a clear way to send in patches to Google. Amazon could put their current version up on Github, accept patches, and treat Google's new drops as another source of possible patches.

Understanding each company's core business is what makes this likely. Apple is in the devices business, with the media business as a small side earner designed to make their devices more attractive. Google is in the Advertising business, with their Android business designed to make searching everywhere, continuously more likely. Amazon is in the shopping business, migrating from physical goods to media, with Kindle a way to drive this. A tablet that they can sell audio and video to as well as eBooks makes more sense to them if it as widely distributed as Kindle playback apps are now.

Is Netflix picking the right disruption?

The decision to split Netflix into two companies, with the poorly-named Qwikster getting the DVD by post business and Netflix keeping the streaming business has caused a lot of fuss. Bill Gurley suggests that this is due to the very different licensing regimes the two businesses work under, and Clayton Christensen has praised this an a rare example of a company pre-emptively disrupting itself.

However, Reed Hastings gave rather non-plussed responses to those who complained about not having the two queues (DVD and streaming) integrated. As danah said,

It may seem logical to split the world based on business models from the inside of a business, but if you want your business to succeed, you should be focusing on understanding your users' mental models. And those aren't organized along business lines. They're organized around movies that they like, obtained by the means that is appropriate to the particular context of that user. People understand Netflix through its database of movies and the ratings that they've spent time providing, not its distinct queues.

Hastings clearly isn't thinking about this from our point of view - we want to watch something, and are much less focused on the particular medium. Instead of separating the two modes, Netflix should be uniting them further - help us book cinema tickets too, or buy Blu-ray discs. Encourage us to bring in information on favourite films and TV shows from Facebook, Amazon, GetGlue. They still could do this in an exemplary way by having Netflix and Qwikster share users' information through public APIs that others could use too. Activity Streams was made for this.

Being on the users' side in this way is another disruption, and indeed several startups at TechCrunch Disrupt had this mindset - what Doc Searls's VRM project calls '4th party tools' - ones that mediate between the customer and the vendor on the customers behalf. Cake Health mediate between you and insurers, TalkTo between you and local shops, FlickMunk between you and cinemas, and u4them as a way to donate to others medical bills.

The deeper currents of disruption of the film and TV industry are showing up in music. At SF Music Tech last week, turntable.fm was on everyone's lips, as the site that has got us all sitting round playing music for each other again, like the older label execs fondly remember from the 1970s. What it has done is apply the semi-overlapping publics and sharing models of twitter to listening to music together.

The other critique of Netflix that I saw was Megan McCardle saying that they were freeloading on the studios by only paying the marginal cost. Someone has to fund the creation, she pleads with us. Again, the answer was assumed at SF Music Tech, in the form of Kickstarter, which explicitly encourages people to fund production, and not just at marginal cost either. A key part of a successful Kickstarter project is widely spaced payment options, the special deals that are really about showing support with largesse. That these are power-law distributed seems odd at first sight - why would people pay more? But in fact it makes perfect sense. Income and wealth are power-law distributed in the US too, so people can pick the level of patronage that fits their income.

Independent artists like Pomplamoose and Zoë Keating are not served by the commodity pricing of Spotify - many are removing their songs from the catalog; they'd rather host them for download themselves. Zoë reports that her Bandcamp site, which lets you pay as much as you like for an album, has received payments of $8 to $500. Because they want to support her.

Cory Doctorow once said that "If big-budget movies might turn into opera, then long-form narrative books might turn into poetry." Opera has long understood the power-law distribution of wealth, and seeks donations in the same kind of structure.

In some ways, this power-law distribution of price is visible throughout commerce in the US. You can pay anything from $1 to $5000 for a burger, with price points inbetween, similarly for housing, transport, drinks, clothing, shoes, you name it.

The old economic choice between a commodity business that's all about margins, or a fashion business that is about competing for the most popular spot is finding a new accommodation. The discovery mechanisms like Turntable, Spotify, Pandora, and yes even Netflix, need to connect to these artist empowering patronage sites, as well as the commodity playback from the industrial aggregators. They need to lead people to Kickstarter and Bandcamp too. They need to be convenient, comprehensive and supportive of those creating art.

Update:Zoë Keating on Spotify, Apple and Independents (and lettuce)

Google Plus must stop this Identity Theatre

Bruce Schneier in Beyond Fear coined a phrase:

one of the goals of a security countermeasure is to provide people with a feeling of security in addition to the reality. But some countermeasures provide the feeling of security instead of the reality. These are nothing more than security theater. They're palliative at best.

The Common Names debâcle at Google Plus is a variant of this, where the supposed protections are manifestly not working. Google's stated policy on this is that you should use your 'common name' - normatively defined to have exactly two words in it, in a naïve English speaking way, that fails in a huge number of common English cases, let alone other languages.

Vic Gundotra has said

he is trying to make sure a positive tone gets set here. Like when a restaurant doesn't allow people who aren't wearing shirts to enter.

so it is explicitly designed to exclude 'people not like us' from the space.

Early users can set the tone for a network, but one that has aspirations to include most people will need to support multiple different communities within it. If you want a positive tone, you have to work at it, and empower the tummlers to maintain it. Teresa Nielsen-Hayden put it well:

1. There can be no ongoing discourse without some degree of moderation, if only to kill off the hardcore trolls. It takes rather more moderation than that to create a complex, nuanced, civil discourse. If you want that to happen, you have to give of yourself. Providing the space but not tending the conversation is like expecting that your front yard will automatically turn itself into a garden.

2. Once you have a well-established online conversation space, with enough regulars to explain the local mores to newcomers, they’ll do a lot of the policing themselves.

More from Teresa and from John Scalzi.

The initial flavour of Google Plus, because it was seeded by Googlers and other geeky folk they invited, was like pre-Eternal September Usenet - it had a cultural coherence because we were all geeks. As it grew to 25 million users, this could not hold.

Blogs deal with this by making it clear who the site owners are, and empowering them to manage commenters. Twitter does it by not showing you comments unless you chose to see the commenter, or if they address you directly. Google Plus is an uneasy hybrid of the two.

You can delete and block commenters on your postings, like a blog, and if you reshare someone's post, it starts a new comment thread, like a blog. However, anyone can @ or + your name and drag you into another comment thread via notification, and then you get notified of other follow-ups too, making griefing and harassment all too easy.

Enforcing 'common names' does nothing to help this; it just means your trolls and griefers will be using plausibly American-looking names that may or may not be their own, while those with unusual names, will either be excluded outright or easily preyed on by the griefers reporting them, which is what I suspect happened to Violet Blue tonight.

Once you are suspended, the verification process is crude and manual, and also easily gamed. Kellan warned about this problem:

If you’ve never run a social software site … let me tell you: these kinds of false positives are expensive.

They’re really expensive. They burn your most precious resources when running a startup: good will, and time. Your support staff has to address the issues (while people are yelling at them), your engineers are in the database mucking about with columns, until they finally break down about build an unbanning tool which inevitably doesn’t scale to really massive attacks, or new interesting attack vectors, which means you’re either back monkeying with the live databases or you’ve now got a team of engineers dedicated just to building tools to remediate false positives. And now you’re burning engineer cycles, engineering motivation (cleaning up mistakes sucks), staff satisfaction AND community good will. That’s the definition of expensive.

And this is all a TON of work.

And while this is all going down you’ve got another part of your company dedicated to making creating new accounts AS EASY AS HUMANLY POSSIBLE. Which means when you do find and nuke a real spammer, they’re back in minutes. So now you’re waging asymmetric warfare AGAINST YOURSELF.

This is the hole Google is now in. A surprisingly large number of people I know, who've been discussing civilly online for years, have fallen foul of Vic's Procrustean name rules. When they point this out, they're harrassed by 'Real named' dickheads telling them to shut up and change their name, both in public and by being +-summoned by the trolls, and they have to find Google plus's well-hidden blocking tools rather quickly. Or give up and go elsewhere.

Now, Google has announced that they are verifying some people's names, to prevent impersonation. Trouble is, they haven't said how . Twitter verifies celebrities via an opaque process. Amazon does it by checking your name matches a Credit Card. Google Search uses rel="me" and rel="author" microformats. What Plus does is unknown. One of my profiles is verified, possibly because I went through the verification process on Google Knol before.

This is also Identity theatre - Google saying 'trust us', rather than revealing the rel="me" link from the person's page that we already know.

Vic Gundotra needs to stop digging this hole. Scrap the normative 'common names' policy, add a coherent name verification and linked-site verification so we can tell the people we already know, and make moderation tools visible and available so we can curate the conversations ourselves.

With this, and an apology to those already ensnared by the existing process, he could maybe prevent Plus from being spoken of only alongside Wave and Knol.

More on this:

• my.nameis.me has testimonials on the nuances of names
  
• jwz is shocked by othering of the anonymous
  
• The EFF's case for pseudonyms
  
• Journalists tempted by slumming

David Cameron should heed Douglas Adams and ORG

Widely reported today are David Cameron's comments to parliament on riots and social media:

Mr Speaker, everyone watching these horrific actions will be stuck by how they were organised via social media.

Free flow of information can be used for good. But it can also be used for ill.

[And when people are using social media for violence we need to stop them.]

So we are working with the Police, the intelligence services and industry to look at whether it would be right to stop people communicating via these websites and services when we know they are plotting violence, disorder and criminality.

I have also asked the police if they need any other new powers.

(the bit in square brackets was in his press statement, but not read in the Commons)

This particular line of reasoning was magnificently rebutted by Douglas Adams in 1999:

Newsreaders still feel it is worth a special and rather worrying mention if, for instance, a crime was planned by people 'over the Internet.' They don't bother to mention when criminals use the telephone or the M4, or discuss their dastardly plans 'over a cup of tea,' though each of these was new and controversial in their day.

I was encouraged recently when the UK Govt abandoned web blocking plans in the Digital Economy Act. Understanding that the internet is there for common carriage (a mere conduit, as the EU puts it) is important. Even on its own terms this threat makes little sense: if people are plotting riots on social media, that is surely exactly the evidence you need to convict them under the UK's statutory Conspiracy law. The telephone, the M4 and cups of tea are much harder to use as sources of evidence.

The Open Rights Group, has a typically measured and thoughtful response to this.

Cameron should be careful, or he'll look to posterity like William Cobbett ranting about the pernicious evils of tea.

Should 'Money' be an adjective, not a noun?

I've been following Ben Laurie's thoughts on Bitcoin and now his new paper on An Efficient and Practical Distributed Currency.

He envisages a group of 'mintettes' that can agree on the distribution of coins, transfer them between individuals, and mint new ones, deciding between themselves how to distribute these. I like the idea of the different mintettes having different Public Good type ideas of where the newly created coins get assigned. The key here is to grow the coin supply at a rate that is lower than the growth of value held, so holders of your coinage get some appreciation, and distribute the new money to worthy causes, or to clients of that mintette.

In effect you're doing an end run around Gresham's law, in the same way that the Brazilian Real did - and not how the US Govt is doing with dollar coins. This is the bit that the libertarian summer camp got backwards - although they traded with gold, they set prices in US dollars.

We do have a precedent for this, and it is an encouraging one. In effect, each company stock is a private currency. The success of Silicon Valley has been helped by the ability of companies here to mint this money-like stuff, and distribute it to stakeholders and investors alike. The difference is that they create new tranches of 'coins' at board meetings, though stock option vesting is a bit like the smooth currency growth that Bitcoin and Ben envisage. Again the goal is growing the money supply at a rate below demand, so that those holding it are rewarded.

In effect we already have things that are more or less like currencies, and these new ones have some encouraging precedents.