Standards are the links of the Social Web

Mike Arrington wrote a plea for better social software on Sunday:

The online social landscape today sort of feels to me like search did in 1999. It’s a mess, but we don’t complain much about it because we don’t know there’s a better way.

Everything is decentralized, and no one is working to centralize stuff. I’ve got photos on Flickr, Posterous and Facebook (and even a few on MySpace), reviews on Yelp (but movie reviews on Flixster), location on Foursquare, Loopt and Gowalla, status updates on Facebook and Twitter, and videos on YouTube. Etc. I’ve got dozens of social graphs on dozens of sites, and trying to remember which friends puts his or her pictures on which site is a huge challenge.

What enabled Google to solve the search problem was a common standard for expressing pages and the links between them, so that they could index the webpages and derive a metric for which ones were more important. They didn't do this by replacing the web with a structured database that they curated, they worked with the standards in use to make sense of it.

To solve the social conundrum we need the equivalent - agreed standards in widespread use so that we can generalize across sites. Fortunately, we have these. We have OpenID and OAuth for delegated login; we have XFN, other microformats and Portable Contacts for public and private people connections; we have Feeds and Activity Streams for translating social actions between sites.

This enabling social infrastructure means that we'll be able to have a new generation of sites that enhance our web experience through social filtering without our connections being centralised in a single company's database.

Once we get used to the experience of being able to delegate login, personal connections and activity updates, we'll look askance at developers who insist we create yet another profile and invite all our friends by email to experience their site; it'll be like a website without links.

Farewell to Google

I'm no longer working for Google. I had an interesting time there and worked on lots of fascinating projects with great colleagues, so this is a small look back at some of them.

My first taste of Google was to work on orkut, before starting the project now known as Google Profiles, which was first launched in Google Maps, and is now seen across Google and the wider web. I then worked on the engineering side of OpenSocial, before its launch. Realising that Google had thousands of engineers, but very few comfortable speaking in public, I became a Developer Advocate, working to bridge external and internal developers, explaining the Social web to Google and OpenSocial and more to the wider web community.

I've spent most of my time working on building and promoting open web standards, both inside the company and out. I helped launch the Social Graph API, promoted OAuth and OpenID, helped converge Portable Contacts with OpenSocial, and explained how the Open Stack fits together. I helped promote Microformats within Google and without and am very pleased to see them showing up in Rich Snippets in search. The Activity Streams effort continues this web-wide work to build social infrastructure to make the web more social.

I'll still be working on web standards through the groups above, the Open Web Foundation, the Open Rights Group, and more. Professionally, I'll be coding, writing and speaking on the social web via several new projects. I hope to see many of you this week when I'm talking to the SFAMA on Thursday night, and hosting the Microformats 4th Birthday party on Friday.

If you want to get hold of me, I'm kevinmarks on most social networks, domains and of course Twitter. Or just google me.

Notes on Charlene Li's Future of Social Networks SF AMA talk

Last night I went to an interesting talk by Charlene Li at the SF American Marketing Association -here are my twittered notes. See also, Charlene's slides.

says @charleneli: Theme is "social networks will be like air" - her better phrasing of my "Social Cloud" idea
says @charleneli: in future we'll say "wasn't it quaint that we had to go someplace to be with our friends"
says @charleneli: "I want Amazon to have a 'friend's reviews button on there - or anywhere else they could be"
says @charleneli: we'll have a feed of the presedential debates with our friends tweets on - like I did in 2004: http://bit.ly/IRCdebate
says @charleneli: universal login with OpenID lets you tie your IDs together, and sites can import friends from your networks
says @charleneli: I had to friend my co-author Josh 35 different times on different sites - Portable Contacts should save us from this pain
says @charleneli: Profiles where they are useful - eg LinkedIn profiles showing up in Lotus Notes via email
says @charleneli: your friends activities in context with GetGlue.com's plugin - Iron Man wikipedia page and IMDB page shows friends reviews
says @charleneli: 2 sets of standards exist Facebook's own protocols and the OpenStack backed by Google, MySpace, Plaxo, Yahoo and more
says @charleneli: advertising has evolved - content targetting for demographics; Search marketing for intent; behavioural targetting
says @charleneli: how many of you have gone to a social network site and remember seeing an Ad? or clicked on one?
says @charleneli: Who wants to be a fan of FiberOne on Facebook?
says @charleneli: people want to tell each other about things they care about - need new ads for this
says @charleneli: examples of new Ad types - branded virtual gifts, shown to you as your friends gave or received them
says @charleneli: SocialVibe has profile sponsorships that donate to your favourite charity eg colgate ad to leukemia
says @charleneli: the Tipping Point argued that there are influencers that can make a product go viral [I disagree see http://bit.ly/watts ]
says @charleneli: social graphs and interests, culture of sharing and online and email behaviour can create context for ads
says @charleneli: vendors who identify influencers include 33across, lotame, media6 degrees, unbound technologies
says @charleneli: network neighbourhood modelling in interesting - homophily is a good predictor for clusters - you are like your friends
says @charleneli: Google tracks who I email most - very useful to me: "In Google I Trust" http://bit.ly/BtvV
says @charleneli: Media6 identifies you by profiles you view on SNSs - shows ads to your friends based on your purchases
says @charleneli: Media6 gets 3-7x increase in response rates on banner ads through this homophilic targetting - no PII involved
says @charleneli: Influencer strategies are a misnomer, btu clustering works
says @charleneli: People will demand greater contol over when, where, how profiles + friends are used. Detailed permissions - a UX nightmare
says @charleneli: remember when people didn't trust callerID? Now if you turn it off, people won't take your call
says @charleneli: setting up lists of who can see your pictures is a pain - have to categorize people - reclassifying is hard
says @charleneli: there's a need to better articulate and detect sub-groups of friends so this is less of a chore
I pointed out the power of asymmetric friending eg http://bit.ly/publics and @charleneli and audience agreed that it reduces awkwardness
says @charleneli: people will pay real money for virtual gifts
[ChrisSaad @kevinmarks asymmetic is good, the term friending is not great. I prefer follow or subscribe ]
@ChrisSaad agreed "following" is a better term for this
Audience: when will people profit from us using their profiles? @charleneli says we all have our own CPMs
[clynetic @kevinmarks What is CPM?]
@clynetic CPM is marketingspeak for 'cost per thousand' - I suppose CPA ( cost per action) is better
says @charleneli: don't give up your social capital for short term gain me: don't be the Amway guy at the party
says @charleneli: behavioural targetting is often faulty, as behaviours change
says @charleneli: social media advertising experiments are waiting for turnaround
says @charleneli: GYM (Hotmail for M) will test social media integration with webmail
says @charleneli: Facebook Connect and Open Stack gaining traction with media co's
says @charleneli: Social shopping experiments start - we want our friends recommendations
says @charleneli: identify where social network data and content shoudl be integrated in your sites
says @charleneli: leverage existing identity and social graphs where your audience is
says @charleneli: get your privacy and permission policies aligned with an open strategy
says @charleneli: find your trust agents - in google I trust? do you trust facebook?
says @charleneli: the media buyers are still trying to buy demographics or content, not better targetting

Cycling to new layers of freedom

Dave Winer used the public beta of Google Friend Connect to reflect on tech industry cycles:

A new generation of young techies comes along, takes a look at the current stack, finds it too daunting (rightly so) and decides to start over from scratch. They find that they can make things happen that the previous generation couldn't cause they were so mired in the complexity of the systems they had built. The new systems become popular with "power users" -- people who yearn to overcome the limits of the previous generation. It's exhilirating! [...]
The trick in each cycle is to fight complexity, so the growth can keep going. But you can't keep it out, engineers like complexity, not just because it provides them job security, also because they really just like it. But once the stack gets too arcane, the next generation throws their hands up and says "We're not going to deal with that mess."

Now, I may be a few years behind Dave, but I think he is throwing the baby out with the bathwater, or the stack out with the cycle here. Back when I started out, to get my computer to generate sound, I had to make my own D to A converter to attach to the parallel port, and for non-character graphics, my hardware hacker friends swapped the character generator ROM for RAM, and I had to code in assembler to swap the display data in time.

Now my son thinks nothing of mixing 10 polyphonic Midi tracks in an afternoon or editing hi-def video (and yes, it's on an OS I helped to make capable of that).

Dave's revolutionary impulsiveness has a germ of truth, but what really happens is that successful technologies become invisible infrastructure for the next things that build on them.

I no longer need to write assembler, heck I no longer need to write C code. Dave's very URL - scripting.com - shows how we have built up layers of utility to work upon.

HTTP, HTML, JSON, Atom and Javascript are infrastructure now. Our deepest role as developers is to build the invisible infrastructure for the next generation to take for granted, so they imagine new abstractions atop that. Dave did it with feeds.

What we're doing with the Open Stack — OpenID, OAuth, PortableContacts and OpenSocial— is part of this evolutionary cycle too. We're combining building blocks into a simplified whole that makes sense to people who want their websites to become social.

It comes down to what you can take for granted as the baseline to build the next exciting cycle on.

Open Source and Social Cloud Computing

Tim O'Reilly has written an excellent review post on Open Source and Cloud Computing which says, among other things:

The interoperable internet should be the platform, not any one vendor's private preserve.

So here's my first piece of advice: if you care about open source for the cloud, build on services that are designed to be federated rather than centralized. Architecture trumps licensing any time.

But peer-to-peer architectures aren't as important as open standards and protocols. If services are required to interoperate, competition is preserved. Despite all Microsoft and Netscape's efforts to "own" the web during the browser wars, they failed because Apache held the line on open standards. This is why the Open Web Foundation, announced last week at OScon, is putting an important stake in the ground. It's not just open source software for the web that we need, but open standards that will ensure that dominant players still have to play nice.

The "internet operating system" that I'm hoping to see evolve over the next few years will require developers to move away from thinking of their applications as endpoints, and more as re-usable components. For example, why does every application have to try to recreate its own social network? Shouldn't social networking be a system service?

This isn't just a "moral" appeal, but strategic advice.[...]

A key test of whether an API is open is whether it is used to enable services that are not hosted by the API provider, and are distributed across the web.

I think this API openness test is not strong enough. As I wrote in An API is a bespoke suit, a standard is a t-shirt, for me the key test is that implementations can interoperate without knowing of each others' existence, let alone having to have a business relationship. That's when you have an open spec.

The other thing I resist in the idea of an internet operating system is that that the net is composable, not monolithic. You can swap in and implementations of different pieces, and combine different specs that solve one piece of the problem without having to be connected to everything else.

The original point of the cloud was a solved piece of the problem that means you don't have to worry about the internal implementation.

Thus, the answer to "shouldn't social networking be a system service?" is yes, it should be a Social Cloud. That's exactly what we are working on in OpenSocial.

Digital publics, Conversations and Twitter

Last week, I left the Web 2.0 conference to listen to Mimi Ito, danah boyd and their colleagues talk about their research on Digital Publics.

Now if you haven't been paying attention, that plural of 'public' there may throw you. Surely things are either 'public' or 'private'? As danah explains:

Just as context is destabilized through networked publics, so is the meaning of public and private. What I learned from talked to teens is that they are living in a world where things are "public by default, private when necessary." Teens see public acts amongst peers as being key to status. Writing a public message to someone on their wall is a way of validating them amongst their peers. Likewise, teens make choices to go private to avoid humiliating one of their friends.

Yet, their idea of public is not about all people across all space and all time. They want publics of peers, not publics where creeps and parents lurk.

Bly Lauritano-Werner (17, Maine):

My mom always uses the excuse about the internet being 'public' when she defends herself. It's not like I do anything to be ashamed of, but a girl needs her privacy. I do online journals so I can communicate with my friends. Not so my mother could catch up on the latest gossip of my life.

Properties of technology have complicated what it means to be in public. We are all used to being in publics that don't include all people across all space and all time. Many of us grew up gossiping with friends out in public and stopping the moment that an adult walks over. This isn't possible when things are persistent. And it's really hard to be public to all peers and just keep certain people out. So teens are learning how to negotiate a world where the very meaning of public and private have changed. Again, this is a good thing. They're going to need these skills in the future.

The day before, at Web2Open, I had heard something similar in the Troll Whispering session. Christy Canida explained that when someone posts something trollish or otherwise dubious on her site, they get put in a state where only they can see their posts, but no-one else can (except Christy and the other conversation monitors). This damps down the flame responses until Christy and co have time to review, and maybe release them, but in their view the post is on the site, but no-one is responding.

This varying view of the web, depending on who you are, seems odd at first, but it is in fact a recognition in code of what actually exists in human attention. We don't all read the same web, we see our own reflections in what we seek through searches or filtered by our homophily-led reading.

Which is where Twitter comes in. Like Jeff, I've been twittering more than blogging recently, and while immediacy is part of it, a far stronger thing is that I have a sense of public there - a public of people I choose to follow and who chose to follow me. Everyone who uses Twitter sees a different, semi-overlapping public, which maps closer to our individual idea of the digital public we are speaking to, and listening to; one that maps more closely what the socialogist and theorists have been describing for a while.

The Social Cloud

My talk from LIFT is here for you to watch below (20mins, needs flash):


The others are up at the LIFT Video site