New python toys

Bob has released PyObjC 1.2, which lets you build Cocoa apps in Python. I made my zero configuration IRC app with this last year - I must check out the new version.

Bob is also looking for a blogging tool that lets him automate markup
- he should try Ecto, which Ado added a great new feature to - it will call your Python code to filter the text before posting (also perl, shell or AppleScript).

New year's resolutions

My New Year's Resolution is to blog more.

St Luke's News

A while ago I overhauled the St Luke's website, and added a blog, but it was only tonight that I finally got together with Michael Penfield to show him how to use it. St Luke's Chapel News will have regular updates from now on.

If you're looking for an Anglican service in Silicon Valley this Christmas Eve, do come along - it is a little gem of a church in the Los Altos Hills, and has the services that feel right to the expatriate English like me.

Shiny new beginning

Congratulations to Adam and the ops team for the efficient and successful move of all the Technorati servers to their new home.

This gives us a solid hardware foundation to build new Technorati services on for the future, and Joi has news of some.

Obfuscatory marketing

I'm disappointed by Ed Felten's 15 line p2p app.
Ed is a great communicator, and a smart programmer, but in this case he has abdicated explanation for a sound-bite.
He could have written his minimal p2p app clearly and used it to teach, but instead he fell into the Obfuscator's trap of optimising for lines of code, to make a marketing splash.

Peer to Peer communication is a natural part of programming these days; writing code that looks as cryptic as that fails to make the point clearly.

Squared Circle

Flickr's Squared circle tag. It's pretty, but there is this odd feeling that there is a joke here I'm not in on.

charming leakage

When I saw Suw's announcement that she and Chris had met and were in love I was happy for them, but not surprised. Sure, I had seen them talk in chatrooms, but the real clue was some information leakage from iChat AV.
It's presence information shows you when people are online, and when they are available for audio and video chat. So for a couple of weeks beforehand, I would glance at my buddy list and whenever Suw & Chris were online, they'd both have their A/V icons dimmed, as they were together, video-chatting. Congratulations.

Ocean's boiled to order

I was asked last night where the phrase 'Boil the Ocean' came from in Tantek and my presentation on lossless xhtml - Can your website be your API?

I wasn't sure, so I searched a bit, and found this:

Will Rogers’ response to a reporter’s question on how he would deal with the Nazi U-boats:
"Boil the ocean."
"But how would you do that?" the reporter continued.
Without a beat Rogers replied, "I’m just the idea man here. Get someone else to work out the details."

Value destroyed

Fortune.com:

Harder to fathom is the MSN TV 2 Player's inability to play music that has been downloaded through a PC from MSN's own Music Store. The Player had no problem with PC-created MP3 files and tunes ripped into the Windows Media Audio format from CDs but choked on music bought from MSN itself. An MSN representative said the incompatibility arises from the digital rights management (DRM) copy-protection system employed by the MSN Music Store to fend off pirates.

For the Fallen

For Remembrance Day, I have a special audioblog. This is Laurence Binyon, reading For The Fallen from the British Library's splendid collection The Spoken Word – Poets

With proud thanksgiving, a mother for her children,
England mourns for her dead across the sea.
Flesh of her flesh they were, spirit of her spirit,
Fallen in the cause of the free.

Solemn the drums thrill; Death august and royal
Sings sorrow up into immortal spheres,
There is music in the midst of desolation
And a glory that shines upon our tears.

They went with songs to the battle, they were young,
Straight of limb, true of eye, steady and aglow.
They were staunch to the end against odds uncounted;
They fell with their faces to the foe.

They shall not grow old, as we that are left grow old:
Age shall not weary them, nor the years condemn.
At the going down of the sun and in the morning
We will remember them.

They mingle not with their laughing comrades again;
They sit no more at familiar tables of home;
They have no lot in our labour of the day-time;
They sleep beyond England's foam.

But where our desires are and our hopes profound,
Felt as a well-spring that is hidden from sight,
To the innermost heart of their own land they are known
As the stars are known to the Night;

As the stars that shall be bright when we are dust,
Moving in marches upon the heavenly plain;
As the stars that are starry in the time of our darkness,
To the end, to the end, they remain.

BBC MP3s

Tom, our man on the inside at the Beeb:

There's this show on Radio 4 called In Our Time that's concerned with the history of ideas. Each week Melvyn Bragg brings together three guests (serious guests at the top of their fields) and they have a discussion around the major themes. It's kind of awesome if you're interested in science or history... Starting from shortly after last week's episode, now you can download and listen to the whole programme in non-DRM'd, easy to understand, iPod (and other media player) -compatible MP3 format!

Finally my campaign is bearing fruit. Get that In Our Time Electricity episode now.

Counting blogged votes

Having talked about vote links for well over a year now, it was about time someone started counting them, especially with tomorrow being election day in the US, and it might as well be me.
Presenting the Technorati Blogger Vote Count.
Instructions on how to add votes to your blog are there. Note that you can vote for or against each candidate individually.

What's wrong with this picture?

David Weinberger says:

When your wifi card doesn't work under XP, after spending three hours futzing with drivers, I suggest you try this:
Control Panel > Administrative tools > Services. Look for Wireless Zero Configuration. Click on it. If it's stopped, start it. If there's no start or stop button, double click on it and change "Startup type" to "Automatic."
Or you could get a Mac which, because it is a closed environment, tends to be easier to live with.

No David, the Mac is easier to live with because it's designers don't assume they control (or should control) all networking. Stuart, who invented Zero Configuration Networking, would say on seeing that Configuration dialog:

Which part of "zero configuration" did they not understand?

Audio recording

Dave says::

This is an old-style Morning Coffee Notes, the kind we did before we were doing audio blog posts, before they were called podcasts, back when we couldn't find any software to record our voices (seriously, PCs came with microphones, but search high and lo, hither and yon, there was no software to actually use the microphones in a most basic way).

Dave, you should have asked, instead of searching. Here's some free code to record audio using QuickTime from any Windows device. The code dates back to 1992, and was working on Windows in 1997.
This is what AOL use to do the Audio/Video chat in the latest AIM releases (that iteroperates with iChat).

Value destroyed is not owned

Tim Oren writes on DRM as a warning sign:

Copy protection DRM always destroys end user value, in both convenience and robustness. When you see DRM in a business plan or analysis, it is always there to benefit someone other than the end user. Find out who, it will indicate where power lies in a content value chain.

The mere presence of DRM indicates a failure to deliver end user value. If the information object were to lose value when extracted from the bundle or service from it was derived, DRM would not be felt necessary. Therefore the presence of DRM suggests a vendor that is behind the curve, failing to find a new value to deliver as their chokepoint disappears in the digital world.

Tim is correct that DRM destroys value, but he is mistaken in his implication that it can just be Customer value that is destroyed. The 0th law of economics is that a trade only happens when both parties see themselves as gaining from it. If the Customers see less value, they pay less for the product, and its value is thereby reduced. This kind of short-sightedness is foolish enough with conventional goods, but is especially stupid with digital media, when you have no measurable marginal cost of goods and can carry over Customers' excitement into other goods.

How about mass video editing?

Mark Cuban has some ideas for improving TiVos. This reminded me of an idea I had while watching the Olympics. TiVo collects data on which programs have been watched, which bits were fast-forwarded, and which were played more than once or in slow motion.
Imagine if it took something like the Olympics, or a baseball or football game, and collated everyone's replay speeds, and then offered up various highlights packages- the most viewed 5 minutes; most viewed hour and so on. This would naturally edit out all commercials, and the commentators padding, and show which bits of action people as a whole found interesting.

Losing language sales

This weekend we attended the Home School Geography Club, which was about Viet Nam, and enjoyed it immensely. Our friends the Hamiltons hosted it and, among many other fascinating things, taught us as few words in Vietnamese, and fed us Phô .
However when writing out the Vietnamese sheets, Bich was adding the accents by hand on the printout. It turns out that the Vietnamese keyboard inclued with OS X is far too hidden for anyone but experts to find(System Preferences, International, Input Menu Tab, and check the ones you want in a scrolling list).

This reminded me of an idea I had for an Apple ad campaign to highlight OS X's language support.

What I suggest is a poster campaign, showing a localised Mac screen running Mail with large type saying

Macintosh speaks your languages

Except that you do it as a teaser.

Start with the least common (in the US) languages Apple localises to, eg Korean, and work your way up the demographic to English, changing posters once a week or more often

매킨토시는 너의 언어를 말한다
マッキントッシュは言語を話す
Macintosh говорит ваши языки
Macintosh fala suas línguas
Macintosh spricht Ihre Sprachen
Macintosh parle vos langages
Macintosh habla sus lenguajes
Macintosh speaks your languages

Each time you change a poster and add a language, you switch the outer UI (menus etc) to that language. You deliberately place the posters in non-ethnic areas, so they are cryptic to most.
(Obviously, you get native speakers to translate the slogan instead of using Sherlock like I did).

47 million Americans speak a non-English language, according to the 2000 Census. 26 million also speak English well, 21 million are less proficient. Millions more learn a foreign language in school.

Imagine the media buzz these cryptic posters would generate, and the feeling of pride the bilingual people would have when they see an ad in their language, out in public.
In other countries, you do the same thing with a different language order.

Heckling the debate in irc

Dave Winer put up an mp3 of the debate; David Weinberger organised an irc chat to heckle it.
I combined the two:

You can call this audioblogging with comments or maybe it is something else.Note that if you open it in QuickTime Player, you can search the text for keywords like 'flu' or 'bin Laden'.

Direct link: http://homepage.mac.com/kevinmarks/johodebate.mov

Hackaton at Technorati next Wednesday

As we'll have a huge concentration of web geeks just up the road at Web 2.0 next week, we decided to have a hackathon at Technorati on Wednesday night.

Bring your laptops and brains, and hack on our API and other web code.

RSVP promptly

Getting semantic with Tantek

Tomorrow night, Tantek and I are talking at SDForum about Semantic XHTML:

Can your website be your API? - Using semantic XHTML to make your structures clear

XML formats gained popularity as a backlash against the messiness of HTML mixing structure and presentation, and leniency for sloppy markup. With XHTML+CSS now widely supported in mainstream browsers, and gaining converts even amongst those most focused on representation, these objections lose their force, and the resistance to more and more ad-hoc specialized schemas grows. How far can we get specifying structure in pure XHTML -valid XML - styling it with CSS for presentation, and making it parsable for meaning?

4 million served

Technorati just passed the 4 million blogs indexed point. We get about the same number of posts a week.

Sharks in a tank

We went to Monterey Bay Aquarium on Sunday and saw the baby Great White shark they have in the tank there - video below:

They say this is the only Great White to eat in captivity, and that the longest any lasted previously was 16 days. How annoying for the Bond villains with the shark tanks to have to restock every couple of weeks, and then to have the sharks refuse to eat when they drop victims in.

The tip if you go to see it is to go in the Members' entrance (membership is a good deal if you go even twice a year) and go straight to the 'Endangered Wildlife' section, which leads to the bottom of the Outer Bay tank, where the shark spends its time. If you go upstairs to the auditorium seating part of the Outer Bay tank upstairs you are very unlikely to see the shark clearly past the crowds.

Audio misunderstandings

IEEE Spectrum has an article on audio compression that manages to combine some useful info with sloppy misconceptions and questionable assertions. I annotated it to point these out. Do join me in correcting it for them.

Can't give money away

I'm told by James and Jim of VoteOrNot that the chances of winning $100,000 are good as their sign-up rate is lower than they expected.

Vote early and vote often.

Unacknlowedged legislators

I'm guest-blogging over at Many2Many; this is my first effort.

Shelley wrote that poets are the unacknowledged legislators of the world, and this dream lies behind a lot of blogging, though the literary archetype is perhaps Peter Wiggin rather than Byron.

The challenge for social software is to construct frameworks for people. Suw and Adina have recently discussed the analogies with architectural spaces; Joel about how having lots of people involved changes design.

I spent the holiday weekend building sandcastles, watching waves closely to decide which one to jump into, and reading Churchill's description of how political organisation evolved in the UK.

What I hope to do while guest-blogging here is to talk about how we build enduring frameworks that enable people to grow new, surprising institutions together.

Dewey dubiety

David Weinberger partially defends the Dewey decimal system. I see his point, but a system that gives Phrenology a top-level number (139) but sues people promoting it is doomed to an early death when there is a free and open alternative to refer to topics easily.

Dive Into Python

Mark Pilgrim's Dive Into Python is now available in book form . Reading it in print reminded me how good it was, and it has grown a few chapters since I first read it online.

If an aversion to reading online has kept you back, go buy it now. It is a very impressive piece of work - it manages to explain the Python language, and, more importantly the idioms and customs of Python programming, through a series of well-chosen and interesting examples. The chapter on test-driven development shows why this makes sense, how Python supports it, and how it leads to better code.

Mark writes as he does - look at his excellent Universal Feed Parser, with its thousands of test cases for a concrete example of the power of test-driven development.

Python is my favorite programming language. Mark does a great job of explaining why. Try it, you'll like it.

Call Off the Search

Explaining Technorati and blogging to people has been tricky. Once I explain we keep track of what people are saying and linking to in weblogs, their first reaction is to say 'Oh you're a search engine' (silent subtext: Google will eat you for breakfast).
This is where it gets tricky, because search is a task, not a goal.
Jeff Bezos and John Battelle help explain this better:

[Bezos uses] "discovery" as an umbrella term which incorporates search. I think in the end when I use the word "search" I really mean "discovery" as Jeff uses it. What's discovery? Well, much more in the book, but in the end, it's search plus what happens when the network finds things for *you* - based on what it knows of you, your actions, and your inferred intent.

Last week I watched Steve Jobs explain Technorati's advantage over Google - he was talking about Safari's RSS search, but Technorati searches millions of blogs for you within minutes of them updating, not just the RSS feeds you have already subscribed to.

But again, searching for keywords is missing the point.

The great thing about weblogs is when you discover someone. Someone who makes sense to you, or someone who surprises you with a viewpoint you hadn't thought of. Once you have found them you can subscribe to their feeds and see how they can keep inspiring or surprising you.
You can even start a blog, link to them, and join the conversation,
The continuity of viewpoint within a blog is key - you can see more about them than just the one comment, and you can keep discovering and growing with them. Conversely, being aware that what you are writing is 'on your permanent record' means that you write more carefully for a blog than for an email.

Blogcritics sent me a CD to review - Call Off the Search by Katie Melua. Rosie loved it, but the title song sums up what I'm getting at here: "Now that I've found you I'll call off the search."

Blogging is about what you discover, not about what you search for.
How you can follow the conversations and make new discoveries is what I'm working on. [updated 2014 - original Steve Jobs link was broken 5 times over by Apple: it linked to homepage.mac.com (Which they killed) hosting a Quicktime reference movie (which they killed) to a streaming Quicktime movie (which they killed) of a Steve Jobs keynote (which is now offline) explaining Safari RSS search (Which they killed).

Millions counted

Yesterday, Technorati hit 3 million blogs, and Apple hit 97 million iTunes songs.
So, if every blogger buys one song, the 100 million prize can be won.

No prize here, but the 3 millionth Technorati blog was Mi eterna ciclotimia

Citation and deep linking

Dorothea points out some problems around piecemeal citation:

[...] there’s no automated way to add callouts to one individual paragraph without adding callouts to all of them.

A more subtle explication of the problem: I could, if I chose, add individual id attributes to paragraphs on CavLec I thought especially worthy of notice. But who’s to say that my idea of noteworthy paragraphs meshes with any other blogger’s? Nobody, that’s who. (Not least because it’s an open question whether any paragraphs on CavLec are noteworthy.) The only way to ensure that anyone who wants to link to noteworthy paragraphs can do so is to assume that all paragraphs are potentially noteworthy.

Worse, even if I do add id attributes, there’s no way for a would-be linker to get at them for linking purposes except by inspecting my HTML code. Green hash marks may be crufty, but they address a genuine issue, one we might call “identifier invisibility.”

The way around this is to do what I just did - copy in the piece you are citing and link to the whole. It's a little cumbersome, but it has the benefit of resilience (the original might vanish or be re-edited). A way to take this technique further is to use QuickTopic Document Review, as I did for AKMA's speech for example. This both adds the paragraph citation links, enables inline comments, and archives a copy of the cited source elsewhere, protecting against it changing or vanishing and thus invalidating the citation link.

This is the same issue as discussed by Jon Udell last month for MP3's.
If you want to cite an MP3 in a stable way, you can do it by copying a fragment and saving it locally, and linking back to the original source file. We don't try to dynamically insert chunks of text from other people's servers into the middle of our prose; why do it for media?
What is missing here is the rich media equivalent of QuickTopic Document Review, which mirrors media and adds annotation. Building something to enable this would be a fine project for the Internet Archive.

Dashboard is nice, but how DO I make Safari sidebars?

Dave Hyatt said something interesting about the new dashboard widgets:

Just to prove a point that there are many ways to think about this new feature, here's another take on what Dashboard is. From a browser geek's perspective, the Dashboard is a collection of HTML sidebar panels liberated from the browser window and placed anywhere on your screen. The "Web pages as widgets" concept is really just a logical extension of the Web sidebar panel metaphor fused with Exposé.[...]
However the sidebar metaphor suffers from usability problems, such as the inability to scale up to many panels as well as being constrained by the browser's window width. It's also hard to view multiple panels at once. The panels are also tied to a particular application (the browser) despite frequently having no connection to the application itself.

Which makes sense for sidebars that are decoupled from the current page context. However, what is missing here (and as far as I could tell at WWDC last week, is not available at all in Safari) is a way to put a sidebar in the browser that is informed by the page context therein and can interact with it. As Hyatt says, this is possible in IE on Windows, and in Mozilla derivatives. Can we do this within Safari, using Dashboard or AppleScript, or anything other then hacking the nibs inside Safari itself?

DropDV: Convert MPEG to DV

More than 10 years ago, Apple added MPEG playback to QuickTime. In order to work around QuickTime not having support for bi-directional difference frames, and some issues with MPEG's format being a bitstream rather than a byte stream, this was done my making it a special media type rather than showing up as separate video and audio tracks. As a side effect of this, exporting audio from MPEG movies didn't work. This was one of those features that was annoying to few enough people and just enough work that it has been continually deferred from release to release (hey, I spent 6 years there and never got to it either). There have been MPEG stream manipulation tools from Sparkle onwards, but they have always needed a deep understanding of the format, a degree in computer science and lots of patience to use.
Now that there are video cameras and digital cameras that record MPEG directly, this is suddenly a lot more annoying, but fortunately there is an answer:
DropDV: Convert MPEG to DV: DropDV is a Mac OS X droplet which converts MPEG video into DV video streams. This allows the video to be edited in iMovie, Premiere, Final Cut, or any other DV video editing system.
Features
Handles both video and audio
Uses high quality bicubic scaling for the best video image
Decodes in YUV color space, other tools use RGB.
Supports both NTSC and PAL output
A Simple, drag-and-drop interface

At least pick a socialist who can write

A clunky new version of some classic prose via the C of E::

The more usual 'Give us this day our daily bread', from the Lord's Prayer, becomes: 'You are giving us our daily bread when we manage to get back our lands or get a fairer wage'.

'Yea, though I walk through the valley of the shadow of death, I shall fear no evil.' has been removed from the 23rd Psalm in favour of 'Even if a full-scale violent confrontation breaks out I will not be afraid, Lord'.

George Orwell, in Politics and the English Language warned about this sort of thing:

Now that I have made this catalogue of swindles and perversions, let me give another example of the kind of writing that they lead to. This time it must of its nature be an imaginary one. I am going to translate a passage of good English into modern English of the worst sort. Here is a well-known verse from Ecclesiastes:

I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all.

Here it is in modern English:

Objective considerations of contemporary phenomena compel the conclusion that success or failure in competitive activities exhibits no tendency to be commensurate with innate capacity, but that a considerable element of the unpredictable must invariably be taken into account.

This is a parody, but not a very gross one. [...] It will be seen that I have not made a full translation. The beginning and ending of the sentence follow the original meaning fairly closely, but in the middle the concrete illustrations -- race, battle, bread -- dissolve into the vague phrases "success or failure in competitive activities." This had to be so, because no modern writer of the kind I am discussing -- no one capable of using phrases like "objective considerations of contemporary phenomena" -- would ever tabulate his thoughts in that precise and detailed way. The whole tendency of modern prose is away from concreteness.

Social software arbitrage

Jonas:

As seen on eBay - we've gots 'em. Nine brand new, shiny, gmail invites are sitting in my inbox waiting to further dilute the value some people are apparently willing to assign to this...

It's not free, however. If you're interested in one, comment here and let me know what you're willing to do for it. Not to me (though I am more than ready to trade for a few good massages), but to someone else. A random act of kindness, maybe? Work in a soup kitchen? Help out at a needle exchange? Or maybe you're doing that already - you'd be the ideal recipient.

Something for Nothing

Something for Nothing: The Free Culture AudioBook Project is a great essay on Copyright as seen through the lens of AKMA's project ot make an audio book from Lessig's Free Culture.

Technorati Developers Salon

May 19th from 6pm to 8pm was the Technorati Developers Salon.
see what people have said about it.

Free Culture readathon

I joined in AKMA’s "free culture readathon":

Anyone feel like recording a chapter of Lawrence Lessig’s new book?

The license pretty clearly indicates that, so long as we’re not making a commercial venture of it, we can make a recording of (“perform”) the text. There are a Preface, Introduction, fifteen chapters, a conclusion and an afterword.

I recorded the Preface of Free Culture which has me referring to Code and Other Laws of Cyberspace as 'my first book', which is a kind of lèse majesté, or Lessig majesté.

I used the Audacity Open Source Audio editor for this, which works very nicely, and reminds me of SoundEdit from long ago.

Here's a QuickTime version for people on slow modems.

[updated 2014 to fix dead links thanks to Apple being crap] Here's the previously unpublished dance remix I made then:

Life imitates code?

At Technorati, I've been writing 'spiders' - little bits of code that scuttle over the web indexing pages when roused. We normally have hundreds running around at once. So I was interested to see these spider hatchlings in the garden on Thursday, swarming over a web, trying to make sense of it.

Big picture of baby spider cluster

How prior would you like that art?

Jeneane's Phonecon made me smile, but the early days of telephony were full of inventive people trying out new services and business models.

With the Patent Office moving toward rejecting the Eolas plug-in patent on prior art grounds, maybe it's time for them to reconsider the Acacia and SightSound patents that take an obvious idea and add the word 'digital', and then go around shaking down anyone doing rich media online.

Alexander Graham Bell invested heavily in a company to send opera over the telephone for a fee in the 19th Century.

Live music over the phone was happening in 1877, and remote playback by telephone in 1888 and even remote paid playback of recordings over the phone on demand, from 1909.

Black, White, Grey and mediAgora

I know I've been quiet on the mediAgora front recently, but the Grey Album case is in many ways a perfect example of the mediAgora principle of rewarding Creators of both derivative and original works.

Imagine, if you will, a parallel universe where The Beatles 'White Album' and Jay-Z's 'Black Album' had been released under mediAgora licences.

Along comes Danger Mouse, and mixes the two to make the 'Grey Album', and releases it for sale under a mediAgora license too.

So what happens?

He list both monochromatic albums as 'source works'. Everyone who buys 'The Grey Album' has to own a copy of the two source albums too. If they already do, they just pay Danger Mouse; if they own The Beatles but not Jay-Z they pay him and Danger Mouse.

As DM is generating incremental sales in this way, he gets promotion fees from the other two.

And all those bloggers pointing to the Grey Album? They get promotion fees from Danger Mouse, insofar as they generate sales (and have bought a copy themselves).

End result - every Customer has 3 great albums, and all Creators involved get paid the price they set.

And even Glenn Miller and the orchestras George Martin and the Beatles sampled could be rewarded too.

Technorati, Xanadu and other dreams

Tim Oren says nice things about Technorati:

As we all know, Ted Nelson meant hypertexts to have bidirectional links. But due to a laboratory accident in Switzerland, we ended up with this lame thing. Mechanisms such as Google link search and Technorati are just hacks, ways to leverage Moore's Law to ameliorate a fundamental flaw in our hypertext data architecture, crawling the Web faster and faster to aggregate all of our trackbacks.

Yesterday, David Sifry convinced me that's just wrong. What Nelson missed, with his focus on 'literary' architectures, is that networked hypertexts are inhabited by people. Links are not just citations. They are gestures in a social space, parts of conversations or other interactions. There's an inherent value in looking at the dynamics of the record as it is created.

Obviously I agree with the broad thrust of this or wouldn't be working at Technorati. However, I think the one-way nature of links was necessary for the Web to achieve what it did. The globally connected nature of the web as a small world network is built on a scale-free distribution of linkage. If all links are required to be two-way, this rapidly becomes unwieldy and cumbersome - imagine if the front page of Apple.com showed all the inbound links to it. The unidirectionality created the permission-free linking culture the web depends on, and reversing those links in a useful way is an interesting problem we're having fun solving - the hot products page is an example of this.

In fact, that page addresses another issue, if obliquely. Shelley asked how being a community member affects your writing:

[W]eblogging [is] different than Big Media, because it puts publishing in the hands of the people. I have to presume they think this is a good thing because webloggers can write what they want, and aren't censored. Unlike Big Media, we aren't accountable to an editor, or big companies, or important politicians.

But I guess we're accountable to each other, and that's the most dangerous censorship of all -- it's the censorship of the commons.

Indeed. I think this is a good thing. The fact that when blogging we are accountable for our writings and their public history acts, in general, in a good way - it makes us stop to think about our reactions before they 'end upon our permanent record'. Shelley's own campaign against comment spammers that violate community norms in this way is an example. David Weinberger in 'Small Pieces Loosely Joined' put it this way:

A human being raised in isolation would not be identifiably human in anything except DNA. Sociality grants a mute herd of brutes their souls and selves.

The example of what happens when anonymity is allowed in Amazon reviews, leading to all kinds of dubious behaviour was revealed in the NYT , and picked up by auctorial bloggers like Cory and Neil Gaiman. The bloggers' comments show up in context with the rest of their writings, so you can gather whether you are likely to agree with them generally too.

For example, when Lago attacks Vote Links for reinforcing hegemony I can see that he is the same person who threatened to offer Joi a reading list, but then didn't, so I can offer him one instead:

DRM a sign of Disney's malaise

Mitch Wagner:

DRM doesn't work and consumers don't want it, so of course it's very appealing to big business, who are also in a big rush to sell other, equally practical products, such as anchovy flavored ice cream and bicycles with square wheels.

We learned that DRM doesn't work in the late 80s, only back then it was applied to software and we called it 'copy protection.'

Not much to add, but it's good to see the meme spread.

Vote Links

A while back I made a proposal for 'Vote Links' - a way to indicate that just because you are linking to something, you are not necessarily endorsing it (which is the default assumption by search engines and other dumb robots).
My original proposal used a nonstandard attribute which would make it hard to validate.
Tantek has helped me create an XHTML compliant Vote Links specification, which we'll be talking about tonight at the Technorati Participant Session and the XHTML Semantics session at ETCon.

Technorati at Etcon

We've got Dave's talk at etcon this morning. Here's the newest thing - the recent amazon products page

See what amazon products people have blogged about recently.

Power laws and blogs: first, the classic power law chart - note how smooth the curve is - no saturation due to no barriers to entry.

In fact, if you count up the total number of blogs with links you see a different picture - look how the 'little' blogs totally outweight the top few.

Technorati is Hiring

I don't think I've mentioned it here yet, but as Director of Engineering at Technorati,we're hiring.

The Mass Media bubble bursts

Doc and I were taking about this the other day, and tonight I remembered how well Douglas Adams explained the end of Mass media five years ago:

Because the Internet is so new we still don�t really understand what it is. We mistake it for a type of publishing or broadcasting, because that�s what we�re used to. So people complain that there�s a lot of rubbish online, or that it�s dominated by Americans, or that you can�t necessarily trust what you read on the web. Imagine trying to apply any of those criticisms to what you hear on the telephone. Of course you can�t �trust� what people tell you on the web anymore than you can �trust� what people tell you on megaphones, postcards or in restaurants. Working out the social politics of who you can trust and why is, quite literally, what a very large part of our brain has evolved to do. For some batty reason we turn off this natural scepticism when we see things in any medium which require a lot of work or resources to work in, or in which we can�t easily answer back � like newspapers, television or granite. Hence �carved in stone.� What should concern us is not that we can�t take what we read on the internet on trust � of course you can�t, it�s just people talking � but that we ever got into the dangerous habit of believing what we read in the newspapers or saw on the TV � a mistake that no one who has met an actual journalist would ever make. One of the most important things you learn from the internet is that there is no �them� out there. It�s just an awful lot of �us�.

Read the whole thing. Regularly.

Orlowski trolls again

In his latest rambling, shambling semi-coherent collection of innuendo, semi-sourced smears and out of context quotes with no attempt at fact-checking or giving anyone the chance to reply, Orlowski says:

"What sort of people go to these?" he asks, rhetorically. The panel on Political Blogging doesn't even include a Political Blogger. We can help answer that one: andsometimes a picture is worth a thousand words. (Taken at the $500 a head BloggerCon conference).

Now that is a picture with me in the middle. Admittedly Adam Curry, Elizabeth Spiers, Jeff Jarvis and Charlie Nesson are there too, but what is he trying to say? That we use laptops in public?

He even repeats his widely debunked Googlewashing hogwash.

Fortunately we don't need to worry about him giving space for responses - you can read the rebuttals here.

RSS Winterfest - I'm speaking

I'm speaking at RSS Winterfest this morning, with Anil Dash and others on 'RSS 2.0 and Atom'

Tune in

Categorising blogs

Why I Write - George Orwell :

There are four great motives for writing:
1. Sheer egoism.
2. Aesthetic enthusiasm.
3. Historical impulse.
4. Political purpose.

Seems like one way to categorize blogs.

Technorati beta test

Dave Sifry just announced what I've been working on with him: We focused 100% of our time on completely refurbishing our underlying event engine - essentially taking a Volkswagen engine out and putting a Ferrari engine in.

Head on over to http://beta.technorati.com and try it out.

Or recursively see who has linked to it at the beta Technorati beta cosmos

Alienaid: Iowa Caucases

I think these were explained by Lewis Carroll:

What I was going to say,' said the Dodo in an offended tone, `was, that the best thing to get us dry would be a Caucus-race.'

`What IS a Caucus-race?' said Alice; not that she wanted much to know, but the Dodo had paused as if it thought that SOMEBODY ought to speak, and no one else seemed inclined to say anything.

`Why,' said the Dodo, `the best way to explain it is to do it.' (And, as you might like to try the thing yourself, some winter day, I will tell you how the Dodo managed it.)

First it marked out a race-course, in a sort of circle, (`the exact shape doesn't matter,' it said,) and then all the party were placed along the course, here and there. There was no `One, two, three, and away,' but they began running when they liked, and left off when they liked, so that it was not easy to know when the race was over. However, when they had been running half an hour or so, and were quite dry again, the Dodo suddenly called out `The race is over!' and they all crowded round it, panting, and asking, `But who has won?'

This question the Dodo could not answer without a great deal of thought, and it sat for a long time with one finger pressed upon its forehead (the position in which you usually see Shakespeare, in the pictures of him), while the rest waited in silence. At last the Dodo said, `EVERYBODY has won, and all must have prizes.'

Media Heresy: Compression is becoming redundant

Yesterday, Ross sent Bambi Francisco to talk to me, and she asked an interesting question: "Is there a Moore's Law for compression?" My answer didn't all make it into her article, so here is an expanded version for my 'media heresies' series.

"Is there a Moore's Law for compression?"
In the sense of compression getting uniformly better over time? No.
Compression has different constraints - it is primarily based around fooling human perception systems by sending less information. Compression generally has two phases - a lossy phase where the data is transformed into a less accurate version by exploiting limitations of human vision or hearing, and a lossless phase where redundancy is squeezed out mathematically (this phase is like using .zip).

With more computing power, more elaborate transformations can be done in the first phase, and more complex mathematical compression can take place in the second phase, and still give the computer enough time to achieve a useful frame rate, but overall compression standards do not improve at anything like Moore's Law speed.

I'd say video compression is maybe 2-4 times as efficient (in quality per bit) than it was in 1990 or so when MPEG was standardised, despite computing power and storage having improved a thousandfold since then.

However, what does happen is that the Moore's Law effects on computing power, and the Moore's Law cubed effect on storage capacity mean that compression becomes less relevant over time.

You can now buy an off-the shelf computer that can edit uncompressed High Definition TV for under 10% of the cost of an HD tapedeck.

Consider that the iPod has gone from 5GB to 40GB in under 18 months - a factor of 8. The MP3 compression iTunes uses is about 8:1, so that means you could fill the new iPod with uncompressed audio and store as much as you did in the old one. Apply that rate of doubling another few times and think about pocket TiVos. Farfetched? I'm not so sure - Computer users have been watching DVDs on laptops for a while now; hand-held DVD players are being bought for children in the backseat and people who travel. According to my friends at Best Buy, they sold out all the portable DVD players they had this Christmas - they had hit a sensible price point.

The deeper point is a trend based one. If storage continues to improve in capacity per dollar at 3 times the rate of computing power, compression becomes wholly redundant - the CPU running the bit-manipulation is the bottleneck. The HD editing computers work this way - they have DMA (direct memory access) hardware in the disk interface and the screen interface, and the computer's job is to get out of the way.

The other reason compression is a bad idea in the long run is precisely because of its success in removing redundancy. If you have uncompressed audio or video, a single bit error will likely go un-noticed. If you are unlucky and it is the high bit of a sample, you will get a transient click in the sound, or a brightly coloured dot in the wrong place in video, but it will soon pass and be covered by a correct bit.
If you have a single bit error in a compressed stream it will make the rest of the frame, or possibly many frames, corrupt. In the worst case it can destroy the rest of the file from then onwards.
For archival content this kind of fragility is not what you want.

RIAA's fake cops harrass based on racial stereotypes

LA weekly story:

'A large percentage [of the vendors] are of a Hispanic nature,' Langley said. 'Today he�s Jose Rodriguez, tomorrow he�s Raul something or other, and tomorrow after that he�s something else. These people change their identity all the time. A picture�s worth a thousand words.'

Langley is Western regional coordinator for the RIAA Anti-Piracy Unit.