Epeus' epigone

Edifying exquisite equine entrapments

Wednesday, 14 January 2004

Media Heresy: Compression is becoming redundant

Yesterday, Ross sent Bambi Francisco to talk to me, and she asked an interesting question: "Is there a Moore's Law for compression?" My answer didn't all make it into her article, so here is an expanded version for my 'media heresies' series.

"Is there a Moore's Law for compression?"
In the sense of compression getting uniformly better over time? No.
Compression has different constraints - it is primarily based around fooling human perception systems by sending less information. Compression generally has two phases - a lossy phase where the data is transformed into a less accurate version by exploiting limitations of human vision or hearing, and a lossless phase where redundancy is squeezed out mathematically (this phase is like using .zip).

With more computing power, more elaborate transformations can be done in the first phase, and more complex mathematical compression can take place in the second phase, and still give the computer enough time to achieve a useful frame rate, but overall compression standards do not improve at anything like Moore's Law speed.

I'd say video compression is maybe 2-4 times as efficient (in quality per bit) than it was in 1990 or so when MPEG was standardised, despite computing power and storage having improved a thousandfold since then.

However, what does happen is that the Moore's Law effects on computing power, and the Moore's Law cubed effect on storage capacity mean that compression becomes less relevant over time.

You can now buy an off-the shelf computer that can edit uncompressed High Definition TV for under 10% of the cost of an HD tapedeck.

Consider that the iPod has gone from 5GB to 40GB in under 18 months - a factor of 8. The MP3 compression iTunes uses is about 8:1, so that means you could fill the new iPod with uncompressed audio and store as much as you did in the old one. Apply that rate of doubling another few times and think about pocket TiVos. Farfetched? I'm not so sure - Computer users have been watching DVDs on laptops for a while now; hand-held DVD players are being bought for children in the backseat and people who travel. According to my friends at Best Buy, they sold out all the portable DVD players they had this Christmas - they had hit a sensible price point.

The deeper point is a trend based one. If storage continues to improve in capacity per dollar at 3 times the rate of computing power, compression becomes wholly redundant - the CPU running the bit-manipulation is the bottleneck. The HD editing computers work this way - they have DMA (direct memory access) hardware in the disk interface and the screen interface, and the computer's job is to get out of the way.

The other reason compression is a bad idea in the long run is precisely because of its success in removing redundancy. If you have uncompressed audio or video, a single bit error will likely go un-noticed. If you are unlucky and it is the high bit of a sample, you will get a transient click in the sound, or a brightly coloured dot in the wrong place in video, but it will soon pass and be covered by a correct bit.
If you have a single bit error in a compressed stream it will make the rest of the frame, or possibly many frames, corrupt. In the worst case it can destroy the rest of the file from then onwards.
For archival content this kind of fragility is not what you want.
Posted by Kevin Marks at 01:33

2 comments:

Kevin Marks said...

Patrick B: favorited this.
via twitter.com

February 11, 2015 12:39 pm
Kevin Marks said...

Kevin Marks: mentioned this in Confirmed: compression doesn't....
via known.kevinmarks.com

July 01, 2016 12:09 pm

Post a Comment

Newer Post Older Post Home
Subscribe to: Post Comments (Atom)

This is my personal blog. Any views you read here are mine, and not my employers'.

Atom Feed

Support the Open Rights Group
My photoKevin Marks Me on Twitter
Me on G+

People's thoughts I read:

Daily

Rosie
San Jose Young People's Theatre
Dave Weinberger
Doc Searls
Gonzo Engaged
AKMA
Cory & friends
Denise Howell
Charles Wiltgen
Shelley Powers
James Lileks
Suw Charman
Halley Suitt

Weekly

Andrew Marks
Blogsisters
Arts & Letters Daily
Bricklin, Frankston & Reed
Steve Yost
Jeneane Sessum
Brian Micklethwait et al
Tom Matrullo
Gary Turner

Sporadically

Small Pieces
Stuart Cheshire
RageBoy
Nonzero
Neil Gaiman
Thomas Vincent
Brad deLong
Andrew Odlyzko
ProSUA

No to Mickey Mouse Computers

powered by blogger

Blog Archive

  • ►  2023 (1)
    • ►  September (1)
  • ►  2017 (2)
    • ►  May (1)
    • ►  April (1)
  • ►  2015 (7)
    • ►  November (2)
    • ►  May (3)
    • ►  April (1)
    • ►  January (1)
  • ►  2014 (3)
    • ►  October (1)
    • ►  April (2)
  • ►  2013 (5)
    • ►  June (1)
    • ►  May (1)
    • ►  April (2)
    • ►  March (1)
  • ►  2012 (8)
    • ►  December (1)
    • ►  May (1)
    • ►  April (1)
    • ►  March (1)
    • ►  January (4)
  • ►  2011 (11)
    • ►  December (1)
    • ►  November (1)
    • ►  September (2)
    • ►  August (2)
    • ►  July (1)
    • ►  April (2)
    • ►  January (2)
  • ►  2010 (16)
    • ►  November (1)
    • ►  October (1)
    • ►  September (3)
    • ►  June (1)
    • ►  May (2)
    • ►  April (2)
    • ►  March (2)
    • ►  February (2)
    • ►  January (2)
  • ►  2009 (22)
    • ►  November (2)
    • ►  October (2)
    • ►  September (2)
    • ►  August (3)
    • ►  July (2)
    • ►  June (2)
    • ►  May (2)
    • ►  April (1)
    • ►  February (2)
    • ►  January (4)
  • ►  2008 (29)
    • ►  December (2)
    • ►  November (3)
    • ►  August (1)
    • ►  July (3)
    • ►  June (3)
    • ►  May (5)
    • ►  April (2)
    • ►  February (3)
    • ►  January (7)
  • ►  2007 (45)
    • ►  November (3)
    • ►  October (4)
    • ►  September (4)
    • ►  August (10)
    • ►  July (3)
    • ►  June (8)
    • ►  April (2)
    • ►  March (6)
    • ►  February (3)
    • ►  January (2)
  • ►  2006 (119)
    • ►  December (13)
    • ►  November (8)
    • ►  October (16)
    • ►  September (10)
    • ►  August (3)
    • ►  July (6)
    • ►  June (24)
    • ►  May (3)
    • ►  April (10)
    • ►  March (7)
    • ►  February (8)
    • ►  January (11)
  • ►  2005 (101)
    • ►  December (10)
    • ►  November (13)
    • ►  October (9)
    • ►  September (8)
    • ►  August (7)
    • ►  July (7)
    • ►  June (8)
    • ►  May (12)
    • ►  April (7)
    • ►  March (6)
    • ►  February (1)
    • ►  January (13)
  • ▼  2004 (53)
    • ►  December (8)
    • ►  November (5)
    • ►  October (6)
    • ►  September (7)
    • ►  July (5)
    • ►  June (3)
    • ►  May (2)
    • ►  March (3)
    • ►  February (7)
    • ▼  January (7)
      • Orlowski trolls again
      • RSS Winterfest - I'm speaking
      • Categorising blogs
      • Technorati beta test
      • Alienaid: Iowa Caucases
      • Media Heresy: Compression is becoming redundant
      • RIAA's fake cops harrass based on racial stereotypes
  • ►  2003 (196)
    • ►  December (12)
    • ►  November (14)
    • ►  October (21)
    • ►  September (23)
    • ►  August (19)
    • ►  July (11)
    • ►  June (14)
    • ►  May (9)
    • ►  April (22)
    • ►  March (20)
    • ►  February (16)
    • ►  January (15)
  • ►  2002 (224)
    • ►  December (15)
    • ►  November (21)
    • ►  October (22)
    • ►  September (12)
    • ►  August (11)
    • ►  July (28)
    • ►  June (19)
    • ►  May (29)
    • ►  April (18)
    • ►  March (19)
    • ►  February (16)
    • ►  January (14)
  • ►  2001 (13)
    • ►  December (2)
    • ►  November (11)

Contributors

  • Kevin Marks
  • Kevin marks