Fear and Loathing in Online Video: The Codec Conundrum Continues

My last post described, what Google's decision to drop H.264 from Chrome in favor of its own WebM format means for online video publishers, and concluded that this situation creates even more complexity video publishers, along with the potential of a massive increase in video publishing costs. It's been over a week now since Google made that announcement and commentary across the web all led to a similar conclusion – that Flash video is the winner in this latest codec conundrum, and now the safest bet for online video publishers producing video for the web.

Many have both lauded and lambasted Google's decision while many others have shrugged it off as not being a really a big deal.

Regardless of the debate around "open" and "closed" standards and patents or codecs and containers – the net result of Google's decision affects the entire ecosystem of video content creators, publishers, developers and advertisers and has created an atmosphere of fear and loathing in online video.

Google replies to criticism 

Google defended its decision in follow up post on the Chromium Blog: More about the Chrome HTML Video Codec Change, where Google Product Manager Mike Jazayeri addressed some of the key questions.

Regarding Google's decision to support WebM for HTML (video) tag, Jazayeri provided the following explanation:

"We believe there is great promise in the (video) tag and want to see it succeed. As it stands, the organizations involved in defining the HTML video standard are at an impasse. There is no agreement on which video codec should be the baseline standard. Firefox and Opera support the open WebM and Ogg Theora codecs and will not support H.264 due to its licensing requirements; Safari and IE9 support H.264. With this status quo, all publishers and developers using the (video) tag will be forced to support multiple formats.

This is not an ideal situation and we want to see a viable baseline codec that all browsers can support. It is clear that there will not be agreement to specify H.264 as the baseline codec in the HTML video standard due to its licensing requirements. Furthermore, we genuinely believe that core web technologies need to be open and community developed to enable the same great innovation that has brought the web to where it is today. These facts led us to join the efforts of the web community and invest in an open alternative, WebM."

Jazayeri admitted that H.264 has broader support in the publisher, developer, and hardware community today but due to its licensing requirements and patent royalties, which could potentially increase over time, it could not back it as the baseline in the HTML video standard. He reiterated Google's vision to create "open innovation" for video on the web, and that its significant investment in making WebM an open platform for development was proof of that.

But Google was not abandoning H.264 altogether, Jazayeri said, and pointed out that the majority of H.264 videos on the web are viewed in plug-ins such as Flash and Silverlight, and H.264 will continue to be supported in Chrome through a plug-in as well. He admitted that this decision will force publishers to create multiple versions of their videos, but argued that it was already the case since Firefox and Opera never supported H.264 in the HTML (video) tag.

Additionally, Google is already doing with YouTube videos and with the proliferation of mobile devices, platforms, and connectivity types across the web, he added that most content providers already produce multiple versions of their videos. Google is confident that WebM will emerge as a viable and compelling solution for publishers and the WebM Project team will soon release plugins that enable WebM support in Safari and IE9 via the HTML standard (video) tag.


Jazayeri's final point acknowledged the elephant in the room:

"Bottom line, we are at an impasse in the evolution of HTML video. Having no baseline codec in the HTML specification is far from ideal. This is why we're joining others in the community to invest in WebM and encouraging every browser vendor to adopt it for the emerging HTML video platform. Our choice was to make a decision today and invest in open technology to move the platform forward, or to accept the status quo of a fragmented platform where the pace of innovation may be clouded by the interests of those collecting royalties. Seen in this light, we are choosing to bet on the open web and are confident this decision will spur innovation that benefits users and the industry."

Fear, loathing and confusion

Jazayeri's post generated over 240 comments which spoke to the clear division within the business and developer communities, with many who have spoken out against Google's principles of "open" calling it inconsistent and hypocritical, including:

• MG Siegler: Google Clarifies Their H.264 Stance, Hands Keys Of Web Video’s Future Back To Flash
• Dan Frommer: Sorry Google, But "Open" Is A Crock 
• John Gruber: Daring Fireball: The Practical vs. Idealistic Scenarios for the Near-Term Future of Online Video
• Tim Sneath: An Open Letter from the President of the United States of Google - MSDN Blogs
• Prasad Naik: Google Drops H.264 in Chrome. What does it mean?

While on the other side of the argument, others have come out in defense of Google's decision:

• Haavard - Is the removal of H.264 from Chrome a step backward for openness? and Haavard - John Gruber's fundamental misunderstanding
• John Dowdell; VIDEO debate, cutting to the chase « jd/adobe
• Thom Holwerda: Microsoft, Opera's Haavard Respond to Google's H.264 Move and Google: H.264 Stifles Innovation

But some have gone into great detail to provide unbiased analysis of the situation:

• l@antimatter15: The Ambiguity of “Open” and VP8 vs. H.264 – Blog.
• Steve Yelington: The video tag mess, and why Google's interests are (mostly) our interests | yelvington.com 

The video industry responds

Many voices within online video industry have spoken up as well, mainly to educate consumers and publishers. Brightcove launched its Jumpstart Your Online Video Strategy for 2011 webinar series and Sorenson Media's David Dudas provided, A Simple Explanation Of The Huge WebM Versus H.264 Debate in a recent article on Business Insider.

Dudas offered the following analogy:

"You can think of the H.264 and WebM video formats as modern day VHS and Betamax tapes, and think of the Web browsers on iOS and Android devices as the respective tape decks  (left as an exercise for the reader to determine which is which). Imagine coming home from the video store with your favorite new move on VHS tape and trying to cram it into a Betamax deck. Wouldn't exactly work, would it? Well, that's the same experience consumers will have in the near future as they try to watch H.264 video on Chrome-powered devices, or WebM video on Safari-powered devices. The format wars are alive and well my friends.

So what does this mean to you, the business person who depends on online video as a core part of your business? Quite frankly, it means you're on the hook to ensure cross-device compatibility for all your video content."

Dudas offered a few real world examples of publishing video to Hulu and Netflix to describe the complexity that content publishers face to get their videos to "just work" on each of the platforms. Hulu provides a rich viewer experience on the web, but it doesn't work on the iPad. In contrast to Netflix, which works on almost any device.

But making it "just work" isn't easy, as Dudas said:

"The complexities of device compatibility go far beyond the H.264 vs. WebM debate: each device supports different frame sizes, data rates, codec profiles, adaptive streaming protocols, digital rights management frameworks, and so on. The encoding phase of content production - and, to a lesser extent, the delivery - is where all of these things are either done right or done wrong. Doing it wrong means your encoded content may not play back, or - if it does play back - it may look absolutely terrible. We're not sure which is worse. On the other hand, doing it right means your customers will be delighted rather than frustrated, captive rather than fleeting, which translates to more time and money spent with your business."

On ProVideo Coalition.com, Allan Tépper said that Google has thrown a monkey wrench into video distribution and complicated video producer's workflow, and echoed Dudas' advice:

"What does this mean for us content producers? The goal of any intelligent content producer is to create material which is visible on all popular computers, mobile devices, set top boxes & HDTV sets with onboard streaming, including Apple’s iOS (AppleTV, iPad/iPhone/iPod Touch) and Google’s Android phones, tablets (including most non-Apple ones), and GoogleTV. Unless Google relents (which I hope but do not expect to happen), content producers who want to offer a universally accepted, seamless experience on the mentioned devices will be now need to encode web video in at least two video códecs (i.e. H.264 and WebM or Theora) with the required web code to auto-negotiate among raw H.264, raw WebM (or Theora), and H.264 embedded in Flash. Google has thrown a monkey wrench into our workflow and best practices, and I’m not the only one complaining."

Will Google drop H.264 in YouTube?


That's the big question that's been on everyone's mind. The Guardian’s Charles Arthur asked Google directly and was told that this change is related to Chrome and not YouTube. Google supports many formats on YouTube, and has an opt-in HTML5 trial for YouTube. Browsers that support the video tag in HTML5 can choose either the H.264 video codec or the WebM format (with VP8 codec).

Arthur went on to ask a number of other related questions that posed by commenters of his previous post, Google's WebM v H.264: who wins and loses in the video codec wars?, and summed up Google's answers:

• "YouTube isn't giving up H.264. At all. You can, if you're determined, get WebM/VP8 content on YouTube (both to contribute and download). There's the possibility that it is re-encoding all its content into WebM - just as it did to H.264 in June 2007, when the iPhone was about to arrive. That took something like three or four months to do. The library is bigger now, but so is Google's processing power."
• "The lack of any date, or inclination, on the part of YouTube indicates that this is purely a decision made by the Chrome team to push things along."
• "This isn't going to affect the mobile side - so iPhones, iPads, iPod Touches are not going to go dark."
• "Hardware to decode VP8 might be on the horizon - but Google can't do much to push it along except improve the codec, and do things like, well, drop H.264 support in its browser. (Pretty much everyone will see no difference, because the H.264 decoding will be handed off to the operating system.) I asked how many people are working on improving the VP8 codec, since that's sometimes helpful to know: is it a priority? But Google didn't want to discuss team sizes. Hm."

WebM's growing support

While this situation has become more complicated for video publishers, many within the tech industry believe that support for WebM will grow as the format gains wider adoption among software and hardware vendors. This past week, the Free Software Foundation, a non-profit corporation that advocates free software ideals and an early supporter of WebM, publicly urged web site owners to distribute videos in the WebM format, and abandon H.264.

This infographic by Niels Leenheer shows the current and future state of H.264 vs. WebM.

From: http://html5test.com/

Changes for Google, Apple and HTML5

Whether or not Google's claim of open video is genuine or not, or if its stand against H.264 in Chrome is really a swipe at Apple's dominance in the mobile browser market, it's interesting to note that both Google and Apple have seen a change in leadership within the last week – with Steve Jobs taking a leave of absence with Apple COO Tim Cook stepping in at Apple and Google Co-founder Larry Page replacing Eric Schmidt as Google's new CEO.

It's also interesting to note that just a few days after the World Wide Web Consortium (W3C), the international community that oversees the HTML standard, introduced a new HTML5 logo  to promote next-generation Web technology, a related standards group, the Web Hypertext Application Technology Working Group (WHATWG), a community of web browser manufacturers and people interested in evolving the HTML, declared that HTML5 was dead and that the HTML specification will henceforth just be known as "HTML" referred to simply as HTML.

As for the online video industry and what the future holds, the codec conundrum continues.

What Does Google's Decision to Drop H.264 from Chrome Mean For Online Video Publishers?

Just when it appeared that the online video industry was moving toward a video standard, with H.264 video playback in HTML5 – more chaos ensued with Google's announcement of its plans to phase out support for H.264 in its Chrome browser, in favor of open source formats like its own WebM or Theora. Like shot heard round the world, the news spread quickly throughout the web sparking a flurry of commentary and debate on the reasoning behind Google's decision. The overwhelming response within the online video industry is that this move by Google will set the adoption of HTML5 video back even further, and for online video publishers, this creates even more confusion and potentially a massive increase in video publishing costs.

The stakes are high in the battle for a video standard for the web and mobile devices. It's a war that's been brewing for several years with open source advocates pushing for a free and open codec unrestricted of proprietary licenses for the HTML5 video element, and an industry that has been built on a widely adopted yet proprietary and licensed video codec. But the lack of a single codec standard that is supported across web browsers is not only a serious issue for publishers but a stalemate for the industry.

WebM is a relatively new media file format Google introduced last May at the i/O developers conference, as a royalty free open source license packaging its VP8 video codec and the open source audio codec Vorbis, as part of Google's wider open source initiative called WebM Project. The goal of the WebM project, Google says, is "dedicated to developing a high-quality, open video format for the web that is freely available to everyone." WebM was launched with support by Mozilla, Opera, Adobe, Google and more than forty other publishers, software and hardware vendors. Absent from the list companies at launch was of course, Apple and also Microsoft, which back H.264 in Safari and IE9. However, Microsoft has said it will support VP8 as well. Google acquired the VP8 codec when with its acquisition of video codec maker On2 Technologies last year.

H.264/AVC video format is the digital video coding industry standard H.264/MPEG-4 Part 10, or AVC  used widely in set-top boxes, media player and other personal computer software, mobile devices, Blu-ray Disc players and recorders, game machines, personal media player devices and still and video cameras. Apple has become most outspoken proponent of AVC, and backs it as the standard video codec used for its HTML5 video compatible mobile iDevices like the iPhone, iPad and iPod Touch, as well as for iTunes videos. Several months ago, the MPEG LA reversed its decision and freed up the license of H.264, and will not charge royalties for any content that is free to consumers. It was speculated by many that the move by the MPEG LA was spurred on by Google's release of VP8 as an open source royalty free video codec.

But how much of Google's decision to abandon H.264 is based on its goal to "enable open innovation" or to capture future market share?

In a short post this past week on the Chromium Blog: HTML Video Codec Support in Chrome, Google Product Manager Mike Jazayeri said that its decision was a move to make Chrome consistent with the codecs already supported by the open Chromium project:

"We expect even more rapid innovation in the web media platform in the coming year and are focusing our investments in those technologies that are developed and licensed based on open web principles. To that end, we are changing Chrome’s HTML5 (video) support to make it consistent with the codecs already supported by the open Chromium project. Specifically, we are supporting the WebM (VP8) and Theora video codecs, and will consider adding support for other high-quality open codecs in the future. Though H.264 plays an important role in video, as our goal is to enable open innovation, support for the codec will be removed and our resources directed towards completely open codec technologies."

Some interesting commentary on Slashdot pointed to more scrupulous plans that Google is manipulating the market for selfish reasons to reap hundreds of billions in WebM devices. Also, on Daring Fireball, John Gruber asked 5 Simple Questions for Google Regarding Chrome's Dropping of H.264 which many of us would like to know the answers.


But what about all the millions of websites that content publishers around the world have adopted H.264 as HTML5's video element over the last year?
 Jazayeri added:

"These changes will occur in the next couple months but we are announcing them now to give content publishers and developers using HTML (video) an opportunity to make any necessary changes to their sites."

Okay, so what this means is that by default, the Chrome browser will automatically playback videos encoded in either the WebM or Theora open source standards, but will need a plug-in just like the Flash Player plug-in to play anything encoded in H.264 – which is actually the more widely adopted standard used throughout the industry. H.264 and can be decoded by hardware in many devices, like tablets and mobile phones.

So now content publishers will have to go back to encoding multiple versions of their videos in different file formats and codecs, primarily in H.264, WebM and even Flash. Or for many publishers, take the easy way out and go back to delivering video in Flash, since the Flash Player is baked into Chrome and Flash video will play on all the browsers.

Many are speculating that this is all about  timed announcement to steal Apple's thunder of its Verizon iPhone announcement, at least in the short term, and also upset Apple's domination in the mobile device and digital media industry. Others are calling Google hypocritical for dropping H.264 and not Flash, which requires a plug-in.

ReadWriteWeb calls this current situation, A Stalemate of Standards: What H.264 Means for the Average User, and spoke with blip.tv Co-founder Justin Day who said that Google's move is a step back to the Dark Ages and also Michael Critz, a freelance interactive and motion graphic designer, who agreed that this is a regression for the average user.

According to Day:

"I think from our standpoint this looks like a regression. We're all for open formats, but they should be chosen based on their merits, not merely their license. This move means that Chrome users will suffer from a worse user experience because they will need to rely on Flash fallback."

Last month blip.tv announced it has adopted HTML5 as the standard video player on its site after years of serving Flash video. Day says that there really is no competition between WebM and H.264 for the content producer because of the low cost tools available to create higher quality video using H.264.

As a content creator, Critz agreed:

"There isn't a desktop WebM conversion tool that worth a pile of beans to what's available now for H.264. I'm often traveling for video shoots. If I'm on a shoot in Miami and working all day then I get back to my hotel room I know I can depend on my Turbo.264 encoder to give me hardware accelerated H.264 encoding that I can use online, in Flash, on my producers' iPad, and on my clients' iPhones and Blackberry phones."

In a detailed report on Ars Technica, Peter Bright wrote that, Google's dropping H.264 from Chrome a step backward for openness and creates more work and cost for video publishers.

Bright said:

"Video distributors wanting to support both Flash and HTML5 users will have to encode twice; once in H.264, for Flash users, and again in WebM, for HTML5 users. This doubles the computational cost, doubles the storage requirements, and as an added bonus will tend to hurt quality. This is inconvenient for a small site with one or two videos; for sites like SmugMug it's an enormous headache. They can either suffer the doubled costs and complexity, or ignore HTML5 altogether and stick with Flash.

It looks like sticking with Flash and ignoring the (video) tag is indeed what SmugMug may end up doing. And who can blame them? Flash works for almost every Internet user, and Flash supports H.264."

Jan Ozer also commented while Chrome only has a small percentage of the web browser market it will be interesting to consider what's to come. He advised producers and publishers to check their wallets because they'll find less money in it, now that we're well on our way to a Two-codec world.

So how much will this disrupt the browser market share?


As far as browsers go, they are waging their own war and actually, Chrome is a small player compared to Internet Explorer and Firefox which is second to IE. But in just about two years since its launch Chrome's market share is currently about 13.5%, which is about twice that of Opera and Safari combined.

Both Firefox and Opera lack native H.264 support, so combined with Chrome, the browsers without H.264 support total 44.05%. Internet Explorer has the largest market share with 44.53%, Safari has 5.8%

Along with the browsers, the opponents on each side of the format war are Apple, Google and Adobe continue to wrestle control over video formats, and I've said before the battle continues and publishers and consumers are caught in the struggle. Now with even more fragmentation within the online video ecosystem no support for common video codecs across the industry, we are even further away from reaching an online and mobile video standard anytime soon.

For video publishers, it's clear that there will be additional costs involved in delivering video to all web browsers and mobile devices. For companies in the video encoding business, this must be your lucky day.

¡Viva La H.264! MPEG LA Liberates H.264/AVC For All Free Video Content Forever... But What Does It All Mean??

The video codec format war heated up last week, with the announcement by the MPEG LA, that it has freed up the license of the H.264 AVC (Advanced Video Coding) codec, and will not charge royalties for any content that is free to consumers. The MPEG LA is the firm that manages the patent pool for AVC, and earlier this year, it announced it would not charge royalties for such video through December 31, 2015 (PDF link). But now and forever, it has made it free for the life of the license and will only charge for AVC encoded video content sold to end users in any form for a fee, on a title or subscription basis.

H.264/AVC video format is the digital video coding  industry standard H.264/MPEG-4 Part 10, or AVC  used widely in set-top boxes, media player and other personal computer software, mobile devices, Blu-ray Disc™ players and recorders, game machines, personal media player devices and still and video cameras. Apple has become most outspoken proponent of AVC, and backs it as the standard video codec used for its HTML-5 video compatible mobile iDevices like the iPhone, iPad and iPod Touch, as well as for iTunes videos, and Microsoft also announced support for H.264-encoded HTML5 video in IE9 earlier this year.

Many have speculated that the move by the MPEG LA was spurred on by Google's release of VP8 as an open source royalty free video codec, a new rival to H.264/AVC and potential replacement to the open source Theora. Google announced the release of the VP8 video codec under the WebM Project at the i/O developers conference in May, with broad industry support. Absent from the list of supporters was, of course, Apple.

The tug of war between these two competing video codecs has reached an inflection point according to Streamingmedia.com's Tim Siglin, who says:

"There is little doubt that the most recent announcement by MPEG LA is, in part, a counteraction against the potential inroads that WebM may make in the online video space."

The codec issue is a big dilemma for online video publishers says Jan Ozer. He recently authored a recent survey for StreamingMedia.com, Supporting the iPad and HTML5, and found that less Than 50% of publishers ready to implement HTML5 video,

According to Ozer:

"54% of respondents who were considering HTML5 support rated the lack of single codec either a serious concern, or a very serious concern. Lack of HTML5 browser penetration was rated even more of a concern, but that will resolve in time. The codec issue appears to be a permanent problem which will force producers to encode in as many as four or five different formats in the short term, and at least two in the long term."

Rounding out the analysis from Streaming Media is Dan Rayburn who acknowledged that while the MPEG LA's royalty free license is good news for content owners, it does not address the future market needs for subscription based content services and device manufactures who will continue to pay to license the codec.

Rayburn also agreed that the lack of a codec standard is a major sticking point for the industry:

"The news also does nothing to address what I consider to be a bigger issues which is the need for browsers to give us a single codec we can call use. Companies like Mozilla and Opera are still going to have to pay for a license if they want to support H.264 which means we are not any closer to have much in the way of video standard on the Internet."

As far as browsers go, they are waging their own war. Safari supports H.264, Firefox, Opera and Chrome support WebM, and IE9 will support both as well as its own VC-1. Mozilla has refused to support H.264 as a codec for the HTML5 video tag in Firefox, and it's unlikely it will in the future. Theora plays natively in FireFox 3.5, Chrome as of version 3.0.182.2 and Opera as version 10.50.  All browsers can play H.264 encoded video presented in Adobe's Flash plug-in, but as everyone knows, Flash doesn't play within Apple's iOS on its iDevices. According to current global browser market share and trends of the five major browsers, IE 6.0-8.0 leads with 52.68%, followed by FireFox 3.0+-3.5+ with 31.49% and those numbers are growing steadily as people migrate to the newer versions. Chrome has grown to 9.80%, Safari has about 5.09% and Opera is 1.90%.



Ultimately, it's all not just about patents and licenses, there's a much bigger back story. I like this post by Brightcove CEO Jeremy Allaire, The Future of Web Content – HTML5, Flash & Mobile Apps, which provides a detailed industry analysis on the codec format and browser wars. He says video is a cornerstone issue and the emergence of the mobile device industry has brought the issue to the forefront:

"With massive growth in hand-held web browsing from smartphones, iTouch devices and the pending iPad product, this has raised a deeper issue for media publishers who are eager to have their content be accessible to end-users. In particular, it is the show-down between Apple, Google and Adobe over who can control video formats on these devices that is creating challenges. Again, this is not about “what is the right technical solution”, it is about the political economy of who controls the formats that in turn lead to owning downstream audience and monetization opportunities."

While many big video websites, like YouTube and Vimeo, have rolled out their own HTML5 video players, we are held at bay by the big browser developers that can't agree which video codec to support in HTML5. The MPEG LA's move has brought us closer to a standard - with the two dominant video codec standards AVC and WebM - the tug of war continues and we're caught in the struggle. Until there is broad support for all codecs and containers across the industry, we won't be reaching a web video standard anytime soon.

About MPEG LA, LLC

MPEG LA is the world leader in alternative technology licenses, enabling users to acquire worldwide patent rights necessary for a technology standard or platform from multiple patent owners in a single transaction as an alternative to negotiating individual licenses. MPEG LA’s initial licensing program for MPEG-2 digital video compression helped produce the most widely employed standard in consumer electronics history, and the MPEG LA® Licensing Model has become the template for addressing other technologies. Today MPEG LA manages licensing programs consisting of essential patents in 57 countries. MPEG LA is an independent licensing administrator; it is not related to any standards agency and is not an affiliate of any patent holder. For more information, please refer to http://www.mpegla.com.

Related:

• To Infinity and Beyond: MPEG LA Extends H.264 Internet Video Moratorium Indefinitely - StreamingMedia.com
• MPEG-LA Announces No royalties on Free Internet Videos - Ever
• iPad and HTML5 Adoption - Supporting the iPad and HTML5
• Larry Kless' Weblog: Google Releases VP8 as Open Source Royalty Free Video Codec, Launches WebMproject.org with Broad Support from Online Video Industry
• Analysis: Royalty-free H.264 may clear way for HTML5 video standard | Browsers | iOS Central | Macworld

Related WebM reading:

• What Does the Webmproject Mean for the Future of Video on the Web
• VP8, WebM, and HTML5 video « Zencoder Blog
• Diary Of An x264 Developer » The first in-depth technical analysis of VP8