Fear and Loathing in Online Video: The Codec Conundrum Continues

My last post described, what Google's decision to drop H.264 from Chrome in favor of its own WebM format means for online video publishers, and concluded that this situation creates even more complexity video publishers, along with the potential of a massive increase in video publishing costs. It's been over a week now since Google made that announcement and commentary across the web all led to a similar conclusion – that Flash video is the winner in this latest codec conundrum, and now the safest bet for online video publishers producing video for the web.

Many have both lauded and lambasted Google's decision while many others have shrugged it off as not being a really a big deal.

Regardless of the debate around "open" and "closed" standards and patents or codecs and containers – the net result of Google's decision affects the entire ecosystem of video content creators, publishers, developers and advertisers and has created an atmosphere of fear and loathing in online video.

Google replies to criticism 

Google defended its decision in follow up post on the Chromium Blog: More about the Chrome HTML Video Codec Change, where Google Product Manager Mike Jazayeri addressed some of the key questions.

Regarding Google's decision to support WebM for HTML (video) tag, Jazayeri provided the following explanation:

"We believe there is great promise in the (video) tag and want to see it succeed. As it stands, the organizations involved in defining the HTML video standard are at an impasse. There is no agreement on which video codec should be the baseline standard. Firefox and Opera support the open WebM and Ogg Theora codecs and will not support H.264 due to its licensing requirements; Safari and IE9 support H.264. With this status quo, all publishers and developers using the (video) tag will be forced to support multiple formats.

This is not an ideal situation and we want to see a viable baseline codec that all browsers can support. It is clear that there will not be agreement to specify H.264 as the baseline codec in the HTML video standard due to its licensing requirements. Furthermore, we genuinely believe that core web technologies need to be open and community developed to enable the same great innovation that has brought the web to where it is today. These facts led us to join the efforts of the web community and invest in an open alternative, WebM."

Jazayeri admitted that H.264 has broader support in the publisher, developer, and hardware community today but due to its licensing requirements and patent royalties, which could potentially increase over time, it could not back it as the baseline in the HTML video standard. He reiterated Google's vision to create "open innovation" for video on the web, and that its significant investment in making WebM an open platform for development was proof of that.

But Google was not abandoning H.264 altogether, Jazayeri said, and pointed out that the majority of H.264 videos on the web are viewed in plug-ins such as Flash and Silverlight, and H.264 will continue to be supported in Chrome through a plug-in as well. He admitted that this decision will force publishers to create multiple versions of their videos, but argued that it was already the case since Firefox and Opera never supported H.264 in the HTML (video) tag.

Additionally, Google is already doing with YouTube videos and with the proliferation of mobile devices, platforms, and connectivity types across the web, he added that most content providers already produce multiple versions of their videos. Google is confident that WebM will emerge as a viable and compelling solution for publishers and the WebM Project team will soon release plugins that enable WebM support in Safari and IE9 via the HTML standard (video) tag.


Jazayeri's final point acknowledged the elephant in the room:

"Bottom line, we are at an impasse in the evolution of HTML video. Having no baseline codec in the HTML specification is far from ideal. This is why we're joining others in the community to invest in WebM and encouraging every browser vendor to adopt it for the emerging HTML video platform. Our choice was to make a decision today and invest in open technology to move the platform forward, or to accept the status quo of a fragmented platform where the pace of innovation may be clouded by the interests of those collecting royalties. Seen in this light, we are choosing to bet on the open web and are confident this decision will spur innovation that benefits users and the industry."

Fear, loathing and confusion

Jazayeri's post generated over 240 comments which spoke to the clear division within the business and developer communities, with many who have spoken out against Google's principles of "open" calling it inconsistent and hypocritical, including:

• MG Siegler: Google Clarifies Their H.264 Stance, Hands Keys Of Web Video’s Future Back To Flash
• Dan Frommer: Sorry Google, But "Open" Is A Crock 
• John Gruber: Daring Fireball: The Practical vs. Idealistic Scenarios for the Near-Term Future of Online Video
• Tim Sneath: An Open Letter from the President of the United States of Google - MSDN Blogs
• Prasad Naik: Google Drops H.264 in Chrome. What does it mean?

While on the other side of the argument, others have come out in defense of Google's decision:

• Haavard - Is the removal of H.264 from Chrome a step backward for openness? and Haavard - John Gruber's fundamental misunderstanding
• John Dowdell; VIDEO debate, cutting to the chase « jd/adobe
• Thom Holwerda: Microsoft, Opera's Haavard Respond to Google's H.264 Move and Google: H.264 Stifles Innovation

But some have gone into great detail to provide unbiased analysis of the situation:

• l@antimatter15: The Ambiguity of “Open” and VP8 vs. H.264 – Blog.
• Steve Yelington: The video tag mess, and why Google's interests are (mostly) our interests | yelvington.com 

The video industry responds

Many voices within online video industry have spoken up as well, mainly to educate consumers and publishers. Brightcove launched its Jumpstart Your Online Video Strategy for 2011 webinar series and Sorenson Media's David Dudas provided, A Simple Explanation Of The Huge WebM Versus H.264 Debate in a recent article on Business Insider.

Dudas offered the following analogy:

"You can think of the H.264 and WebM video formats as modern day VHS and Betamax tapes, and think of the Web browsers on iOS and Android devices as the respective tape decks  (left as an exercise for the reader to determine which is which). Imagine coming home from the video store with your favorite new move on VHS tape and trying to cram it into a Betamax deck. Wouldn't exactly work, would it? Well, that's the same experience consumers will have in the near future as they try to watch H.264 video on Chrome-powered devices, or WebM video on Safari-powered devices. The format wars are alive and well my friends.

So what does this mean to you, the business person who depends on online video as a core part of your business? Quite frankly, it means you're on the hook to ensure cross-device compatibility for all your video content."

Dudas offered a few real world examples of publishing video to Hulu and Netflix to describe the complexity that content publishers face to get their videos to "just work" on each of the platforms. Hulu provides a rich viewer experience on the web, but it doesn't work on the iPad. In contrast to Netflix, which works on almost any device.

But making it "just work" isn't easy, as Dudas said:

"The complexities of device compatibility go far beyond the H.264 vs. WebM debate: each device supports different frame sizes, data rates, codec profiles, adaptive streaming protocols, digital rights management frameworks, and so on. The encoding phase of content production - and, to a lesser extent, the delivery - is where all of these things are either done right or done wrong. Doing it wrong means your encoded content may not play back, or - if it does play back - it may look absolutely terrible. We're not sure which is worse. On the other hand, doing it right means your customers will be delighted rather than frustrated, captive rather than fleeting, which translates to more time and money spent with your business."

On ProVideo Coalition.com, Allan Tépper said that Google has thrown a monkey wrench into video distribution and complicated video producer's workflow, and echoed Dudas' advice:

"What does this mean for us content producers? The goal of any intelligent content producer is to create material which is visible on all popular computers, mobile devices, set top boxes & HDTV sets with onboard streaming, including Apple’s iOS (AppleTV, iPad/iPhone/iPod Touch) and Google’s Android phones, tablets (including most non-Apple ones), and GoogleTV. Unless Google relents (which I hope but do not expect to happen), content producers who want to offer a universally accepted, seamless experience on the mentioned devices will be now need to encode web video in at least two video códecs (i.e. H.264 and WebM or Theora) with the required web code to auto-negotiate among raw H.264, raw WebM (or Theora), and H.264 embedded in Flash. Google has thrown a monkey wrench into our workflow and best practices, and I’m not the only one complaining."

Will Google drop H.264 in YouTube?


That's the big question that's been on everyone's mind. The Guardian’s Charles Arthur asked Google directly and was told that this change is related to Chrome and not YouTube. Google supports many formats on YouTube, and has an opt-in HTML5 trial for YouTube. Browsers that support the video tag in HTML5 can choose either the H.264 video codec or the WebM format (with VP8 codec).

Arthur went on to ask a number of other related questions that posed by commenters of his previous post, Google's WebM v H.264: who wins and loses in the video codec wars?, and summed up Google's answers:

• "YouTube isn't giving up H.264. At all. You can, if you're determined, get WebM/VP8 content on YouTube (both to contribute and download). There's the possibility that it is re-encoding all its content into WebM - just as it did to H.264 in June 2007, when the iPhone was about to arrive. That took something like three or four months to do. The library is bigger now, but so is Google's processing power."
• "The lack of any date, or inclination, on the part of YouTube indicates that this is purely a decision made by the Chrome team to push things along."
• "This isn't going to affect the mobile side - so iPhones, iPads, iPod Touches are not going to go dark."
• "Hardware to decode VP8 might be on the horizon - but Google can't do much to push it along except improve the codec, and do things like, well, drop H.264 support in its browser. (Pretty much everyone will see no difference, because the H.264 decoding will be handed off to the operating system.) I asked how many people are working on improving the VP8 codec, since that's sometimes helpful to know: is it a priority? But Google didn't want to discuss team sizes. Hm."

WebM's growing support

While this situation has become more complicated for video publishers, many within the tech industry believe that support for WebM will grow as the format gains wider adoption among software and hardware vendors. This past week, the Free Software Foundation, a non-profit corporation that advocates free software ideals and an early supporter of WebM, publicly urged web site owners to distribute videos in the WebM format, and abandon H.264.

This infographic by Niels Leenheer shows the current and future state of H.264 vs. WebM.

From: http://html5test.com/

Changes for Google, Apple and HTML5

Whether or not Google's claim of open video is genuine or not, or if its stand against H.264 in Chrome is really a swipe at Apple's dominance in the mobile browser market, it's interesting to note that both Google and Apple have seen a change in leadership within the last week – with Steve Jobs taking a leave of absence with Apple COO Tim Cook stepping in at Apple and Google Co-founder Larry Page replacing Eric Schmidt as Google's new CEO.

It's also interesting to note that just a few days after the World Wide Web Consortium (W3C), the international community that oversees the HTML standard, introduced a new HTML5 logo  to promote next-generation Web technology, a related standards group, the Web Hypertext Application Technology Working Group (WHATWG), a community of web browser manufacturers and people interested in evolving the HTML, declared that HTML5 was dead and that the HTML specification will henceforth just be known as "HTML" referred to simply as HTML.

As for the online video industry and what the future holds, the codec conundrum continues.

What Does Google's Decision to Drop H.264 from Chrome Mean For Online Video Publishers?

Just when it appeared that the online video industry was moving toward a video standard, with H.264 video playback in HTML5 – more chaos ensued with Google's announcement of its plans to phase out support for H.264 in its Chrome browser, in favor of open source formats like its own WebM or Theora. Like shot heard round the world, the news spread quickly throughout the web sparking a flurry of commentary and debate on the reasoning behind Google's decision. The overwhelming response within the online video industry is that this move by Google will set the adoption of HTML5 video back even further, and for online video publishers, this creates even more confusion and potentially a massive increase in video publishing costs.

The stakes are high in the battle for a video standard for the web and mobile devices. It's a war that's been brewing for several years with open source advocates pushing for a free and open codec unrestricted of proprietary licenses for the HTML5 video element, and an industry that has been built on a widely adopted yet proprietary and licensed video codec. But the lack of a single codec standard that is supported across web browsers is not only a serious issue for publishers but a stalemate for the industry.

WebM is a relatively new media file format Google introduced last May at the i/O developers conference, as a royalty free open source license packaging its VP8 video codec and the open source audio codec Vorbis, as part of Google's wider open source initiative called WebM Project. The goal of the WebM project, Google says, is "dedicated to developing a high-quality, open video format for the web that is freely available to everyone." WebM was launched with support by Mozilla, Opera, Adobe, Google and more than forty other publishers, software and hardware vendors. Absent from the list companies at launch was of course, Apple and also Microsoft, which back H.264 in Safari and IE9. However, Microsoft has said it will support VP8 as well. Google acquired the VP8 codec when with its acquisition of video codec maker On2 Technologies last year.

H.264/AVC video format is the digital video coding industry standard H.264/MPEG-4 Part 10, or AVC  used widely in set-top boxes, media player and other personal computer software, mobile devices, Blu-ray Disc players and recorders, game machines, personal media player devices and still and video cameras. Apple has become most outspoken proponent of AVC, and backs it as the standard video codec used for its HTML5 video compatible mobile iDevices like the iPhone, iPad and iPod Touch, as well as for iTunes videos. Several months ago, the MPEG LA reversed its decision and freed up the license of H.264, and will not charge royalties for any content that is free to consumers. It was speculated by many that the move by the MPEG LA was spurred on by Google's release of VP8 as an open source royalty free video codec.

But how much of Google's decision to abandon H.264 is based on its goal to "enable open innovation" or to capture future market share?

In a short post this past week on the Chromium Blog: HTML Video Codec Support in Chrome, Google Product Manager Mike Jazayeri said that its decision was a move to make Chrome consistent with the codecs already supported by the open Chromium project:

"We expect even more rapid innovation in the web media platform in the coming year and are focusing our investments in those technologies that are developed and licensed based on open web principles. To that end, we are changing Chrome’s HTML5 (video) support to make it consistent with the codecs already supported by the open Chromium project. Specifically, we are supporting the WebM (VP8) and Theora video codecs, and will consider adding support for other high-quality open codecs in the future. Though H.264 plays an important role in video, as our goal is to enable open innovation, support for the codec will be removed and our resources directed towards completely open codec technologies."

Some interesting commentary on Slashdot pointed to more scrupulous plans that Google is manipulating the market for selfish reasons to reap hundreds of billions in WebM devices. Also, on Daring Fireball, John Gruber asked 5 Simple Questions for Google Regarding Chrome's Dropping of H.264 which many of us would like to know the answers.


But what about all the millions of websites that content publishers around the world have adopted H.264 as HTML5's video element over the last year?
 Jazayeri added:

"These changes will occur in the next couple months but we are announcing them now to give content publishers and developers using HTML (video) an opportunity to make any necessary changes to their sites."

Okay, so what this means is that by default, the Chrome browser will automatically playback videos encoded in either the WebM or Theora open source standards, but will need a plug-in just like the Flash Player plug-in to play anything encoded in H.264 – which is actually the more widely adopted standard used throughout the industry. H.264 and can be decoded by hardware in many devices, like tablets and mobile phones.

So now content publishers will have to go back to encoding multiple versions of their videos in different file formats and codecs, primarily in H.264, WebM and even Flash. Or for many publishers, take the easy way out and go back to delivering video in Flash, since the Flash Player is baked into Chrome and Flash video will play on all the browsers.

Many are speculating that this is all about  timed announcement to steal Apple's thunder of its Verizon iPhone announcement, at least in the short term, and also upset Apple's domination in the mobile device and digital media industry. Others are calling Google hypocritical for dropping H.264 and not Flash, which requires a plug-in.

ReadWriteWeb calls this current situation, A Stalemate of Standards: What H.264 Means for the Average User, and spoke with blip.tv Co-founder Justin Day who said that Google's move is a step back to the Dark Ages and also Michael Critz, a freelance interactive and motion graphic designer, who agreed that this is a regression for the average user.

According to Day:

"I think from our standpoint this looks like a regression. We're all for open formats, but they should be chosen based on their merits, not merely their license. This move means that Chrome users will suffer from a worse user experience because they will need to rely on Flash fallback."

Last month blip.tv announced it has adopted HTML5 as the standard video player on its site after years of serving Flash video. Day says that there really is no competition between WebM and H.264 for the content producer because of the low cost tools available to create higher quality video using H.264.

As a content creator, Critz agreed:

"There isn't a desktop WebM conversion tool that worth a pile of beans to what's available now for H.264. I'm often traveling for video shoots. If I'm on a shoot in Miami and working all day then I get back to my hotel room I know I can depend on my Turbo.264 encoder to give me hardware accelerated H.264 encoding that I can use online, in Flash, on my producers' iPad, and on my clients' iPhones and Blackberry phones."

In a detailed report on Ars Technica, Peter Bright wrote that, Google's dropping H.264 from Chrome a step backward for openness and creates more work and cost for video publishers.

Bright said:

"Video distributors wanting to support both Flash and HTML5 users will have to encode twice; once in H.264, for Flash users, and again in WebM, for HTML5 users. This doubles the computational cost, doubles the storage requirements, and as an added bonus will tend to hurt quality. This is inconvenient for a small site with one or two videos; for sites like SmugMug it's an enormous headache. They can either suffer the doubled costs and complexity, or ignore HTML5 altogether and stick with Flash.

It looks like sticking with Flash and ignoring the (video) tag is indeed what SmugMug may end up doing. And who can blame them? Flash works for almost every Internet user, and Flash supports H.264."

Jan Ozer also commented while Chrome only has a small percentage of the web browser market it will be interesting to consider what's to come. He advised producers and publishers to check their wallets because they'll find less money in it, now that we're well on our way to a Two-codec world.

So how much will this disrupt the browser market share?


As far as browsers go, they are waging their own war and actually, Chrome is a small player compared to Internet Explorer and Firefox which is second to IE. But in just about two years since its launch Chrome's market share is currently about 13.5%, which is about twice that of Opera and Safari combined.

Both Firefox and Opera lack native H.264 support, so combined with Chrome, the browsers without H.264 support total 44.05%. Internet Explorer has the largest market share with 44.53%, Safari has 5.8%

Along with the browsers, the opponents on each side of the format war are Apple, Google and Adobe continue to wrestle control over video formats, and I've said before the battle continues and publishers and consumers are caught in the struggle. Now with even more fragmentation within the online video ecosystem no support for common video codecs across the industry, we are even further away from reaching an online and mobile video standard anytime soon.

For video publishers, it's clear that there will be additional costs involved in delivering video to all web browsers and mobile devices. For companies in the video encoding business, this must be your lucky day.

Jeroen "JW" Wijering: HTML5 Video Is Not Quite There Yet

Earlier this year, Mark Robertson and I met with Jeroen "JW" Wijering, Chief Digital Architect of LongTail Video at his office in New York City, to talk about the growing interest and hype around HTML5 video, and the most pressing issues facing its adoption. Jeroen is an online video pioneer and the creator of the ubiquitous JW Media Players, which have generated several million downloads since their launch in 2005.

In a post on the LongTail Community Support Blog (and also a guest post on Reel SEO), Jeroen noted that Apple's launch of the iPad along with Steve Jobs' arguements with Adobe over Flash accelerated HTML5 video development. He wrote that there's a lot of video tag euphoria within today's tech industry and the practical side of HTML5 video development has been overlooked and faces a major threat.

"The video tag is still in its infancy and misses certain core functionalities. As developers demand these features, browser vendors are tempted to implement incompatible solutions instead of agreeing upon standards. These hasty developments, already underway, are setting HTML video up for the same chaos as HTML styling in the pre-CSS era."

Jeroen says, the most pressing issues facing HTML video development are:

• Codecs - H.264 versus OGG debate is ongoing, but all browsers have placed their bets - Firefox and Opera favor OGG, Internet Explorer and Safari choose H264. Chrome plays it safe and does both. Today's HTML5 video format is H264.
• Streaming - HTML5 does not specify a streaming mechanism yet. While this is being worked on (W3C: Fragments, Media Multitrack API), it means that live, DVR and long-form video content cannot be played using a video tag.
• Fullscreen - While a small feature at first sight, fullscreen playback is essential to the success of HTML5 video. Without fullscreen, HTML5 video is mostly useful for presenting short clips.

Jeroen suggests that the addition of both captions and fullscreen support would be big steps forward for HTML5 video. He says that cross-browser support should be practical and compatible, or we risk web development regression.

"Browser vendors should be stringent when building solutions that are both practical and compatible. If not, crossbrowser HTML5 video will be too difficult, not to mention expensive, to implement. This presents the risk of web development regression.

In favor of its advancement, we cannot allow this to happen. Online video will go mobile and big screen. It also needs to become accessible and searchable. HTML5 video will advance the progress in these areas, if developed carefully and intelligently. However, without compatible solutions, online video is in definite jeopardy of a setback."

While Jeroen says that HTML5 video is not quite there yet, as the year comes to a close, there are signs of the maturing market:

• Google's release of WebM  (VP8 royalty-free video codec) several months ago which Jeroen says they'll be keeping an eye on, 
• Support for HTML4 in beta versions of all the major browsers, including Internet Explorer, Firefox, Safari, Chrome, and Opera HTML5
• Advertising solutions from mDialog, Brightcove and FreeWheel, MeeFeedia and Yume coming to market for Android and Apple iOS 
• Longtail's release of its JW player HTML5
• Adobe teaming up with Kaltura for its HTML5 player
• YouTube, Vimeo, Dailymotion all have HTML5 players, and Blip.tv is readying HTML5 to be its standard video player

These are but a few of the many innovations and growth of HTML5 video within the online and mobile video industry. As we move into 2011, we'll continue to see the technology evolve at a rapid pace, as well as the debate.


What is HTML5? (reprinted from HTML5 - Advertising FAQ)

• HTML is the mark-up language used for the World Wide Web. Almost all web pages you visit on the internet are based around HTML code. HTML5 is simply the fifth and latest iteration of this mark-up language that allows for more dynamic, animated and interactive web pages. Up until now web pages have had to embed content or plugins like Flash for items like video players or interactive animation. HTML5 allows for lots of this functionality to be done without an embedded file. This doesn’t mean the end of Flash however because Flash too continues to evolve - it is an alternative. See more on the W3 website. 

Resources:

• Dive Into HTML5 
• HTML5video.org
• HTML5 Video - IEBlog
• Introduction to HTML5 video - Dev.Opera
• 4.8.6 The video element — HTML5'
• HTML5 video Tag  
• HTML5 Rocks

Related:

• HTML5 vs. Flash: How will the battle play out in 2011?
• Why rivals Google and Apple agree on HTML5
• Dive Into HTML5 Video With The Masters – HTML5 ReelSEO Webinar Replay Video
• Apple Didn’t Kill Flash, HTML5 Did
• "The World is Moving to HTML 5" and Other Flights of Fancy
• Lift SEO with HTML5
• Top 5 Security Threats in HTML5

Updated 12/22/2010: Added my buddy Mark to post