HDMI Livestreaming: CPU, Harddive, Thunderbolt ...the search for the bottleneck

After coming yesterday to the conclusion that maybe the CPU in my computer is not fast enough to take in the HDMI signal ( that is merged on the Blackmagick Design with the phono sound input ) via Thunderbolt and therefore causes a challenge during the livestream, I put it up for a test on hardware benchmarking today.

Using Apples Activity Monitor, I wanted to see if my CPU is actually suffering under the data on-slaught it has to deal with while taking in the uncompressed signal and then having to compress it to a streaming format ( in this case "Flash 720p" ) with Telestream Wirecaster.

First i started with BM Media Express, to see how the intake of the signal will work all by itself. While just previewing, the CPU still had around 80% idle space and neither the memory nor the harddrive got any access...except the occasional pages written and read by the operating system. But nothing that broke my computer into sweat. We had the standard preview delay of about 1-2 frames when moving the camera. This seems to result from the general output of the camera. I realized i had the same delay when using my Zacuto EVF, even before the HDMI output was uncompressed clean HDMI. So i guess we just have to live with that delay, which seems to happen on the camera side.

So I moved on to see if the problems get bigger when starting to stream ( means, the signal gets compressed in real time ) and send over the internet to UStream.

This is the CPU usage:

Based on the endresult, which is heavily lagged, I would guess that most gets lost via the internet connection from here ( im located in Kenya now none-the-less ).

HDMI live streaming: alternatives

After having taken a look into the Blackmagic Intensity Shuttle, it pretty fast came to the point where we had to realize that for a field broadcast device the Blackmagic solution lacks the hardware encoding support, leaving it all on the shoulders of the computer. Being in the field, traveling will most likely not let you connect to a super-high-end workstation that can handle the data flow of an uncompressed HDMI signal from the 5D Mark III.

Carrying hardware supported, traditional devices might also add to the load we have to bring along and is counter productive to what I try to achieve in this blog: Making news broadcast light, flexible and less hassle while traveling to remote places on earth.

So whats the alternatives. One that I had my eye on since some time is the Terradek Cube. It does pretty similar things to what the BM Intensity does, just more...and more mobile.

Check it out: http://www.teradek.com/pages/cube-details#-technical-specifications

Costing much more then what the BM solution cost's it seems to come with a pretty sufficient skillset. Most of the encoding seems to be done via hardware, so taking it away from the shoulders of our little computer.

HD Live broadcast: computer specs

Before we go any further, to stem the HDMI uncompressed data stream of a 5D Mark III , try to get a computer with around this specs:

• Quad Core i7-2600 series CPU or better
• 8GB RAM
• 5Mbps or better upload bandwidth
• Dedicated video card, 512MB VRAM

The Realworld : LIVE broadcasting with the 5D Mark III

So, i spent the biggest part of my last week on getting actually a real live-broadcasting session going with a 5D Mark III. I don't talk about a Live-Session for some sorry ass online stream that no body watches, but actually a live quality that can be used by REAL broadcast media ( like TV Stations i.e. ).

I have to admit, i haven't really been completely successful. But I will guide people here through my steps and the caveats i had to work around etc.

First of all, get the latest Firmware update for the 5D Mark III. On the day this was written we are still on the 1.2.1. That gives us clean, uncompressed HDMI output. For streaming via internet from some ass-of-the-world-place I have to admit i could care less about the resolution ( cause we have to compress our signal again a lot to actually get it to places via a BGAN or something ) but we get a CLEAN signal. Means, we dont have all this letterboxes, menu's etc.

I would have loved if the 5D would have an SD outgoing CLEAN signal, but that is not what the camera is meant for and i clearly try to push the boundaries here, so no complaint.

On top of that we need to tell the camera how we want our HDMI signal to be processed. We can do that in the camera Menu. There is an item that let us define what settings the HDMI should use. We have the choice between the 24p and the 60i. Working ona  progressive plattform I choose the 24p for now ( which is actuall of course more like the 23.98p ).

To pipe in our nice, CLEAR and uncompressed HDMI signal to my MacBook Pro (MBP) ( iCore7i, 2.2 Ghz, 8GB Ram ) I took a Blackmagic Intensity Shuttle with Thunderbolt connection.
To stream my media i tried Telestream Wirecast and Livestream ProCaster.

After connecting the cam via HDMI cable to the Intensity Shuttle, and the Shuttle to the MBP via Thunderbolt I start up the included Blackmagic Media Express tool that comes with the Shuttle. Before doing the whole Shebang, I just want to check if I get a general signal from the camera.

After starting up, the screen stays black. So obviously no signal. Withs some research i find out that I have to look out for 2(!!) settings. One is in the Blackmagic driver software, which we can access through the "Settings" menu in OSX the other is in the Blackmagic Media Express. In both menus we have to specify the HDMI signal. Which is of course 23.98 p.
After doing that the screen shows the nice, clean output from the 5D Mark III.

The first thing we have to understand is, that the HDMI out of the 5D Mark III doesn't deliver any sound. Its only the video signal. The Shuttle has next to the HDMI input also a phono-stereo connection ( the white and red plugs we all love from our home-stereo-systems ). Or there is the alternative, to connect the microphone directly to the MBP Line-In.

Not having a Jack-Phono cable at hand first, i just connected the microphone to my Line-In. That at least gave us some audio signal. But the audio signal is heavily delayed.
In general piping out a uncompressed HDMI signal to a computer like mine, puts a lot of pressure on the ressources. I haven't really figured out by now if its my computer or the Intensity Shuttle, but the video in general is slightly delayed for a few frames.
The sound is even worse.

After doing some more research, it turns out that there are many sound issues. Many people seem to have the problem that the sound comes first and the visual lags behind. That comes mainly from the fact, that the video signal has to be dealt with and it takes some time to calculate the operational encoding etc. with such a data heavy HDMI signal.

But in my case, the SOUND actually comes late. It seems that the apple OS is first processing the Thunderbolt signal and then it connects the Line-In. This could be due to calculation limitations on my computer or just a design-"flaw" ( internal technical limitation ). I could not figure that out, for me here in Kenya not having a "newer" MBP with more calc power or more memory.

The only solution I could think of, is to take the sound out of the 5D Mark III and to connect a 3.5 jack to phono cable to the Intensity Shuttle. In that way i hoped that the sound and the video gets mixed into one signal that actually is itself synced to each other...even maybe in its entire form delayd...

But after doing all that...it doesn't. Sound is still significantly delayed in Telestream Wirecast preview. Actually so much, that it is virtually unusable.

BUT: After recording in some testfile...i realized...its actually pretty much in sync. And that is awesome.

The biggest problem still is the amount of data. We have quite big data chunks that need to be converted to a lower res signal ( for online transmission ) and that can actually eat up quite some processing power. Maybe it could be even too much for my little machine, which would end up in quite a delay on the receiving end.

Using Telestream Wirecast i was testing it with UStream to some other computer and the lag was just enormous. But this could originate from quite some different issues...internet speed on the othe side, my own internet speed, encoding/decoding on the receiving end etc.

My assesment so far: online Live broadcasts via DSLR's can work, if the internet speed is fast enough, the computer is a powerful machine that can decode/encode the HDMI signal in a good timeframe and the receiving end of the broadcast isn't a bottle neck either. So, many "IF's" in the end.

UPDATE: based on the BM documentation of the Desktop Video drivers for the whole Intensity series etc. the driver settings ( which, remember, you access via the OSX Settings window ) as apparently the functionality to convert the high res HDMI signal to something more usable for Live-Streaming. Unfortunately, it doesn't really seem to work. Absolutely independent on what I click on the drop down menu on the OSX version of the drivers, its all the same in the end ( on top of it all, the manual screenshot looke like nothing as I get with the 9.7.5 drivers on my mac ):

UPDATE: Looks like there is no real-time down conversion in the Intensity Shuttle during capture. Only during playback by software. That means, you can't let the Shuttle do the dirty work for you and most likely you need a REALLY FAST machine.

Just also for info, here are the specs of the computer i used for this test:

MacBook Pro , model late 2011
iCore7i 2.2 Ghz
8 GB Ram
AMD Radeon HD 6750 M

More bang for the buck...

Again the people at Magic Lantern outdid themselves with a very interesting hack this time. The were able to increase the dynamic range of the 5D Mark III up to 14stops. This itself is pretty impressive. Paired with the big full-frame sensor of the 5D, this should deliver some outstanding image results.

People over at PetaPixel have already taken a peek into what you can do with still photography. The noise in the shades are significantly less and soon we are going to have some impressive HDRI style shooting with one snap.

Although you need RAW frames to do this effect, it limits right now what you can do with a 5D MIII for video. But, it is a first step to some truly impressive performance.

How is this all useful in the real world ? Most likely not so much. Its a nice technical show case, but in reality, shooting short seconds-long clips so i can get the RAW images for video doesnt help me much. Also the fact that that you only have half resolution, based on the the way they implemented that hack, might not be appealing to many stills photographers.

But for me its all signs of things to come...

Canon 70D, the future

The new Canon 70D is announced and Canon already showed around its new focusing system.

I have to admit, judging on the videos this new system which is more related to traditional cam-corder focusing implementations, is really fast. Nothing compared to the slow, sluggish things we have seen in the last years on HD DSLR's when using the backscreen.

Interesting is that it took Canon quite a while to "implement" this technology. Im no engineer, but i have the feeling that the technology, which is around for such a long time, would have been already possible with the last generation of DSLR's. But then you would not be able to sell more in the future i guess...

Canon claims that the 70D was developed with the video professional in mind and not sure if it will be worth it to wait for the 7D Mark II ( which will have most likely most of the traits from the 70D ), but the swivel screen, the fast autofocus system and a worked over UI could be a real unique selling point for someone who wants to upgrade. I would definitely love to get my hand on one of these to see how they perform in REAL life situations.

Update:

So, here is the complete spec for the camera. What struck me first is the lack of an Audio-Headphone jack. Seriously !? A camera with improved AF detection system, a new sensor, a swivel screen and then no audio output via a standard jack connector to monitor the recorded sound ? That sounds to me like a deliberate attempt to "scale" down the camera to make it really of very limited practical use nowadays. After the 5D Mark III I thought we are past the "mixer box" to hear actually what my mic picks up...I hope they "reserve" this feature to the 7D Mark II ( if they cut the swivel screen out of this one, i will have to say that Canon is just trying really hard not to cannabilize their C series camera line ) by limiting their DSLR's of practical elements.

production diary: Nuba moutnains - Sudan ( part 2 )

With a little bit of a delay, the micro doc that we shot is now online.

And while we are at it, i will deliver the 2nd part of the production diary. Last time i gave a quick roundup of how it was to work with the 5D Mark III in a challenging environment. Now i want to go more into the details.

Overheating: While i had some 2 weeks holiday back home in germany, i put my 5DMIII into the service after the battery compartment cover broke ( a little bit of a weak spot of the camera it seems ). So i put my old 7D back into action when i shot some small little piece to test a new tripod setup ( Induro AT213, with Novoflex MagicBalance and a Manfrotto HDV 701 head ). I realized pretty fast that even during a normal sunny afternoon in central europe, my 7D showed the "overheat" warning pretty fast. My 5DMIII is much more enduring, even during 50 degrees heat ( celsious that is ). Both of them have not active fan ( like the C100+ cameras).

Reduced equipment: Again and again i promised myself i will forcefully downgrade my gear. And i was always remembered why when walking up and down steep slopes under scorching sun.  A small mic, a good alround lens ( maybe tele additinal), a leightweight tripod is all you need in the end when you know what you are doing ( a radio mic is so small, i would not even put that into the count ). Especially when working in Video, a lot of people tend to carry far too much stuff into the field. Especially since we can now leave DV tapes at home, we can really travel light. Never understimate advantages to travel light. You are less exhausted, you can't lose or break so many things and for outsiders you look less threatening.

Edelkrone Pocketrig: I made the mistake of taking the Edelkrone Pocketrig with me, just to give it a proper field test. I have to admit, it pretty much failed me. Although i first had a positive view of the design etc. it turned out that it was just so annoying to set up and not very reliable that i put it back into my bag from the 2nd day on and never took it out again. I rather shot with handheld and tried to keep it steady with good breathing techniques.
I know its not a professional tool, but the LACK of "fix locks" for all the hinges and moving parts rendered this tool really useless in demanding situations. You end up adjusting it more then actually filming.

Sound: Next to the on-camera recording, my production partner Tristan, used Zoom recorder to record some natural sound and things that he encountered. That was very helpful in the end production to fill some sequences with better atmosound.

image by Tristan McConnell