July 7, 2016
Scaling Your API Design Workflow
Heavybit member company Apiary analyzed hundreds of thousands of API projects, and from these they selected their Top 5 API Commandments tha...
Welcome to Demuxed, a new podcast for and by engineers working with video. Demuxed is hosted by Matt McClure, Co-founder of Mux, Steve Heffernan, Co-founder of Mux, and Phil Cluff, Director of Media Technologies at Brightcove.
In this first episode, Matt, Steve, and Phil are joined by Reinhard Grandl, Solutions Architect at Bitmovin. The group has a great discussion about NAB 2016, 360 video on the web, and what’s next in online video delivery formats.
About the Guests
Reinhard Grandl is a self-professed video streaming enthusiast. He’s currently a Solutions Architect for Bitmovin, a service that improves video quality on the web while delivering adaptive video streaming with the lowest startup delay, no buffering, and in the best possible quality.
Matt McClure: Welcome to the first Demuxed podcast. This has probably been a year in the making, or at least since before Demuxed 2015. Anyway, today we’ve got Reinhard from Bitmovin, and he’s going to tell us all about 360 Video, etc.
But first we wanted to talk a little bit about Demuxed in general. If you’re not familiar with it, last year was the first year of the conference. But we think that, at this point, it’s more than just the conference but also kind of the global community of video engineers.
We’ve started planning for Demuxed 2016 at this point. The conference dates have been announced. We’re looking at October 13. And October 12 is going to be FOMS.
Phil Cluff: And for those of you who don’t know what FOMS is, that’s Foundation of Open Media Standards. Which is a great group of people.
Steve Heffernan: And software. There’s a lot of S’s in there.
Matt: FOMS is also one of the better video experiences I’ve ever had. I think that’s really what got me interested in the standards. Because if you just hear it, it’s super boring. But that’s where, correct me if I’m wrong, Steve, but I’m pretty sure that’s where MSE and EME kind of got their first kick.
Steve: Yeah, exactly. And this was back in probably 2011, 2012, when those conversations were kicked off. That’s where a lot of those standards got started.
Matt: We’re coming off of NAB. I mean, that’s what, two weeks ago?
If you’re not familiar with NAB, that’s the National Association of Broadcasters.
It’s a great conference where, I guess it’s more of a trade show, but 90% of what you see is going to be things for a local broadcaster.
Phil: Things I can’t afford.
Matt: Things you can’t afford: lights on top of TV towers, other random satellite equipment.
Phil: More drones than you can shake a stick at.
Matt: But yes, there are awesome things like drones and cool online streaming video startups, things along those lines.
But since we’re coming off of it so recently, I figured we’d at least go into what we saw and thought was cool. Did you make it to NAB this year, Reinhard? I know I saw somebody there from Bitmovin.
Reinhard Grandl: Yes, actually that was my first time at NAB. It was pretty impressing, and also we had very nice conversations there. I think I also saw a lot of you guys there, especially at the meetup. So it was a very nice show.
Matt: That’s right, you made it to the Demuxed party for a little while.
Reinhard: Yes, actually I did.
Matt: Cool. The thing about NAB every year, it’s kind of the largest, coming from the online video world, what we see the most is what everybody’s excited about. And it’s always what your customers ask you about, but it’s very rarely what you’re customers actually want.
Three years ago, it was all about HEVC, and everybody wanted to know, “Do you support HEVC?” There’s not a single piece of hardware that actually supported it three years ago. So it was useless, but there’s always kind of that fad.
Steve: And then like two years ago, it was 3D TVs, and those kind of didn’t go anywhere.
It’s hard to know if what you’re actually seeing there is the exciting thing that will actually carry on into being a reality.
Phil: I guess those things this year would be what, 8K? We started seeing a lot of 8K stuff this year. Saw a million dollar TV, and it’s priced per square foot, which was quite terrifying. We have to ask, is VR this next thing that will disappear in the next two years?
Matt: It’s definitely the thing that, we’ll get into this more later, but I think it’s definitely the thing that people aren’t quite ready for, in terms of wide adoption. But it’s already kind of there, right?
I’ll give it to Wowza, I think they had the best swag of the year with their free Google cardboard handout. But I just don’t see a lot of consumers dropping $800, $900. What is even the published price of the production Oculus?
Phil: That’s probably a 1,000 bucks, right?
Matt: I think it’s roughly the same for the Vive.
Reinhard: Vive is even more expensive, right?
Phil: Yeah, because you got those dangly controls you wiggle around, and remotes you throw around.
Reinhard: On the other hand, you get the Samsung one for free.
Matt: With the Vive?
Reinhard: No, if you buy the cellphone, then you get the Gear VR for free.
Phil: Do you want to plug your phone into the front of your VR headset?
Matt: Honestly, coming from my friends in VR, and you guys might have the same feeling about this at Bitmovin, but
my friends that are actually familiar with the VR space constantly call cardboard “poisoning the well.”
It’s ruining the VR experience and they hate it.
They totally turned me on to it, putting a video in this headset, and then all of a sudden experiencing a 360 video was transformative. It was really, really cool. They finally made me believe in the general application of 360 video online.
Steve: But there is a big difference between what 360 video is and what VR video is, right? That’s probably where I’d assume they get caught up, right? .
Matt: This might be worth talking about, honestly. I don’t know if we want to get into that now, but let’s talk a little bit about what else we saw at NAB in general. Lots of AK.
Phil: HDR, lots of HDR, love to talk about HDR.
Matt: Lots of 360 video in general, lots of HDR. Phil, you were telling me about consolidation of suppliers and platforms. Why don’t you tell us a little bit more about that?
Phil: From my perspective, what’s interesting is, yeah, we saw some new, exciting stuff, the 8K VR, the exciting stuff this year.
From the side of the vendors for the encoding space, we’re seeing a huge amount of consolidation right now.
Like Amazon buying Elemental. That means we’re the bottom end of Encoding, Elastic Transcoder. We’re seeing this new Telestream cloud product which is Pandastream re-batched now, we’ve got X-Com buying video, Harmonic buying Thomson.
I think that was the the most interesting one for me, walking around the floor and seeing this big space where Thomson were going to be, and just a sign that says, “Go to the Harmonic booth.”
Matt: That’s an expensive sign. That’s a really expensive 301.
Phil: I think there was a lot going on, and everyone kind of getting their platforms to a state where they are stable and consolidated into one place now. I think a lot of interesting ones is Beamr buying of Vanguard there. I don’t think any of us saw that one coming, right?
That was a surprise for everyone, to get that investment and then go buy Vanguard. Certainly a really interesting place right now, and I guess that means, from our perspective we’re going to see a load of interesting innovations from all these companies next, next year, next NAB or NABC even. That’s in September. I think that’s really interesting.
Matt: On that note, we also talked a little bit about HEVC cooling off a little bit. Do you think that’s because it’s actually kind of becoming a thing now, and it’s less the new hotness and more just a… standard is a bit of a stretch.
Phil: I guess, from my perspective, you’re right. HEVC last year was crazy. Everyone was just talking about HEVC.
Matt: Everyone had their hardware encoder for HEVC.
Phil: Yeah, great. I can’t play it anywhere, but I got a hardware encoder for HEVC. I think the broadcast space and VOTT space, we’re certainly seeing some really big players making bets on HEVC now, which is kind of surprising for me.
But I think it’s looking really different in the browser and phone space; we’re looking at Google now delivering VP9 by default on all Chrome.
So that’s nearly 40% of traffic for YouTube, is VP9.
I think with the direction we’re going with the live-stream media now, what we’re going to see in browsers over next 12 months and what we’re going to see when they actually embed a device in someone’s house, it’s going to be very different.
I didn’t really see much talking about VP9 at NAB. There were a couple of people who had some early hardware accelerated VP9 encoders, but nothing mind-blowing. Whereas, there were a lot of companies doing HEVC and accelerated HEVC as well.
I guess that’s another thing that is becoming really popular right now. Elemental have been doing it for years, GPU accelerated encoding, and they’ve done that in AWS as well.
But I think what’s interesting, as well, you’ve got things like the Intel Media SDK coming out, and the cost now to entry on the accelerated transcode solution is dropping dramatically. The problem is there is no cloud solution there yet. So that will be interesting to watch, I guess, over the next six months, year.
Matt: When we talk about VP9 now, it feels like it’s kind of gone from the Google-YouTube pet to actually being a legitimate delivery option for a lot of customers. Which I wouldn’t have said a year ago, two years ago. It was just kind of “that other format.” It was delivered before H.264, and people kind of started talking about VP9 maybe a little bit. But now it’s legitimate.
Steve: When it was VP8 and when it was certainly that other bastard child codec,but now with VP9, it’s really the only next-generation codec that’s available in a browser. There are no other options. So if you’re really hurting for bandwidth savings, that is the obvious solution, the only solution currently. Makes a lot of sense.
Matt: And hardware, you talked to the YouTube guys, and they are kind of able to bully,bully may not be the right word, but they can throw the Google weight around. “If you want to put this YouTube badge on your TV, then you have to support VP9,” and that’s happening.
P I think the important thing there is, when you look at the live-stream media, you’ve got not only Google framing the weight behind it, you’ve got “beep” Netflix, you name it. They are throwing their weight behind this AV1 codec, which that is going to be a big deal.
I think the interesting thing with VP9 is when you start looking at some of the stuff that’s coming out around, doing VP9 and FMB 4 content, the spec that Netflix put out recently, from what I understand, that’s going into Chrome Canary pretty soon. So you’re going to get to a place where you don’t even need web packaging to deliver VP9.
Steve: That’s interesting. I didn’t know that.
Phil: Netflix put out a spec. Looks like that’s what Netflix are doing, their kind of intermediary storage.
Steve: I had no idea.
Phil: Yeah, it’s pretty cool.
Steve: Are you guys ready?
Reinhard: We recently got a rent for HEVC, but I think VP9 definitely makes sense also for web-based streaming. Especially if you consider that the majority of your users are using Google Chrome, then it might be a good trade-off to go for second encoding and have nevertheless a lot of bandwidth savings as well.
Phil: I think something really interesting is when you look at the sort of timelines that AOM guys are talking about, it’s so aggressive. They are talking about a bitstream freeze by the end of the year, and a filters freeze by the middle of next year, which is really aggressive.
I don’t think the follow up of HEVC is going to be anywhere near that timeline, which I think that’s an interesting thing, and I don’t know if that pushes MPEG in a particular direction about how they go post-HEVC, or what that means.
Matt: Was there a Lytro booth, or did I just miss the Lytro booth?
Phil: Well, they did a talk which was really good. I think Zack went into the presentation and got kind of hands-on, had a chat with their guys, some really cool-looking stuff.
I think that space is really, really interesting right now, actually, the light-field optics space.
As an MPEG working group the other week, they are working from two perspectives on building light-field storage solutions. Because one of the big problems of this new Lytro camera puts out a heck of a lot of data.
It interacts, interacts, interacts with SSDs to record anything, and there is actually a lot of work going on to standardize how we record and compress light-field data. That’s going to be pretty cool.
Matt: You guys seen anything extra special in the HDR front? Aside from HEVC last year, HDR felt like the non-VR buzzword this year, including the people that produced the laser projector, Christie, their demo was.
Phil: It was like a hockey field, wasn’t it?
Matt: They had a full on movie theater, like a 15-seat movie theater, where they showed off their HDR laser projectors and it was honestly pretty phenomenal.
They would show you what black looks like on a normal projector, and it’s kind of grey. “This is what black looks like on our projector,” and it was just pitch black. They had to explain that they hadn’t just turn off the projector. I was impressed.
Steve: Full Rec. 2020 color. Which I’m going to say that like I fully understand what that means.
Matt: All the 2020s.
Phil: My vision is 20/20, is that the same thing?
Matt: My other favorite HDR, I would love to be the marketing person that okayed this, because I know they were giggling the whole time. But I’m standing there looking at this HDR television on the outside of somebodies booth, and it’s glorious.
There is this majestic eagle sitting on a post, and all of sudden it kind of jumps backwards, and a giant stream of poop flies out the back. Another eagle enters the scene, but it’s in super-slow motion.
So for a solid second and a half, there is just this giant white stream coming off the back of this eagle in full 8k HDR.
It was the end of the big butt bunny all over again. Except way more majestic and sad at the same time.
Phil: I think one of the best HDR demos I saw was actually Elementals. They had a really good side-by-side, I don’t know where they got the footage from. It was a fire-tosser, fire-eater. It was really visually impressive. The fire really brought it out well.
I think last year the best HDR demo I saw was actually, believe it or not, “The Lego Movie.” It’s so colorful and so bright that it looks great in HDR. I think that was Dolby showing that off in Dolby Vision last year.
Steve: Anybody else seen anything, other than drones, which are always cool. Some guys were trying to pitch broadcasters: “Wouldn’t this be interesting to you guys, to broadcast drone racing?” They were pitching first-person video drone racing.
Matt: That was a hard sell.
Phil: I think that’s cool when it includes 360 cameras on top of them. When you can just look around, that’s going to be nauseating.
Matt: We’re like, “No, I refuse.” I played that stupid space game with the Oculus DK 1, and I was sick for a week and a half.
Phil: I was happy because, I didn’t get much time on the floor this year, but I did manage to get hands-on with both the Rift production model and the Vive production model, which was great. I was really happy to actually be able to get hands on both, because my pockets don’t stretch to those as toys, unfortunately.
Matt: There was also that 360 camera. It was actually pretty impressive. It was kind of a square. You showed it to me.
Reinhard: The Nokia one? Are you talking about the Nokia?
Matt: Yeah, that’s it.
Reinhard: It was that one as well. That’s not the one I sent you.
Matt: Oh, OK.
Phil: The Nokia one is really cool as well. I like that one. I can’t remember what it was.
Matt: This segues nicely into the general buzz about 360. We walked by the Wowza booth, and we were talking to some of the Wowza guys. They’re like, “We’re not really doing anything special for 360. We’re just delivering video. And it just happens to be…”
Steve: Flattened onto a plane.
Reinhard: Equirectangular .
Matt: All we’re doing is just delivering that, and then you can play it back however you want.
They are still giving out cardboard headsets and talking about VR like it was like a core competency. Just because that’s what everybody wanted to hear.
What did you guys see that was innovative on that front, rather than just, “What Wowza is doing was cool.” It’s just that there was nothing out of their traditional offering. It was just, “Look, you can use this to deliver a VR video.” What were you guys seeing on that front?
Steve: I saw two VR demos there, and that actually kind of surprised me, that I wasn’t able to interact with more. Maybe I wasn’t trying hard enough, but there was one where there was 50 people sitting in different chairs, spinning around in chairs, and it looked like…
Matt: The VR cattle grazing.
Steve: The cattle grazing, yeah. You got in line, you sat in a chair, and people just look off in random directions. It looked really kind of freaky.
Matt: Honestly, it made me question if VR could ever take off, because that’s what we look like under VR. Just a bunch of idiots with a giant block on their face, spinning around in chairs, staring off in different…
Steve: Yeah, but then we got in line and tried it out.
Matt: I couldn’t help but think, as I was spinning in this chair, looking at the ground and the sky, what I look like to everybody else.
Phil: That was actually one of my favorite VR things about NAB, was taking videos of people with VR goggles on. I have so many videos of guys just going “Wow, look at this. It’s over here, do you guys see this?”
Matt: No. Literally, you’re the only one who can see this.
Steve: There is something incredibly disconcerting about standing in a booth just waiting in line to play with some headset, and the guy just turns around and faces you like he is staring at you. But he is actually looking at God knows what.
Phil: Did you guys see the Little Red Riding Hood demo? I think that was actually the demo from the Nokia camera. They shot this Little Red Riding Hood sort of sketch where there was a girl in the woods, in a red dress, and you were kind of immersed in the middle of it.
You looked around and the wolf was there. It was quite cool, certainly visually, it was really well shot. And the resolution was really high,it looked great on the headset, so I was very impressed with it.
A couple of other guys came to the booth and asked me to have a look at some VR stuff they were working on, and it was really interesting. The concept is you’ve often got a huge catalog of content, and your content isn’t going to be 3D, overwhelmingly, if you’re a traditional broadcaster, you probably don’t have a lot of VR content.
Their concept was they would put you in a virtual room to watch 2D content.
So you’re put in this virtual theater where there are people that sit next to you and stuff, all to watch this 2D content on a big screen in a 3D world. It’s kind of crazy.
Matt: I would totally do that on an airplane, though. Reinhard, you might be able to tell us a little bit more about this phenomenon in general, but I was asking somebody I know from the Oculus team about that. All I really want to do is put a headset on in an airplane and pretend I’m in a movie theater.
You can have people walking in front of me with popcorn for all I care, but a big movie theater experience on a plane with a headset on, and I have no idea that there is a 9-month-old screaming behind me.
I’m just in a movie theater, right? That sounds awesome and he was explaining that apparently, when you’re flying and the plane turns, you actually end up turning your head slowly until you’re kind of looking at the bottom right-hand corner, whatever direction the plane is turning.
But you don’t even realize you’re doing it until all of a sudden you’re facing the opposite direction after 20 minutes of watching this movie.
Steve: And staring at your neighbor.
Reinhard: I think that would be fun, I would love to see that.
Phil: I want everyone else to wear the 3D headset so I could watch this happening.
Reinhard: Actually, that’s a good idea to have those things on a plane.
Matt: I guess the question there is, when it comes to re-centering content, I guess this whole thing segues nicely into what you guys see as your special sauce at Bitmovin.
Because from the outsider’s perspective, what I know about Bitmovin is you guys have a player, particularly for segmented streaming, adapted bit-rate streaming, and you guys have an encoded API once again focused on adaptive bit-rate content and segmented outputs.
But I had no idea that you guys were focused so much on VR until actually we started talking about the meetup last month, and just for reference, video should be up soon on YouTube in general.
Phil: He’s staring at me as he says that, knowing that I’ve got the only copy of it all.
Matt: But Reinhard spoke, I guess that was the April SF Video meetup, which if you’re not familiar, sfvideo.org, last Thursday of every month.
He talked about 360 video and delivery and playback at the last meetup, and before we started planning this, I had no idea that you guys were focused on that. So, what made you guys decide to go down that route, initially?
Reinhard: You’re right, we’re coming out of an encoding and player-like service. But it was one of our last Hecodons in 2015 where one of our engineers decided, “Let’s go for a plugin so we can stream 360 degree content with our player.” That’s where it all started and things evolved, and we saw a lot of interest in the industry.
Not that people are using it too much in the production, but they’d like to know that you can deliver that, maybe for a good application in the near future, that they are ready. And that’s exactly what we are doing now. So we get ready for fancy applications which are not fully there yet. We spoke about the Uplink stuff, actually.
I cannot see people sitting in their living rooms with VR devices on their heads and doing things. But there might be some killer application coming soon.
Matt: Really, right now, what you guys see it as is more of a conversation starter, right? Like, “Look at this VR product we have. Also, check out the stuff that you actually would use in production right now.”
Reinhard: It’s really a door opener. We had some live streams in this context already, which are more or less production. But to be honest, they are things you want to talk about,and people want to hear and people want to make sure you are ready for it, but it’s not so much into production right now.
Matt: What do you see as the blocker? Let’s pretend that everybody does have an Oculus or Gear VR and they want to watch 360 content in their living room right now. Up to the headset, what do you see as the current blocker for that being?
Reinhard: I think that there is simply not too much content available right now. It’s hard to record it and also expensive to do that in a really good quality. And up to now, not too many services are having the majority of content in 360 degrees. I think we’re more or less waiting on the studios to produce something.
Phil: Is bandwidth going to be a problem here? Because obviously, like we discussed at the meetup, you’ve got a very big amount of data coming down, but the viewport is only a small segment of that, generally.
We had another conversation about optimizations you could make, about an ending piece of the video. If someone looks left or right quickly, it’s difficult to then have to switch out to those and maybe even stream the middle of it at a higher quality and then kind of radiate out in lower qualities.
But I think, realistically, we’ve got to be talking about 4K video to get a good 3D experience. Is that going to be a blocker coming up?
Reinhard: Definitely. As you mentioned,
you have to go at least for a 4K video frame to have an accepted quality and something which is comparable to what people are used to today.
That’s hard to deliver, especially if your connection is not good enough.
The first major step was to combine DVR and the 360 degree video topic with adaptive streaming, which at least provides you with a smooth video consumption, also for low-bandwidth channels.
But as you mentioned, there are a lot of things which have to be addressed in the context of efficiency. We are streaming the entire frame and just making use of the very small viewport, which I would consider as a huge loss of bandwidth.
There are topics coming up in the future we want to address, and I think by mid to late 2016 we will see a lot of this going on in the context of efficiency in the VR entry of 360 degree space.
Matt: We touched on light-field cameras earlier, and I think we even talked a little bit about during the Q&A session of the meetup, but especially for somebody that only vaguely even knows what a light field is.
Steve: The math is amazing.
Matt: How does that play into what we’re talking about right now? Is that only for CG-generated content?
Steve: It might even be worth taking a step back. I know when I first started learning about this side of the space, the difference between 360 video and VR was not very clear. I don’t know, would you be interested in talking to the differences?
Reinhard: There is a difference, but nobody is mentioning that. There was VR, and everyone is good on that, but actually the 360-degree video is something which exists in real life.
That’s the difference to VR. VR is a virtual space which is totally designed by the computer, for instance, and the 360 degree video is an actual recording of the real life.
Steve: One of the ways I think about it is, with 360 video, it’s the big panorama, the 360 panorama. And say I’m watching the movie, and I look down by bending my neck. I’m looking at the horizon, and I bend my neck down to look down. When I look down, I’m still looking at the horizon, because the entire video is actually moving with me there.
So that’s not right, and with VR it’s actually where, if I move my head side to side, up and down, it’s more as if I’m actually in the space itself. And you can experience that.
Reinhard: That’s right, and it’s a whole different experience, I would say. It feels more like you’re really in there.
Steve: I guess I’m probably wrong about looking down on a 360 video.
Matt: You would still look down. If you tilted your head to the right, you’re still in a fixed point.
Phil: The head is kind of fixed and it wobbles around on its own axis. You’re not going to look differently on an object, you can’t look behind an object or look to the side of an object.
Reinhard: You only have the viewport of the camera which was recording the scene. You can’t change your position in the video, you can only move around your head. But the position will stay fixed.
Matt: Somebody explained to me just a little bit about how, say you had a recording, you spent the metric crap ton of money you would need for 360 Lytro recording, you get a little bit of perspective shift. How exactly does that work?
Reinhard: You might have to ask Lytro for that.
Phil: We really should have invited Lytro along for this thing.
Steve: I think the camera they were talking about at NAB, it’s like a one meter, by one meter sphere, and it’s meant for actual VR recording. So when we’re talking about 360 video, like you were saying, your head is in one place and you can’t actually move with in space.
But their camera takes in all of the light-field information and within that one meter by one meter space, so you at least have that amount of room that you can move around, I didn’t try it out myself, but I can imagine that there is a big difference there between 360 video and actually being able to feel like you’re really there in the space. That would be really interesting to try out.
Phil: I think what Lytro are going at right now is allowing decisions later in the production chain. Lytro is objective as far as what they talked about at NAB, to make decisions later in the production chain for the editor to say, “Actually I didn’t really want the camera there. I wanted it six inches to the left or six inches to the right. And I would’ve got it better angle on what I saw.”
I think they very much focus in that area right now, rather than being in a space where you can replicate being in a VR or 360 headset. I think that’s way down the chain, way, way down the chain. Like I said,
with MPEG and others talking about compression standards for stuff, it’s going to be a long time before we can realistically see a place where you can stream that information and be able to move your head.
Matt: Everything I heard about in terms of bandwidth, delivery, three gigs of frame or something crazy like that for some of the Lytro content, call me crazy, but I don’t know if that’s necessarily tenable.
Phil: “Storage is getting cheaper,” that’s what we always say.
Matt: Considering I can’t get Google Hangout to work properly for 30 minutes at a time on a 100 mg connection, I’m skeptical.
Phil: That’s all that rubbish VP9 encoding.
Matt: But I am curious, this has kind of been a constant thing, and we touched on this really briefly before. But being able to efficiently deliver this stuff, and you talked about wasted viewport, since you’re only using this tiny, tiny section of the video that you’re actually looking at in any given time.
You mentioned this at the meetup, your plans, but why don’t you talk to us a little bit about the winds that you see in terms of efficiency and delivery, and how does your Bitmovin forte of adaptive bit-rate and segmented streaming fit into that?
Reinhard: One obvious fit is, as you mentioned, the adaption to the bandwidth in context of different quality layers. That’s what we already do, and another obvious thing is to have a better codec which we already talked about. So having HEVC or VP9 would help a lot actually.
Then it comes to the point where you want to deliver only that what’s really used by the viewer. Imagine the entire equirectangular video, the flat video of the entire sphere. The viewport is only a small fraction out of that, and if we just would deliver this small fraction, it would be huge in bandwidth savings.
But once the user moves left or right, then it would be black, which is not the best experience. It would also be necessary to deliver parts of the frame on the very next and on the very right to the user, maybe in a lower quality.But if he or she turns, at least there is a picture. And then adopt to the movement of the user as well.
Phil: I guess that comes down to how good the player is, how quickly it can adapt and how quickly you can jump up bit-rates. That has interesting knock-on effects on your encoding, traditionally rendition switch happens on an open tool, to be able to rendition-switch quickly to someone turning their heads actually has knock-on effects right down the encoding chain, right?
Reinhard: Yeah, that’s right. Definitely the majority of the work has to be done on the player side,and it will be more complex. We are not there yet. I think nobody is, but that’s definitely the way to go.
Steve: Does 360 video have the same framerate issue as VR video in the sense, as I’ve heard it, VR video requires at least a 120 frames per second to make sure people aren’t getting sick as they turn their heads quickly.
And the frame needs to catch up, that’s what I heard, is essentially the minimum to prevent sickness, right? Does that go for 360 video, too?
Reinhard: I’ve already heard that for 360 degree video, I think if you go for 60 frames per second, it’s totally fine. At least I haven’t been sick for that reason.
Steve: When you’re taking the picture, you said equirectangular. When you’re flatting it like that, do you lose any of the detail around the edges of the flat plane video?
Reinhard: It’s not that you lose information. It’s more that there is redundant information on the top and on the lower part of the frame. So as a landscape, where you see the north and south pole much bigger than it really is, you have some kind of information you wouldn’t need.
So also the equirectangular approach, although it is quite an easy and often-used approach, it’s not ideal in the context of efficiency.
Phil: We talked at the meetup about DRM and DRM challenges when it comes to 360 VR content. Can you talk a bit more about that and why that is so much of a challenge right now?
Reinhard: We also talked about in term of the demonstration at NAB with DRM protected content for 360 degree video, because it’s logically the next step.
Studios will want to protect their content once they are in the phase where they can make money out of it, and actually to achieve that with our solution, you would have to break the DRM.
Because what we do right now and what most of the companies are doing is that we take the information of the decoded frame and render it on top of the actual video player using WebGL. But that requires that you get that decoded information, and DRM is exactly in place to prevent that.
We have to come up with new approaches for that and also to align the industry and talk to browser vendors on what we can do and how we can establish the more or less standardized way to stream 360 degree videos and VR content with DRM protection.
Phil: I think it stands like a great topic for funds, right?
Steve: Actually, yeah. Definitely.
Phil: That is great breakout-group stuff.
Reinhard: We are very happy to cooperate, and of course we cannot do that on our own. To get some partners aligned, that would be great.
Matt: October 12: put it in your calendar.
Phil: And I think there is a great conference happening afterwards. I don’t know when that is.
Matt: Again, October 13, Demuxed 2016 hosted by Ellation HQ, it will be near the Westfield Mall. I always forget what the name of the mall is, but yeah, Westfield Mall, basically.
Steve: In San Francisco.
Matt: In San Francisco, that’s probably a key note to make. Just go to any Westfield Mall that’s near you, ask for Demuxed, they’ll know what you’re talking about.
Phil: If you’re doing good, one will actually just give you a 360 headset and you can join from there.
Matt: And you can go to elevator four and three quarters.
Phil: Elevator four and three quarters?
Matt: It’s a Harry Potter joke.
Phil: I got it, that’s a station platform.
Matt: Yeah, who has trains anymore, anyway?
Reinhard: Just because your trains don’t work in this country.
Matt: OK, I think we’re coming up on our time here. But I just wanted to thank Reinhard for coming out. How much longer are you in San Francisco for?
Reinhard: Thanks very much for the invitation. I will leave on Friday, actually, so only two more days left. I’m glad it worked out.
Matt: Great, I’m so glad we could have you by while you’re in town. Thanks again for speaking at the conference, at the meetup, rather. Who knows about the conference? We’ll see.
Phil: Put your submission papers in, guys.
Matt: By the time we publish this, that should be online. If you’re interested, we should have the meetup talk from Reinhard at SF Video up on YouTube here within the next, I’m going to say week or so. Keep an eye out, and hope to see everybody at Demuxed 2016.