# data[].attributes.attachment_link.readable_text
It seems like every generation has some sort of technology that defines their era, but if you're a millennial, it feels like we've been a part of such a huge tech boom that we've been defined and redefined half a dozen times. From the internet to social media to streaming and now to AI, things keep changing at an increasingly rapid pace.

During this whole time, we've seen filmmakers tackle stories involved in the present day, and some tell stories in the past that seem to echo now.

Christopher Nolan has been amidst all of these movements, from his spyware in The Dark Knightthat seemed to harken to the Patriot Act to the environmental crisis in Interstellar that delivered a warning about the future of our planet, he's someone with his finger on the pulse.

Now, his new movie, Oppenheimer, takes us back to the 1940s to look at the invention of the atomic bomb and the fallout from its creation.

What Does Oppenheimer Have to Say About the World Today?

For Nolan, it's a movie about a creation that changes the course of human history that then haunted the rest of the years that followed with the Cold War and the increasing power humanity had to destroy itself.

In a recent interview with Wired magazine, Nolan talked about these implications. When the interviewer brought up AI, he balked at its likeness to the atomic bomb but did go into how it is changing the world.

At first, Nolan stated, "I think the relationship is an interesting one. It's not the same. But it's the best analogy-which is why I used it in -for the dangers of unthinkingly unleashing a new technology on the world. It's a cautionary tale. There are lessons to be learned from it. Having said that, I do believe the atomic bomb is in a class of its own as far as technologies that have changed and endangered the world."
Credit: Warner Brothers
But he also was willing to dissect the worry around AI and the public perception of the tool. Especially as people worry they'll be out of jobs because of the technology.

Nolan said, "That's part of the problem. Everybody has a very-call it a partisan point of view. The issue with AI, to me, is a very simple one. It's like the term algorithm. We watch companies use algorithms, and now AI, as a means of evading responsibility for their actions."

When prompted to expand on this, Nolan continued, "If we endorse the view that AI is all-powerful, we are endorsing the view that it can alleviate people of responsibility for their actions-militarily, socioeconomically, whatever. The biggest danger of AI is that we attribute these godlike characteristics to it and therefore let ourselves off the hook. I don't know what the mythological underpinnings of this are, but, throughout history, there's this tendency of human beings to create false idols, to mold something in our own image and then say we've got godlike powers because we did that."

So, what, if anything, could AI be used for in filmmaking?

Nolan had an answer for that.

He said, "... The whole machine learning as applied to deepfake technology, that's an extraordinary step forward in visual effects and in what you could do with audio. There will be wonderful things that will come out, longer term, in terms of environments, in terms of building a doorway or a window, in terms of pooling the massive data of what things look like, and how light reacts to materials. Those things are going to be enormously powerful tools."
Credit: Universal Pictures
When asked if he would use AI personally, he gave a thoughtful answer.

"I'm, you know, very much the old analog fusty filmmaker. I shoot on film," Nolan said. "And I try to give the actors a complete reality around it. My position on technology as far as it relates to my work is that I want to use technology for what it's best for. Like if we do a stunt, a hazardous stunt. You could do it with much more visible wires, and then you just paint out the wires. Things like that."

Nolan went on to explain even further, going into the idea of using AI for visual effects. He said, "It's not starting from nothing. It's starting from a much more detailed and data-driven idea. It might finally break the barrier between animation and photography. Because it's a hybrid. If you tell an artist to, say, draw a picture of an astronaut, they're inventing from memory or looking at references. With AI, it's a different approach, where you're actually using the entire history of imagery."

I thought these were very thoughtful answers for a tool that feels like it is still in its infancy, but also for a filmmaker so intent on keeping cinema within our society. The truth is, we don't have a lot of answers about AI because it is so new. That's why it's so important to protect human jobs right now, and then walk the fine line of learning how we can use AI to our advantage.

What did you think of Nolan's responses? Let me know in the comments.

Source: Wired

# data[].attributes.attachment_link.title
Christopher Nolan Opens Up About AI and 'Oppenheimer'

# data[].attributes.attachment_link.description
What does Nolan see happening with AI and is he afraid of it?

# data[].attributes.attachment_link.readable_text
Selected Works Managing Director, Sean Costelloe explains how working with Escape Technology's Sherpa supported Selected Work's launch and organic growth Sponsored by Escape Technology
"When we launched Selected Works in the Summer of 2021, there were just the three of us, having come from a company with over 400 people. We wanted to operate on a much leaner, talent-driven basis to take advantage of the post- pandemic world; to work either remotely or in our offices, while still ensuring that our clients get that tactile, artist-led experience.

"We wanted to do things differently and build a company relevant for the times.

"Our intention was to set up a business that was truly independent - where we could chart our own course - without any outside investment.

"We looked at various options. On- prem storage and physical workstations come with what would have been a crippling upfront cost and you need to additionally think about ongoing physical infrastructure like power and air conditioning.

"We looked at hybrid scenarios and we looked at hosting all our storage, workstations and applications entirely in the cloud with a lighter footprint. "We discussed our ambitions with the team at Escape and how as a new business we wanted to work on an OpEx rather than CapEx basis.

"Locating our facility in the public cloud was becoming the increasingly obvious path to follow and Escape gave us the confidence to go forward.

"Escape's Sherpa platform offered us a truly virtualised facility with storage, workstations and applications all hosted securely within a public cloud with a secure pipeline, all managed by Escape. Better still, Sherpa is a flexible solution and would allow us to scale based on need.
Nike 'Sam Kerr - Flip the Game', Dir. Alaska, Prod Co: Iconoclast, Agency: Wieden & Kennedy Portland Flexible
"We were up and running with Sherpa in less than an hour and we've built organically from there.

"The most exciting aspect of working on the Sherpa platform is the flexibility it offers us to add what resources and applications we want, when we want them and to present them to our artists wherever they are in the world. We can scale dynamically, at the flick of a switch. "How we use our own space is entirely flexible. Our artists work off thin clients with a monitor, tablet and pen or a keyboard and mouse without any physical machine. We can spin up whatever software or machine we need, whether Flame, Houdini, Nuke, Maya...

"With Sherpa, we always have access to the latest versions and the most powerful machines. We can leverage additional cloud computing resources when we need to for faster rendering. The Sherpa platform even automatically switches off the additional processing resource when we're done to avoid overcharging.

"We now have about 40 artists working with us - all working in the cloud - in our offices, around the UK, in Italy, Spain, France, Germany and India and even New Zealand. We have an additional Sherpa studio within a data centre in Singapore to reduce any latency our artists might otherwise experience in Asia.

"Escape takes care of all the hardware and software within the Sherpa studio and always ensures we have access to the latest versions.

"And if we were to ever get less busy, we could shrink and contract. (It's not something we have ever done, but it's nice to have the option.)
Simplicity of Use
"Being in the cloud and using Sherpa's interface is very simple to use. We can dial up additional machines and resources and get additional artists up and running in minutes.

"An artist contacted me to say that their IP address had changed. For security reasons we need to re-whitelist the security address. I could do this on my mobile phone on the train and it took me about a minute. I got the artist back up and working instantly. If I can do it on my phone, then it's got to be simple and user friendly.
Valued Partner
"With Sherpa, Escape has helped us navigate and leverage the incredible power and flexibility you can achieve by working in the cloud. Their long- standing expertise in post and VFX and interpretation of what's on offer from the hardware and software vendors sets them apart. What really impressed me was how quickly we could dial up Sherpa and how intuitive the GUI was and how seamless the pipeline."
Bet365 'Never Ordinary', Dir. Jack Driscoll, Prod Co: Academy, Agency: Drummond Central Meet Sherpa
An easy to manage cloud compute platform designed for VFX workflows brought to you by Escape Technology

Secure, simplified cloud computing infrastructure with transparent cost management tools.

Simple GUI removing the complexity of architecting and managing public cloud infrastructure

Metered costs, fully transparent in real-time

Dial-up, dial down the infrastructure as projects demand, no overspend

Centralised storage in the cloud

Global access to talent

Run your software on studio-grade workstations

Your workflows the way you want

Secure by design

sherpa.escape-technology.com ESCAPE TECHNOLOGY, YOUR SOLUTION PARTNER IN VFX, GAMES AND VIRTUALISATION
We pride ourselves on being the trusted partner and expert technology consultant for the creative industries, delivering workflow solutions in VFX, Visualisation, Education and gaming.

Working with the finest technology vendors and partners in the business, whether you are a boutique studio, gaming start-up or global giant we can help!

We provide comprehensive consulting and support services.

As well as being the leading supplier of hardware and software solutions to the great and the good of the VFX, Visualisation, Architectural, Education and Gaming world we also offer consultancy and comprehensive support packages.
Ladbrokes 'Rocky', Dir. Nicolai Fuglsig, Prod Cop: MJZ, Agency: Neverland ABOUT SELECTED WORKS
Selected Works is a talent led, client focussed, award-winning VFX company based in London. Passionate about our craft and proud to be specialists.

We focus our expertise exclusively on our partners, using our skills, our knowledge, and enthusiasm, to create outstanding work. Dedicated and personalised teams tailored to each project's needs.

The team has vast experience, having run the biggest creative VFX studios in the UK and US. With a long track record of producing acclaimed visual effects, we love what we do and we have fun doing it.

Founded in June 2021, Selected Works utilises the very latest in technology enabling infinite scalability and creative freedom.

To date we have won numerous awards, including Cannes Lions, British Arrows, AICP Show and VES for our VFX.
Main image: Samsung 'Galaxy Book 3 Ultra', Dir. Adam Berg, Prod Co: Smuggler, Agency BBH USA
Share this story

Share Televisual stories within your social media posts.
Be inclusive: Televisual.com is open access without the need to register.
Anyone and everyone can access this post with minimum fuss.

# data[].attributes.attachment_link.title
Selected Works, a Truly Virtualised Facility - Televisual

# data[].attributes.attachment_link.description
Selected Works Managing Director, Sean Costelloe explains how working with Escape Technology's Sherpa supported Selected Work's launch and organic growth

# data[].attributes.attachment_link.title
The 7 Best AI and Neural Engine Features in DaVinci Resolve Studio 18.6

# data[].attributes.attachment_link.description
From face recognition to object removal, here's everything you need to know about the AI and neural engine features in the latest version of DaVinci Resolve.

# data[].attributes.attachment_link.readable_text
For anyone who's been to Las Vegas in the past few months, or simply been online to see images and videos shared across social media, can tell you-the new Sphere arena is a sight to behold. Perhaps now fittingly referred to as one of the newest wonders of the world, the giant state-of-the-art LED screen features a 160,000 square-foot display that features a resolution over 80 times that of a normal high-definition television.

But, as Darren Aronofsky recently discovered when he had the chance to create his new movie Postcard From Earth which is now screening at the Sphere, shooting footage for this insane new format is quite challenging, to say the least.

Let's take a look at how Aronofsky was able to use an 18K camera, which reportedly takes up to 12 people to operate, to shoot his film for this gigantic format.

Shooting Postcard From Earth

In what sounds like a very loose 50-minute film about the history of life on Earth, Aronofsky's Postcard From Earth short film certainly promises to be every bit as ambitious as its technical challenges. Following the birth of human civilizations and exploring the diverse wonders of the world, Arronofsky's Postcard even ventures to glimpse into the future when humanity departs Earth to settle other worlds.

But, back here on Earth and in the here and now, this new film from Aronofsky feels about as futuristic as they come. There are tons of technical challenges that come from creating content for this giant screen format. Not only does the screen reach up to 366 feet tall (which is between 30 to 35 stories depending on how you measure it), but it is also a curving 270-degree screen that covers the size of about four football fields.

On top of that, Aronofsky needed to shoot for an 18K resolution at an incredibly fast 60 frames per second to give audiences in the sphere the best experience possible.

To do that, Aronofsky needed one hell of a camera.

The camera used for Postcards From Earth

Credit: Sphere Studios

The Big Sky 18K Camera

Dubbed the "Big Sky", the camera Aronofsky chose to go with featured a 3160-megapixel HDR image sensor which was three by three inches in size. When compared to your normal 4K camera, this represents a 40x resolution increase. The Big Sky is also reportedly capable of capturing 10-bit footage at up to 120 frames per second in the 18K square format, or 60p at 12-bit.

The Big Sky camera also features an "18K x 18K" (18K Square Format) custom image sensor which, when combined with the Big Sky's single-lens system, can reach the optical requirements for matching the Sphere's 16K x 16K edge-to-edge immersive display plane.

For this project, PetaPixel reports that two primary lenses were designed and used including a 150-degree field of view lens which would be true to the view of the sphere, and a 165-degree field of view lens for shooting when the camera was in rapid motion on a helicopter.

Operating the Big Sky Camera

What's really fascinating though about the Big Sky camera and its 18K resolution recording might simply be how hard it was to use. In a recent interview on Late Night with Seth Meyers, Aronofsky revealed that it took up to a dozen crew members to simply operate the camera and that numerous issues had to be solved when simply trying to move it or keep it from overheating.

"It was great. It was what we were hoping for. So, no one had ever seen it, so we didn't really know what they would respond. And then this moment happened. I actually didn't know that jaw-dropping was a real thing. You hear that word, but it's fun to look around, and people are just like, their jaws are literally dropped, and they're all pointing at different places on the screen. So it's a joy to have audiences go along for that trip." - Aronofsky on Late Night.

Hopefully, the technical challenges didn't appear too daunting to others and the Big Sky might be usable again in the future. Whether Aronofsky decides to make another film for the Sphere might be up to him, but it's exciting to see technology push forward in such bold and fascinating ways.

Related Articles Around the Web

# data[].attributes.attachment_link.title
How Darren Aronofsky Shot His 18K Sphere Film With a 12-Person Camera Crew

# data[].attributes.attachment_link.description
Shooting 18K resolution film at 60 fps is nothing short of a technical marvel. Here's how Aronofsky (and his 12-person camera crew) pulled it off.

# data[].attributes.attachment_link.title
Adobe adds AI-powered features to Elements

# data[].attributes.attachment_link.description
There are now over 20 AI-powered features across Adobe's one-time purchase, consumer-oriented Photoshop Elements and Premiere Elements packages.

# data[].attributes.attachment_link.readable_text
Traditional software is made to appear clever by human programmers. AI apps are much cleverer, and they do it all by themselves. So, can we ever trust them?

Software can be infuriating, challenging to learn, and sometimes unpredictable, buggy and stubborn. You could be forgiven for sometimes thinking that it has a mind of its own.

Despite that, it's pretty safe to say that it doesn't. Unexpected behaviour is more about the observer than the software itself, which is strictly deterministic. Just because it doesn't work the way you think it does doesn't mean it is making things up as it goes along or that it is trying to outsmart you. It is simply a category mistake to think that a traditional software program can "want" to do anything - still less rule the world.

Give a software program the same set of inputs, and you will get the same outputs. That's something that, without realising it, we should be extremely grateful for, especially when it comes to software in aircraft, lifts and medical ventilators. Seemingly random and unexpected behaviour would not be welcome in those examples.

What's in the box?

But AI is different. You can take a conventional software program and unpack it, stepping through each piece of code and figuring out how it works, line by line, decision by decision. AI doesn't have that level of transparency. You could say that AI is the definitive black box. And that's a problem if you want to put AI in charge of your nuclear power station, or anything, for that matter, where human lives are concerned.

Take autonomous cars, for example. Tesla's latest self-driving software is entirely based on a massive resource of neural nets, learning constantly from millions of drivers' inputs. In some ways, it's brilliant. It's only a matter of time before it gets provably better than any human driver. If you're going to say that you can't trust AI because you don't know intimately how it works, then you also have to say that about human airline pilots and neurosurgeons. At some point, you have to say that in order to get things done, you just have to trust the agent that's doing the job.

Maybe the problem with AI is that it hasn't earned our trust yet. Human experts spend years perfecting their knowledge with education and experience.

When we meet a doctor, we don't know anything about their path to qualification, but the fact that they're called "doctor", that they work in a hospital or clinic, and that they have all the trappings of medical doctor-hood, adds up to a powerful reason to trust them. Layered on top of this is their manner. Medical professionals sound confident and reassuring. They tick all the boxes needed for us to trust them implicitly. We don't demand to see our doctors' qualifications before they write a prescription, and most of the time, that's perfectly OK.

But that type of definition of trust doesn't seem adequate with AI. Maybe it's because it doesn't have any sort of "embodiment" that it seems so strange to us. How can we trust it from its body language when it doesn't have a body? A smiley face on a goofy low-resolution screen instead of an actual face doesn't cut it for most of us. We know a "Speak and Spell" is not really speaking - and if we don't, just try holding a conversation with this 1980s relic.

Perhaps what's worse is that Large Language Models (LLMs) are rather good at sounding like they know things. That's because they're trained on a wide range of knowledge, as well as the language that humans would typically employ to discuss that information. They almost have no alternative but to sound confident, even if they're not.

Removal of doubt

These are likely to be transitional issues as the technology improves, but there will always be doubts until an AI model becomes so provably adept at a given set of tasks that it is literally beyond criticism.

But will that ever happen? It might be for some tasks like cooking and ironing. But what about flying a plane with five hundred people on board? In principle, if an AI model works well and does so repeatedly, then we should no more doubt its capabilities than we would a human. But we still don't know what's going on inside. The AI remains a black box - in the sense that if it makes a mistake - we won't know where to look to fix it.

I think it's likely that we will be able to build self-reporting into AI. In some ways, it's a necessary step. It may slow things down a bit (or a lot), but the benefits would far outweigh that, and with progress already so fast, slightly slower rapid progress is not going to hurt anyone.

Phase changes

But there are other ways to impose control over AI-driven activity. Professor Max Tegmark suggested recently that, as it develops. AI goes through a "phase change" analogous to the transition from a gas to a liquid or a liquid to a solid. In each case, the atoms or molecules in a substance go through a massive simplification of their motion. Particles of matter are clearly far less mobile when in solid form than in a liquid or a gas.

Tegmark makes an analogy with a moon shot. In the very early days, a rocket company might do things by trial and error. Essentially, they would fire a rocket at the moon, look at the results, and then adjust to correct the trajectory and try again. They would have got there in the end but would have lost a lot of rockets and several peoples' fortunes.

But it never happened like that. Because most of that trial and error testing became unnecessary when someone worked out the equations concerning gravity, mass and velocity. Knowing these, all you needed to do was populate the equations with the figures for a given rocket - including velocity, direction, position of the moon, etc, and the result would come out correct. The remarkable thing is that once the dynamics of mass and gravity were understood, the whole exercise could be computed on a processor that would be considered outdated in a washing machine.

Reducing dimensions

One way to look at this is to talk about dimensions. Stay with me - I'm not going to invoke 13-dimensional String Theory. It's really quite simple.

When an AI model enters training, it is genuinely clueless. It's like a black box covered in controls (like a synthesiser) with no labels. Tweak one knob - or a hundred - and you'll probably get a meaningless or useless result. But as you do this repeatedly, you'll find settings that do make a difference. So you make a note and put a label on the machine to say, "If you see this pattern of behaviour, it's statistically likely that this (an image or shape or something) is what we're looking at". Do that often, and do it enough, and you'll start to build a recognition machine.

So, eventually, your AI model will "learn" about putting rockets accurately on the moon. But it might take a supercomputer to do it. That's because it doesn't really "know" what it's doing. It's just based on a few good results pointing it in the right direction.

But what if another layer in your AI model could detect and isolate mathematical trends? It's just possible that it would pick up the set of trends that come just before a formula. These AI-generated equations might not look exactly like ours, but they will be functionally the same. In a sense, what's happened is that the "dimensionality" of the problem has been reduced, ignoring potentially thousands of spurious or coincidental data points that can safely be put in the bin because they don't affect anything.

This rapid and drastic reduction in dimensionality is what allows us to let AI find a result and then put the essence of the process onto a much smaller computer than was needed in the early stages. It also means we're much closer to having a verifiable AI. This is what Teghmark means by "Phase change". It's a loose analogy but a helpful one.

It might mean that we can be much closer to a testable, verifiable, and deterministic AI future, where we can see into the black box, see what's coming up next, and take corrective measures if we have to.

# data[].attributes.attachment_link.description
Traditional software is made to appear clever by human programmers. AI apps are much cleverer, and they do it all by themselves. So, can we ever trust them?

# data[].attributes.attachment_link.readable_text
We have been talking a lot about digital editing actors' performances. From deepfaking actors like Bruce Willis to virtually dubbing out curse words or translating films to another language, AI has become Hollywood's favorite friend.

While there are good reasons to use deepfake edit performances to save your film from going way over budget, some actors like Keanu Reeves have a problem with this technology.

During a recent interview with Wired for John Wick: Chapter 4, Reeves spoke against deepfake technology in Hollywood, saying "When you give a performance in a film, you know you're going to be edited, but you're participating in that. If you go into deepfake land, it has none of your points of view. That's scary. It's going to be interesting to see how humans deal with these technologies. They're having such cultural, sociological impacts, and the species is being studied. There's so much 'data' on behaviors now."
Credit: Lionsgate
Reeves made it clear that he isn't against the necessary edits that need to be made to an actor's performance to make them look good while also choosing the footage that best serves the story, but going beyond those edits takes the "agency" away from the actors.

In his fight against the dangers of deepfake technology, Reeves confirmed that he has a clause in each of his movie contracts that prevent studios from digitally manipulating his performances.

"People are growing up with these tools: We're listening to music already that's made by AI in the style of Nirvana, there's NFT digital art," Reeves said. "It's cool, like, 'Look what the cute machines can make!' But there's a corporatocracy behind it that's looking to control those things. Culturally, socially, we're gonna be confronted by the value of real, or the non-value. And then what's going to be pushed on us? What's going to be presented to us?"

"It's this sensorium. It's spectacle. And it's a system of control and manipulation," Reeves continued. "We're on our knees looking at cave walls and seeing the projections, and we're not having the chance to look behind us."

How we use AI in film and TV should be a tool that helps us enhance the stories we are telling, but Hollywood isn't doing that. Instead, Hollywood is seeing AI as a solution to a non-existent problem. They are so committed to nostalgia, IP recognition, and something familiar that they are willing to take away anything real and meaningful, and strange in favor of something profitable.
Credit: Lionsgate
As filmmakers and film lovers, we have to go against Hollywood's mindset of profit over art. Sure, making a profit is nice, but it shouldn't always be the goal behind your project. That way of thinking will ruin the culture and the medium. We must be brave enough to support and make projects that we want to see.

I should clarify that I am not against AI technology helping with film productions. AI tools exist for a reason and should help us make an easier workflow. What is dangerous is building a reliance on these tools. We need human minds and hearts behind these projects. We need human stories that speak to our existence rather than turning moon-eyed for spectacle filmmaking.

Let us know your thoughts in the comments below!

Source: Wired

# data[].attributes.attachment_link.title
Keanu Reeves Warns Us About the Dangers of AI Editing Actors

# data[].attributes.attachment_link.description
Is there a limit to using AI tools to enhance an actor's or film's performance? Keanu Reeves says yes, and this is why.

# data[].attributes.attachment_caption
We still need to be vigilant about AI in writers' rooms

# data[].attributes.attachment_link.readable_text
Generative AI Panel

We had a busy SIGGRAPH - between DigiPro and SIGGRAPH - there is a lot of content we were a part of and that we will be rolling out over the next two weeks.

As part of SIGGRAPH, Mike was invited to be a part of a panel on Reshaping the Film & Television Industry with Generative AI. This was one of three speaking engagements at SIGGRAPH on AI, the first being a workshop (not recorded) on Trust in Al. We have a special fxpodcast discussing that topic coming up in Sept. (more soon).

The Generative AI panel was hosted by Rick Champagne, Global Media & Entertainment Industry Strategy and Marketing, NVIDIA, and joining fxguide's Mike Seymour was Sanja Fidler, VP, AI Research, NVIDIA and Tye Sheridan, Co-Founder & President, Wonder Dynamics.

It was great to discuss the issues surrounding AI and our thanks to NVIDIA for the opportunity.
Click here to go the NVIDIA site: tinyurl.com/NVIDIA23MIKE1
Vision series

Exploring the Transformative Power of AI in Media & Entertainment Panel was also held on Day 2 of SIGGRAPH with Fxguide on the panel. This panel session from Autodesk is not yet online- (we will post it when it is). This panel talked about the possibilities and impact of machine learning on the media and entertainment industry.

On the panel were Phil Barrenger, Head of Operations at Rising Sun Pictures, Nikola Todorovic, Co-Founder and CEO of Wonder Dynamics, Rachael Appleton, Director of Media & Entertainment Business Strategy at Autodesk, and fxguide's Mike Seymour.

Office Hours

Mike also appeared on Day 1 of the Office Hours Live stream - more by accident than design.

(Mike appears at around 6 mins in the video)

You can watch a full replay of the live Office Hours broadcast here

# data[].attributes.attachment_link.title
AI Panels and Talks: Videos from SIGGRAPH with fxguide - fxguide

# data[].attributes.attachment_link.description
AI panels at SIGGRAPH, including Reshaping the Film & Television Industry with Generative AI + Crashing a Live Broadcast.

# data[].attributes.attachment_link.readable_text
According to reports shared on Canon Rumors, the upcoming Canon EOS R5 Mark II is set to double down on 8K video recording, but this time with a "massive" frame rate bump.

Now, there's a lot to unpack here, but the reports seem to indicate that the R5 Mark II is indeed going to come out here soon-most likely in early 2024-and will return as the flagship camera to Canon's mirrorless line.

Let's explore everything we know so far about the Canon R5 Mark II and what this "massive" frame rate bump could mean both for its 8K video and for its potential high-frame rate capabilities. Plus, let's see what filmmakers and video pros like yourself might be able to do with this new camera once it is officially announced.

Canon R5 Mark II 8K Rumors

Really, when the Canon R5 Mark II is officially announced, it should come as no surprise to anyone that the Mark II will keep the 8K tradition established by its predecessor going. The 8K video recording was perhaps the R5's most important-and buzziest-selling point as the market really hadn't seen recording specs that high for a camera at that initial price point.

Even today, with a 45MP full-frame CMOS sensor capable of 8K Raw video at 30fps, the current R5 is still a worthy flagship offering. However, as is the case with any new camera on the market these days, consumers are always looking for that next bigger and better thing, so it would make sense for Canon to double down on the 8K and even kick things up a notch.

Massive Framerate Bumps

That's where the second part to these Canon rumors come in as the report shares details about a "massive" framerate bump coming to the R5 Mark II.

On one hand, we're led to speculate that this framerate bump will have to do with the highest-end 8K video that the camera will perhaps be capable of, in which case we could expect to possibly see 8K video recording at up to 60 fps.

The way the rumors are worded though it sounds also like the high-frame rate capabilities of the R5 Mark II in general could be boosted as well. The current R5 is "only" capable of 4K at 120fps with 10-bit internal video. (Honestly, still great specs not worth scoffing over.)

However, an upgraded R5 Mark II could most likely boost those specs as well, possibly to the tune of 4K video at up to 240 fps with the same, or better, internal recording capabilities.

What to expect with the Canon R5 Mark II

Credit: Canon

What to Expect Next

Regardless of exactly how accurate these reports end up being, we do feel pretty sure that the new Canon R5 Mark II is indeed coming and it will most likely push its 8K video capabilities to new heights.

The latest reports from the same rumors still put the R5 Mark II on an expected announcement roadmap for early 2024, possibly as soon as CP+ in February.

So, until then, we'll keep tabs on this story and let you know as soon as we hear anything official from Canon.

# data[].attributes.attachment_link.title
Upcoming Canon R5 Mark II Rumored to Get 8K With "Massive" Frame Rate Bump

# data[].attributes.attachment_link.description
Are you getting excited for this next version of the Canon EOS R5 Mark II yet?

# data[].attributes.attachment_link.readable_text
A couple of weeks ago I was watching Pulp Fiction (again) and I made a mental note about an aspect of that film I always found interesting. The scene where Bruce Willis' character, Butch the Boxer, is in a cab and it's obvious that the background of the city passing by is fake. I always thought it was green screen. But, as I learned from today's episode of Film Riot, it was actually a projected image. In this case, a rear projected image of passing traffic (there's another similar scene in the film with John Travolta's Vincent Vega driving while high).

I'm used to seeing these kinds of scenes in older movies like the car chase scene in Dr. No.

I also remember starring in a "movie" myself when visiting Universal Studios Hollywood Tour and being the lucky kid chosen to help demonstrate this "groundbreaking" effect. (I think they used rear projection. Or was it a green screen example. I forget. It was a long time ago).

As Ryan points out in the episode, the history of the effect dates back to the early 20th century, with films like the 1930's film and the 1920 film Just Imagine.

But you might be surprised at how many contemporary productions you've seen that used front or rear projection in lieu of green screen. Movies like Terminator 2, Oblivion, and Aliens, to modern TV shows like Disney+'s The Mandalorian.

The Benefits of Projection

You may ask yourself, "With the advancements like green screen and blue screen, why would anyone want to use front or rear projection?" There are a few practical benefits. First and foremost, there's reduced post-production time. You don't have to key out anything and then key in a new background. You also don't have to deal with the pain of keying out hair.

Second, having a practical "effect" like front or rear projection on set, gives you more flexibility in lighting the subject and camera movement.

This is all predicated on you having a high quality projector that can deliver imagery that looks as good as a green screen would. For this episode, Ryan was able to try out the LG Cinebeam. It boasts -4K resolution, a 2M:1 contrast ratio, DCI-P3 with 97% color accuracy, and HDR10 level brightness.

Pros and Cons of Front Projection vs Rear Projection

As the name suggests, rear projection is when a projector casts an image from behind a translucent screen. Front projection places a projector in front of a screen.

The primary benefit of front projection vs. rear projection is that it allows for a more saturated and sharp image. The downside is the throw length. This is the length from the projector to the surface on which you're projection. The light that is cast from the projector could bounce and hit the talent. One of the benefits of the LG Cinebeam is that is has 2" throw length that is able to cast a 90" image. You can bring it as far back as 7.2" for a 120" screen.

At the end of the day, front projection (or rear projection) still remain viable options for film productions looking to save post-production hassle; or if you want to communicate a particular style like the case of Tarantino. And to prove we put our money where I mouth this, this episode was shot using front projection for the usual Film Riot green screen background.

Enjoy the episode!

# data[].attributes.attachment_link.title
Rear and Front Projection in Film Production - Film Riot

# data[].attributes.attachment_link.description
A short break down of this week's Film Riot episode about front projection and rear projection used on film productions.

# data[].attributes.attachment_link.readable_text
In the fast-moving world of artificial intelligence, it can feel like a full-time job just keeping up with all of the new apps and features. While we're very aware of the scarier elements of what AI could mean for the future of creative industries like filmmaking and video editing, we do feel that it's important to stay up on the latest AI tools and innovations.

One AI video tool that is quickly becoming one of the best for creating cinematic-looking videos is Pika Labs. It's basically a cross between Midjourney and Runway, and, with the right workflow, it can be used to quite easily create some pretty cinematic looking videos.

So, following along with a guide laid out by Curious Refuge, let's explore how you can create cinematic AI videos of your own with Pika Labs.

Bring in a Cinematic Image

So, as mentioned above, for this guide we're going to be following a tutorial laid out by the team at Curious Refuge. Developed as the "world's first online home for AI storytellers," Curious Refuge is the channel behind those popular Star Wars by Wes Anderson trailers that we've covered before.

More notably though, Curious Refuge also offers its own AI filmmaking course which, if this is your interest and cup of tea, you can check out here.

To get started though, all you need to do is create an image. This can be something you create with AI or dream up the good old-fashioned way on your own. For this tutorial, our host recommends using Midjourney as it's still one of the best text-to-image AI tools available.

Open Image in Pika Labs

From there you're going to want to head to Pika Labs, which currently operates as a Discord server much in the same way as Midjourney. So, if you're familiar with pulling AI images from prompts in Midjounrey, you can expect to use Pika Labs much in the same way.

And, if you aren't familiar, it's still pretty easy.

Prompt and Direct

All you're going to need to do is enter a prompt and direct your video scene. To upload your image all you need to do is drag and drop your file into the Discord channel. From there you're going to type in the type of action that you'd like to see.

Caleb from Curious Refuge gives some good insights into what makes a good prompt, but you're free to try out as many different approaches as you'd like in this phase. Pika Labs is surprisingly knowledgeable about filmmaking as you'll be able to enter prompt commands for adding camera movement, zoom or rotation, as well as even changing the frames per second.

From the Curious Refuge tutorial, Caleb recommends using these tags at the end as a way to remove any unwanted qualities: "-neg morphing, erratic fluctuation in motion, noisy, bad quality, distorted, poorly drawn, blurry, grainy, low resolution, oversaturated, lack of detail, inconsistent lighting."

Refresh or Shuffle As Needed

From there all you'll need to do is hit enter and wait (approximately around 30 seconds) for your AI video to render and load. You'll then be able to review your AI clip and decide if it will work, or if you'd like to refresh it (to let the AI run the prompt again as is) or choose to shuffle (which will allow you to tweak your prompt before sending it through again).

This is where you can really explore Pika Labs and its AI powers to learn more about what types of prompts work for you and your desired cinematic video looks.

Remove the Watermark

Now, one downside to using Pika Labs - and many other AI apps for that matter - is that your video is going to have a big watermark on it in the corner. This isn't a big deal if you're just trying this out for the exploration of course. (And, if you do want to be ethical, if you're just sharing with friends it wouldn't hurt to leave this watermark on so you can let others know where it was created.)

However, if you would like to remove the watermark the tutorial recommends using Wondershare PixCut as an easy-to-use app for brushing the watermark off your video. This app isn't free though, and if you know your video editing tools you could also do this in a more advanced app like After Effects as well.

Upgrade the Resolution and Export

Finally, to really give your video an upgrade and boost, our guide recommends taking your video into the Topaz AI app to upgrade its resolution and export as a larger file size. Topaz Labs has a pretty solid upgrade offering that allows users to turn your SD video into a higher-resolution 4K or 8K version.

You can also use a different AI model (like the Proteus AI model) to help smooth it out before exporting it. There are several export setting options as well, but ProRes 422 Standard will probably be the best option.

And from there, you're done. You have a simple, yet fully realized, cinematic AI video which came from nothing but your imagination and the complex algorithms of artificial intelligence.

Related Articles Around the Web

# data[].attributes.attachment_link.title
How To Create Cinematic AI Videos With Pika Labs

# data[].attributes.attachment_link.description
Check out this easy guide to getting started with Pika Labs and using AI to generate cinematic video.

# data[].attributes.attachment_link.title
Adobe previews AI dubbing & generative fill for video

# data[].attributes.attachment_link.description
It is also adding the ability to publish directly to Facebook, YouTube, and TikTok in Premiere Pro

# data[].attributes.attachment_link.readable_text
Peaky Blinders creator and writer Steven Knight CBE will be a big draw for visitors but is just one of the headline guests set to take to the stage at The Media Production & Technology Show (MPTS) on 10-11 May.

Knight, who is currently penning a new Star Wars sequel and a remake of Alfred Hitchcock's classic Vertigo, will share insights into his creative processes,and his passion and plans for bringing more productions into the West Midlands.

Knight joins a host of luminaries from UK film and TV at the country's largest event for the broadcast media, production and technology industry. Speakers include Daniela Neumann, who heads Spun Gold TV, BAFTA winning screenwriter Danny Brocklehurst ( Shameless), and Jo Plaete of Metaphysic, the company behind the Tom Cruise deepfakes. Powerhouse doc filmmakers Tom Porter and Michael Ogden and the creative team behind Oscar winning Best Animated Short, The Boy, The Fox, The Mole and the Horse will also take to the stage.
View the programme
Generative AI and real-time production are two game-changing technologies that the MPTS programme keeps an eye on, along with major issues around tackling diversity, training, sustainability and the ethical use of technology.

Charlotte Wheeler, Event Director said, "MPTS is our chance to press pause, assess the impact and speed of change, and collectively make sure our industry is fighting fit for the future."

There are over 100 presentations, masterclasses, panel discussions and educational sessions across the two days, plus a packed show exhibit which covers all bases. In order to accommodate more exhibitors on one floor, the event has now moved into the Grand Hall of Olympia. The additional space was also needed for all the new exhibitors joining MPTS this year. The seminar theatres have increased in size by a quarter, there's more seating and there's a bigger bar!

The biggest increase in terms of activity is related to the boom in Virtual Production. With studios popping up all over the UK and the world and with technology and techniques advancing every day, the need to learn about the latest developments is pressing. BK Johannessen, Unreal Engine Business Director for Broadcast and Live Events at Epic Games keynotes, while at the expanded Virtual Production Theatre visitors can hear case studies and insights from Pixotope, Absen, Move.ai and disguise and get to grips with demonstrations with exhibitors on the show floor.
Find out more here
The industry is experiencing a talent crisis that could last decades and impact on the UK's ability to maintain its place at the centre of international production. That's why one of the biggest themes at the show is encouraging and sustaining next generational talent.

"We need to make sure our sector is a viable, inclusive and attractive one for young people to want to build a career in," said Wheeler. "Not enough students are choosing engineering and technical roles as a vocation at University but the focus should also be on in-house training. MPTS will explore this by also welcoming exciting new talent on the stages to talk about their experiences."

The best part is that no matter when you register, the show and its stellar conference programme is FREE to attend!
Register now here
Wheeler added, "This is our BIGGEST and most exciting programme yet, and we hope visitors enjoy all that it has to offer."

Share this story

Share Televisual stories within your social media posts.
Be inclusive: Televisual.com is open access without the need to register.
Anyone and everyone can access this post with minimum fuss.

# data[].attributes.attachment_link.title
MPTS grows to reflect strength of UK Creative Screen - Televisual

# data[].attributes.attachment_link.description
Peaky Blinders creator and writer Steven Knight CBE is just one of the headline guests at The Media Production & Technology Show on 10-11 May.

# data[].attributes.body
"So imagine what we could do if AI could help us identify all the shots of the day, or all the shots of Paul, or all the medium shots, or the close-up shots."

"I can spend more time crafting a story and less time searching and sifting through video."

Paul Saccone (Senior Director at Adobe)

# data[].attributes.attachment_link.description
Dave Shapton talks about the future of AI with Paul Saccone, Senior Director at Adobe.

# data[].attributes.attachment_link.title
Deepfaking AA members: inside Daisybeck’s alcoholism doc

# data[].attributes.attachment_link.description
Exec producer Pamela Gordon and BBC religion chief Daisy Scalchi hail technology for giving recovering alcoholics a voice

# data[].attributes.attachment_link.title
IBC2023: Blackmagic launches Cinema Camera 6K

# data[].attributes.attachment_link.description
Dave Shapton talks about the future of AI with Paul Saccone, Senior Director at Adobe.

# data[].attributes.attachment_link.title
The future shape of post-production workflows

# data[].attributes.attachment_link.description
AI occupies a domain that is neither obviously analogue nor digital. Can it ever be the basis for a professional professional video recording - or even a post-production workflow?

# data[].attributes.attachment_link.readable_text
Most of the press coverage of "deepfake" videos has focused on how disturbingly funny they are, the security implications of the videos and the threat they pose to democracy. But there's another elephant in the room.
In this vortex of press coverage, one aspect has been overlooked: the challenge that deepfake poses to filmmakers. If you are a documentary filmmaker and have spent many hours to get a politician on tape, it's worrying that some tech pro can (theoretically) get the same footage without leaving their bedroom.

Maintaining authenticity and trust in your filmmaking is more difficult than ever. But there are some techniques for doing so.

The Rise of Deepfake

As has been well covered, deep fakes were originally used for porn. It's a cliché to say that pornography is the driver of every new advance in filmmaking tech, but in the case of deep fakes this is true.

The problem, in 2019, is that deep fakes are now far more common and are being used for far more than porn. Alongside Hollywood actors, politicians are now a popular target. Political deep fakes have been produced with the explicit intention of slanting elections, both in the US and elsewhere. For audiences not used to fact-checking videos, they can be very effective. That's why several analysts have argued that deep fakes are a threat to democracy.

That may or may not be true. But deep fakes are definitely a threat to documentary film-making.

Most Deepfakes until now focus on humour

The Threat For Filmmakers

There are a number of key threats that deepfake presents for filmmakers. The most obvious is the way the deep fakes undermine the authenticity and trust in films made the 'old fashioned' way. But before I get to that, it's worth thinking about another issue: the legal ramifications of the rise of the technology.

At the moment, the legal framework is a mess when applied to deep fakes. Individuals have no recourse to the law if their image is used to make these videos, but equally, filmmakers have only limited protection if their footage is stolen and re-used. If your footage is stolen, not only are you losing many hours of hard work, you are also contributing to the undermining of your own industry. Before thinking about trust and authenticity issues, therefore, you should make sure you protect your footage by using the most secure cloud storage you can afford. This is especially important today, as even the best cloud storage providers like Dropbox and pCloud have shown their infrastructure can be more difficult for hackers to break into.

The major issues with deep fakes, though, is that they threaten to destroy the value of filmmaking itself. If no-one can trust film, no-one will trust your film. Whilst this is particularly of concern to documentary filmmakers, it also has implications for the entire industry.

The implications in terms of trust in filmmakers work could be grave.

Retaining Authenticity

At a basic level, the way to combat the rise of deepfake seems pretty obvious: stress the authenticity of your films. But creating and retaining authenticity has been a concern for filmmakers since the inception of the media and is easier said than done.

As of 2019, proving that your footage is legitimate is still not that difficult. As WIRED has pointed out, deep fakes are still pretty easy to spot because of the slightly unnatural way in which 'people' act in these videos. As the technology develops, though, it's likely that deep fakes will get harder to spot. And what do we do then?

As a filmmaker, it's critical that you establish the fact that you have genuine access to your sources. This can be done by including footage that is (currently) difficult to fake: shots of interviewees outside of the 'talking head' format, or lighthearted moments that a deepfake author is not likely to include.

You might think that sounds like familiar advice and you'd be right. Most film schools have been saying the same thing for decades. In this sense, then, building trust with your audience today relies on exactly the same techniques that we used last year, last decade and last century.

The Opportunities of Deepfake For Filmmakers

For fiction film-making, the rise of deep fakes is both more and less problematic. The opportunities of deepfake in this type of film-making are various: the ability to use post processing to change scenes at a deep level and (eventually) even the ability to 'design' actors from the ground up.

Because of these opportunities, it's worth pointing out that the rise of deepfake is not completely negative for filmmakers. Whilst it threatens to undermine trust in film, it also offers almost unlimited possibilities for creative filmmaking.

In fact, taking advantage of these new technologies is one of the best ways to understand how they work and how best to differentiate your 'normal' filmmaking practice from them. There are plenty of guides available on how to make a deep fake and the process can be very instructive in terms of helping you to understand your current practice.
Header image: Shutterstock

# data[].attributes.attachment_link.title
Deepfakes could destroy trust in filmmakers. What can we do about it?

# data[].attributes.attachment_link.description
Deepfakes could destroy trust in filmmakers. What can we do about it?

# data[].attributes.attachment_link.readable_text
Cover image via

Why manually transcribe things by hand when you can use machine-learning algorithms to get the job done? Let's explore some options.

In my last video, I tested out a tool that uses AI tech to perform basic object removals. The results were quite impressive.

Today I'm testing another tool from the same company (RunwayML) that's all about creating subtitles, which is a bit more beneficial for the masses. And, just like last time, I'm going to be pushing the tech to the limit to see just how well it performs.

Why Use Subtitles?

While not everyone wants to perform object removals, almost everyone wants to create subtitles for their videos. Don't believe me? Well, almost everyone.

A recent survey of US customers shows that 92% view videos with the sound off on mobile. It makes sense when you think of all those people waiting in a line somewhere, mindlessly scrolling their social feeds.

Imagine the millions of people in bed, ignoring their significant other via a mobile screen. Or, the number of teenagers in a classroom ignoring a teacher, vegging out to a video.

There are plenty of scenarios where having the sound on is inappropriate, and subtitles are here to help. The survey also shows that 80% of consumers are more likely to watch a video all the way through when subtitles are available. Those are pretty huge numbers.

Text can help grab a viewer's attention and keep them interested. It can add context and help get a viewer locked into your message quickly. Most folks might swipe on by without text, never to return.

So, how can you add subtitles to a video?

The Technology is . . . There?

Luckily, technology has finally reached a place where creators are no longer required to pay considerable fees to get a transcription of a video made-or, even worse, attempt to transcribe everything by hand themselves. Good luck with that.

As if that wasn't cool enough, auto-captioning for live streams is currently being rolled out to English channels.

These days, YouTube will automatically caption a video soon after it's uploaded. What's more, this feature is available in over ten languages.

How is this possible? Well, Google uses machine-learning algorithms for their auto-captioning feature. It's still imperfect, however, as Google writes in their documentation:

...the quality of the captions may vary. We encourage creators to add professional captions first. YouTube is constantly improving its speech recognition technology. However, automatic captions might misrepresent the spoken content due to mispronunciations, accents, dialects, or background noise. You should always review automatic captions and edit any parts that haven't been properly transcribed.
- Google

Now that you've learned a bit about subtitles, let's see how to create some.

Before people start sending angry letters, let me get one thing straight: captions and subtitles are not the same thing.

Captions are specifically designed for the hearing impaired. They transcribe what is on screen, even describing sounds and various visual occurrences in the text.

Subtitles translate the spoken word. Captions give much more context to a scene than subtitles do.

The Burn-In vs. SRT Files

Before actually creating the subtitles, you'll first want to think about the viewer's experience.

How will they see the subtitles? Will they have to click on a button to activate them? Can you force them to show up on-screen each time for every viewer?

This can be challenging if you share a video across multiple social media platforms.

While there are various subtitle formats, the main distinction for the layman to understand is how you can "burn" some subtitles into your video. This means that the text elements will always be on screen. It's permanent. This differs from the subtitles that you can toggle on and off, as you see available on YouTube.

With such a vast number of people watching videos sans sound, creators will benefit from the burn-in method. Let's jump into Runway and see how the burn-in process works.

How to Create Subtitles in Runway

Step 1: Upload a Clip

The cool thing about Runway is that it's a browser-based tool, allowing users to create subtitles with nothing other than a basic Internet connection.

Step 2: Generate the Subtitles

Once the clip is inside Runway's video editor, please select it and navigate to the Subtitle tab. Select the flavor of English (or Spanish), and press Start.

Like Google, Runway uses machine-learning algorithms to generate the subtitles automatically. In my experience, the generation process was faster than in real-time.

Step 3: Edit and Refine

Once generated, the subtitles show up on the track above your video layer. They can be edited and moved on the timeline as with any other clip.

To customize the look of the subtitles, select the clip and navigate to the Subtitle tab. Users can change the font, weight, size, fill, and background colors.

Step 4: Export

To burn the subtitles into the video, export the clip. To generate an SRT file, select the drop-down menu to the right of the Subtitle Properties.

Tongue-Twisting the Tech

My first test of Runway's subtitle tool was nearly flawless, with only one word mistakenly being duplicated. Unlike YouTube's auto-captioning feature, Runway adds punctuation, giving the subtitles a clean look. The results were awe-inspiring.

Indeed, I expected the results to be near perfect, as the audio quality from my professional microphone is quite good. There were no real background noises or distortions in the recording. I also didn't use any slang and enunciated clearly in my neutral American accent.

To put the machine-learning tech to task, I decided to record several tongue-twisters, as well as sentences with semi-difficult vocabulary. While the punctuation looks nice on Runway, this was one area where the algorithms couldn't follow along.

In one instance, question marks randomly peppered the sentence. This indeed tripped up Runway's subtitle generator on multiple occasions.

Nevertheless, the tech did surprisingly well, overall.

I'm interested to see how advanced this tech will be in just five years. I imagine the machine-learning algorithms will be able to generate subtitles in real-time and provide instant translation in nearly any language.

That's what I love about technology. It keeps getting better.

A few more fun ones for you:
Cover image via Vector Point Studio.

# data[].attributes.attachment_link.title
Is AI Software Faster (and Better) at Creating Subtitles?

# data[].attributes.attachment_link.description
Why manually transcribe things by hand when you can use machine-learning algorithms to get the job done? Let's explore some options.

# data[].attributes.attachment_link.readable_text
We're in Berlin at the IFA trade show for consumer electronics. This is the place where you can find tumble driers in the same exhibition hall as cutting-edge 8K TVs. But it didn't disappoint. The world has changed with the arrival of Samsung's first range of consumer 8K TVs.
It was a rather unusual experience. You can tell that Samsung is a huge company when you pull up in your Uber at what you think is the exhibition building containing the IFA consumer electronics show, and then you realise that this is just the Samsung exhibit.

While we were still processing that, we went into the hall, only to be confronted by the shiniest, sparkliest array of clothes washing paraphernalia ever seen by humankind. I did wonder if for a minute we'd arrived in the wrong place, but then, over in the distance, we saw a sign that said, matter-of-factly "8K".

This is what brought us to Germany in the first place. Even though Sharp has had a range of 8K TVs for around a year, no-one can diminish the influence and market presence of Samsung in the TV space. Samsung is big enough to have its own gravitational field, and when it moves into a new space, everyone has to take notice of it.

Which is exactly what happened with 8K TV today. For the first time, the world's biggest manufacturer of TVs has a range of consumer 8K sets. This wouldn't have happened if Samsung didn't think that 8K has a future.

Sceptics

Even if you're an arch skeptic about 8K, you'd have to admit these displays were gorgeous - if, most of the time, rather over saturated. It's almost as if the manufacturers think customers won't even see their products unless the pictures are straining at the edge of the colour gamut. (And they might be right: I'm sure they've researched this). But as with most TVs of the last decade, if you turn the saturation down to realistic levels, the pictures look even better.

We weren't able to get the full details about the 8K TVs. This show is not like IBC or NAB. It's hard to get an interview with a product manager unless you're the leader of a G7 country. One thing we do know - and Samsung trailed this at CES, but not with a production-ready TV - is that the sets will use AI to help with the upscaling. It wasn't clear on the booth where this was in use, but I'm told the results are astonishingly good and do a pretty decent job of upscaling 4K and even HD to 8K. So early adopters needn't worry about the lack of 8K material in the early days of this new format.

Nevertheless, we will get some responses from Samsung as soon as we can. Meanwhile, check out my video below from the show floor, where I give my instant impressions of Samsung's 8K offerings. (Oh, and please not that I was wrong when I said that QLED is the technology that upscales 4K to 8K. The sets do do this, but I think QLED is the screen display technology).

There is more 8K at the show. I'm told that LG has an 8K OLED panel, but it's not clear if this is just a technology demonstration of a product that's going to be on sale soon.

# data[].attributes.attachment_link.title
8K is now mainstream. These 8K sets will be in the shops soon

# data[].attributes.attachment_link.description
8K is now mainstream. These 8K sets will be in the shops soon

# data[].attributes.attachment_link.readable_text
New technologies are in development that look to look to find faster and more accurate ways of replacing backgrounds in movies and TV.

Green screen has, of course, been around for a rather long time by now. The blue screen method to produce a travelling matte that predates it stretches back to the post war era, while green screens themselves became popular in the 1970s for the likes of weather forecasts produced by TV networks where the chroma key technique they used was less likely to be tripped up.

Their use has, of course, become ubiquitous since, for everything from virtual studios to the latest Holywood special effects montage (sorry, movie), to the portable ones that people can pop up behind themselves on Zoom calls. But, as anyone with a halfway decent GPU in their computer processing a blurred background on said calls without the aid of one can tell you, the need for them is slowly diminishing.

At the high- (and increasingly middle-) end, LED volumes simply throw a pre-rendered background live behind the actors with the ultimate goal that this will enable everything to be shot in-camera with little or no post production required.

Meanwhile other teams are investigating the use of AI to further take chroma key technology on to the next stage.

Magenta? Devine...

According to New Scientist ($) Netflix researchers have developed a new technique called Magenta Green Screen (MGS). In this technique, the actors are filmed against a background of bright green LEDs, while simultaneously being lit from the front with red and blue LED lights. Together, these create a magenta glow on the actors, effectively enabling them to pop more effectively out of the background.

So, now take your RGB recording digital camera and film what's in front of you. The net result is that you get a green channel that records only the background, and red and blue channels that record only the foreground. The green channel can be swapped with the desired background footage with "even potentially tricky areas, such as transparent bottles or the area around strands of hair, working without problems. "

The difficulty lies in that you are then left with annoyingly magenta-tinged actors, which is where the AI comes in. This is trained on a picture or scene of them shot under normal lighting, and uses this reference picture to recolour them. It's fast but not real-time fast yet, though there is a way to average the red and blue channels to approximate green enough for monitoring on-set.

It sounds complex and probably at the moment is, but Netflix is keeping on pushing it with the hope that it can reduce some of the current restitutions working with green screen as well as the donkey work involved in tidying up a shoot in post.

What androids dream of...

Apologies for the blatant Philip K Dick reference, but it's a way of introducing the VFX start-up Electric Sheep that is taking AI and using a different route to solving the green screen issue.

It says that the only way to deliver the detail needed for big screen releases is to manually cut around foregrounds frame-by-frame a process; the aforementioned donkey work which it says takes around 6 hours for 1 second of footage and costs the industry over $1.5 billion per year.

That is a lot of money and could explain why the nascent company has just held an oversubscribed pre-seed funding round of half a million dollars.

It's developed a cloud-native solution, Spotlight, that it says is an impressive 360x faster, processing 1 second of footage in less than a minute. And the results look very accurate too.

It's not saying what the 'existing solutions' are, but the Electric Sheep detail around the hair is impressive

Steven Sullivan, a partner at one of the three venture capitalists funding the company, Spatial Capital, (and former Senior Technology Officer of LucasFilm to boot) said: "Electric Sheep's cutting-edge cloud and AI platform is poised to revolutionise several core VFX workflows. They're taking a smart and pragmatic approach to accelerate some of the most tedious post-production tasks."

Exactly how it works the company is not yet saying, and it's obviously not aiming it at live work, but it does reckon that flawless background removal negates the need for green screens on set entirely and it will be interesting to see what it comes up with as it ramps its technology up.

# data[].attributes.attachment_link.title
The quest to replace green screen accelerates

# data[].attributes.attachment_link.description
New technologies are in development that look to look to find faster and more accurate ways of replacing backgrounds in movies and TV.

# data[].attributes.attachment_link.readable_text
It's that wonderful time of the year where thousands flock to Adobe's largest conference and party to celebrate the brand's dominance over the photo and video software world.

While Adobe Max can be a lot of fun and full of helpful seminars and panels for those looking to grow their skills, the real highlight is the Adobe Sneaks program which showcases the latest, and often most ambitious, projects which Adobe is finally ready to share with the world.

From this year's Adobe Max we have lots to report and explore, but perhaps the most noteworthy so far is the news that Adobe is announcing new Generative Fill applications specifically for their video editing tools Premiere Pro and After Effects.

So, let's take a look at this new "Project Fast Fill" and explore how it is set to bring the power of Firefly (Adobe's AI product) to unlock human-prompted generative AI inside Adobe's video editing tools.

Introducing Adobe Project Fast Fill

While we haven't seen too much about this bombshell new feature just yet, rest assured we'll all be surrounded by tutorials and rant and rave reviews of Adobe's new generative AI capabilities soon enough. All we know so far is that this new tool is set to bring the power of Firefly generative AI to video for the first time.

Using the same technology which currently powers Generative Fill in Photoshop, as we've long speculated, Project Fast Fill will offer an early look at what human-prompted generative AI could enable inside Adobe video editing tools including Premiere Pro and After Effects.

We can assume that this Project Fast Fill will work much in the same way as Generative Fill to ease the addition, removal and expansion of content with simple Firefly-powered text prompts. Most likely unlocked as a subtle-use tool at first for short insert shots or sequences, but could very well lead to a bold new future of full generative AI video editing.

A look at Project Fast Fill in action

Adobe

Export to Social in Premiere Pro

The other big news from Adobe Max specifically for video editing might be an export to social feature coming to Premiere Pro with native integration support for posting to TikTok. This will be in addition to Adobe's existing support for Facebook, Vimeo, X (formerly known as Twitter), and YouTube.

All you'll need to do now is log in to your video or social media accounts from right inside Premiere Pro (the new beta version) and you'll be able to create your posts, add captions, hashtags, keywords, etc... and then upload them directly without ever having to leave Premiere Pro.

Here's an overview of the new features and what you can publish to each platform:

TikTok (new!): You can now export your Premiere Pro video as a TikTok draft or publish directly to TikTok.

Facebook: Along with the existing ability to post to public pages, you can now publish your Premiere Pro videos directly to Facebook pages.

YouTube: In addition to the existing capability to publish your videos directly from Premiere Pro to YouTube, you can now do the same to create YouTube Shorts.

On top of Premiere Pro, customers can also schedule and directly publish videos to TikTok, Facebook, LinkedIn, X, and Pinterest all within Adobe Express -Adobe's all-in-one app for fast and easy social media creation.

Other Updates for Video Editing

While we'll likely have more to share about all of these updates coming out of Adobe Max once the conference is done and wrapped up, here are the rest of the highlights coming to the video editing tools that Adobe currently has on its roster including Premiere Pro, After Effects, and Frame.io.

AI-powered Text-Based Editing improvements in Premiere Pro and Roto Brush in After Effects are now generally available, putting more creative power in the hands of users to experiment, ideate, and create in new ways.

The debut of Share for Review with Frame.io in Adobe's digital video and audio tools where Premiere Pro's "Share" button will now intuitively introduce Premiere users to Frame.io with the click of a button for faster content sharing and collaboration.

Additional innovations include a massive 5x timeline performance advancement in Premiere Pro for faster and smoother editing, new color preferences, and improved tone mapping to make it easier to get great color in addition to dozens of other requests from the creative community.

Be sure to stay tuned as we share the rest of the news and highlights as we dive deeper into these new generative AI features coming to Premiere Pro and After Effects.

# data[].attributes.attachment_link.title
Adobe Announces Project Fast Fill to Bring the AI Power of Firefly to Video

# data[].attributes.attachment_link.description
Let's take a look at how human-prompted generative AI is coming to Premiere Pro and After Effects, plus other new upgrades and integrations from Adobe Max.

# data[].attributes.attachment_link.readable_text
AI-powered tools can help video editors immensely by automating certain key repetitive tasks, freeing them to do more creative things.

The Genie is out of the bottle and it isn't going back in. Artificial intelligence (AI) has revolutionised the way video content is created, and it has become an indispensable tool for many video editors. Many of you are purists, and we love and respect that. However, there's no getting away from the fact that with the help of AI tools, video editors can save time, automate tedious tasks, and (arguably) enhance the creative process. It also dramatically lowers the learning curve for new content creators.

So, below I explore a range of AI powered tools that can speed up and improve your video editing process, whether you're a beginner that wants to tell a story with video or an expert that wants to save time/ become more productive.

Adobe Sensei is an incredibly powerful AI tool that comes with Adobe Premiere Pro, After Effects, and other Adobe Creative Cloud applications. It can analyse footage and automatically apply edits such as colour correction, noise reduction, and image stabilisation. Adobe Sensei can also analyse dialogue and generate automatic captions and transcripts. It even has the ability to match footage from different sources with a single click. Oh, and as I found out earlier today when a client asked for a video to be delivered in 16:9 and 1x1, Sensei's "Auto Reframe" tool can be a lifesaver!

Lumen5 is an AI-powered video creation platform that allows users to turn blog posts, articles, or other written content into genuinely engaging videos. With Lumen5, video editors can save time by automating the video creation process (yes, really), and the platform's AI algorithms can help identify the most important parts of the text and turn them into stunning visuals.

Okay, so I saw a demo of this on Instagram and had to check it out! is a video creation platform that uses AI to help users create professional-looking corporate videos quickly and easily. With Animoto, video editors can upload their own footage or choose from the platform's library of stock footage, music, and images. The platform's AI technology then takes over, automatically selecting the best clips and transitions to create a polished video.

We fell in love with this tool at first sight! Colourlab.ai is an AI-powered colour grading platform that allows users to enhance their video footage quickly and easily. With Colourlab.ai, video editors can upload their footage and choose from a range of presets or create their own custom look. You want your project to look like a still from The Joker? No problem - the look matching software will get you most of the way there. The platform's AI technology takes over, analysing the footage and applying colour grading adjustments to create a polished and professional-looking video.

Storylines.video is a brand new (still in beta as of today but very much working) AI-powered text based (with more to come) video editing platform that helps users create engaging corporate or other dialogue focused videos in minutes. With Storylines.video, video editors can upload their footage and choose from a library of pre-built templates. The platform's AI technology then takes over, analysing the footage and automatically editing it to create a polished and engaging video. The platform also offers a range of customization options, allowing users to tweak the final product to their liking. It also integrates with all major NLEs allowing everyone (especially you purists out there) to finish up in Adobe Premiere Pro, AVID Media Composer, Apple FCP, Da Vinci Resolve etc OR simply export to MP4 if you're in a hurry. If that isn't enough it has a very smart collaboration tool that allows projects to have instant client and team feedback. If you want to help this product and provide feedback on the Beta version they have a pretty active Discord channel open .

Transformative tools

So what have we learned? AI tools have transformed the video editing process, making it easier, faster, and more accessible than ever before. And that is a great thing IMO. Adobe Sensei, Lumen5, Animoto, Colourlab.ai, and Storylines.video are just a few of the many AI tools available to video editors. Whether you're creating social media videos, corporate productions, product demos, or full-length films, these tools can help you save time, automate tedious tasks, and enhance the creative process. So why not give them a try (most have a free version to test) and see how they can help take your video editing to the next level? Oh, and no doubt I've missed a few tools so feel free to add them to the comments below, or shout at me for allowing AI to take over the world.

Update: Ssemble & ChatGPT

As is the way of these things, more tools are emerging all the time. App developer Ssemble, for instance, is one of the first to integrate ChatGPT into its products with its ChatGPT script writer plugin and subsequent Script to video plugin. The idea is that users can easily generate full video scripts by simply entering their desired topic into the app hooked into ChatGPT. This spits out a script that can be edited if desired (tightening it or potentially removing the bits where the AI starts hallucinating). This is then loaded into the Script to Video Plugin that adds relevant stock videos or images, subtitles, and an AI-generated voiceover to match the content of the script.

We've not had a chance to test it out yet so can't attest to the final output quality, but it certainly won't be the last editing product to try and use OpenAI tools and their like to speed up the video production process.

# data[].attributes.attachment_link.title
5 great AI tools for video editors in 2023

# data[].attributes.attachment_link.description
AI-powered tools can help video editors immensely by automating certain key repetitive tasks, freeing them to do more creative things

# data[].attributes.attachment_link.readable_text
A new open standard holds out the hope of being able to verify what is real, even when it's virtual.

Despite all the skill that goes into creating them, both artistic and technical, it's not hard to see that deepfakes present a problem. We have long given up on trusting the veracity of the still image, and now the moving image is on the same journey towards suspicion. And for all the entertaining uses that go viral on social media, there are plenty of darker ones that spring to mind; everything from fraud to revenge porn to even the potential destabilisation of entire governments.

Political misinformation, hate speech, harassment...It's a problem and it's a growing one. Talk to any news broadcaster and the issue is right at the top of their agenda. How when you're in a constant competition with other broadcasters to be first on the air with your news, do you ensure that you're not being hoodwinked? Is that video of that politician confessing his admiration for Problematic Historical Figure A real? Did they really say that? And is that celebrity really doing that with a member of the Royal Family?

Newsrooms have to make decisions about such things fast and the volume of deepfakes is increasing all the time, with what are referred to as 'non-consensual and harmful deepfake videos crafted by expert creators' roughly every six months.

There is, of course, advice on how to detect them. offers an eight step program, pointing out that there is usually incongruence in some dimension of a deepfake. This is true and we can usually pick out something that is wrong with them even if we're not always consciously aware of it. But the deepfake technology is improving all the time and there is definitely a certain amount of objectivity regarding steps such as "Pay attention to the cheeks and forehead. Does the skin appear too smooth or too wrinkly? Is the agedness of the skin similar to the agedness of the hair and eyes?"

Of course, you can use the same technology that created the deepfake, usually a form of AI-powered Generative Adversarial Network where two AIs work on the problem together, to detect them. But this can be a) slow and b) ends up working to just make the original fakery better. What is needed is a guarantee of authenticity that can be quickly checked, and that's where the (Coalition for Content Provenance and Authenticity) reckons it comes in.

Digital provenance

The C2PA was launched in February 2021 and counts Adobe, Arm, the BBC, Intel, Microsoft and Truepic among others among its founders. Twitter joined a few months later. While there are various theoretical ways of detecting deepfakes - blockchain features prominently in many proposals - the C2PA's recently published specification is the first of its kind to provide a working technical specification for digital provenance.

Essentially it provides a series of statements, called assertions, that cover a range of areas surrounding a digital asset such as capture device details, author, edit actions, and more. These are wrapped up into a digitally signed entity called a claim alongside W3C Verifiable Credentials of the individual people and companies involved, and then all bundled together into something termed a manifest by a hardware or software component called a claim generator. This is all cryptographically protected and the basis of making any decision regarding the veracity of the resulting digital file is the identity of the entities associated with the cryptographic signing key used to sign the claim in the active manifest.

In other words, it provides an exact record of who created the asset and who has worked on it since. As a news organisation, or even as a consumer, you can then decide whether you trust it.

As an example, say a news cameraperson uses a C2PA-enabled capture device during a newsworthy event they are covering. The assets are then brought into a C2PA-enabled editing application, and after editing it, they are sent from the field to an editing suite back at base. An editor then makes additional edits also using a C2PA-enabled application before the finalised asset is moved into the content management system of a news organisation, which is also C2PA-enabled, and broadcast and/or posted to social media.

At each stage of the journey the manifest can be checked to see that all the companies and people associated with it are trustworthy. And the end consumer can even do that themselves on social media with a C2PA-enabled application. The manifest is hard bound to the asset using a cryptographic key and the W3C credentials are verified, making spoofing, impersonating, and other chicanery if not impossible then at least very, very difficult.

Is it perfect? No. Bad actors can subvert the system, and when it comes to media organisations some are definitely more trustworthy than others; several have been found recently passing off fake news stories that fit their own political agendas. It is also login to need buy-in from a lot of software and hardware vendors in the digital ecosystem. But is it a system that is better than what we currently have? Yes. Because, frankly, at the moment pretty much the only thing protecting us from deepfakes is our own instinct regarding what looks right and what doesn't. And throw enough computational power at that and even that instinct will be fooled more often than not.

# data[].attributes.attachment_link.description
A new open standard holds out the hope of being able to verify what is real, even when it's virtual.

# data[].attributes.attachment_link.title
Feature: The future of virtual production - Televisual

# data[].attributes.attachment_link.description
Virtual production is changing the way that high-end TV and films are being made, but how is it likely to evolve? Michael Burns reports

# data[].attributes.attachment_link.readable_text
Final Pixel and CUBE Studio team up to launch an accessible virtual production offering, designed for UK brands and creative agencies.

Named Content Factory, the new and accessible virtual production package looks set to transform the future of content creation. An efficient solution, providing access to a library of virtual backgrounds, camera, lighting, crew, props, post-production and more, it empowers creators to capture multiple campaigns in one day within CUBE Studio's 7,500 sq. ft. state-of-the-art Virtual Production Studio.

Final Pixel, an award-winning global Creative Virtual Production and Innovation Works with expertise in producing, world-building technology development and education and UK-based Virtual Production Studio, CUBE Studio, have teamed up to launch the first Content Factory, which combines virtual production expertise, techniques and technologies into an accessible, comprehensive and highly approachable package of services designed to enable anyone to embrace and easily integrate virtual production into their content, branding and advertising strategies.

Designed with a focus on digital, influencer and product advertising, Final Pixel's Content Factory bridges the gap between ad-level budgets and one-day virtual production shoots, empowering creative visionaries to take their first steps into this new era of immersive content creation and embrace virtual production technology to bring their ideas to life.

Stop time, control the weather

Hosted at CUBE Studio's 7,500 sq. ft., state-of-the-art virtual production studio, and providing access to a library of immersive and photoreal virtual environments, highly skilled crew, camera, lighting and a range of props, this groundbreaking package allows creatives to capture multiple campaigns in just one day.

Delivering a triple-threat of speed, efficiency, and enhanced creative capability, virtual production represents a paradigm shift in the way commercials and brand campaigns are being created. With a growing demand across the advertising space for a streamlined virtual production solution, Content Factory enables brands to deliver Hollywood quality storytelling, without the blockbuster spend.

Hanah Draper, Global Head of Production and Final Pixel, said: "Content Factory means you can stop time, control the weather, the season, the country and reach creative ambitions - but without breaking the bank" adding "We've unearthed a way of democratising virtual production and simplified it into a nice, digestible package."

Very first studio to offer the Content Factory package

Adoption of virtual production techniques has been accelerated in recent years, as productions place increasing importance on sustainability. Calling for smaller crew sizes, reducing travel and enabling virtual collaboration throughout the production process, Content Factory allows creators to embrace sustainable production techniques while exploring innovative technology solutions.

Home to the first Final Pixel Content Factory, and the exclusive UK destination, CUBE Studio, located just outside of London, provides the perfect stage for creators and brands to level up their storytelling and redefine what's possible.

James Hakesley, CEO and Executive Producer at CUBE Studio said: "We've always been passionate about empowering content creators to push creative boundaries through access to the very latest technical and visual innovations, and the launch of Content Factory will make virtual production more accessible than ever before.

Editing, sound mixing and colour grading

"We're proud to be partnering with Final Pixel, as the very first destination to offer the Content Factory package which, we believe, will be a catalyst for growth for virtual production across the UK."

CUBE Studio is home to a 10.5×5.5m LED volume, crafted with Unilumin Upad IV LED panels, which feature HDR image quality, a pixel pitch of 2.6 mm and new DDC (dual drive configuration) technology, plus Brompton Technology processing, to deliver visual excellence.

Designed with comfort and experience at the forefront, CUBE Studio features:

5mx5m drive-in vehicle access to the LED stage, with unloading bay

Viewing platform with views of the LED stage

Production office, boardroom, kitchen and refreshment facilities

Spacious greenroom and breakout lounge for talent and clients

Dressing rooms and hair and make-up suites

On-site parking

Sohonet dedicated high-speed internet access

Fresh air ventilation system

Equipment and setbuild area

Just off the M4, 15 minutes from Heathrow and access to London via Elizabeth Line

Encompassing support from pre-production through to post-production, this new offering includes editing, sound mixing and colour grading to Final Pixel and CUBE Studio's high standards. Follow the link to find more about Content Factory.

Support ProVideo Coalition

Shop with

# data[].attributes.attachment_link.title
Content Factory, an accessible virtual production package by Jose Antunes - ProVideo Coalition

# data[].attributes.attachment_link.description
Final Pixel and CUBE Studio team up to launch an accessible virtual production offering, designed for UK brands and creative agencies. Named Content Factory, the new and accessible virtual production package looks set to transform the future of content creation. An efficient solution, providing access to a library of virtual backgrounds, camera, lighting, crew, props, [...]

# data[].attributes.attachment_link.title
Topaz Photo AI Review: Does It Make a Difference on Large Prints

# data[].attributes.attachment_link.description
Topaz Photo AI claims to sharpen, remove noise, and increase the resolution of your photos. But does it make a visual difference on a printed photo? Let’s find out. Topaz Photo AI claims to sharpen, remove noise, and increase the resolution of your photos. But does it make a visual difference on a p…

# data[].attributes.attachment_link.title
How Virtual Production Worked On-set of the Lion King - fxguide

# data[].attributes.attachment_link.description
How to shoot 110 setups in a day without going into overtime.

# data[].attributes.attachment_link.readable_text
Wooden Camera is now taking pre-orders for their Elite Accessory System for RED KOMODO-X, which includes new accessories as well as industry-trusted favorites that have been optimized for the camera.

Last July ProVideo Coalition revealed how the Wooden Camera Elite Accessory System for Sony Venice II and Rialto II was being discovered by filmmakers and now it's time to announce that Wooden Camera is taking pre-orders for their Elite Accessory System for RED KOMODO-X.

According to Wooden Camera, "this collection offers an Elite System for users who want to outfit their camera with a full cinema setup as well as a Core System for users who want to start with the basics. This system includes new accessories as well as industry-trusted favorites that have been optimized for the KOMODO-X."

"This camera packs an unprecedented punch in a tiny package," said Dominick Aiello, Sr. Dir. of Accessories. "So we designed a modular accessory system that allows crews to augment the compact camera body dynamically into a full-fledged cinematography system without compromising size or weight. Highlights of the collection include an ARCA Baseplate, a new Cage System, the Ultra Handle System, a new D-Box and new Side Grips."

Here is a description of the different elements included:

ARCA Baseplate: The ARCA Baseplate was designed for dual 15mm rods and allows for maintaining optimal lens height for studio-style builds. This Baseplate comes with a quick-release lever to be used with the ARCA Riser when lightweight builds are necessary. This Baseplate includes a Bridge Plate Dovetail for use with either 15mm or 19mm Bridge Plates.

Cage System: Utilizing the tie-in points at the top of the camera body and the added Side Rails, this system provides much-needed mounting points on the rear and sides of the camera. This system enhances configurability and offers a sturdy base for rigging out the camera body with accessories. A built-in Accessory Rail Clamp adds a quick-release function to certain compatible accessories, like Accessory Plates or our B-Box. Complete the Cage System by adding the Top Cheese Plate, which connects directly to the camera, allowing the camera's contact pins to pass trigger through the plate. This offers connectivity options like the Ultra Handle RED Riser with its 3-pin run-stop ability, or simply expanding on rigging options with the Rigging Top Plate.

Ultra Handle System: A modular, integrated system that comes with a riser allowing the contacts on the camera to pass a 3-pin connector with trigger ability through the handle, thereby allowing third-party options to start and stop the camera roll. Also included in the system is a ¼"-20 Support Bracket for added stability, a non-slip 4" Handle with durable Bocote hardwood inserts for comfort. The Handle has two pass-through holes for accessing the mounting bracket with the Hex Wrench while the handle is attached. This Ultra Handle System also comes with two 3" Handle Extensions, a 1" Extension, and a Right Angle Extension, as well as 24x color rings (six each in red, blue, green, and yellow).

D-Box: The D-Box offers hot swap abilities when power is connected to the DC-In and an on-board battery. There are additional 12V 2-pin, a D-tap port, and a 3-pin 24V power options. When connected to the EXT port of the camera, Run/Stop is enabled on both the D-Box as well as the B-Box. On the assistant's side of the camera there is a DC-In port for connecting additional power items like block batteries and an EXT Out option is also included to pass connectivity options to a B-Box to be connected to the system, as needed.

Side Grips: These 2 grips include cold shoe mounts at the top of each handle, allowing users to mount accessories like mics for compact shooting scenarios. The lightweight grips include ⅜"-16 and ¼"-20 mounting points on the outside of the grips to allow for mounting any additional accessories.

"As our Engineers started to design the basic elements that are included with every collection, the need for new products and different designs emerged. The result is a full line of accessories that incorporate new features and improvements upon legacy products that align with the new design philosophy of Wooden Camera," says Greg Smokler, VP of Cine Products for Creative Solutions.

Support ProVideo Coalition

Shop with

# data[].attributes.attachment_link.title
Wooden Camera's modular accessory system for RED KOMODO-X by Jose Antunes - ProVideo Coalition

# data[].attributes.attachment_link.description
Wooden Camera is now taking pre-orders for their Elite Accessory System for RED KOMODO-X, which includes new accessories as well as industry-trusted favorites that have been optimized for the camera. Last July ProVideo Coalition revealed how the Wooden Camera Elite Accessory System for Sony Venice II and Rialto II was being discovered by filmmakers and now [...]

# data[].attributes.attachment_link.readable_text
Epic Games has published a guide for virtual production (VP): The Virtual Production Field Guide. It is a downloadable PDF intended as a foundational information document for anyone either interested in VP or already leveraging these techniques in production.

Virtual production is a broad term for a wide range of computer-aided filmmaking methods that are intended to enhance the creative process and increase efficiency. Epic Games are at SIGGRAPH next week and, along with a set of major partners, they will be showing a range of technology and approaches to VP, from Real-Time Live to specific talks.

The use of these real-time tools can turn the traditional linear pipeline into a parallel process where the lines between pre-production, production, and post-production are blurred, making the entire pipeline more fluid and collaborative. The field guide both explains terminology and includes interviews with key players in this space including Ben Grossmann (Magnopus: Lion King), Director Wes Ball, Director Sir Kenneth Branagh, Glenn Derry (Fox VFX Lab), Producer Connie Kennedy, DOP Bill Pope ( The Matrix trilogy, The Jungle Book, and Alita: Battle Angel), DOP Haris Zambarloukos ( Thor, Murder on the Orient Express) and the Third Floor's Virtual Production Supervisor, Kaya Jabar.

Below is an extract from the guide.
Get your own full copy here.
Virtual Production Supervisor Interview: Kaya Jabar

Kaya Jabar has worked at The Third Floor for nearly three years. She worked as a virtual production supervisor on projects such as Game of Thrones, Dumbo, and Allied. Jabar loves being able to flesh out ideas and carry them over to realistic shooting scenarios, evaluating available equipment, and coming up with new configurations to achieve impossible shots.
Can you describe your work as a virtual production supervisor?
I started in games, then joined The Third Floor as a visualization artist, and later transitioned into virtual production three years ago. My first show was Beauty and the Beast where part of The Third Floor's work was doing programmed motion control and Simulcam using visualized files. We used MotionBuilder for rendering, and the live composite was mostly foreground characters. Next, I did Allied with Robert Zemeckis and Kevin Baillie.

In the last few years, I worked on The Third Floor's team, embedded with the production crew on Game of Thrones. I did previs in season seven. Then in season eight, I supervised motion control and virtual production, primarily running the main stages and prototyping production tools. Anything to do with mounting flame throwers on robot arms, driving motion control rigs, virtual scouting, custom LED eyelines, and fun stuff like that. I also used a bit of NCAM real-time camera tracking and composites on that show.

Now I'm back in the office focusing on our real-time pipeline and looking at how virtual production can be further leveraged. At The Third Floor, we're generally on productions quite early, building and animating previs assets to storyboard the action quickly. It's been pivotal on a few shows where you can have visualized content speedily and use it to make decisions faster to achieve more impressive things on camera.
Which tools do you use?
We use Unreal in our current pipeline for real-time rendering, so we're leveraging that and building tools like our virtual scouting and virtual camera tools around it as we evolve the workflow. By leveraging Unreal, our goal is to be able to work with clients, directors, and cinematographers in ways that get them closer to a final pixel image, so they can make more informed decisions and be more involved in the process.
How does your work overlap with the visual effects pipeline?
It tends to flow both ways, for example, on Game of Thrones, we pre-animated a lot. So I was responsible for all the dragon-riding virtual production, which was incredibly complex. It was three pieces of motion control equipment all in sync and slaved to the final comp.

We would previs everything, get that approved quickly, and then send our scenes to the finals house. And then they'd send us exports back, so I could use those exports to do any NCAM work or motion control work I needed. It empowered us to pull quite a bit of animation work forward in production.
Do you think virtual production will increasingly deliver final pixel imagery?
At least for OTT [over the top, streaming media] such as TV and lower budget productions, we're heading that way. For the final season on Game of Thrones, we had six months' turnover between finishing filming and delivering six hours of content. For those kinds of shows where the production house isn't the VFX house as well, I think you can push previs assets further, due to the fact that you can cross-pollinate between final VFX houses, which they can give Alembic caches for all your work and so on.

Much attention is being drawn to virtual production lately. I've been working on it for over four years, and other people have since Avatar, but it was never conceivable that you would get final pixel in-camera. It was always a prototyping tool. With improving GPUs, especially the NVIDIA RTX cards and global illumination, I can see it being brought more on set.
How do you keep the virtual production department as efficient as possible?
By understanding the brief and having the virtual production supervisor involved early. On my most successful shows, I've been involved from the beginning and had somebody there who was auditioning the technology. You need to be able to look at it from the standpoint of: is this something that will take seven days to set up for one shot, or can we use iteratively without a team of 15 people?

I worked on Dumbo as well, and it was essential to look at all the different hardware and software providers and run them through their paces. A lot of it is making sure data flows into the effects pipeline. There's a lot of duplication of work which can be frustrating, where you've made all these decisions in one software package, and they don't flow over to the other software packages or the other supervisors, etc.
How do you bring new crew members up to speed with virtual production?
We need to understand the problems the crew is trying to solve and make sure that the tech addresses those rather than pushing a technical idea and way of working that doesn't answer the original brief. If somebody is looking for a quick way to storyboard in VR, then we should focus on that. Let's not put a virtual camera on them, or any of those other inputs, overwhelming them with tech.

I'm a big believer in flexibility and being able to tailor experience and UI design to address different approaches and working with different directing styles and preferences, etc. It's being able to understand what they're trying to do, so we can do it more quickly by not involving final animation.
What is The Third Floor doing to stay on the leading edge?
All our supervisors get deployed on the shows as boots on the ground. We not only talk to the creatives but also the technicians. I speak to every single motion control operator and every person who is prototyping solutions and approaches to understand the questions they're being asked. On Thrones, that collaboration and continuously innovating on everyone's ideas and input led to some very successful results, for example, for dragon riding shots to use a Spidercam motion base and a Libra camera, which had never really been used in sync together.

We are often active across many other parts of a show, not just virtual production. A lot of our tools are bespoke per show, so we will create a specific rig that we know they are using or other components custom to the project. But we then also build up experience and knowledge with each project and find ways to generalize some of the tools and processes that can overall streamline the work.
Did virtual production help inspire the significant leap in visual effects scope for the final few seasons on Thrones?
Game of Thrones was this unique ecosystem of people who had worked together for so long that they'd gained each other's trust. Season six had about seven major dragon shots. Joining on season seven, my biggest challenge was doing 80 major dragon shots with entirely new equipment. What worked there was the relationship The Third Floor had with the entire production: the executive producers, DPs, the directors, and all the key departments. The VFX producer trusted us to come up with new solutions.

They said, "This is impossibly ambitious, so let's all sit in a room and talk to all the heads of departments. Let's figure out what VFX can do, what virtual production can do, and what the final VFX vendors can provide for them to accomplish that." That type of relationship and collaboration happens, but for TV it's unique because you need to work with the same people again year after year. If you don't get along, usually you don't get asked back.
What do you think about the future of virtual production?
I'm quite excited about final pixel in-camera, at least for environments and set extensions. More departments are working in real-time engines. I think it would be fantastic to walk onto a set and have every camera see the virtual sets as well as the built sets. We're showing the camera crew and the director what they're going to see, but we need to find a way to engage the actors more as well.
Has virtual production, including the work you've done, raised the scope on the small screen?
I genuinely hope so because I think there is a lot of virtue in planning. What we achieved on Game of Thrones with the modest team that we had, was purely because everybody was willing to pre-plan, make decisions, and stick to them. If you use visualization the way it's meant to be used, you can achieve absolute greatness in small scale.

There are so many ideas in the creative world, and I would like to be able to help directors and others who don't have a $100 million budget. I'm looking forward to smaller movies getting the tools they need to achieve the vision which was previously reserved just for Hollywood blockbusters. It's exciting to democratize the process.

# data[].attributes.attachment_link.title
Virtual Production Guide: Kaya Jabar, Third Floor - fxguide

# data[].attributes.attachment_link.description
The Third Floor's Kaya Jabar talks virtual production as part of the new free field guide to VP.

# data[].attributes.attachment_link.title
8K live cinematic streaming with RED Connect Module by Jose Antunes - ProVideo Coalition

# data[].attributes.attachment_link.description
Unlock the capabilities of up to 8K live cinematic streaming with the new RED Connect solution, which enables users to real-time stream RAW R3D files direct from the V-RAPTOR and V-RAPTOR XL cameras. "We are extremely excited to officially launch RED Connect and the new module," said Jeff Goodman, RED Digital Cinema vice president of [...]

# data[].attributes.attachment_link.readable_text
Cover image via

The streamer's mysterious boat fantasy series featured heavy use of virtual production techniques, especially 'the volume.' But the episodic also looked to evolve the use of new technology to help themselves and future filmmakers.

As a filmmaker, you'll want to start learning about virtual production techniques, whether the latest version of the Unreal Engine or how to shoot in a LED volume. Cinematographer Nik Summerer is less of a technical artist than one who understands when things aren't right. He also wasn't versed in the use of the volume, so he had some catching up to do when his new show, 1899, was booked into an as-yet unbuilt LED volume studio in Germany to shoot it.

Their volume and the ideas on how it would work had to be created from scratch, but that had some advantages. They could take what had happened elsewhere and make it better, evolving it for themselves and others in the future. That meant experimentation, innovative additions like a turntable within the staging to allow for different angles, fast reverse shots, and the ability to bring in real-world SFX like rain. These were the first examples of those in the world of volume.

Nik explains where they were regarding the volume's global use at the time. "Because the only other show using a volume at that time was The Mandalorian, we used them for most of our research. Their advice helped with the basic design, including the height, the choice of screens for the ceiling, and basic questions to do with the space we needed."

But they had a bigger problem; their volume hadn't even been built. It was to be constructed in Babelsberg Studios near Berlin, which would take most of 2020.

The First Volume Turntable

While the Berlin site set about constructing Dark Bay Studio, which would hold the volume, the crew decamped to London to learn how to shoot in such a new environment, albeit a smaller version set up just for them by their VFX vendor Framestore. While they went through their process and how practically they would make 1899, someone jokingly said they needed a turntable so they could take reverse shots quickly.

It turned out to be a great idea and timely, too as construction was able to adjust to having a brand-new element right in the middle of the set. Nik immediately saw the advantage of this new addition. "We talked about how to shoot in a volume, and this idea for a turntable came up. As normal, you want to shoot in one direction and then turn around and do the reverse shot. What if you could quickly reset everything and do that within a volume stage; the volume is the easy part as it's just a digital asset."

The Mandalorian had done the same thing but just flipped everything from the left side to the right side. You then move any props accordingly; depending on the complexity of the scene, it could be done relatively quickly. But the more complex the scene then, the longer the time it takes; Nik saw this as a problem.

"Regardless of how you do it, in my opinion, you are breaking the atmosphere of a scene. The actors are in a certain mood; everybody is working at that moment, so breaking it apart isn't ideal. The question was how we could do it quickly and turn around at the same speed as you would in a normal location."

The production's SFX team would build something that could hold a few tonnes of weight as you had to support the props from the boat, the crew, and the cast. It would spin at a regulated speed with a diameter of 21m and take about three minutes to turn around.

All the lights are not mounted on the ship, so they don't have to be change, just swapped over, but all the cabling has to be disconnected to let it spin freely.

Extending the use of the turntable could include walking shots and near-instant different angles, as it can stop anywhere around the circle. The turntable is one of a kind, although there are bound to be others soon as the numerous advantages are.

More Traditional Shoot with Rain

The turntable allows the production to have the comfort of a traditional way of filmmaking within the latest volume tech. With new users of a LED volume becoming potentially intimidated by this new way of doing things, offering them a connection to their usual way of working is a great idea.

But there was more innovation that looked to evolve the new virtual technology. Again this was the SFX department working out a way of incorporating standard features on the set. Being a drama about huge passenger liners in the middle of the Atlantic, there was a need to bring rain onto the set. With all the sensitive electronics involved with the LED panels and now the turntable, this was a complex problem to overcome.

The showrunner and director Jantje Friese and Baran bo Odar wanted to put a pivotal fight scene on the ship's deck in the middle of a storm, so they wanted rain and lots of it. Nobody could see how they could achieve so much water within the confines of so much technology. But the SFX team again came up with the answer to control the rain rig and the water piping system in a way that wouldn't affect any other electrical systems.

Advice on volume shooting

It was important for the 1899 production team not to limit themselves to their usual shoot regime while in the volume. They insisted on not compromising in that regard inside the volume space. So they started with traditional dolly moves and even a Technocrane on the set. It was actually on the second day of shooting that the director thought that a handheld camera movement was what was needed. This was to put the audience more 'in the story,' but it changed his concept of the whole thing.

"We learned that there was a sweet spot for shooting within the volume, and you wanted to be around 4.5m away from the wall, but other than that, you had to treat it like a typical location shoot and not be limited by things. You can use all the shots, but you have to be careful with wider shots as they might need more work with VFX later to fix certain things.

"We didn't have any major problems with Moiré, which everyone had warned us about, but we found shooting anamorphic certainly helped."

Nik's take from his experience

There's nothing better than learning on the job, and that's what Nik and his crew certainly did. But his takeaway from his experience should help others to deal with the challenges. "You need to familiarize yourself with the digital background. As in what the wall can give you because it's also your lighting instrument or part of it.

"You need to spend a fair amount of time in there to understand the benefits of the wall, the quality of light you are getting, and the color temperatures you're getting. Also, what the limits are like proper skin tone rendering, what you can do to improve this with supplemental lighting, what lights are in the foreground, for example."

Nik spent around 55 days of the shoot in the volume and would use the system again, depending on the project. But 1899 has moved this technology on and, by doing so, has made it perhaps less fearful for other productions to take advantage of it.
1899 is currently in its first season on Netflix.
For more on virtual production, check ou

# data[].attributes.attachment_link.title
Netflix's 1899 Evolves Virtual Production

# data[].attributes.attachment_link.description
The streamer's mysterious boat fantasy series featured heavy use of virtual production techniques, especially 'the volume.' But the episodic also looked to evolve the use of new technology to help themselves and future filmmakers. The streamer's mysterious boat fantasy series featured heavy use of virtual production techniques, especially 'the volume.'

# data[].attributes.attachment_link.readable_text
Shooting a dawn scene for 11 hours or moving the mountains behind the actors are some of the magic tricks introduced by Virtual Production in filming The Mandalorian. Here is how it's being done!

A peek behind the scenes of The Mandalorian, in February 2020, revealed how Virtual Production was being used to shoot the saga. Over 50 percent of The Mandalorian Season One was filmed using this ground-breaking new methodology, eliminating the need for location shoots entirely. Instead, actors in The Mandalorian performed in an immersive and massive 20' high by 270-degree semicircular LED video wall and ceiling with a 75'-diameter performance space, where the practical set pieces were combined with digital extensions on the screens. But it all started almost half a century ago, when George Lucas imagined a space-opera in the sky.

When George Lucas returned to the Star Wars saga, in 1997, with what was to be The Star Wars Trilogy Special Edition a theatrical anniversary edition of the original trilogy, in order to celebrate the 20th anniversary of the release of Star Wars: Episode IV A New Hope, he could not have guessed that decades later the Star Wars universe would still be the test bed for some of the most exciting developments in Virtual Production: The Mandalorian is the new playground for all the magic Hollywood can deliver.

The 1997 renewal of the original saga - which started in 1977 - was done by digitally remastering the image and sound to, as George Lucas said in a September 2004 interview with the Associated Press, "finish the film the way it was meant to be". The Special Edition versions introduced special effects which were not viable when the films were first released and added new digital sequences made possible by the improvements in CGI technology and other production techniques that occurred in the twenty years that passed since the original trilogy was released.

From the Dykstraflex to Virtual Production

A name always connected with the continued quest of "what's possible" by George Lucas is Industrial Light & Magic (ILM), the motion picture visual effects company that the filmmaker founded in 1975. A division of the Lucasfilm film production company, the birth of ILM coincides with the start of production of Star Wars. ILM is responsible for many of the developments that came after, from the computer-controlled motion camera Dykstraflex - named for lead developer John Dykstra - to THX sound quality or the evolution of the original blue screen composite shot technology, used for so many of the space battles in the saga.

Industrial Light & Magic (ILM) is now part of Disney, as the company bought Lucasfilm in 2012, and acquired ILM in the process. Through that process, ILM continues to be involved with the Star Wars saga, in its new chapter, The Mandalorian, which is a test bed for the technologies now available to filmmakers. The video published last year showed a behind the scenes look at the groundbreaking virtual production technology used on The Mandalorian, Season One.

Digital 3D environments created by ILM played back interactively on the LED walls, edited in real-time during the shoot, which allowed for pixel-accurate tracking and perspective-correct 3D imagery rendered at high resolution via systems powered by NVIDIA GPUs. The environments were lit and rendered from the perspective of the camera to provide parallax in real-time, as if the camera were really capturing the physical environment with accurate interactive light on the actors and practical sets, giving showrunner Jon Favreau, executive producer and director Dave Filoni, visual effects supervisor Richard Bluff, and cinematographers Greig Fraser and Barry Baz Idoine, and the episodic directors the ability to make concrete creative choices for visual effects-driven work during photography and achieve real-time in-camera composites on set.

A game engine, Unreal, and The Mandalorian

Four years ago, in 2018, Industrial Light & Magic introduced the use of real time rendering (with Unreal Game Engine from Epic Games) and digital LED displays as a virtual set in the production of The Mandalorian. The end-to-end virtual production solution, named ILM StageCraft, is a production-hardened technology that provides a continuous pipeline from initial exploration, scouting, and art direction, traditional and technical previsualization, lighting, and of course, real-time production filming itself, with the innovative StageCraft LED volumes.

As ILM noted at the time, "the technology and workflow required to make in-camera compositing and effects practical for on-set use combined the ingenuity of partners such as Golem Creations, Fuse, Lux Machina, Profile Studios, and ARRI together with ILM's StageCraft virtual production filmmaking platform and ultimately the real-time interactivity of the Unreal Engine platform."

"We've been experimenting with these technologies on my past projects and were finally able to bring a group together with different perspectives to synergize film and gaming advances and test the limits of real-time, in-camera rendering," explained Jon Favreau adding, "We are proud of what was achieved and feel that the system we built was the most efficient way to bring The Mandalorian to life."

The Mandalorian, Season Two, and StageCraft 2

This April, Industrial Light & Magic returns with another video, this time revealing the key aspects of the Virtual Production of The Mandalorian, Season Two. For the second season of Lucasfilm's hit Disney+ series, The Mandalorian, Industrial Light & Magic reengineered their StageCraft virtual production platform rolling out version 2.0 in which ILM introduced among other things, Helios, Industrial Light & Magic's first cinematic render engine designed for real-time visual effects.

According to ILM, "engineered from the ground up with film and television production in mind, Helios offers incredible performance, high fidelity real-time ray tracing, the ability to rip through scenes of unparalleled complexity, all while leveraging ILM's unrivaled color science, and was designed from the start to work seamlessly with ILM StageCraft. The purpose-built, production-hardened platform allows filmmakers to explore new ideas, communicate concepts, and execute shots in a collaborative and flexible production environment."

Also, according to ILM, "the new stages are vast improvements over the original ground-breaking LED volume developed for the first season of The Mandalorian in 2018. Physically, the new stages are larger, utilizing substantially more LED panels than ILM's original stage and also offering both higher resolution and smooth wall to ceiling transitions - this directly results in better lighting on set as well as many more in-camera finals. ILM's proprietary solutions for achieving groundbreaking fidelity on the LED walls at scale allows for higher color fidelity, higher scene complexity, and greater control and reliability."

5,000 visual effects shots

The tools StageCraft offers are only part of the process of filmmaking, and that's the reason why ProVideo Coalition shares with its readers a third video, Behind the Magic: The Visual Effects of The Mandalorian Season 2. This new video, published last March, shows visual effects work on The Mandalorian created in all five of ILM's studios (San Francisco, Singapore, Vancouver, London, and Sydney) as well as a contingent of other vendors under ILM's supervision.

Industrial Light & Magic states that "the season's 8 episodes encompassed nearly 5,000 visual effects shots in addition to all of ILM's real-time effects work done for use during principal photography. The effects team leveraged virtually every trick in the book, from miniatures and motion control to traditional puppeteering, advanced animatronics, spectacular special effects, and photo-real CG. Here's a sampling of ILM's post visual effects work done for the series that garnered ILM 13 Visual Effects Society Award nominations."

Support ProVideo Coalition

Shop with

# data[].attributes.attachment_link.title
The Mandalorian: a test bed for Virtual Production by Jose Antunes - ProVideo Coalition

# data[].attributes.attachment_link.description
Shooting a dawn scene for 11 hours or moving the mountains behind the actors are some of the magic tricks introduced by Virtual Production in filming The Mandalorian. Here is how it's being done! A peek behind the scenes of The Mandalorian, in February 2020, revealed how Virtual Production was being used to shoot the [...]

# data[].attributes.attachment_link.readable_text
The next SMPTE Hollywood Section meeting explores "deepfakes", and how the emerging field of synthetic humans can be used for both entertainment and nefarious purposes.

Under the title "Digital Humans and Deepfakes: Creative Promise and Peril", the next SMPTE Hollywood Section meeting promises to be a memorable moment everyone should attend. The Hollywood Section of SMPTE, the organization defining the future of storytelling, will examine the promise, and potential peril, of digital humans and so-called deepfakes at its monthly meeting on Tuesday, November 19, in Hollywood. Held in tandem with the Radio, Television, Digital Newsroom Association (RTDNA), the free event will include a panel discussion of experts in the emerging field of synthetic humans.

https://youtu.be/9NkKj0aNB0s

Deepfakes are believable human images synthesized through artificial intelligence techniques from completely real and totally non-real or "fake" elements. In Hollywood, digital humans, convincing enough to fool audiences, have been the holy grail of visual effects for decades. Both techniques are designed to trick the viewer, but, whereas digital humans are constructed to entertain, deepfakes can be used to mislead and misinform, often for non entertainment purposes.

Deepfake videos doubled the last 9 months

Everywhere you look online these days, deepfakes are present. According to an article on BBC News, "new research shows an alarming surge in the creation of so-called deepfake videos, with the number online almost doubling in the last nine months. There is also evidence that production of these videos is becoming a lucrative business."

https://youtu.be/UlvoEW7l5rs

The trend started, apparently, when someone, back in December 2017, used deep learning technology to edit the faces of celebrities onto people in pornographic video clips. The term is used for both the technology used and the videos created, which can be anything from a funny transformation to a video used for political propaganda. The multiple examples shown in recent months reveal how far the technology has evolved, and how dangerous it can become if used for nefarious purposes.

We're on the verge of a revolution that will mix Artificial Intelligence, machine learning and immersive experiences made possible by Augmented Reality and Virtual Reality, to take us to what can well be the uncanny valley. The deepfake experiences we've seen until now are mostly bidimensional, but when you add VR to the equation, you get a multidimensional experience that enhances the illusion. The "holographic" promise of Star Trek may be just around the corner. I recently met Mona Lisa, smiling and waving at me, just by using my Oculus Rift S headset...

Digital humans and deepfakes

The movie-industry, that thrives on a make-believe world, is now worried about the potential peril represented by "deepfakes". SMPTE Hollywood and the RTDNA will offer a totally real presentation on deepfakes and digital humans. The panel will describe the history of digital humans and deepfakes, the challenges involved in creating them convincingly, and if/how news and entertainment professionals can spot a deepfake.

https://youtu.be/AQvCmQFScMA

Presenters include Chaos Group Lab head of research and development Christopher Nichols, who leads the Digital Human League, sponsor of the open source Wikihuman; Corridor Digital's Niko Pueringer, who has produced short-form Internet content for more than a decade and is an expert in creating and detecting deepfakes, and Shruti Agarwal, a Ph.D. student in the Department of Electrical Engineering and Computer Sciences at University of California, Berkeley, who is conducting research in multimedia forensics.

https://youtu.be/GZIcciDZtDA

Freelance journalist Debra Kaufman (USC Entertainment Technology Center, New York Times, Los Angeles Times, Wired, Reuters, Bloomberg American Cinematographer, International Cinematographers Guild Magazine) will moderate the discussion. Kaufman and Linda Rosner are producers of the event.

Taxi Driver starring Al Pacino

To illustrate this piece of news, we picked a few examples that are directly connected to Hollywood, from Taxi Driver starring Al Pacino to Terminator 2 starring Sylvester Stallone, or Willem Dafoe in the Coin Toss sequence in No Country for Old Men. These deepfake videos, created by Patreon author named ctrl shift face, are a good example of what is possible to achieve right now with the technology. The author is "creating entertaining deepfake videos", which he says are "windows to parallel universes". The videos and the behind-the-scenes on some of them might help to educate people about what "deepfakes", but they are a frightening sign of how easy it can be to fool people... Taxi Driver with Al Pacino!?

https://youtu.be/QhxTTshL3b0

The SMPTE Hollywood Section, November meeting takes place Tuesday, November 19, 2019. 6:30 p.m. at Hollywood American Legion Post 43, 2035 N. Highland Ave., Los Angeles, CA 90068. The title is "Digital Humans and Deepfakes: Creative Promise and Peril". Follow the link to register.

Early arrivals have an extra activity: a guided tour of the historic American Legion #43, built in 1929. Tours will include the modern projection booth with 35/70 mm FILM and featuring Christie 4K DIGITAL Projection. Please indicate on EVENTBRITE your choice of a tour at 5, 5:30 or 6 pm. As always, SMPTE Hollywood Section meetings are FREE and non-members are welcome.

Support ProVideo Coalition

Shop with

# data[].attributes.attachment_link.title
SMPTE Hollywood explores Deepfakes: creative promise and potential perils by Jose Antunes - ProVideo Coalition

# data[].attributes.attachment_link.description
The next SMPTE Hollywood Section meeting explores "deepfakes", and how the emerging field of synthetic humans can be used for both entertainment and nefarious purposes. Under the title "Digital Humans and Deepfakes: Creative Promise and Peril", the next SMPTE Hollywood Section meeting promises to be a memorable moment everyone should attend.

# data[].attributes.attachment_link.readable_text
It is hard to believe that Paul Debevec published his PhD just ten years ago. His work in defining high dynamic range images (HDRi) was almost immediately adopted by the major effects companies and expanded into one of the most significant visual effects areas of the last ten years. Debevec's latest paper, which builds on HDRi and his now famous light stage, has just been presented at Siggraph this year. This is a stage rig that can reproduce the exact lig
Background
Most HDR images are made by photographing multiple exposures of mirror balls or large chrome ball bearings mounted on a tripod. This produces a very high dynamic range image image once the separate exposures are combined into one HDR file. This process, often referred to as capturing a Lightprobe, has effectively recorded the light hitting that point from almost every angle, making it very useful in CG|. HDR images are also made by stitching together multiple images into a panorama see the Batman Begins story. But there is no reason why HDR images can not simply be multiple combined exposures of a normal scene or shot.
1994 and before
In 1994, some work that Debevec had earlier done as a high school student with his own photography in dark rooms played a significant role in some work he wanted to do for a friend. As a teenager, he had set off fire crackers with very long exposures "what looked to me like dots of light - would be on the film as great streaks of light," says Debevec. Years later as he got into CGI he realized that such effects were not seen in CG images or if they were they were rather "heavy handedly added after the fact."

A friend of Debevec's, Brian Mirtich, had a chance to have an image from his research on dynamic simulation featured on the front cover of Berkeley Engineering Magazine. The problem for Debevec was that "still photographs of these falling bowling pins and spinning coins did not convey the motion inherent in the research", so he started thinking about the problem of adding motion blur and simulating long exposures. "I wanted somehow to get that magic effect of the streaking of those bright regions," says Debevec, and this was the moment that he realized that as you average together 20 or 30 conventional images to simulate motion blur you lose optical streaks. The specular highlights are clipped at a 100% level and because they are averaged down, the highlight becomes washed out or averaged out. But this doesn't happen in the real world since there is no clipping.

Debevec decided to ask Brian to render two passes of his animation, one that was just the diffuse pass of the bowling pins or coins that were flipping round, and another that was just the specular pass. In those days the render was merely a Phong shader, so "the specular was never that strong in a high dynamic range sense," says Debevec. "But what I did was convert these specular highlights to floating point images, making the specular much higher, and then added it all back together only converting it to 8 bit for the final output." In converting the images to floating point, it allowed the specular highlight to be above 100 %. As the image is averaged, the specular highlights do not grey out and instead remain strong and produce vivid streaks. Thus simulating much more exactly the fire crackers of Debevec's youthful home photography. And HDR was born.

Working nearby Debevec was Greg Ward. In 1985, Ward began development of the Radiance physically-based rendering system at the Lawrence Berkeley National Laboratory. Because the system was designed to compute photometric quantities, it seemed unacceptable to Ward to throw away this information when writing out an image. He settled on a 4-byte representation where three 8-bit mantissas shared a common 8-bit exponent. "When I first started developing Radiance in 1985-86, I had in mind separating the pixel sampling from the filtering steps in the calculation, which required passing floating-point data to do it right," explains Ward. "Since none of the image formats or toolkits at the time seemed to handle floating-point very efficiently, I decided to roll my own."

"Had we spent a little more time examining the Utah Raster Toolkit, we might have noticed therein Rod Bogart had written an experimental addition that followed precisely the same logic to arrive at an almost identical representation as ours". He explains further, "the Utah Raster Toolkit, which I had only looked at briefly back then, had a poorly documented feature where the alpha channel acted as an exponent, just as I ended up doing in the Radiance RGBE format.". Bogart later went on to develop the EXR format at Industrial Light and Magic. Another motivation for Ward was that he hoped to use his calculations to come up with lighting values. "throwing away the absolute values I had gone to so much trouble to compute seemed wasteful," says Ward. "In the long run, using floating-point (HDR) images had many benefits in the post-processing phase, including tone-mapping possibilities that I began to explore in the late '90s after seeing some of the exciting work done by Jack Tumblin and Jim Ferwerda."

. The Online Wikipedia names Ward as the "founder of the discipline of high dynamic range imaging." But in Ward's own words "I never thought I founded anything....I think credit should really go to Paul Debevec for HDRI, as my part in it was mostly limited to providing some useful tools so he could implement his ideas without too much programming." Greg Ward, for all his modesty, produced both the breakthrough open source Radiance software package and defined the Radiance image file format, which was initially the format of choice for everyone working in the field of HDR. While the file format stores floating point images in much less space, there were some things that could be improved. "Although RGBE is a big improvement over the standard RGB encoding, both in terms of precision and in terms of dynamic range, it has some important shortcomings," says Ward "Firstly, the dynamic range is much more than anyone else could ever utilize as a color representation. It would have been much better if the format had less range but better precision in the same number of bits."

HDR was immediately of interest to film makers due to its ability to record or sample extremely accurately lighting conditions and to allow previously unheard of levels of realism. It fit perfectly with new models of global illumination and attempts to model radiosity. For the first time we were able to film a set of exposures and combine them into one file - which could then be used to produce a set of 3D lights (normally 20 to 30) that would very accurately reproduce the lighting that was on set.
1996/1997
Debevec actually worked on two pieces of research that for a while were considered by most people to be the same: Photogrammetry, the ability to reconstruct models from still images, and HDRi. Debevec's first film, The Campanile Movie made in the spring of 1997, was actually photographed in a manner to completely avoid HDR issues. "We choose a cloudy day to avoid needing multiple different exposures," he says.

The Campanile Movie used image-based modeling and rendering techniques from his 1996 Ph.D. thesis "Modeling and Rendering Architecture from Photographs" to create photorealistic aerial cinematography of the UC Berkeley campus. To get the aerial shots around the top of high the Berkeley Tower, Debevec and UC Professor Charles Benton actually flew a kite and took pictures from it for the photogrammetry and texture maps. In the end, the 150 second film was created with just 20 images from the eight rolls they shot. Few people know that the original storyboards had the Campanile falling over and crushing the hero, but that never made it into the final film.
1999
It took less than 2 years for the work in The Campanile Movie to make its way into mainstream feature films. Masters student George Borshukov, who worked on the project, went on to work at Manex Entertainment where he and his colleagues applied The Campanile Movie's virtual cinematography techniques to create some of the most memorable shots in the 1999 film The Matrix. The backgrounds in the famous rooftop "bullet time" shots in the film used photogrammetric modeling and projective texture-mapping. Beacause the bullet time cameras all face each other, a virtual version of the roof top was required. VFX supervisor John Gaeta had seen The Campanile Movie and immediately saw its potential.

The technology was also used to produce the landmark short film at Siggraph 99. The New York Times described the film as "An orgy of technical expertise... a thousand Ph.D.'s software engineers and animators burst into applause." The short is set in an image based model of St. Peter's Besilica featured hundreds of steel spheres and falling monoliths and was described by the film makers as "an abstract interpretation of the conflict between Galileo and the church, drawing upon both science and religion for inspiration and imagery."

Fiat Lux not only explained the films goal, but it was also the motto of the University of California. The ancient church built in AD326 and rebuilt by Renaissance architects in the 16th century was digitally rebuilt from just 10 photographs. To the Church were added slow motion mirrored spheres and solemn black doiminoes all bathed in softlighting. The 4500 frame animation took a hundred 167Mhz xcomputers and 20 PII working full time for three days to render.
2000-2001
Films such as Mission Impossible II and and many others started using HDR imagery and around 2001 Debevec's HDR Shop program became available, allowing anyone to make HDR images and use them in production.

As the techniques became understood, the research work split into several related and yet different paths. Companies such as the Pixel Lieberation Front (PLF) used the Photogrammetry side of the initial research very effectively in films such as Fight Club. In Godzilla the animators at Centropolis needed a good low-res matchmove set for the big lizard to run down. A clever reverse solution was provided by PLF to build and match each building in order to place it by hand into the match moved lived action footage - allowing for interactive effects to be added.

Inspired by Debevec's work, several companies developed photogrammetry products. Canoma, originally written by Robert Seidl and Tilman Reinhardt, was sold to Metacreations and then Adobe. One product which faired much better was RealViz Imagemodeler which was recently updated to version 4. RealViz also released Stitcher for building panoramas. Another successful program is PhotoModeler Pro, a Windows based software program from Eos Systems, which allows the creation of accurate, high quality 3D models and measurements from photographs. PhotoModeler Pro 5 is widely used in the fields of accident reconstruction, architecture, archaeology, engineering, forensics, as well as 3D graphics. The latest PhotoModeler Pro 5 offers fully automated camera calibration, NURBS Curve and NURBS Surface modeling, supporting a variety of file export formats and enhanced photo texturing.

While these companies advanced digital Photogrammetry, it is worth noting that it was not long after George Eastman helped invent photography as we know it that the first use of the term Photogrammetry was made. The term was first used in 1893, by Dr. Albrecht Meydenbauer, who went on to found the Royal Prussian Photogrammetry Institute. But it has been with the advent of digital cameras that the techniques have had wide spread application.

In 2000, ILM started working on a format for HDRi data. A few HDR formats existed but some were extremely large in size and none really stood out as a default industry standard. ILM had rejected 8 and 10 bit formats due to the lack of dynamic range necessary to store high contrast images. 16 bit formats tended to not account for 'over range' elements - i.e. specular highlights - and tended to clamp the highlights. 32 bit float tiff files were enormous in terms of storage.

File formats are simply "containers". The full TIFF format was built on the original PIXAR Log Encoding TIFF, which was a log 11bits per channel file format, developed in house at Pixar by Loren Carpenter to maximize the use of their film recorder. 96 bit TIFF is a 'dumb' 24 bits per pixel format which poorly compresses the file with a fairly ineffective compression such as gzip. The Radiance file format came from research started in 1985 that was freely distributed beginning in about 1994. The original PIXAR TIFF had 3.6 orders of magnitude (3600:1) in 0.4 % steps (1% in Luminance is edge of human perception). As noted earlier, the Radiance format has 76 orders of magnitude, which is say 62 more than you need for any level of light. This handles imagery from the shadow under a rock on a moonless night all the way up to the Sun itself. It is therefore vastly over specified for HDRi work.
2002
In 2002, Industrial Light and Magic published C source code for reading and writing their OpenEXR image format, which has been used internally by the company for special effects rendering and compositing for a number of years. This format is a general-purpose wrapper for the 16-bit half data type, which has also been adopted by both NVidia and ATI in their floating-point frame buffers. Authors of the OpenEXR library include Florian Kainz, Rod Bogart (of Utah RLE fame), Drew Hess, Josh Pines, and Christian Rouet. OpenEXR has since grown to be the dominant format for HDRi files. The format can store negative and positive values at about 10.7 orders of magnitude (the human eye can see no more than 4 ). The OpenEXR specification offers extra channels for alpha, depth, and others, which makes it invaluable for high end effects work. As it is simply a 'container' , the float format that can be used for other things - such as optical flow - XY floating point confidence level. Debevec thought that ILM's release of Open EXR format was "really a great thing...they were putting the seal of approval of one of the best visual effects houses in the entire world on thinking about things this way and doing things in this sort of way".

One of the first films ILM used their new EXR format on was the 2002 Time Machine. "The new floating point allows us capture a larger dynamic range, " said ILM's Scott Squires - VFX sup in a cgw article, "because its floating point we can get more than 16 bits of colour per channel". At the same time - on the same film in fact Digital Domain was working on their own floating point point pipeline, but based on RGBE file formats. This lead to Digital Domain first adding f-stop tools to Nuke. At this stage Nuke was still exclusively an inhouse Digital Domain tool.

There is also a Microsoft format scRGB, but it is not widely accepted because the number of bits it uses provides poor accuracy. Prior to OpenEXR, one of the best float formats was the SGI LogLUV format, which was released in 1997. It was developed by Greg Ward while he was at SGI in 1997. Ward set about correcting the mistakes he felt he had made made with RGBE (Radiance file format) in hopes of providing an industry standard for HDR image encoding. This new encoding was based on visual perception and designed so that the quantization steps matched human contrast and color detection thresholds much more closely. It is a good format and was used in many 3D aps prior to OpenEXR.

Ward himself still uses both the RGBE and SGI LogLUV file formats. "For HDR photos, I prefer LogLuv TIFF because it's fast, accurate, and reasonably compact," says Ward. "I still use Radiance's RGBE format with Radiance, as retooling the whole package around LogLuv would be a ton of work for very little benefit. Either format provides sufficient accuracy for lighting design and human perception. OpenEXR is an excellent format for visual effects, as it has significantly higher accuracy equivalent to an extra decimal digit of precision over RGBE and LogLuv. This isn't strictly necessary for perception, but provides a guard digit for extensive compositing and blending operations as required in your industry. The only drawback of OpenEXR is that you can sometimes run out of dynamic range, but I believe the circumstances where this is a problem are rare. OpenEXR is also quite complicated compared to Radiance's format, and comparable to TIFF."

"]

There is a new format JPEG HDR - which uses a concept from the earliest days of moving from Black and White video to Colour NTSC Video. The use of 24-bit RGB to high dynamic range (HDR) images can be hindered by excessively large file formats sizes - with no backwards compatibility. Maryann Simmons, at Walt Disney Feature Animation, and Greg Ward proposed a simple approach to HDR encoding that parallels the evolution of color television from its grayscale beginnings. A tone-mapped version of each HDR original is accompanied by restorative information carried in a subband of a standard 24-bit RGB format. This subband contains a compressed ratio image, which when multiplied by the tone-mapped foreground, recovers the HDR original. The tone-mapped image data may be compressed, permitting the composite to be delivered in a standard JPEG wrapper. To naive Jpeg reading software, the image looks like any other, and displays as a tone-mapped version of the original. To HDR-enabled software, such as Photosphere, written by Ward, the foreground image is merely a tone-mapping suggestion, as the original pixel data are available by decoding the information in the subband. Thus the format provides HDR images - that can be read by normal Jpeg readers. The image below is such a jpeg HDR image - it should be displaying normally in your web browser- but taken into Photosphere - this image can have its exposure adjusted - a full range of fstops. This format could easily be the ultimate format for domestic HDR cameras, once camera technology advances, a Jpeg-HDR would be the ideal home photoformat - full back compatible and truly adjustable if you took the wrong exposure - while still only being extremely small in file size.

While others explored Photogrammetry or brilliantly extended the HDR formats, Debevec moved on to do his Light Stage research. The Light stage has evolved from its first appearance at Siggraph almost 6 years ago. At Siggraph 2000 Debevec presented Light Stage 1.0 as a way of capturing the appearence of a person's face under all possible directions of illumination. The original verison allowed a single light to be spun around on a spherical path so that a subject could be illuminated from all directions as regular video cameras were used to record the actors appearance. It literally worked by someone pulling ropes. It took a minute to capture one image of a face and the subject could not move, so often multiple attempts were made to get a good result.

Debevec and his team soon moved to LightStage 2.0 and the technology became more useable. The first major production to use the Light stage for a feature film was Spider-Man 2. In the first Spider-Man film, the villain wore a mask and rode a hover-board, making technical challenges much less daunting. In Spider-Man 2, the villain Dr. Otto Octavius (Alfred Molina), more commonly known as Doc Ock, has four octopus-like tentacles and doesn't wear a mask. Thus, the digital effects team at Sony Pictures Imageworks (SPI) had to add CG tentacles to the actor in 178 shots and create photorealistic Doc Ock digital doubles for 38 shots.

At this point in Light Stage's development, the reflectance field of a human face could be reliably captured photographically. The resulting images can be used to create an exact replica of a subject's face from various viewpoints and in arbitrary, changing lighting conditions. To create Doc Ock's face, Molina was seated in a chair and surrounded with four film cameras running at 60 frames per second. Above his head was an armature that rotated around the chair. On the armature, strobe lights fired at 60 frames per second. At the end of eight seconds, they had 480 images from each camera of his head in one position, but with different lighting conditions in each.

To create photorealistic movement for the actors' faces, SPI developed a system based on motion capture. However, it was not a motion-captured performance. The actors were recorded using Vicon motion capture cameras. The actors wore 150 tiny markers in his face, each marker a 1.5 diameter, retro-reflective sphere flattened on one side. From this a library of faces and expressions was built up.

To match the lighting on the set, the crew took high dynamic range images (HDRI) using fish-eye lenses to create 360-degree environments. Once a CG character's head was surrounded with those global environments, a software algorithm could calculate which of the 480 images from each camera to blend together to create the character's face. If the set was dark on one side, for example, the algorithm wouldn't access images taken on that side of the actor's face.

In the end, the HDR capture gave the lighting and the Light Stage provided the actors face which matched the lighting conditions. This was mapped on to a CG face with the appropriate expression according to the script. The Lightstage was still only grabbing one very complex shot of the head - and not the actor actually acting. The work was breakthrough, but still required someone to animate the action.

Several other important events happened around this time, which was a truly expansive period for HDRi. Lightstage 3 was shown for the first time at Siggraph 2002, with the now-familiar 156 LED dome. This recorded the performance in both visible and infra red light to produce an infrared based alpha or matte channel. Gone was the swinging rib of Lightstage 2 and in its place was a fixed dome.

Also around this time, Ward - who had briefly worked at Shutterfly - took 3 months off between positions and built a robust cataloging tool which includes an HDR generator called Photosphere. Photosphere is free like HDRshop, but unlike HDRShop Ward developed the program for the Mac OSX platform. While the program does many things, it introduced a series of advances for HDR. First, it reads the exposure information from the file header and thus does a dramatically faster job of calibrating the camera and producing the necessary camera response curve calibration. This was always a very slow and time consuming process on HDRshop. The technique was also adopted by Adobe when it introduced HDR creation to Photoshop CS2. "I read the exposure from the Exif header, which CS2 does also," says Ward, "but then I apply Mitsunaga and Nayar's polynomial solution rather than the SVD method of Debevec and Malik. I'm not sure which is more accurate, but for sure the poly solution is faster".

The other innovation of Photosphere was the automatic alignment and stabilizing of HDR images one to another. This feature was also incorporated in CS years later, but Photosphere is about 6 times faster. "Ironically, I offered to license my facilities to Adobe, but they hate not owning things outright," explains Ward. "They ended up spending nearly two years developing their own methods, which don't perform quite as well. The alignment algorithm is slow but passable, and their camera response recovery sometimes does strange things."

Greg Downing, having worked at REALVIZ in France on their Stitcher and Image Modeler teams, developed a method for HDR Stitching which was featured in Debevec's 2002 Siggraph image based lighting course. Downing would later move to Rhythm & Hues to help develop their second generation HDR pipeline and work as a lighting TD, including main character lighting, on Garfield the Movie.
2003
HDRi was used effectively for the Mysique character in X-Men 2 using a pipeline of image captures with a Sigma fisheye lens, Photoshop, HDRShop, and image-based lighting. HDRi had also been used briefly in the first X-Men movie where Senator Kelly dies and becomes a pool of water.
2004
At last year's Siggraph Version 2 of HDR shop was released which includes support for OpenEXR and using CameraRAW information rather than just Jpeg images to make extremely high quality HDRs. HDRShop 2.0 implements multi-threaded operations which speeds up processsing.
2005
Debevec's team has broken through the critical limitation that stopped the actor from being able to act, presenting a paper entitled "Performance Relighting and Reflectance Transformation with
Time-Multiplexed Illumination" at Siggraph 2005. The latest version, Light Stage 5 (there was not a Lightstage 4 ever shown) has turned into a giant bubble with masses of LEDs around the outside - all pointing in to the centre from every direction. With a high speed camera shooting an incredible 4800 times a second, the actor is filmed delivering a line of dialogue in real time with every possible lighting environment.

Since film runs at 24 frames per second Debevec explains "there is actually spare time in there to do other things." The actor acts and the lights flash incredibly quickly as the camera captures it all at over 2000 times a second. When done, pick any HDR lighting reference plate - the desert, the ocean, the original St. Peter's Besilica, whatever - and the computer can play the actor's actual performance with the correct lighting. It is not doing image processing or colour correcting but instead discards all the lights that wouldn't have been in the reference and plays the performance showing only the correct - exact - lighting to match. It even creates a black and white matte by running a few passes of the over 2000 captures that had the actor in perfect silhouette.

As you can imagine there are a host of limitations. The camera technology only allows for the actor to be captured at 800 x 600 resolution and only for 4.3 seconds before the RAM of the high speed camera fills up and you have to break and download the data to a hard drive. But various optical flow techniques are built into the process to allow for motion blur to be re added to the image and solve the otherwise extremely short exposure time of each "frame". There are also some issues with noise in the shadow areas and issues such as when the actor casts self-shadows, such as with a hand moving across their won face. Thse can produce a sort of slight ringing shadow.

In the closing paragraphs of this year's Siggraph paper, the authors hint at a hybrid system applied to "traditional studio lighting arrangements." Such a system would for the first time allow for NO lighting decisions to be made on-set during effects work and *any* lighting to be perfectly dialed in during post-production - matching the lighting of the environment the actor is composited into. In Light Stage 1 the team acquired 1800 light directions in about 1 minute but now the team can capture the performance with any lighting in real time.

The real world on-set version could perhaps be a system that would record fill, key light and rim light separately on set. This would allow a compositor to re-light a live action performance in exactly the same way a 3D artist does. According to Debevec, "it is analogous to an audio recording which records the various instruments of a band separately and then they are mixed together in audio post." The system would not simulate the lighting - or colour correction the image in a DI sense - it would literally show you the same performance with any combination of lights on or off. The result would not only look real, it would be *the* real lighting for that new composite.

At Siggraph 2005 the same team is planning on showing initial tests of recording not only the lighting of the performance and all lighting info in one pass, but also recording a combination of multiplex light sources and a geometry capture. Thee actor is recorded with all the lighting and all the geometry of the performance, allowing the team to derive light effects like venetian blind effects across the face or derive ambient occlusion passes and specular passes. Channels can be produced that allow the live performance to be adjusted like 3D such as making the skin look wet or more diffuse or colored - allowing for what Debevec calls "virtual make up" design for the first time ever.

As for the current time limit on each take, Debevec notes that "there is now a 24GB option for the camera which would allow for 25 seconds of capture at 800×600", so as technology advances, so to will the process become ever more viable.

In the years ahead this work will undoubtedly spread from the lab into mainstream film making. With the speed at which past innovations have been adpoted for practical situations, it most likey won't be long before this new technology enters mainstream film making.
Applications using HDR
In 2005 Autodesk released Toxik, unlike its older brother Flame, - Toxik was designed from the ground up to support HDR - OpenEXR pipeline. This includes full OpenEXR colour grading and compositing.
By far the most common compositor for OpenEXR film projects is Shake, who's floating point file option has allowed for its integration into most commerical feature film pipelines all around the world.
As mentioned above HDR tools were first added to NUKE in 2002 and these have been widely available for several years now.
Most 3D packages support both the importing of light probes and now most renders also output OpenEXR floating point files, most notably Renderman - which is the most widely used render engine for floating point feature pipelines.
HDRi APPLICATION OVERVIEW
HDR Shop v2

There are two versions of HDR Shop - the free version 1 and an updated commercial version that includes OpenEXR support, Ability to read RAW files directly, scripting and multi-threaded solutions. HDR shop - unlike many other program - allows for complex operations once the HDR is made. These include tools to help combine two captures in order to remove the Photographer in the reflection.

HDR Shop 2.0 is available under a commercial license or an academic license. HDR Shop 2.0 is not available for free download for non-commercial/personal use. The price for a commercial license for the executable program for a two-year term is $600 for a single-user.
http://www.hdrshop.com/
RADIANCE

Radiance is still today currently the only rendering system that seriously targets lighting and delighting design applications. For a while, LightScape (which became 3D Studio Viz) was a useful tool, but they've since abandoned lighting for more commercial markets. There's just not as much money in building design, as visual effects. The program's author Greg Ward jokes that "they spend billions of dollars constructing these things (buildings), but the design part is given short shrift every time".

Radiance has been used on quite a few of the designs done by Arup in London who used Radiance at various stages. You can also check out the work done by Visarc on their website, as they use Radiance almost exclusively. Recently, Ward himself has been working to help with the design of the new New York Times building designed by Renzo Piano Architects.

As far as Ward knows, Radiance has not been used on any big budget special effects films, as it emphasizes accuracy over beauty and makes the wrong compromises for film. "Numerous times people have asked me to tweak this or that to make the output more beautiful at the expense of physical accuracy," says Ward, "but that would make it just another rendering tool rather than what it is: a unique, validated lighting simulation tool."
http://radsite.lbl.gov/radiance/
PHOTOSPHERE

A brilliant image catalogue program, written by Greg Ward, for the Mac OSX that includes very fast and very accurate HDR tools, such as automatic HDR stabilization, image/camera calibration and multiple file export options. It is a technically accurate and much more powerful version of Photoshop album or iPhoto - but without some of the GUI sparkle.

http://www.anyhere.com
(not www.photosphere.com - which is an unrelated image library company)

PHOTOSHOP CS2

Photoshop now includes HDR creation tools, but they are best accessed via Photoshop and not the companion Bridge program. see the fxguide Photoshop CS2 HDR article
http://www.adobe.com/products/photoshop/main.html
EASYPANO PANOWEAVER

A cost effective PC stitching program that allows virtual tours via Tourweaver. Panoweaver is a professional tool that can stitch digital images and creates 360-by-360 panoramas, and then produce virtual tours, or output panoramas up to 10000*5000 pixels
http://www.easypano.com/index.html RESEARCH PAPERS
Light Stage 5
http://projects.ict.usc.edu/graphics/research/LS5/index-s2005.html

Performance Geometry Capture
http://projects.ict.usc.edu/graphics/research/SpatialRelighting/

Light Stage 3
http://www.debevec.org/Research/LS3/

Light Stage 2
http://projects.ict.usc.edu/graphics/research/afrf/

Light Stage 1
http://www.debevec.org/Research/LS/
REFERENCES

Reinhard, Erik, G. Ward, S. Pattanaik, P. Debevec,

High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting,
Morgan Kaufmann Publishers, San Francisco, 2005.

Seetzen, Helge, W. Heidrich, W. Stuezlinger, G. Ward,
L. Whitehead, M. Trentacoste, A. Ghosh, A. Vorozcovs,
"High Dynamic Range
Display Systems," ACM Trans. Graph. (special issue SIGGRAPH 2004), August 2004.

Ward, Greg, Maryann Simmons,
"Subband Encoding of High Dynamic Range Imagery,"
First Symposium on Applied Perception in Graphics and Visualization (APGV),
August 2004.

Ward, Greg, "Fast, robust image registration for compositing high dynamic range photographcs from hand-held exposures," Journal of Graphics Tools,
8(2):17-30, 2003.

Larson, G.W.,
"LogLuv encoding for full-gamut, high-dynamic range images,"
Journal of Graphics Tools, 3(1):15-31 1998.

Larson, G.W., "Overcoming Gamut and Dynamic Range Limitations in Digital Images," Proceedings of the Sixth Color Imaging Conference, November 1998.

Larson, G.W. and R.A. Shakespeare,
Rendering with Radiance: the Art and Science of Lighting Visualization,
Morgan Kaufmann Publishers, San Francisco, 1998.

Ward, G., "The RADIANCE
Lighting Simulation and Rendering System," Computer Graphics, July 1994.

# data[].attributes.attachment_link.description
It is hard to believe that Paul Debevec published his PhD just ten years ago. His work in defining high dynamic range images (HDRi) was almost immediately adopted by the major effects companies and expanded into one of the most significant visual effects areas of the last ten years. Debevec's...

# data[].attributes.attachment_link.readable_text
Next-gen studio, ZeroSpace in Brooklyn, NY has leveraged NVIDIA multi-GPU workstations for high-fidelity real-time workflows.

Part service provider and technology incubator, ZeroSpace specializes in a first-of-its-kind studio specializing in digital, virtual, and live content. With more than 50,000 square feet of rentable studio space, the innovative production and creative studio is home to one of the largest LED volumes on the East Coast, as well as a core team of 20 world-class artists, engineers, and technicians. Along with providing in-camera visual effects (ICVFX) services to clients. ZeroSpace offers performance capture, full-body photogrammetry, and volumetric capture. It is the only studio with both a fixed installed subway and car process in New York. The company also has an in-house virtual art department (VAD), and a real-time art department (RAD) team offering 3D and VFX services; all the while developing its own creative IP and proprietary technology.

Much of the studio's robust technical capabilities are achieved through strategic partnerships, and leveraging cutting-edge equipment and pipelines. The ZeroSpace engineering team is constantly exploring ways to improve internal workflows and move the overall virtual production industry forward. Most recently, this technology-first approach drove the studio to adopt Z by HP's new Z8 Fury desktop workstation, which supports up to 56 cores in a single CPU and up to four high-end GPUs.

"We received our first Z8 Fury in late June and it's already been a major lifesaver for many reasons. With it, we were able to simultaneously capture multiple performers using markerless motion capture for a crossover music video brand event," shared Evan Clark, ZeroSpace Head of Research.

Powering past software limitations

When a customer approached the company to do a motion capture shoot with a five-member band in an unrestrictive environment, the team at ZeroSpace was up for the challenge. However, the studio's AI-assisted markerless motion tracking software was only capable of capturing two performers simultaneously.

"We were trying to come up with a workaround. We worked closely with our partner, who was able to send us an early beta version of new software that could split the performer models on multiple GPUs, and then recombine them downstream. It was Linux-based, and we were already running Ubuntu on our Z8 Fury, so we decided to install it and give it a try. Like magic, we were able to split the workload across three NVIDIA RTX™ A6000 GPUs, allowing us to track five people simultaneously at 60 frames a second; it made a significant difference," Clark noted. "We were able to shoot all the content we needed in a single day; this was a game changer as anyone in the business knows that when it comes to film shoots, time is money."

Though the Z8 Fury supports up to four NVIDIA RTX GPUs, the team opted to leverage one of the card slots for a 25-gig networking adapter to run the array of 12 machine-vision FLIR cameras with sufficient bandwidth to receive all the frame data from the performers. Additionally, optical motion capture was used in tandem for prop tracking the performers' musical instruments. All data was fed through Epic Games' Unreal Engine, where it was synchronized and recorded. Clark found building out the new workflow to be relatively straightforward and had the setup tested and running in about a day. Typically, the studio's real-time pipeline for motion capture requires two or three machines; using a single box simplified the approach.

"If it wasn't for the Z8 Fury, we would've been out of luck because most of the machines we have in-house are all single NVIDIA GPUs. Fortunately, it gave us the absolute best baseline to work with; otherwise, we would have had to record the band in separate takes, which would have compromised their performance and taken away from the final result."

Evan Clark, ZeroSpace Head of Research.

Advancing real-time fidelity for ICVFX

Clark and his team have also been testing the Z8 Fury for ICVFX applications, including experimenting with the SMPTE ST 2110 protocol and Unreal Engine graphics splitting. He mused, "Pushing the level of quality on an ICVFX volume while maintaining high frame rates is the Holy Grail. How can you get enough definition to really make it look realistic without having to leverage camera tricks or any type of post-VFX work? There are limitations when you're trying to scale a real-time render engine to millions, -if not billions or trillions of pixels, but you need to get that level of fidelity to convince the eye that what it's seeing is real. With the density of the Z8 Fury and some of the new Unreal Engine features, we can run bigger scenes and have a really solid mechanism to split and re-merge the scene on a single machine in a very deterministic way."

Created in partnership with 4Wall Entertainment, ZeroSpace's sizable in-house LED volume is nearly 40 panels wide by eight panels tall, measuring 65 feet on the curve, with a total resolution of slightly more than 8K. Having multi-GPU machines for wall rendering makes it easier for the team to split up workloads, thereby reducing the amount of geometry run by a single game engine instance. This load redistribution enhances performance, enabling faster frame rates and better overall visual quality since the machine is managing a small subset of the task with the entirety of the compute resource.

"In-game engine ray-marching and ray-tracing techniques are really starting to shine because of multi-GPU workload distribution. With that additional resource overhead, you can allocate for dynamic lighting in a scene and be able to change scene parameters in real-time. As these types of scenes improve and the technology evolves, dynamic changes will be able to be made more effectively on stage during a shoot," said Clark. "Image-based lighting has also resurfaced as a technique. and requires significant processing and computing for pixel mapping to make a film shoot possible on a volume. When you have sufficient compute, it unlocks creative handcuffs for your artists."

Building on a strong foundation

Established in 2018 as a downtown Brooklyn immersive art space, ZeroSpace evolved into a production and incubation operation in 2020. At the outset of its virtual production journey, the team built custom machines, in some cases using consumer-class GPUs in the workstations. Opting to standardize on the Z8 Fury workstations freed the engineering team from building and maintaining workstations, allowing them to focus instead on workflow innovation.

"One major benefit of adopting Z by HP workstations is the level of support we receive. If we run into an issue, the HP technician is out right away. On film shoots, every minute is burning hundreds of thousands of dollars, so fast machine repair and replacement is a major advantage; you can't do that with custom machines," concluded Clark. "Our HP machines are the most reliable in the building, and the level of support provided by HP is fantastic."

The Z8 Fury configuration included:

3x NVIDIA RTX A6000

Intel Xeon W7-3455 24C / 2.5GHz

128GB (8x16GB) DDR5-4800 ECC RAM

2x1TB NVMe

Dual 10/25G NVIDIA Mellanox ConnectX-6

You can learn more about the extreme power and performance of the HP Z8 Fury, here.

ZeroSpace is a customer of HP and was provided with a machine for testing purposes.
© Copyright 2023 HP Development Company. L.P.

# data[].attributes.attachment_link.title
HP Z8 Fury G5 Supercharges ZeroSpace Virtual Production Pipeline - fxguide

# data[].attributes.attachment_link.description
Next-gen studio, ZeroSpace in Brooklyn, NY has leveraged NVIDIA multi-GPU workstations for high-fidelity real-time workflows.

# data[].attributes.attachment_link.readable_text
Final Pixel and CUBE Studio team up to launch an accessible virtual production offering, designed for UK brands and creative agencies.

Named Content Factory, the new and accessible virtual production package looks set to transform the future of content creation. An efficient solution, providing access to a library of virtual backgrounds, camera, lighting, crew, props, post-production and more, it empowers creators to capture multiple campaigns in one day within CUBE Studio's 7,500 sq. ft. state-of-the-art Virtual Production Studio.

Final Pixel, an award-winning global Creative Virtual Production and Innovation Works with expertise in producing, world-building technology development and education and UK-based Virtual Production Studio, CUBE Studio, have teamed up to launch the first Content Factory, which combines virtual production expertise, techniques and technologies into an accessible, comprehensive and highly approachable package of services designed to enable anyone to embrace and easily integrate virtual production into their content, branding and advertising strategies.

Designed with a focus on digital, influencer and product advertising, Final Pixel's Content Factory bridges the gap between ad-level budgets and one-day virtual production shoots, empowering creative visionaries to take their first steps into this new era of immersive content creation and embrace virtual production technology to bring their ideas to life.

Stop time, control the weather

Hosted at CUBE Studio's 7,500 sq. ft., state-of-the-art virtual production studio, and providing access to a library of immersive and photoreal virtual environments, highly skilled crew, camera, lighting and a range of props, this groundbreaking package allows creatives to capture multiple campaigns in just one day.

Delivering a triple-threat of speed, efficiency, and enhanced creative capability, virtual production represents a paradigm shift in the way commercials and brand campaigns are being created. With a growing demand across the advertising space for a streamlined virtual production solution, Content Factory enables brands to deliver Hollywood quality storytelling, without the blockbuster spend.

Hanah Draper, Global Head of Production and Final Pixel, said: "Content Factory means you can stop time, control the weather, the season, the country and reach creative ambitions - but without breaking the bank" adding "We've unearthed a way of democratising virtual production and simplified it into a nice, digestible package."

Very first studio to offer the Content Factory package

Adoption of virtual production techniques has been accelerated in recent years, as productions place increasing importance on sustainability. Calling for smaller crew sizes, reducing travel and enabling virtual collaboration throughout the production process, Content Factory allows creators to embrace sustainable production techniques while exploring innovative technology solutions.

Home to the first Final Pixel Content Factory, and the exclusive UK destination, CUBE Studio, located just outside of London, provides the perfect stage for creators and brands to level up their storytelling and redefine what's possible.

James Hakesley, CEO and Executive Producer at CUBE Studio said: "We've always been passionate about empowering content creators to push creative boundaries through access to the very latest technical and visual innovations, and the launch of Content Factory will make virtual production more accessible than ever before.

Editing, sound mixing and colour grading

"We're proud to be partnering with Final Pixel, as the very first destination to offer the Content Factory package which, we believe, will be a catalyst for growth for virtual production across the UK."

CUBE Studio is home to a 10.5×5.5m LED volume, crafted with Unilumin Upad IV LED panels, which feature HDR image quality, a pixel pitch of 2.6 mm and new DDC (dual drive configuration) technology, plus Brompton Technology processing, to deliver visual excellence.

Designed with comfort and experience at the forefront, CUBE Studio features:

5mx5m drive-in vehicle access to the LED stage, with unloading bay

Viewing platform with views of the LED stage

Production office, boardroom, kitchen and refreshment facilities

Spacious greenroom and breakout lounge for talent and clients

Dressing rooms and hair and make-up suites

On-site parking

Sohonet dedicated high-speed internet access

Fresh air ventilation system

Equipment and setbuild area

Just off the M4, 15 minutes from Heathrow and access to London via Elizabeth Line

Encompassing support from pre-production through to post-production, this new offering includes editing, sound mixing and colour grading to Final Pixel and CUBE Studio's high standards. Follow the link to find more about Content Factory.

Support ProVideo Coalition

Shop with

# data[].attributes.attachment_link.title
Content Factory, an accessible virtual production package by Jose Antunes - ProVideo Coalition

# data[].attributes.attachment_link.description
Final Pixel and CUBE Studio team up to launch an accessible virtual production offering, designed for UK brands and creative agencies. Named Content Factory, the new and accessible virtual production package looks set to transform the future of content creation. An efficient solution, providing access to a library of virtual backgrounds, camera, lighting, crew, props, [...]

# data[].attributes.attachment_link.readable_text
Cover image via

The English episodic is a cinematic homage to the Spaghetti Westerns of the late last century. It also posits how the cinematographers of that era might now film them using modern technology.

The show's cinematographer Arnau Valls Colomer had a time problem in shooting The English. He had 90 days to shoot six one-hour episodes of this American land-rush drama. He was also playing with an idea to skew the western aesthetic and move away from the wild west, the big skies, the solitude, and the wilderness. Luckily he changed his mind.

The Western Genre - an appreciation

Arnau not only dismissed the idea of a reinvention of the Western genre, but he also went full tilt into an appreciation of it and past treatments, one in particular. The English tell the story of the Europeans who joined the land grab of the 1800s in states like Oklahoma, Nebraska, and Wyoming - 'English' became a collective term for the many who sought this new land - the Native Americans called them 'locusts.'

Emily Blunt plays an English Lady on a mission to find her son's killer. She meets Chaske Spencer, an Indian scout looking to secure the few acres of land the American army promised him after his service. But the time was full of dangers and general lawlessness, including the continued violent depopulation of the American Indian.

Because of production and Pandemic reasons, Spain would substitute for the US but not the southern part of the country where the Spaghetti westerns were made but further north to duplicate the mid-west and the charting of the main character's journey north. Spanish native Arnau knew that his country's seasons and western and northern areas could complete the pretense.

"Where we shot was not so much the area of the Spaghetti Western. Our story happens between New Orleans and Nebraska, and after reading the script, I wondered how we would do it in Spain. The mid-west of America is so vast it seemed impossible to replicate it.

"But we decided to do it in Castilla-La Mancha where it was more like the south mid-west and more yellowy and flat. We then moved further north to Castile and León, where it's more rocky and green with more mountain areas."

The production added more digital mountains to complete the picture, and Arnau used silhouettes and framing techniques to hide the join. They cleverly knitted together landscapes to move the story on.

But why not use Virtual Production techniques to dial in these landscapes as they would be digital assets stored on local workstations? The new technology wasn't for them; however, the 'volume' of screens was never considered for The English. Luckily, Spain, as Arnau discovered, had locations that were largely without buildings as the old American mid-west was back in the late 1800s; the space was easy to sell to the viewers, and any builds could be done relatively easily.

Vintage lenses

But how was Arnau going to imitate while improving the cinematography of the Sergio Leone and Clint Eastwood era?

The key for him was to decide early how his cameras would move; he had three Sony Venice cameras married to large anamorphic lenses. A fast-moving camera design would perhaps not take in the landscape and linger on the views and the important relationship between the leads. A static camera would allow for both and also for the script to breathe. But more importantly, a static camera increased the speed of the set-ups vital to keeping to that schedule of six hours in 90 days.

A static camera would also act as a witness to the action, not chasing or following, but there was another advantage for Arnau. A camera that didn't move around too much allowed him to use some old but heavy lenses, especially some big anamorphic Panavision lenses from the 1980s. He was then helping to recreate that look with an ingredient that could have been in the original films, the glass.

These lenses also produced beautiful flares and bokeh but with some unusual focus breathing effects, which changed the shape of the bokeh depending on where the focus is. With such a set-up and a Sony Venice digital camera shooting Super 35mm 4K, Arnau produced stunning images on view, with some of the classic Western tropes like silhouettes, long shadows, and breathtaking widescreen scenery.

Modern production methods also helped when he needed to shoot night scenes. A night scene 40 years ago in the deserts of southern Spain would need a studio and a painted backdrop. Trying and shoot them for real would result in pitch blackness, and any light would disappear. But Arnau again mixed old and new methodology and shot the scenes in a studio so he could control the light but then composited the background from a day-for-night scene shot on location.

Aiming for the Technicolor Look

This homage to the old Westerns and this tribute to their look of them took Arnau into the grade where he decided to concentrate on the color of the skies and the skin and not use techniques like masking that wouldn't have been available back in the day.

He was trying to find a balance between the old-fashioned Technicolor style and the new image. Once he had found that the process became simpler, "We were just looking at the primary colors with density and contrast and not going too deep, keeping it simple."

As brilliant and technically impressive as virtual production is, using it for a cinematic western genre is perhaps missing the point. It has its place in every situation, some cityscape backdrops in The Batman, driving plates in Station 11, and balcony scenes in a rom-com, for example. Its original use in The Mandalorian was in peak Covid days, and the landscape was alien, so it needed deep treatments and live VFX.

The big Virtual Production shows like 1899 will gain the plaudits, but the majority of uses of the 'volume' will be for lesser shows and lesser reasons; that's where the significant growth will be.

For more on virtual production

# data[].attributes.attachment_link.title
Virtual Production is Great But it Doesn't Beat the Real Thing

# data[].attributes.attachment_link.description
The English episodic is a cinematic homage to the Spaghetti Westerns of the late last century. It also posits how the cinematographers of that era might now film them using modern technology.

# data[].attributes.attachment_link.readable_text
Our live seminar & roundtable on using HDR as a storytelling tool that took place on Friday night is now available to view online here https://www.youtube.com/watch?v=bJsz0foS0NI.

The evening featured a group of high-end cinematographers, including triple Oscar winner Vittorio Storaro, colourists and DITs

Chaired by Televisual's MD, James Bennett, the session explored the theoretical and practical aspects of HDR production with a stellar panel of luminaries.

Part 1 - a 20minute presentation looking at how luminance impacts colour tonality and with some observations about the electrical department's approach to Woody Allen's A Rainy day in New York

Simone D'arcangelo, DIT, cinematographer and CEO of Wildcat (credits include Café Society and A Rainy Day in New York

Anthony Raffaele, Colourist (credits include Café Society and A Rainy Day in New York)

Pablo Garcia, Head of Colour at Mission (Colour Supervisor for His Dark Materials and Downton Abbey)

Part 2 - a 20minute specially recorded presentation by Vittorio Storaro, ASC, AIC (triple Oscar-winning cinematographer with credits including The Last Emperor, Apocalypse Now & A Rainy Day in New York)

Part 3 - a 75minute panel discussion with colourists, DITs and cinematographers covering HDR preparation, on-set and grading best practice with D'arcangelo, Raffaele, Garcia now joined by:

Asa Shoul, Colourist at Warner Bros De Lane Lea (credits include Mission Impossible - Fall Out and The Crown & Devs)

Roberto Schaefer, ASC, AIC, Cinematographer with credits Quantum of Solace and Finding Neverland)

Angus Hudson, BSC, Cinematographer with credits as additional DoP Star Wars: Episode VIII - The Last Jedi and Mary Queen of Scots, and cinematographer for You Should have left

And Eben Bolter, BSC a cinematographer with credits including Avenue 5 and The Feed

Head to https://www.youtube.com/watch?v=bJsz0foS0NI to watch the sessions

Share this story

Share Televisual stories within your social media posts.
Be inclusive: Televisual.com is open access without the need to register.
Anyone and everyone can access this post with minimum fuss.

# data[].attributes.attachment_link.title
HDR as a Storytelling Tool: seminar now online - Televisual

# data[].attributes.attachment_link.description
A live seminar HDR as a storytelling tool is now available to view online featuring high-end cinematographers, colourists and DITs

# data[].attributes.attachment_link.readable_text
As we reported in 2020, disguise is a platform for creatives and technologists to imagine, create and deliver spectacular visual experiences. The company expanded its focus from primarily live events such as mega live concerts to complex onset projection projects on films such as Solo: A Star Wars Story, and now to LED soundstage volumes.

As LED Stages have grown in size, the technology has had to develop. In the latest advance from disguise, the company is building on the work on Epic Games Unreal nDisplay to offer cluster rendering. The company is also actively facilitating the latest trend in multi-volume display setups, such as fxguide reported ILM has separately been doing on season 2 of The Mandalorian.

Cluster renders

disguise's cluster rendering separates the physical binding of rendering and screen connectivity with LED volumes and it works in concert with Epic's nDisplay. In fact, cluster rendering was partly developed thanks to an Epic Games' MegaGrant. The system allows the real-time LED displays engine configuration, synchronization, content distribution, and computer server management to be offloaded to disguise software. This allows scaling of LED stages as the scene on the screen can now be split several ways over multiple render systems.

Cluster rendering puts a separation layer between rendering and technical delivery. It allows teams to cut their LED stage content up into multiple slices, distributing the workload across multiple machines. All slices are then delivered to the LED screens as one coherent piece, without introducing awkward latency issues.

Clustering can be split over multiple render systems by cutting the canvas, by object or by plates. Clustering by objects is the process of rendering background objects on one node and rendering the foreground object on a different node. The new disguise xR only solution allows team to divide the virtual world into any number of plates which are then each rendered on different nodes.

The system provides near-linear scaling with render complexity, along with correctly mapping to the actual studio space.

disguise aims to separate the rendering of content engines such as UE4 from the technical delivery of final pixels to the LED volume. By separating these two parts of the system in virtual production, disguise can scale them independently. Given the complexity and growing size of LED volumes, it is becoming important to scale up without exploding cost, complexity or sync/render problems from adding additional render points to the LED sound stages.

disguise does this by conceptualizing the two roles of creative content and LED render as independent parts of the systems. "Our rx nodes perform the rendering for 'channels' of content, which can be a hero viewing frustum of a camera, an out-of-frustum render, or any additional passes needed for AR, lighting, etc. Those channels are rendered on the nodes, running UE4, and streamed over our RenderStream protocol into the disguise software," explains disguise's Peter Kirkup who heads up Global Technical Solutions. The disguise rx system is their dedicated system for hosting third-party render engines such as UE4.

As with their original live concert work, at the heart of the LED workflow is the company's disguise Designer software, which manages how the whole LED stage will be visually mapped, including an LED overhead or floor panels. Once a project arrives in disguise Designer software, and the system has calibrated the tracking system, it can be mapped to all the video surfaces using the production team's specific content mappings. The content mappings can reproject content from the camera's frustum but can also be mapped using other techniques. The software also supports spherical reprojection, pixel-accurate feed mapping, or direct mapping into the UVs of a UE4 object. These channels are then composited with any other content (such as plate playback) and the final LED pixels are calculated for the screens.

All the channels can be fed through the ACES pipeline and can have independent input (and output) transforms - so the UE4 render running in ACEScg could be composited with a PQ (Perceptual Quantization) plate shot on-site, and output as a DCI-P3 signal to the LED processing. As the system does the composite in linear space, users can set the input and output transforms per content layer.

The system can then drive the physical LED panels through a powerful multi-server output mapping system - meaning that segments of the LED screen can be connected to different disguise 'actor' systems and all the synchronization is taken care of by using the company's internal server to server communications (d3net) together with an external sync source. The wall is then running in sync with both the camera and the volumetric tracking system that maps the position of the camera.

An actual setup would be more accurately mapped as:

Overall, what this means is that disguise can run any sized LED volume using any amount of render nodes, and the two can scale independently. Channels can be sliced up and rendered across a group of nodes using the UE4 nDisplay architecture. For example, the hero frustum of a camera could be allocated to 3 nodes to provide enough render power to hit the required framerates and image quality, "enabling the creatives to 'turn it up to 11' with no need to scale back on content because there's not enough power in a single GPU," explains Kirkup. "If creative team hits a limit on the GPU rendering, adding another render node to the system provides near-linear scaling of render power - whilst not affecting the technical delivery of pixels to the LED screens".

Multi-volume displays

As stages evolve there is a new move to multi-volume display stages. This allows for actors to move between volumes and not just be encased inside one circular LED set of screens. But as both complexity and sheer numbers of screen expand, solutions such as disguise render clusters become vital to provide the vast amount of scaled performance.

It is relatively easy to get video output from the Unreal engine and push it to a set of LED panels. "When you start wrapping the LED volume around something you expect the lighting, the reflections, and all the objects to correctly appear spatially in the right place," explains Kirkup. "You can't just do that so much anymore. Uh, because as you move, you're not moving the camera in a plane and re-projecting onto complex surfaces. You're now moving within volumes of volumetric space. disguise thinks of the studio space as a 3D volume and projects content into that volume according to various frustums needed by production," comments Kirkup. The system starts in a normal LED space with a pixel map, which represents the production needs. Camera frustums are then defined which represent hero cameras, and additional render passes can be added to cover environmental lighting and out of frustum LED pixels. Without this as the real camera moves it would not get the right light play and surface reflections correctly across the physical elements in the scene.

LED panel lights alone are not all that is used in an LED volume. The disguise system also incorporates DMX controls for the non-LED studio lighting system. This combined with the correct spatial reconstruction inside disguise's spatial mapping system is key to the LED sound stage volumes becoming a truly immersive space. "We're not talking about the technical implementation too widely, just yet," comments Ed Plowman, CTO at disguise. "But we are trying to merge the lighting into one. A volumetric representation is not a straight mapping of the UE planes to the panels. A volumetric mapping uses environmental mapping techniques and that's the precursor to being fully able to track the light from the virtual environment onto the stage and match physical to virtual lighting and virtual to physical lighting." From their extensive concert work, disguise has done a lot with DMX and lighting control systems. "We understand how to control the lights, where they are, what they're doing, what the settings are, as we have done so much work previously on lighting visualization," he adds. "This allows us to address how to compensating for physical lights in the virtual space."

For multi-volume display setups in a studio fall naturally out of this advanced model, "From disguise's perspective, we don't treat it as one volume or some other type of volume, they are all just screen surfaces that are being mapped based on whatever the camera's sees," comments Kirkup. While the multiple volumes need to be accurate in volumetric lighting, multiple screens viewed differently from different angles are at the core of disguise's company DNA. "And I think that's why this has become really exciting for disguise," Kirkup adds. "We feel that we've really got something very unique in our understanding of how these things can be manipulated and interact." Tom Rockhill, CSO at disguise, adds that not only does the company understand stage from a mathematical perspective, but the company also knows from their live concert work, "the need to get something right. Because you've got talent and everything set at that moment, and you don't have time to go start messing around the video files. That's why we again think that migrating this workflow more into film production is a natural and valuable step for the company."

Not all LED screens are created equal

Naturally, there are many screens currently being installed, and some are just effectively rough live or dynamic backgrounds which may or may not be accurate. For example, an LED wall behind a car for a 'through the window' shot. In these cases, if it looks OK it probably is, but disguise offers a far greater level of accuracy and colorimetry confidence. It offers a DOP a precise reproduction tool. But to do this there are a host of issues to be addressed.

LED panels are actually well set up for handling HDR comments Plowman, but he also points out that to get LED capture volumes displaying correctly requires a lot of work to be carefully done in the calibration of the whole pipeline. "Basically, I've got three separate color representation systems that I have to tie together to get a stable output," he explains:

1. the colorspace of the render,

2. the camera's colorimetry (everything including the dyes in the camera's CMOS chip) and

3. the LED processors themselves, that control and modify the LEDs on the back of the LED panels themselves.

This last aspect has often been overlooked points out Plowman, but it is why there has been a rise of companies producing specialist LED processors such as Brompton Technologies and Megapixel VR (Helios). "The biggest problem we have at the moment is not getting LED panels that were designed for this purpose. You're getting LEDs that were originally designed to be an exotic giant TV set. The original mainstay for these things was building giant jumbotrons." LED panels may be said to be exact in their color reproduction, but this assumes averaging. In other words, some LEDs might be up in value some down but the final aggregate is to spec. This may not work well in cases where only part of a screen is visible. It also means there can be real spikes in the spectral response curve over the range of visible light frequencies. These spikes again are assumed to cancel each other out and average to a manufacturer specification.

If that was not enough, LEDs were also designed to be viewed at a fair distance, and from directly facing the screen primarily. Many current LED calibration systems for LED stages also align their color spaces by pointing the actual camera directly at a section of the screen and then resolving a uniform color representation. What disguise is doing for its volumetric calibration is calibrating and adjusting for the angle the camera is to the screens. This is far more important than it might first sound. If the LEDs in a panel are not all perfectly flat, if say the red LED is slightly higher in the actual plastic mold, then even with the LED technically displaying flat uniform white, as the camera moves to a glancing angle on the screens, the color will tint and look pinkish. disguise solves for the actual observation point of the camera relative to the beam coming out from the light source, - from the LED itself. The term Volumetric Calibration is so much more than a marketing term, it accurately reflects a calibration for where the camera is and its precise angle to the screens, even in complex and multi-volume LED stages.

Even pitch varies

The most common question regarding any new LED stage is the pitch of the LED screens. A smaller pitch means the LEDs are denser, have less chance of moire, and are generally more expensive. Ed Plowman points out that even when it comes to looking at this most universal of metrics for LED stages, things are not always straightforward.

The pitch of the LED screen reflects the density of the LEDs as it indicates the separation between LEDs. While this may seem unambiguous, Plowman explains that "density pitch is actually measured as the pitch of the packaged devices on the sub-assembly, not how the actual LEDs are arranged." The quoted number of LED pitch is between the outside of the packaging of the LED not the LED itself. "If you've got a sparse field of LEDs in packaging, you could have a "two-millimeter" pitch LED wall but get a drastically different output." In other words, a production may think they are getting a standard 2.84mm pixel pitch LED wall, but if the panel maker is using inferior LED assemblies, it would look more like a 3mm+ pitch screen in reality. "When people think of an LED, they think of a thing in a plastic bubble on the end of two-wire connectors, you apply a current and it lights up. The LEDs in virtual production are embedded inside a package. They are sold as a sub-assembly, which creates the panel. Pitch density, is actually measuring the density of the LED package with a lens in front of it, not the density of the LEDs inside and their arrangement."

Tom Rockhill believes there's a massive disparity between various LED stages, "there's definitely a big sliding scale of expertise, - and just ambition quite frankly," he comments. "Some people are just happy to use LED panels to capture reflections on a car and are just not building precise calibrated systems." The issue with those simpler stages is that one is the possible lack of consistency. There could be odd fall off of colors, as there is not detailed color management and this, can make it very hard for VFX crews to match and reproduce elements exactly. In some respects, parallels can be drawn with green screens, while VFX-post teams will normally find a way to finish a shot from poor green screens, when more care is taken onset it can dramatically affect schedules and the ease of achieving the creative vision of the DOP and Director. disguise aims to provide reliability and incredibly high confidence about what is being shot when working on real-time LED stages.

Update:

The Carlyle Group & Epic Games take major interests in disguise.

disguise announced last week that it has new backing from The Carlyle Group which will allow the company to enter a new phase of its growth development. The company statement stated that "disguise has experienced tremendous growth over the past five years. It has led the drive for immersive and spectacular concerts and live events, become a market leader in xR (Extended Reality) and a key player in Virtual Production for Film and Television. In the past year, over 150 disguise xR stages have been built in more than 35 countries and disguise xR has powered more than 200 events."

disguise will now be supported by The Carlyle Group, one of the world's most diversified global investment firms with $246 billion of assets under management who will take a majority stake in the company, and by Epic Games who take a minority stake. In the press release the company confirmed that "led by CEO Fernando Kufer, disguise employees will also retain a significant minority share, with Founder Ash Nehru still the largest shareholder amongst them. Existing Private Equity partner Livingbridge will also reinvest in the transaction.

"The virtual production market is forecast to grow substantially, and we believe that disguise is uniquely placed to benefit from the accelerated demand for LED-based visual experiences and capture further market share. In partnering with the disguise management team, we will look to leverage our significant expertise in scaling media technology companies as we support the company to become a global leader."Michael Wand, Managing Director and Co-Head, Carlyle Europe Technology advisory team.

Epic Games will take a minority stake in disguise, enhancing an already strong partnership between the two companies. Production companies and broadcasters are embracing the benefits of virtual production techniques, built on photorealistic real-time graphics engines such as Epic's Unreal Engine, and LED infrastructure for immersing presenters and performers in virtual environments.

# data[].attributes.attachment_link.title
Disguising Virtual Production Rendering - fxguide

# data[].attributes.attachment_link.description
A deep dive on disguise's cluster rendering and advances for virtual production

# data[].attributes.attachment_link.readable_text
The film used to launch Red Bull's newest F1 car to the world's press, showed it on a fantastic road trip across America but also on the Moon... without leaving the studio in the North of England.

At the beginning of 2023, Final Pixel created a promotion that took Oracle Red Bull Racing's RB19 from Manhattan Nevada to Manhattan New York - all from the comfort of a studio in Wakefield, Yorkshire. The car in which Max Verstappen looks set to win the Championship made the short journey to Production Park's state of the art virtual production studio in Wakefield in January. A unique journey was about to begin, for the car and all those involved.

As Final Pixel notes in an Instagram post, "Earlier this year we creatively produced end-to-end, from initial concept and creative all the way to Final Pixel for Red Bull Racing & Red Bull Technology using Virtual Production. With our partners at XPLOR we took a Formula 1 car on a road trip without leaving the studio. The film was used to launch the car to the world's press, driving a new era of innovation and immersive brand engagement using VP."

The Formula 1 car was driven through the desert, the populated roads of Las Vegas at night, over a Miami highway, past the Kennedy Space station, 23,8855 miles to the moon and back, then to the New York Classic Cars Club over Brooklyn bridge - without leaving the studio in the North of England. The shoot blended real archive footage, video plates and drone footage with Unreal Engine produced scenes, CGI and VFX.

Virtual Production was the perfect solution

Phil Adlam, the Head of XPLOR at Production Park said about the shoot: "Hosting an F1 car at Production Park for the first ever Formula 1 virtual production shoot was a dream job for our team at XPLOR. We have some of the best VP and Unreal Engine technicians in the world, and it was fantastic to see what was achieved with our partners Final Pixel and Red Bull Racing."

"It's also great to work on a project that shows the exceptional potential of what we can deliver from our studios here at Production Park, and we are excited to see how we can develop it as a valuable resource right at the heart of Yorkshire's creative sector."

Michael McKenna CEO & Director of VP at Final Pixel said: "Virtual Production was the perfect solution here because it allowed us to do things with the car and show it in a way that otherwise would have been totally unachievable. We took Oracle Red Bull's RB19 through the Nevada desert, into Miami and past the Kennedy Space station - all without leaving the comfort of the studio in Wakefield!

Impossible to shoot on location

"Not only did virtual production aid the sustainability initiative of a carbon net zero production, it also solved numerous pain points for the client that rendered the creative brief, a road trip across America, impossible. The embargoed Oracle RB19 car was of great confidentiality and had not been seen by the public anywhere, so was impossible to shoot on location. The technological advancements that we have built a business out of pursuing made the impossible possible. It is Final Pixel's mission to open this opportunity up to the many brands and storytellers that will benefit from this craft''

Production Park is also home to the Academy of Live Technology, which offers degree programmes as well as a range of short courses and bespoke training in everything from virtual production to stage rigging and lighting design.

Final Pixel Academy is the global training arm of Final Pixel, providing online and in person training from experts working in this specialised approach to filmmaking.

Support ProVideo Coalition

Shop with

# data[].attributes.attachment_link.title
The first ever Formula 1 Virtual Production shoot by Jose Antunes - ProVideo Coalition

# data[].attributes.attachment_link.description
The film used to launch Red Bull's newest F1 car to the world's press, showed it on a fantastic road trip across America but also on the Moon... without leaving the studio in the North of England. At the beginning of 2023, Final Pixel created a promotion that took Oracle Red Bull Racing's RB19 from [...]

# data[].attributes.attachment_link.readable_text
Enter the Volume

"The Mandalorian" was a pretty good show. On that most people seem to agree. But while a successful live-action Star Wars TV series is important in its own right, the way this particular show was made represents a far greater change, perhaps the most important since the green screen. The cutting edge tech (literally) behind "The Mandalorian" creates a new standard and paradigm for media - and the audience will be none the wiser.

What is this magical new technology? It's an evolution of a technique that's been in use for nearly a century in one form or another: displaying a live image behind the actors. The advance is not in the idea but the execution: a confluence of technologies that redefines "virtual production" and will empower a new generation of creators.

As detailed in an extensive report in American Cinematographer Magazine (I've been chasing this story for some time, but suspected this venerable trade publication would get the drop on me), the production process of "The Mandalorian" is completely unlike any before, and it's hard to imagine any major film production not using the technology going forward.

"So what the hell is it?" I hear you asking.

Meet "the Volume."

Formally called Stagecraft, it's 20 feet tall, 270 degrees around, and 75 feet across - the largest and most sophisticated virtual filmmaking environment yet made. ILM just today publicly released a behind-the-scenes video of the system in use, as well as a number of new details about it.

It's not easy being green

In filmmaking terms, a "volume" generally refers to a space where motion capture and compositing take place. Some volumes are big and built into sets, as you might have seen in behind-the-scenes footage of Marvel or Star Wars movies. Some are smaller, plainer affairs, where the motions of the actors behind CG characters play out their roles.

But they generally have one thing in common: They're static. Giant, bright green, blank expanses.

One of the most difficult things for an actor in modern filmmaking is getting into character while surrounded by green walls, foam blocks indicating obstacles to be painted in later and people with mocap dots on their face and suits with ping-pong balls attached. Not to mention everything has green reflections that need to be lit or colored out.

Advances some time ago (think prequels-era Star Wars) enabled cameras to display a rough pre-visualization of what the final film would look like, instantly substituting CG backgrounds and characters onto monitors. Sure, that helps with composition and camera movement, but the world of the film isn't there, the way it is with practical sets and on-site shoots.

What's more, because of the limitations in rendering CG content, the movements of the camera are often restricted to a dolly track or a few pre-selected shots for which the content (and lighting, as we'll see) has been prepared.

This particular volume, called Stagecraft by ILM, the company that put it together, is not static. The background is a set of enormous LED screens such as you might have seen onstage at conferences and concerts. The Stagecraft volume is bigger than any of those - but more importantly, it's smarter.

See, it's not enough to just show an image behind the actors. Filmmakers have been doing that with projected backgrounds since the silent era! And that's fine if you just want to have a fake view out of a studio window or fake a location behind a static shot. The problem arises when you want to do anything more fancy than that, like move the camera. Because when the camera moves, it immediately becomes clear that the background is a flat image.

The innovation in Stagecraft and other, smaller LED walls (the more general term for these backgrounds) is not only that the image shown is generated live in photorealistic 3D by powerful GPUs, but that 3D scene is directly affected by the movements and settings of the camera. If the camera moves to the right, the image alters just as if it were a real scene.

This is remarkably hard to achieve. In order for it to work, the camera must send its real-time position and orientation to, essentially, a beast of a gaming PC, because this and other setups like it generally run on the Unreal engine ( Epic does its own breakdown of the process here). This must take that movement and render it exactly in the 3D environment, with attendant changes to perspective, lighting, distortion, depth of field and so on - all fast enough so that those changes can be shown on the giant wall nearly instantly. After all, if the movement of the background lagged the camera by more than a handful frames it would be noticeable to even the most naive viewer.

Yet fully half of the scenes in "The Mandalorian" were shot within Stagecraft, and my guess is no one had any idea. Interior, exterior, alien worlds or spaceship cockpits, all used this giant volume for one purpose or another.

[gallery ids="1949115,1949123,1949122,1949124"]

There are innumerable technological advances that have contributed to this; "The Mandalorian" could not have been made as it was five years ago. The walls weren't ready; the rendering tech wasn't ready; the tracking wasn't ready - nothing was ready. But it's ready now.

It must be mentioned that Jon Favreau has been a driving force behind this filmmaking method for years now; films like the remake of "The Lion King" were in some ways tech tryouts for "The Mandalorian." Combined with advances made by James Cameron in virtual filmmaking, and, of course, the indefatigable Andy Serkis's work in motion capture, this kind of production is only just now becoming realistic due to a confluence of circumstances.

Not just for SFX

Of course Stagecraft is probably also the most expensive and complex production environments ever used. But what it adds in technological overhead (and there's a lot) it more than pays back in all kinds of benefits.

For one thing, it nearly eliminates on-location shooting, which is phenomenally expensive and time-consuming. Instead of going to Tunisia to get those wide-open desert shots, you can build a sandy set and put a photorealistic desert behind the actors. You can even combine these ideas for the best of both worlds: Send a team to scout locations in Tunisia and capture them in high-definition 3D to be used as a virtual background.

This last option produces an amazing secondary benefit: Reshoots are way easier. If you filmed at a bar in Santa Monica and changes to the dialogue mean you have to shoot the scene over again, no need to wrangle permits and painstakingly light the bar again. Instead, the first time you're there, you carefully capture the whole scene with the exact lighting and props you had there the first time and use that as a virtual background for the reshoots.

The fact that many effects and backgrounds can be rendered ahead of time and shot in-camera rather than composited in later saves a lot of time and money. It also streamlines the creative process, with decisions able to be made on the spot by the filmmakers and actors, since the volume is reactive to their needs, not vice versa.

Lighting is another thing that is vastly simplified, in some ways at least, by something like Stagecraft. The bright LED wall can provide a ton of illumination, and because it actually represents the scene, that illumination is accurate to the needs of that scene. A red-lit interior of a space station, and the usual falling sparks and so on, shows red on the faces and of course the highly reflective helmet of the Mandalorian himself. Yet the team can also tweak it, for instance sticking a bright white line high on the LED wall out of sight of the camera but which creates a pleasing highlight on the helmet.

Naturally there are some trade-offs. At 20 feet tall, the volume is large but not so large that wide shots won't capture the top of it, above which you'd see cameras and a different type of LED (the ceiling is also a display, though not as powerful). This necessitates some rotoscoping and post-production, or limits the angles and lenses one can shoot with - but that's true of any soundstage or volume.

The size of the LEDs, that is of the pixels themselves, also limits how close the camera can get to them, and of course you can't zoom in on an object for closer inspection. If you're not careful, you'll end up with Moiré patterns, those stripes you often see on images of screens.

Stagecraft is not the first application of LED walls - they've been used for years at smaller scales - but it is certainly by far the most high-profile, and "The Mandalorian" is the first real demonstration of what's possible using this technology. And believe me, it's not a one-off.

I've been told that nearly every production house is building or experimenting with LED walls of various sizes and types - the benefits are that obvious. TV productions can save money but look just as good. Movies can be shot on more flexible schedules. Actors who hate working in front of green screens may find this more palatable. And you better believe commercials are going to find a way to use these as well.

In short, a few years from now it's going to be uncommon to find a production that doesn't use an LED wall in some form or another. This is the new standard.

This is only a general overview of the technology that ILM, Disney and their many partners and suppliers are working on. In a follow-up article I'll be sharing more detailed technical information directly from the production team and technologists who created Stagecraft and its attendant systems.

# data[].attributes.attachment_link.title
How 'The Mandalorian' and ILM invisibly reinvented film and TV production | TechCrunch

# data[].attributes.attachment_link.description
"The Mandalorian" was a pretty good show. On that most people seem to agree. But while a successful live-action Star Wars TV series is important in its

# data[].attributes.attachment_link.readable_text
"The entire motivation for Pinscreen is a comprehensive system where you can enable interaction with virtual humans," explains Pinscreen founder Hao Li.

Pinscreen is at the forefront of Neural Rendering and digital human research that uses Machine Learning (ML). The LA-based company is now the full-time focus of Professor Li, who has just resigned as the director of the Vision & Graphics Lab at the USC Institute for Creative Technologies, and Associate Professor at USC Viterbi School of Engineering.

The company has two key streams of research, both involve technology built on their PaGAN and PaGAN II Generative Adversarial Networks (GANs). The first involves making digital humans from data as simple as a single JPEG image to puppet as an avatar. The second R&D area involves the end to end process of making high-quality Agents using both cutting edge rendering technology and state of the art AI which infers what a face should look like and uses Neural Rendering. By combining real-time rendering and technology similar to DeepFakes approaches, Pinscreen is set to deliver stunningly real and interactive digital humans.

Hao Li will be presenting the company's latest work as part of the second day of the online Real-Time Conference (RTC), June 9th. His presentation is part of the session co-chaired by fxguide's Mike Seymour with Facebook's Christophe Hery. Building from the successful April RTC session on TelePresence / Avatars / Agents / Digital Humans, the June 9th event will focus on how ML can help in the creation and application of digital humans. Experts from New Zealand, USA, Israel, and Australia will all discuss the cutting edge new approaches to digital humans using advanced AI. Machine Learning, GANs, and Deep Learning looks to greatly influence face replacement, de-aging, digital makeup and character creation as these new Neural Rendering AI technologies become a part of a professional digital human pipeline.

Generative rendering using GANs and other similar approaches have shown to be able to produce remarkably accurate and realistic results, but full generative approaches have almost no artist control parameters. Neural rendering aims to use these ML tools but introduces controls to guide the process. Full computer generative rendering avoids long render times and a vast amount of artist time needed to create traditional 3D models. Generative rendering swaps laborious artist time for lengthy computer training time, but the final rendering is able to run very quickly. It can do this as it infers the lighting, performance, and responses for the final result. This comes at the price of not having fine-grained artist controls or keyframe parameters that an artist can adjust. Neural Rendering, based on similar ML, aims to sit between traditional CGI approaches and fully automated generative rendering approaches. Neural rendering uses deep neural networks to solve digital people while still enabling some explicit control of the final render. This translates to artist enabled control over illumination, camera parameters, posing, appearance, motion, or lipsync dialogue.
A PaGAN impersonation of the actor Sean Bean
Neural rendering techniques are diverse, and they build on seminal work on GANs by Ian Goodfellow (now Director of ML at Apple) by combining GANs with Variational Autoencoders (VAEs). A typical neural rendering approach takes as input images corresponding to certain scene conditions (for example, viewpoint, lighting, layout, etc.), builds a "neural" scene representation from them, and "renders" this representation with new scene properties to synthesize a new face, body or scene.

Face Replacement

To understand the far-reaching consequences of Pinscreen's research into end to end agents it is best to start with face replacement. This type of technology avoids the typical CGI pipeline of model/texture/light/render and instead the ML program infers the image of what the new face would look like if it was positioned and lit by the original background face. What is important to understand is that the new face is neither modeled as a mesh nor lit with 3D lights to illuminate it. The face is implicit in look, expression, and position. In other words, it never explicitly works out the new face's geometry, rigging or 3D position. Instead, it learns both the original background face and the face of the new subject. It then statistically approximates what the next frame should look like.

This statistical approach first became widely known due to the non-commercial Deepfake program. Pinscreen does not use the Deepfake software but like it, their proprietary PaGAN II core software infers the next frame during a face replacement and can produce exceptional results thanks to years of research into ML. In a face replacement such the Sean Bean demonstration above, there is a one to one mapping. One target and one subject, each trained on for over a day and then rendering in less than a second a frame.

Li's plan is not just an end to end digital human creation system, but one where faces are generated using AI and rendered with a neural rendering approach. "In terms of scale, we're actually building a whole platform that allows the entire avatar to live on a cloud and to be streamed directly to people".

Pinscreen originally built PaGAN to solve the problem of creating a 3D mesh face from just a single Jpeg picture. "PaGAN was written a couple of years ago when we introduced this method, it was to intended to use just a single input image, but have the ability to generate photorealistic faces," say Li. "But, we've actually progressed from there,...One of the first things we changed is the ability to not just learn from a single picture, but from multiple pictures or even videos as training inputs. He did this in order to facilitate faces that are more personalized and higher quality. The second thing we have done is to focus on real-time. If you want to make a virtual assistant, you can't use pre-rendered content as people have done with things like Deepfakes. You have to generate something that can instantly produce the face so that you have the ability to interact with it," he explains.

The next step was for Pinscreen was to build an approach that doesn't just map from a known person to another known person but allows the program to handle a 'Many to One' mapping. The many to one mapping means that "any person can appear, with specific training, and we can turn their face into a photorealistic face". The first demonstration of this new instant approach was shown publicly at the world economic forum in Davos, where Pinscreen showed their real-time face-swapping demonstration. fxguide reported on this demonstration in Switzerland earlier year. For this demo, any person could use the system without pre-training, and yet still to be able to generate a photorealistic face interactively.

To accomplish this, the system needs to be able to track a face while isolating expressions. "The important thing normally is knowing all the details of the face that is being mapped, ...which normally means training data," he says. The original deepfake approach to face replacement is a one to one method, with training on both the input subject and the target. With Pinscreen's live approach, there was not time to train on the subjects face, since it needed to work on anybody who walked up. "We dealt with this approach by training another network to have the ability to handle lots of people's faces, and then map all those faces to a very specific intermediate representation. From that (intermediate) we can infer to a specific target subject that we want". In summary, to achieve a real-time instant face swap, that appeared to need no training, first the process tracks someone's face (similar to original PaGAN I). The process analyzes their face and expressions, but it has been pre-trained on how a large number of people would map to a special internal intermediate specific 'character' at the core of the software. And this internal data model, "allows us to do, at inference time, handle any person and generate a compelling likeness of the target subject" Li explains. Having the PaGAN II neural network do this implicitly allows Pinscreen to both be fast (real-time) and appear to work without source training data and yet still produce a very realistic and accurate final output. While PaGAN I worked with anyone (even from a mobile phone app), the results were less realistic. With PaGAN II, it is actually pre-trained so it can handle everybody and yet Pinscreen has the ability to infer(render) in real-time, with most of the aspects of the face being implicitly solved and completely skipping traditional CGI methods. The new PaGAN II pipeline provides a higher quality output but still runs in real-time.

If one is willing to forgo real-time, such as for visual effect pipeline, a third approach is also possible. A direct GAN would be an inflexible generative render. To produce a super high-quality neural render the PaGAN II can still be used but with three key differences. First, all the optimizations for real-time can be removed, these often had minor quality loss issues for the sake of speed. For example, a deeper more complex network can be used. Secondly, there is no need for the 'Many to One' stage as time can be taken to train on both the Subject and Target faces, but thirdly and most importantly, it is how the team trains PaGAN II. As many academic papers explain, how one trains a Neural Network is very important. "A lot of the deep learning papers, discuss how you train a Network," says Li. "They explore how you augment the data? What is your strategy and how do you package batches of training data," explains Li. "All these things lead to different results, so we now have a very specific way of training." Given that PaGAN II was built to be a Neural Render which allows user intervention, the team can address issues that a straight generative network could not. "We tweak the network, and we designed the network to handle that," he says. Without the adjustments, "you cannot guarantee that the expressions of the final person don't just look weird. And that weirdness is not necessarily blurry artifacts. For example, it could be eye gaze or weird compositing blending artifacts. These are the things that we're focusing on being able to now get right".

Sean Bean Demo

To demonstrate the high-end PaGAN II approach we inferred an imitation of Sean Bean (L) by fxguide's Mike Seymour(R). (While Mike is no actor, and an Australian (!), he did impersonate Bean's distinctive UK accent, for this - he asks forgiveness).

The Sean Bean / PaGAN II demo required about a day of training but then renders at a rate of just under a second a frame. This is still fast by 3D standards, but much slower than the 30Hz real-time PaGAN demo shown in Switzerland.

Agents

While Avatars and face replacements such as this are driven in by a source person, they could also be driven by an AI agent, not unlike a Siri or Alexa. Over the past year, Pinscreen has concurrently developed an entire pipeline for such graphical agents. This includes Natural Language Processing (NLP), AI chatbots, motion synthesis, class simulation, real-time body and hair simulation, and more.

"We now have on the AI side, an entire pipeline that can do a state of the art agent," says Li proudly. "That includes vocal recognition, response generation for freestyle or everyday conversation, as well as the ability to actually generate speech(audio), -so from text to speech to lip animation".

On the left is a neural rendered real-time hybrid agent image, with a base generated in UE4, including interactive, dynamic cloth, and her face enhanced via PaGAN II Machine Learning.

Audio Speech

Pinscreen is very interesting for its digital agents to be more than just purely rule-based agents. They are working on hybrid bots that respond based on a rule-based approach and also uses AI generative methods to simulate freestyle conversation discussion and questions. This would allow Pinscreen's agents to provide more accurate small talk and semi verbal human-style responses. Li is keen for their agents to provide a range of verbal and semi-verbal cues found so commonly in natural conversation, such as ahh, ha, um, and huh. These almost throw away things, like micro facial gestures, add realism, intention, and emotion, just as much as larger facial clues such as broad smiles or frowns.

"The nice thing about this pipeline is that one anyone can personalize it. And it is so fast, that we can generate an accurate voice and there is still time for us to compute other components such as generate the right facial expressions, emotion, and lipsync.

Influencers

Pinscreen is already producing agents that are being used to model clothes and will hopefully soon be launched to become virtual Influencers in Japan. These autonomous agents are all running in real-time for interactivity, and unlike many stylised Japanese 'Manga' characters, these use both the UE4 game engine and the PaGAN ML to produce highly real-time realistic agents. The Pinscreen custom agents can move, talk and emote. The ML runs in the game engine to provide dramatic quality for a real-time application, including cloth and hair simulations. "We are all in on the Unreal Engine for this," Li comments. "The real-time clothing they wear was shown at SIGGRAPH Asia in December last year and the new Pinscreen AI Agents will be released soon".

To achieve their aim of an end to end solution, Pinscreen is had to apply its ML Neural Rendering approaches not only to faces, but bodies, hair and clothes. The company is now starting to produce avatars and agents as full humans.

Talks & Demos

Pinscreen is very focused on real-time and Pinscreen are a key presenter at RTC, next month. "In June I'll talk about how we've gone from 'deep fakes' to virtual assistants to virtual connectivity. Plus there are a couple of new things that we hope to show on real-time telepresence and maybe even a surprise or two!", concludes Li.
For RTC Registration Click Here
Note: those who can pay, you will be invited to contribute. If you can't support RTC financially at the moment, then the conference still welcomes you to the June RTC FOR FREE!

fxguide is a proud media partner of the RTC. Fxguide's Mike Seymour, along with Facebook's Christophe Hery, are curating the June 9th session on Digital Humans | Artificial Intelligence | Machine Learning | GANs.

# data[].attributes.attachment_link.title
Pinscreen's Advanced Face AI & Neural Rendering - fxguide

# data[].attributes.attachment_link.description
Pinscreen: From Social Distancing to Virtual Presence with Advanced AI

# data[].attributes.attachment_link.description
Whether you have an Oculus, HTC Vive, PSVR, here are some of the best VR movies you can watch. Get started with what's the best VR today.

# data[].attributes.attachment_link.readable_text
DreamWorks Animation's The Croods is the first animated feature hit of the year, grossing $330m worldwide after just two weeks. The Kirk DeMicco and Chris Sanders-directed film is visually spectacular, thanks in part to a refined Point-Based Global-Illumination and physically based lighting approach by the studio, and a newly developed cross-over between effects and animation. We talk to the film's visual effects supervisor Markus Manninen.

Watch a scene from The Croods by DreamWorks Animation.

fxg: What was behind the new PBGI approach on The Croods?

Manninen: We wanted to create a world that was more tangible and believable, not necessarily realistic, but something that immersed the audience with this family on this journey. In live action films and in visual effects, they've spent years and years obviously getting very good at - trying to make materials seem like real materials and light behave like lighting on the set. Although our world is slightly different, everything is designed to have an art-directed aesthetic, it still became a process of look-developing the movie in a way we hadn't done before.

fxg: So did you go and shoot real HDRs for example?

Manninen: Yes, we would take reference from real life and shoot HDR images for using in our image based pipeline that we developed for this show. In terms of Point-Based Global-Illumination - we had used it in pieces but now we used it to get not only environment sphere lighting but really using it as a singular paradigm for also area lighting, and making it interactive and very art-directable.

fxg: Where did you shoot HDRs?

Manninen: We were lucky - an on-set supervisors from a visual effects facility had joined us and was actually part of developing the front-end pipeline with me a few years ago. He said, 'This is what I used to do in my previous job, why don't I help?' and so we developed a whole toolset around HDR. It was something we hadn't taken advantage of previously and so we went through the same process everything goes through in capture and getting it into the lighting pipeline. We developed a physical kit and a process and we now have a standard for the studio. It means we can even now do, for example, commercials a lot more efficiently because we can put our characters into plates.

Then we went location scouting for interesting lighting scenarios. One guy in Venice suggested that we go to the morning fog on the beach. We didn't want to have a lot of man-made lighting sources so what we were looking for were naturalistic lighting conditions. So on the beach we shot the overcast IBLs and HDRis. It was all fogged out but with super soft lighting that you get at Santa Monica during the early hours of the day.

Then we went to Griffith Park and shot a cave, which is actually the 1960s Batman cave - which the car drives out of. We shot the cave interior there which was fun. We also at the same time shot a sunset and we didn't have the city in view there. Then we shot in Ventura at the beach there for a beach sunset. We also had the brightest moon in the last 10 to 20 years a few years ago and that night we went up into the hills again and shot moonlight HDRis for our night sequences. It was literally a live action scouting session, but just for lighting and not locations.

Meet Guy, a clip from The Croods.

fxg: That realistic approach reminds me of recent animated films that have been getting live action filmmakers and cinematographers involved as consultants, like DOP Roger Deakins.

Manninen: Well Roger was part of this movie also. We started off with having two to three months with him while he was in between projects and he came in and worked through things. At the time, it was so early that we hadn't locked story. The production designer, Christophe Lautrette, Roger and I spent a month literally planning out the lighting approach for the movie. The movie did change and so we had to adapt while Roger wasn't here but it really helped having those conversations around the lighting scenarios we wanted to embrace and the lighting quality we were looking for. Photo reference was a way to communicate these. Then as the movie became more comical we actually changed our color schemes a little to be brighter and more colorful and fantastical than what we had started out with. But the ideas remained. And who doesn't want to work with Roger Deakins, by the way...

fxg: How did that new lighting approach plug into DreamWorks' existing lighting tools?

The developers behind DreamWorks Animation's Light tool were recently awarded with a Scientific and Technical Achievement Award from the Academy. Read fxguide's behind the scenes look at the tool here.

Manninen: What makes our lighting tools amazing is that they really complement our process. Not just the process of lighting our films but literally our process of evolving our techniques, changing our looks. Every movie has a distinct look and we want to try new things. The Light tool has become an open framework for us developing these new techniques and still being able to have a consistent approach to lighting. Lighters can go from show to show with the same tool but produce these different images. The Light tool's been around for a long time, but it's as relevant today as it was back then. The techniques and methodologies have changed but the framework is still there.

One of the other things we took to the next level on Croods was our rigged-based lighting approach. We've been developing this over the years, and we had a very aggressive goal for our film which had a lot of locations (it's a road movie and in animation it's often something you avoid, because every sequence has multiple locations and characters!).

So using this new HDR/IBL lighting we developed a new surfacing workflow that allowed us to normalize our surfacing - every asset in terms of its materials, diffuse, specularity etc, actually behaved the same. It's like a poor-man's physical-based shading where we had predictable response from your materials. That allows us to throw in any assets in any scene and it would behave the way we would expect it to behave. That in conjunction with our light rig approach - where we can set up one shot and have it propagate to any shot in the sequence - that means we can get the whole sequence up and I can see every shot in the sequence and make choices about which shot I want to 'hero' up versus which shots come out of the lighting almost a hundred per cent.

fxg: Aside from lighting, how did you deal with the huge amount of effects in the film especially things like the tar and volcanoes and water?

Manninen: The idea was that the world is the villain in the film, and all these natural phenomena are things we had to accomplish. We also wanted to end the movie big as well. One of things we've done over the years is establish a department called the Character Effects Department - they partner with the Animation Department to accomplish the additional simulations and procedural work such as hair and cloth. On Croods we took that to the next level where Character Effects literally became the animation department for anything that the characters also touched. So they took on all the interaction with the world, the foliage for example.

As we came into the last eight months of production, the challenge we had was how do we make this tar sequence which revealed the back story of Guy and Grug - how do we create that and leave enough budget of the tar sequence? Damon Crowe, my character effects supervisor, came up to me one day and said, 'Why don't you let us do the tar without doing the traditional mud style simulations we've done in the past?'

He pitched me this idea of using deformation animation and cloth sims as an approach to it - more or less keyframing the tar. What was really appealing to me was, not only did they show a test that was very successful, but even more so the moment itself was not a very procedural moment - it was two guys stuck in tar sinking which was a quiet moment in the movie, but we wanted to keep the tar alive. So having the hand-control of placing everything, we'd created these really nice bubbles and subtle indents moving around, and we could art-direct them and focus on the character behavior.

Watch a 'hunting' clip from The Croods.

fxg: Any other particularly challenging effects sequences?

Manninen: The head of effects Matt Baer did a great job. There was one effects sequence we had that wasn't going to be in the movie and it was a lot about water and the family being washed away by a mudslide/water flood. We had our effects developers working on a new water pipeline, but then the sequence went away - to our great regret! We put the development on a shelf for almost a year, and then another water sequence came back into the movie, with different water, but it didn't mean we couldn't use the technique. We had wonderful, layered detail with close-up water for interaction with the characters. We now use Houdini as a package and write our own things on top of it. So for water we do most of the sims and even rendering outside of Houdini but use Houdini as the interface to the tools we've written. Houdini has a fantastic ability to deal with lots of geometry and particles.

fxg: It sounds like as a visual effects supervisor on an animated film there's a lot to oversee - what do you see as your main role?

Manninen: It is a little different than live action because really everything we create is a computer graphic effect more or less, whether it's a performance of a character or lighting or effects work. My job is to take a holistic look at the whole film and make sure that the producers and directors get the film that they want. I spend most of my time thinking about the final image - how that's going to come together, how all the stuff we do, from the beginning of designing the movie to the end - how that will come together in the lighting department and the final compositing of the shot.
All images and clips copyright 2013 DreamWorks Animation LLC.

# data[].attributes.attachment_link.title
HDRs for an animated film? Behind the scenes of The Croods - fxguide

# data[].attributes.attachment_link.description
How DreamWorks Animation employed Point-Based Global-Illumination on The Croods, relied on HDRs to inform lighting and developed a new cross-over between effects and animation.

# data[].attributes.attachment_link.title
Exploring 8K: Does It Matter? Yes. Should You Embrace It? Maybe.

# data[].attributes.attachment_link.description
Move over 4K, 8K is the way of the future. But the question remains, does it matter today? Should you start shooting videos in 8K? Let's talk about that.

# data[].attributes.attachment_link.readable_text
HeyGen AI software is a ground-breaking AI technology, bringing video avatars and translations/dubbing to the average prosumer production. If you need a talking head video for marketing, training and how-to videos, then this is the place to start! Aside from all the other features, options, templates for generating content from pre-made avatars and images, the real juice with this software is the ability to clone yourself from a video & audio clip is truly incredible.

As I've been saying in all my AI Tools updates: AI Tool technology is advancing at such a rate that we're measuring it by days now, not even weeks, months or years. The developers at HeyGen are a perfect example of such rapid development, that I've had to change this product review several times in the past month; because either their advancements in technology, approach to developing these avatars, and their pricing structure of everything has changed almost daily.

It really took off when I saw a video posted on LinkedIn from the CEO of HeyGen teasing the capabilities of their new "Avatar Lite" beta, which I promptly got on board and applied to start testing - and I got this response in email the same day. The rest is a very, very short history!

While I'll outline several features from this AI Tool, the biggest focus for me is on the Video Avatars - which have been evolving rapidly as I stated above. For instance, I made this video about a week ago and it's already outdated in the features, quality and naming of the various tools. The "Avatar Lite" used to take 3-5 business days to generate a usable video avatar as shown (mostly with hands-on techs refining the process), but now is currently automated to generate an "Instant Avatar" in mere minutes! You can also now "Finetune" your Instant Avatar (for an additional fee of $59/mo - which we'll discuss later in this article) and comes back to you in under 24 hrs.

Here's a completely AI-generated video showing the process - including the voice translations feature mentioned below from only a couple weeks ago:

Video Productions from Templates

Depending on your skill level and requirements, there are many ways to start generating content in HeyGen's studio. You can select one of dozens of starter templates you can modify directly in the portal and even switch your provided avatars from their library. The user interface is really straightforward and easy to navigate and make adjustments.

There are dozens of pre-loaded avatars and voices available to choose from.

You can also just make a video with an avatar on a green background and composite them directly in your NLE of choice. This example was from a generic avatar and voice generated in HeyGen and then composited in After Effects for an example social media short video.

Animated Faces from Photos & Images

This was the first step in discovering how much fun this software could be. I discovered it a few months ago and played around with various photos and images rendered from Midjourney. The process has changed a little since then, but the quality has improved a great deal.

They also have an option to generate AI characters with a text description directly inside HeyGen's interface. It came up with some interesting results but note that only the faces/heads get animated and not the entire torso when you generate avatars this way.

It's a quick process - simply upload your image and apply an AI voice (or cloned voice from your ElevenLabs API) and then create a video with your text input. Just upload your photo or rendered image to start (making sure the face is full visible and central to yoru image).

Here's a few examples from my headshot photo and a couple Midjourney images:

Check out the example below where I'm setting up the green screen studio and our studio mannequin "Leana" complains. That was done from an iPhone photo in HeyGen using this same process.

Video Avatars & Voice Cloning

This is where we split off from the rest of the pack - and what got me excited about using HeyGen for regular marketing and instructional purposes at my day gig at a biotech company. It really has generated a lot of interest with our product marketing folks.

The first step is to make sure you have a good video and audio recording to work from. You can just put up a tripod and shoot yourself or your subject in a casual or business environment with a steady background and clean audio for your submission. You shouldn't move around or make sudden gestures or facial expressions and let the video run for a full uninterrupted 2-5 minutes for the best cloning results.

When you create your avatar, you have to submit a video authorization (from the subject directly) for security purposes. This keeps the site safe from nefarious activities.

In this first video I generated from my home studio office was a baseline to build my other experiments on:

For more flexibility in my avatars, I set-up the green screen studio to shoot more tests of myself, reading the same 2-1/2 minute script from a teleprompter for my comparisons. Setting up the greenscreen after a few years since the first shutdown for Covid took awhile to dial everything in, so our mannequin "Leana" got a bit impatient standing there all day. (also animated with HeyGen) 😉

The process is really simple and I don't need to outline all the steps here because it's easy to follow their instructions from the web site and they have multiple video tutorials on the site they've created. You can use either a prerecorded voice audio file or TTS using a built-in voice or select a clone you've generated. I've downloaded several from ElevenLabs to generate many of my test videos but now prefer using the built-in third-party API to generate directly inside of HeyGen and I can access whatever ElevenLabs voices I have in my account through the voice manager.

So for this example, I applied one AI generated VO audio from ElevenLabs.com to create three different versions of the same script to see how they compared - or differed from each other. Keep in mind that these three avatars aren't just dressed differently, they were sourced from three separate videos that I shot on the green screen at separate times. Applying the prerecorded AI voice from ElevenLabs assured the Avatars would sync properly. I could not get this same result had I run the ElevenLabs API to generate the VO on the fly repeatedly as there would be variations in the voices.

In this example, I ran the same script and composite in Premiere just exchanging the green screen composites from After Effects in the same sequence.
Instant Avatar vs "Finetuned"
The Instant Avatars you get with your plan are adequate for most purposes (you can purchase more if needed), but the Finetuned Avatars do have better mouth and lip sync performance, as seen in my testing.

In this example video, I used ElevenLabs to produce the audio track which I uploaded to HeyGen when I created the video avatars, so they have the exact same audio track for true side-by-side comparison. Notice the accuracy of the lip sync is improved on the Finetuned version on the right.

Better yet - I've found that using the ElevenLabs API link directly inside of HeyGen, I get much better lip syncing and mouth movements on BOTH the Instant and Finetuned avatars.

This is only the beginning... watch this tech closely in the coming months!

Translations & Dubbing

There are two ways of generating translations in HeyGen. One is to input translated text into the video avatar producer and select a multilingual voice from your ElevenLabs API and create a clean video avatar from there.

The other method lets you upload any video clip with a subject running at least 30 seconds facing the camera and it will generate a new video for you with a clone of the actor's voice and lip syncing capabilities automatically in a few minutes. Here's their rundown on the process in video form from the HeyGen website:

I've tested several video clips and the results have been amazing! Check out the intro video at the top of the article to see more examples I've created.

Here's an example clip that I created from a scene from Pulp Fiction with Christopher Walken and translated into Spanish and French. You can see where this could be really helpful for video dubbing and regionalizations in the future.

Pros & Cons

While I have been a major fanboy the past month or so over these new features and capabilities, I'd be remiss to not point out some things that I hope get resolved or updated in future versions of the HeyGen software tools - and pay structures.

The tools are evolving quickly - to the point that I think most of this review will be obsolete by year-end. And with that, possibly positioned to be bought up by a bigger brand or another round of financing encourages the developers to make a leap toward world domination. (only slightly kidding) 😉

I would like to see the ability to control the Instant Avatars more with gestures, facial expression, etc. When the voices get more energetic, the faces should reflect that as well. Mabe just an "exaggeration/enhancement" slider or something.

The talking photos could use more control as well - like the way the puppet tool works in After Effects, where you can define the points that move or at least define the boundaries of the head/hair so the whole head moves - not just the face.

And pricing seems to be all over the place currently - but that might be due to the changes in product offerings as they develop. For instance, the $99/yr for a voice clone that I feel is sub-par to what you can generate in ElevenLabs. (which I'm really thankful for the application of the ElevenLabs API which produces the best of both worlds in one easy step). The monthly fee for the base service is fair, especially when 3 Instant Avatars are included with the $59 Creator package. The "Finetuned" option is an additional $59/mo for EACH AVATAR you upgrade this option for. That means if you upgrade all three Instant Avatars you create, that's an additional $150 mo just to continue to use them. I guess if you don't need them any more, just cancel the upgrade plan for each one, but I'm not really seeing that much value in the little bit of difference that the "Finetuning" provides at this point for most customers - but professionals will justify the additional cost to get a better level result.

Support ProVideo Coalition

Shop with

# data[].attributes.attachment_link.title
AI Tools: HeyGen AI Video Avatars & Translations by Jeff Foster - ProVideo Coalition

# data[].attributes.attachment_link.description
HeyGen AI software is a ground-breaking AI technology, bringing video avatars and translations/dubbing to the average prosumer production. If you need a talking head video for marketing, training and how-to videos, then this is the place to start! Aside from all the other features, options, templates for generating content from pre-made avatars and images, the [...]

# data[].attributes.attachment_link.readable_text
Do you care about 8K video? Is there something you feel is lacking from all of the 4K capable mirrorless cameras on the market and their video resolutions?

Ever since Canon announced its 8K R5 mirrorless camera, the industry has been in an arms race to reach this elusive video recording resolution. Yet, in all the craziness of 8K, we might have gotten ahead of ourselves and failed to ask not the how-but the why.

According to reports, Panasonic has indicated that they feel that the consumer market for 8K mirrorless cameras is not enough to justify chasing that resolution right now... and it might stay that way for a bit.

A Look at the Current 8K Landscape

With big names like Canon and Sony appearing to be all in on making their recent 8K mirrorless cameras the flagship cameras in their lineups, it would make sense for them to double down on 8K being the next big thing. After all, when 4K video hit as the step up from Full HD, it was indeed a huge deal and changed the game for so many cameras and companies which chose to embrace it.

However, early results do tend to agree that 8K isn't as revolutionary as 4K. Besides cameras like the Canon R5, the Sony Alpha 1 and the Nikon Z9 there aren't too many more currently on the market. (There are a few of course, and a few more coming, but it's not 8K or bust right now.)

Will Panasonic Release an 8K Mirrorless Camera?

Could the Panasonic S1H get an 8K upgrade?

Credit: Panasonic

While there have been rumors for some time about Panasonic eventually joining the 8K ranks with their flagship LUMIX S1H successor likely to be announced sooner rather than later, these reports do seem to indicate that the next Panasonic LUMIX line camera will not be 8K because demand is simply "not enough" to justify that recording resolution.

Now, does that mean Panasonic will never release an 8K camera? Hardly, in fact, I'd be willing to be it's still very likely we'll get an 8K mirrorless from Panasonic in the next couple of years. That's just how technology goes.

However, the biggest news from these reports is really about the consumer marketplace and understanding what we-the video professionals-actually want. 8K is undoubtedly better than 6K or 4K, but manufacturers might just now be learning that the difference isn't as much as you'd think.

The Future of 8K

Now, it's impossible to tell the future. Plus, with the ever-changing landscape brought in by AI, the future of film and video production might look completely different in the next few years anyway. But that being said, it's very likely that technology will advance at a rate that will allow for mirrorless cameras to reach 8K (or possibly higher) recording rates in smaller bodies and at cheaper prices.

The question will be more about what we in the video industry want to do with these higher resolutions and just how much flexibility we want in our shooting coverage and edits to work with.

# data[].attributes.attachment_link.title
Do Videographers Actually Care About 8K Video?

# data[].attributes.attachment_link.description
Some people believe that 8K is the future of video, and that it will eventually replace 4K as the standard resolution. Others argue that 8K is overkill for most people, and that the benefits of 8K are not worth the extra cost and complexity.