# data[].attributes.body
In this first week we had a talk by Lauren Newport-Quinn, a graduate from the EPP course.

Her presentation gave a useful insight into emerging technologies which are working closely with AI.

# data[].attributes.body
VP combines real cinematography with specific virtual elements, usually consisting of lighting/screens that have a pre-programmed displays in order to enhance the realism of the production both during shooting and in post.

“Virtual Production combines virtual and augmented reality with CGI and game-engine technologies to enable production crews to see their scenes unfold as they are composed and captured on set.” - Moving Picture Company (MPC), from The Virtual Production Field Guide presented by Epic Games

VP brings post-production talent into pre-production processes, opening up new avenues for post production skills. Post-production knowledge holds a lot of value for optimizing new workflows since it is useful to know how certain effects will look once edited.

# data[].attributes.attachment_link.readable_text
Kristin Baver Kristin Baver is a journalist who loved science fiction before she could even write her own name. (Seriously, she was a card-carrying member of the Star Wars Fan Club when she had no other real reason to own a wallet.) Now she gets paid to pen stories and book reviews, interview fellow fans, writers, and other interesting people, and aspires to one day craft a Boushh disguise and join the ranks of the 501st Legion.

When The Mandalorian opens on a frigid ice planet, the titular bounty hunter zeroing in on his next prize, you can almost feel the gusts of wind howling past the unforgiving terrain. The effect is visceral, a seamless marriage of tried-and-true practical sets and costuming coupled with an innovative new technique that brings visual effects to the forefront of the production process.

"We've been experimenting with these technologies on my past projects and were finally able to bring a group together with different perspectives to synergize film and gaming advances and test the limits of real-time, in-camera rendering," showrunner Jon Favreau has said.

It's staggering to watch the final effect unfold, as a team of artists and engineers known as the Brain Bar act as mission control just a few yards away from the Volume, a curved cocoon of glowing LED screens ready to transport those standing inside of it literally anywhere. "It's exactly the same sort of technology as the large LED screens you see in Times Square," Bluff says. "What we wanted to do was shoot on a small stage with small physical sets that could be wheeled in and out fairly quickly and then extend those physical sets on the wrap-around LED wall." And by moving visual effects to the beginning of the filming process, StageCraft enriches the performance of the actors, and the experience for directors and cinematographers using the new methodology for more precise storytelling in a completely fabricated galaxy.

It was just about six months before filming began that showrunner Jon Favreau, executive producer Dave Filoni, and DP Greig Fraser joined forces with ILM, Epic Games (maker of the Unreal Engine), and production technology partners Golem Creations, Fuse, Lux Machina, Profile Studios, NVIDIA, and ARRI to unlock this innovative achievement. While ILM had pioneered virtual production tools and had worked successfully with LED technology on previous Star Wars films, StageCraft was still very much in its infancy at the time, a virtual reality platform that helped storytellers scout fabricated environments to set up their shots. Propelled by the support of Lucasfilm president Kathleen Kennedy and the sheer will of Favreau, who was always pushing the collaborative team to try new things and bringing together some of the brilliant minds capable of making it happen, the crew took their next steps into the larger world by crafting the prototype of the Volume.

Fresh off projects like The Jungle Book and The Lion King , Favreau was passionate about employing new technology to enrich storytelling as he began his work on The Mandalorian . But the scheduling constraints of a television show - and a Star Wars show that had to satisfy the planet-hopping scope fans have come to expect while making it feel entirely authentic and accessible no less - meant whatever the team came up with had to appear realistic and be able to be shot on a Los Angeles soundstage without the traditional challenges of location shooting. "One of the things we wanted to do is move away from green screens and make the scale of a Star Wars TV show work," Bluff says. "And we knew that we needed a technological innovation to push the boundaries and provide a solve for the production. Through the collaboration with Jon Favreau, Greig Fraser, ILM, Epic Games, and others we landed on the idea of utilizing video wall technology."
"The moment everybody realized this was going to work."
Bluff, like many fans, discovered Star Wars as a child enthralled by the galaxy far, far away and the stories that unfolded there. "I fell in love with Star Wars as a kid. I would watch the movies over and over again and I would lie in front of the TV drawing the characters. I desperately wanted to get into any sort of creative environment, particularly the film world, through drawing." When Jurassic Park broke new ground with realistic computer graphics animation, Bluff saw his opening. "I understood that the doors were about to be blown off the industry and it was going to be a race to learn this technology because it was in its infancy. So I took that opportunity with both hands!"

"It was that pioneering spirit from the early days of ILM and the birth of computer graphics and the birth of the Pixar computers that really got me into it," adds Kim Libreri, the chief technical officer of Epic Games, who collaborated closely with Bluff and the crew. "Honestly, ILM and George's legacy had a big influence on me choosing my career path." One night when he was studying in college in the UK, Libreri had a dream about a realistic video game that involved flying a snowspeeder over Hoth during the battle with the AT-ATs from Star Wars: The Empire Strikes Back . "It was totally real. I was in the movie, but it was a game. I thought, one day it will be possible that a video game will look as good as a movie. And that has driven me my entire career."

Bluff went on to land a job with ILM as part of the team of matte artists on Star Wars: Revenge of the Sith; Libreri also went on to work as a visual effects supervisor with ILM before moving on. And they're both thrilled to be working on new Star Wars stories. "For me coming back to Star Wars in the role that I'm in advancing technology, I hope it inspires kids that were my age when I [discovered Star Wars ]," Bluff says.

Nearly two decades after his first role working behind the scenes building the galaxy, Bluff recalls those early StageCraft tests in the spring on 2018, when the crew working on The Mandalorian had mere weeks to come up with the wraparound screen prototype. "We got the go ahead to start actively developing the tech sometime in the end of March 2018. The idea was that on the 10 th of June the same year we would do a test on the stage with a much smaller LED Volume wraparound screen with our picture camera, a prototype of a costume, and many different LED environments to test the technology."

"The theory I had was that the textures and the lighting from content had to be unmistakably real," Bluff says. Some attempts using the procedural and computer-built landscapes fell flat. "There were a lot of test environments that actually didn't work, which was a beneficial part of the process. More often than not they were environments that were built in a more make-everything-from-scratch approach."

When they put high definition images of real deserts and snowy landscapes, evoking Tatooine and Hoth, on the Volume, the effect felt tangible, yet somehow distant. With the action happening so far away from the landscape background, it behaved more like a traditional matte background. "When the camera's moving 10 feet, the mountain's not going to move. So that really only proved one thing, which is if we can put photo-real imagery on the screen, we will believe that the character is in that environment in a certain lighting situation."

On the second-to-last day of testing, the ILM team had concocted a 3D interior environment using thousands of overlapping images from a real location on Angel Island in San Francisco. Parallax had to be built in so when the background shifted on screen the directional positioning followed. Finally, an actor dressed in the Mandalorian's costume was placed in the center of the curved Volume. "That was our 'Eureka!'moment," Bluff says. "It wasn't a wide-open space, it was a much smaller environment and everyone on set believed it was real.

"The Angel Island set was actually a collection of rooms with doorways in between, and we had the camera dollying back and on screen it gave the impression that we were moving backwards traveling from room to room," Bluff says. "We had the Mandalorian in the center of frame walking and he was completely integrated because the lighting from the CG environment moved with him as the camera tracked backwards. Nobody could tell that he wasn't in that room. And that was the moment everybody realized this was going to work."

"Everybody was scared there would be flickering and all sorts of technical problems," Libreri adds. "And people were literally gob smacked at how good it looked. It was just mind-blowing. Everybody sat on the stage going, 'Oh my god, I think this is the future. I think we've all together, as a team, worked out what is going to be the template for making visual effects in-camera.' The birth of a new way of working."
"Everybody was standing on a desert"
Months later, the Los Angeles set of The Mandalorian was a bustling hive of activity. At the epicenter, the Volume, a curved 20-foot high 270-degree LED video wall made up of 1,326 individual LED screens topped with an LED ceiling. Inside the 75-foot diameter performance space, a variety of physical props and partial sets, created by production designer Andrew Jones and his team, could be swapped out to match the screens and quickly transform the innovative arena into a variety of planets and interior locations.

StageCraft's genius lies in the way it brings together the collaborative creative forces on the first Star Wars live-action series and allows them to work together in real-time. The green- and blue-screens that were once standard for shooting real actors and props before digitally swapping out the background for computer generated environments would have caused a nightmare for the post-production effects team with a central character like the Mandalorian, whose gleaming beskar armor would have reflected back the very hues intended to be cut out from the final frame. In that way, StageCraft is a boon for cast, crew, and fans alike. "Wrap around green screens cause confusion for both actors and crew, limiting spontaneity and on-the-fly creativity," Bluff says.

But beyond the time and cost savings of eliminating the step of meticulously replacing the green screens, StageCraft allows for better lighting on set, with environments reflected back in Mando's armor without a laborious post-production detailing work. And StageCraft's magic effectively allows creators to produce high-resolution visual effects in real time thanks to the game engine technology. "Everything we are doing we would have always done in post-production and it would have always looked exactly the same," Bluff says. "That was the goal. We never wanted to compromise the quality of the show. In fact, to the contrary we wanted to improve on the quality because our challenge was a main character with a fully reflective costume.

"This approach was a game-changer for us, not only by eliminating the green screens but by providing everyone on set with a VFX set extension that typically isn't created until weeks after the shoot wraps. Instead it was there on the day, people could interact with it and immerse themselves - the actors, the camera operators, the director of photography. Everybody was standing on a desert or in a forest or in some hangar somewhere. There was no need for the questions of: where am I going? What is this location? How deep does this go back? How tall is the ceiling?

"With the addition of camera tracking technology, wherever the physical camera was moving as it was photographing the actors, we were able to offset the background in real time as if the camera was looking through a magic mirror into a [computer-generated] world that we built," Bluff says.

"It's like we've put them inside a video game," Libreri adds, although the photo-realistic images go beyond the scope of current gaming technology. "The fact that we're able to go to these locations, I think they got better content, better performances. It's like going back to the days when George [Lucas] was out there in Tunisia," or more recent Star Wars films that took the crew of Star Wars: The Rise of Skywalker to the sands of Jordan. "You get that feeling back," Libreri says. "It feels like classic Star Wars ...and it's going to have resonance for many years."
"Schooled by George"
Such groundbreaking innovations are often the culmination of years of small, hard-won victories that propel filmmaking technology ahead inch by inch. For instance, Favreau and other filmmakers had already been experimenting with the technology, making films like Favreau's The Jungle Book and others, like Gravity before it, possible, when LED screens were first used by Lucasfilm for lighting on Rogue One: A Star Wars Story. At the time, the quality of the images meant most of the screens still had to be replaced by higher-resolution effects in post-production on the Star Wars standalone film. "We built on what was done for Rogue One, we both utilized a wraparound LED screen but at the time they were limited to playing back pre-rendered environments from a single camera perspective," Bluff says. "Their intention was to focus on capturing the interactive lighting from the screens knowing that the content playing back would be replaced in postproduction for a high-fidelity version which would maintain the exact same lighting."

Jon Favreau introduced his own ground-breaking VR advancements developed on both The Jungle Book and Lion King, but wanted to take the effect to the next level. "It takes an enormous amount of courage and vision to believe that we could pull this off," Bluff says. "And there were many times when the folks involved in building out the technology wondered whether or not it was going to be feasible," he admits. But Favreau never faltered. "Jon helped us pull through and prove the technology out."

Libreri first got involved in discussions when Filoni was exploring using Unreal Engine and similar platforms for animated storytelling. "We talked about the idea that a TV show like Clone Wars and some episodic animated content could be done in engine," Libreri says. Tracking physical cameras in a virtual environment was something that Favreau had helped advance with his work on The Jungle Book and more recently, The Lion King . "As you start to look at all these different projects, Jon was always in the conversation and always pushing for what this next advance was going to be." For years, "nobody quite had the answer," Bluff says. "It took everybody coming together and talking about their experiences and what Jon wanted to do whether it was Greig Fraser the DP, whether it was Kim Libreri at Epic, myself from ILM or of course Jon and Dave the filmmakers themselves. And it was through those conversations that we came up with this idea of how we wanted to try to shoot The Mandalorian ."

For Bluff, StageCraft is the crowning glory of two decades in the industry. "It's been the most collaborative experience I've ever had in 20 years working in this industry," he says.
The Brain Bar
Day to day, the real-time rendering and manipulation of the environments being beamed on screen comes down to an elite crew of visual effects artists and ILM engineers known as the "Brain Bar."

"They are basically air traffic control," Bluff says, although huddled together just feet from the Volume, they resemble NASA's stoic mission control room. "They are the ones that are operating the massive screens. They are bringing up all the different environments that you would see, that you would shoot against. They're able to move mountains quite literally. They can rotate the world. They can move us from one end of the hangar to the other end of the hangar. They can add extra lighting into the scene that of course would appear to have an effect on the actors on the stage so they do many, many, many things to continue to make the camera believe the magic trick."

None of it could have been possible without the vision of George Lucas, and the inspiring achievements of Industrial Light & Magic and other visual effects wizards who continued to strive for more realistic and authentic effects since the days of stop-motion and matte painting.

"I don't believe that the technology would have worked as well as it did without the special place that Star Wars has in everybody's heart," Bluff adds. "You can see it in the work that they did and the love that they had for everything that we did. Everybody brought something to this project that you rarely find anywhere else and it all goes back to George Lucas and the wonderful movies he made decades ago."

Go deeper into the making of The Mandalorian in the new docuseries Disney Gallery: The Mandalorian, now streaming on Disney+!

And you can watch Industrial Light & Magic's exclusive behind-the-scenes video on the virtual production of The Mandalorian, Season One, below!
Associate Editor Kristin Baver is a writer, host of This Week! In Star Wars, and all-around sci-fi nerd who always has just one more question in an inexhaustible list of curiosities. Sometimes she blurts out "It's a trap!" even when it's not. Do you know a fan who's most impressive? Hop on Twitter and tell @KristinBaver all about them.
Site tags: #StarWarsBlog, #DisneyPlus, #TheMandalorian

TAGS: Disney Gallery: The Mandalorian, ILM, ILM Stagecraft, Industrial Light & Magic, The Mandalorian

# data[].attributes.attachment_link.title
This is the Way: How Innovative Technology Immersed Us in the World of The Mandalorian | StarWars.com

# data[].attributes.attachment_link.description
Kristin Baver Kristin Baver is a journalist who loved science fiction before she could even write her own name. (Seriously, she was a card-carrying member of the Star Wars Fan Club when she had no other real reason to own a wallet.)

# data[].attributes.body
These consist of a curated selection of audio stories (accompanied by 3D scans), amalgamated into an “immersive collage” that can be watched in a projected dome space or experienced in VR.

Often these technologies can be paired with AI and AR to further enhance the user experience. This opens up many new possibilities to re-use archive in new ways and tell stories in places that no longer exist.

Many of these immersive technologies have been used in public spaces in conjunction with billboards, in order to promote tourism or advertise products.

3D Cat promoting tourism in Shibuya, Japan:

https://www.timeout.com/tokyo/things-to-do/tokyo-now-has-three-3d-billboards-with-cute-realistic-animation

AR screen spaces near Tottenham Court Road:

https://www.outernet.com

# data[].attributes.body
The 16K scanner established an innovative, faster workflow for 65mm dailies allowing directors to view rushes faster and in more depth than ever, optimizing the time used in post-production.

R&D scanner speed = 1.6FPS @ 16K

Previous/other scanners = 0.4FPS @ 4K

# data[].attributes.attachment_link.readable_text
The new documentary Apollo 11 doesn't tell space buffs anything they didn't already know, but it certainly shows them things they've never seen. There's no omniscient narrator, no present-day talking heads opining about the historical import of the old footage we're seeing. The 93-minute film relates the story of the moon mission's launch, landing on the lunar surface, and return to Earth solely through imagery and sound painstakingly sourced from the Apollo era.

While director and editor Todd Douglas Miller has done a masterful job of shaping a daunting amount of material into a tense and cohesive narrative, the movie's most glorious selling point is the surprising sensuousness of the footage itself, much of which has never before been released to the public. During the Apollo program a crew supervised by film editor Theo Kamecke shot dozens of reels of the Apollo launches and the earthbound circuses surrounding them in the ultra-high-resolution Todd A-O 70 millimeter format-part of a deal between NASA and MGM Studios to make a feature documentary on the moon shot for theatrical release in the early 1970s. NASA used the Todd A-O format to shoot a lot of their in-house footage from the period, too.

Kamecke used some of his material for a now-obscure 1972 release called Moonwalk One that fell victim to the public's shockingly fast dropoff in interest in subsequent missions after Apollo 11. The leftovers were filed away at the Motion Picture, Sound, and Video branch of the National Archives and Records Administration in College Park, Maryland. Which is where this breathtakingly beautiful footage documenting one of the greatest achievements in human history sat unseen for more than four decades. Not until Miller asked NARA archivist Dan Rooney what he had on hand from Apollo missions was the imagery that now forms the core of Miller's electrifying new movie uncovered.

Air & Space / Smithsonian spoke with Miller by phone in New York City the week before he came to Washington, D.C. to open Apollo 11 at the Lockheed Martin Imax Theatre at the Smithsonian's National Air and Space Museum. This transcript has been condensed and edited for clarity.

Air & Space: You had agreed to make this film before you knew about those 165 reels of 65 millimeter film from the Apollo era. Didn't that discovery come after you had already taken on a commission to make a documentary about Apollo 11?
Miller: That's correct. Initially the plan was to scan all 16 and 35 [millimeter film] that was available and then any other assets: audio, TV, anything. Anything the various NASA facilities and certainly the National Archives had in the vaults. It wasn't until we started researching what exactly was available which happened a couple summers ago. We sent our archive producer Steven Slater, who is based in the UK, to College Park. And he worked with the archive at NARA. And it was shortly after that trip that we knew about the existence of the 65 millimeter collection. Once word got out that we had developed a prototype scanner to deal with that, some of the various other NASA facilities, namely Marshall, had some 70 millimeter [film] that we also scanned for them.
You had to develop a new scanner for the 70 milimeter film?
Yeah. There wasn't really any good technology out there to do a telecine or any kind of scanning. We got it in 16k resolution.

APOLLO 11 [Official Trailer]

I read that great Vanity Fair feature about your movie back in December, which talked about how some of this footage was used-albeit in a very different way than you used it-in a documentary called Moonwalk One, a movie obscure enough that I had never heard of it. Did you know that movie before you decided to make Apollo 11? Obviously, there was a huge amount of footage for you to cull from, but you were you wary of trying to create a new film from some of the same materials?
Oh yes, Moonwalk One is one of my all-time favorites. It's become a cult classic among space nerds. I'm of the mind that if I'm going to make a film about a subject, I want to devour everything there is to know about it. Not only reading every biography-Michael Collins' book Carrying the Fire, for instance, certainly had a big impact on me and on this film. There was another film that was made [in 1968] called Bridge to Space that was really great. And [Al Reinert's 1989 documentary] For All Mankind, obviously. I love them all for various reasons.

Moonwalk One in particular is an anomaly in that the actual production history of it involves a company run by Francis Thompson. [Thompson, who died in 2003, was a pioneer in the field of large-format and multiscreen films that were the clear precursor to the modern IMAX format.] I'm just an absolute fan of large-format films from the '50s and '60s. These guys really experimented with fractured narrative. It was a lot of films that were played at expos and special venues. Cinerama: Three cameras, three production screens. Francis himself directed a film called To Be Alive! that plays out the 1964 World's Fair in Queens and that movie was edited by Kamecke. It was shown in Cinerama, but they did a cut-down into 65 millimeter and it won an Academy Award. The interesting thing about Moonwalk One was that it was a coproduction between NASA and MGM. MGM backed out about a year into preproduction and Francis took another job but he recommended Theo to direct Moonwalk One.

[Kamecke] retired from filmmaking and became a sculptor in upstate New York. And he passed away [in May 2017] as we were making the film. I dedicated Apollo 11 to him and to Al Reinert.

It's just astonishing that footage of this quality-and historical import, obviously-was unseen for so long.
What's interesting is the actual Panavision collection of large format actually predates Apollo 11 and the Moonwalk One guys by years. NASA, through Technicolor down at the Cape, had a couple of cameras, old Mitchell 65 cameras, and they were documenting with these cameras as early as Gemini. Certainly all the [early] Apollo missions-8, 9, 10-all had large format coverage. It was almost like a dress rehearsal. A lot of the same shots that you see in our film from Apollo 11, there are very similar shots [from] the other missions and they just screwed them up. But they nailed it on 11, which was great.

The cool thing about Moonwalk One was they had two cinematographers. One was this guy, Urs Furrer, they brought in for the day of the launch. They called him "The Bear." And he would hand-hold this giant Mitchell camera 45 years before [director Christopher Nolan and cinematographer Hoyte van Hoytema] were doing that on things like and Interstellar.

The consensus among the part of the Air & Space staff that has seen Apollo 11 and that collectively has a strong familiarity with most of the Apollo footage that has been released over the decades is that there's just an astonishing variety of new material in your movie. And you just said that all the Apollo missions were filmed in high quality. Did you give yourself license to use film from the other Apollo missions, or is everything we see in Apollo 11 from Apollo 11?
We did use some shots from other missions. We were very lucky to have Mike [Collins] and Buzz [Aldrin] and Neil [Armstrong's] sons Rick and Mark all involved very early. They were so gracious with their time, and with me pestering them with questions. "Did we get it right? Did it look like this?" This is a roundabout way of answering your question, but something Neil talked about all the time-and he said this at [the National Air and Space Museum] on the 40th anniversary of the mission-when he was asked about the most indelible moment of the mission, he said it wasn't the first step, or landing, or the launch. It was going to the moon and seeing the moon with a solar corona and having it have this sort of 3D look about it. And I asked Buzz about it. We showed him some shots we were thinking about using from some of the other Apollo missions. He was like, yeah that's it. That's exactly right. Same thing with Mike. They all remember that.

I'd read about that for years, and I've wondered why I've never seen it portrayed in either a fiction or nonfiction film. So we used that from another mission. And then probably the most important one was the TLI sequence, the trans-lunar injection. They light the candle and they're going to the moon after orbiting the earth for a couple revolutions. All the astronauts talk about this; it's one [of the] beautiful scenes. Nine times out of ten they TLI-ed right into a sunrise. It's this amazing moment. So we just happened to find on Apollo 9 that they actually shot that. And it just looked amazing, the sunrise over the earth. So the minute we saw that, again, I said "Buzz? Mike? Is this what it looked like?" So the TLI is from Apollo 9.

The re-entry sequence is taken from an unmanned mission. That was actually an idea that we had pretty early on, because the 11 one is not all that great. And Buzz forgot to turn on the onboard audio, by his own admission. [Laughs.] So as an homage to all the other Apollo missions, we have a collage of everyone's voice and put it all in there during their re-entry, predominantly on 17, because they said some really great stuff and it's accurate to a T on when it happened.

One of the things that we're doing is documenting all this in a production journal. Because it is really important to the space community. I talk endlessly with Al Reinert about this because he got blasted by the space community about For All Mankind, for using composite shots from other missions. It's polarizing to some people, and we want to be very transparent about why we did it. Because Apollo 11-it wasn't their mission to document everything. That was for 10 and some of the other missions. [The crew of Apollo 11] were there to land and come home safely.

What's really great is that Buzz shot one of the greatest landing sequences of all the Apollo missions. And again, he forgot to turn on the camera during lunar liftoff. [Laughs.] So at the Sundance Film Festival, Rick Armstrong said, "Why didn't you show the flag bending over?" And I said, "Because that's from Apollo 14." There's a situation where I didn't want to do a composite. Everybody thinks that the flag fell over on 11 when it's actually from 14. And I think it's a cool part of the historical record that Buzz forgot to turn the camera on. [Laughs.]
Mike Collins shot that footage of the lunar module coming up from the surface of the moon?
Yes! And I think that is one of the greatest things any human being has ever shot in all of cinema. And we show it as an unbroken shot, just like he shot it.

There's definitely an entire thing we're putting together on not only the research but also our decisions to use or not use certain things.

Was there anything you were hoping to discover in all those reels of film from NARA or from Marshall there and you just couldn't find?
Stanley Kubrick hiding in a corner somewhere.

Apollo 11 is now playing at the Smithsonian' s National Air and Space Museum' s Lockheed Martin IMAX Theater and the Steven F. Udvar-Hazy Center' s Airbus IMAX Theater, and at cinemas nationwide.

# data[].attributes.attachment_link.title
"Apollo 11" Takes a Fresh Look at the First Moon Landing

# data[].attributes.attachment_link.description
In the new documentary "Apollo 11," director Todd Douglas Miller uses rarely-seen archival footage to make the lunar mission feel vivid, tense, and recent.

# data[].attributes.body
Pre-production:

Writers rooms to format text

Concept stages, enabling quick mock ups of lighting/costume/set

Production:

Generating actors in a virtual space, utilising AR and large screens to create an immersive 3d environment

Post-production:

Colourisation of archive footage

Restoration of footage that has been damaged or is missing content

Upscaling and format renewal

Audio editing (sync, ADR)

Pros:

If utilized correctly we can harness this technology to work in conjunction with industry standard software, streamlining processes that require less human involvement, its output can then be evaluated with human context and QC’d.

AI can also be used to mitigate ‘budget vs schedule vs expectations’ issues by providing predictions for the length of time required for post production processes.

Cons:

Consent is required for using AI to replicate actors image and likeness

Regurgitating and adapting content without original IP owner’s consent can lead to data protection/copyright issues

If used to replace too much of the creative process, jobs are put at risk and the value of creative work is diminished

# data[].attributes.body
This week focused on 360 video/audio and its niche within industry.

We explored its use as a tool to create 3D environments. This can be used in conjunction with AR and AI to create backgrounds for movie sets and immersive experiences for exhibitions and education.

# data[].attributes.body
360 Video Captures Video on multiple cameras and then combines the footage to be able to be viewed in VR or as a 360 immersive experience.

New forms of workflow required for 360 video:

AI involvement in order to make the final output more seamless and interactive.

Data management is essential to collect this much data from the different lenses and make sure it is in the correct order

Editor must perform an initial "rough stitch" to piece together footage from the lenses

Final work includes smaller changes like removing the rig to hold the cameras/mounts/drone

Technology used:

Meta camera is one of the leading companies in the 360 video industry, capable of producing 12k footage and popularly used for sporting events and large scale VR productions.

https://meta-camera.io

Mistika VR is a well known program used to edit and stitch footage together to create a coherent 360 video.

https://www.sgo.es/mistika-vr/

Las Vegas Sphere immersive experience:

https://pacificdomes.com/msg-sphere-las-vegas/

Art gallery using 360 Video for virtual tours:

https://theninebritishart.co.uk/exhibitions/jeremy-gardiner-south-by-southwest-2020/

A company that generates virtual property tours from 360 video by using AI:

https://help.ricoh360.com/hc/en-us/articles/360061734594-AI-Video-Maker#:~:text=AI%20technology%20automates%20the%20entire%20process&text=The%20AI%20is%20able%20to,and%20automatically%20generates%20a%20video.

"The AI is able to identify points of interest that customers like from millions of 360˚ property images and automatically generates a video." - Ricoh 360 Tours

Video describing current 360 post production workflow:

https://www.screenskills.com/starting-your-career/first-day-in-post/

# data[].attributes.attachment_link.description
About the Work
From the lights of skyscrapers and homes, to the neon street lights and headlamps of cars and trains in motion… fragments of the city float like stardust out of the darkness against the curtain of the night sky. This new vision of Tokyo is explored with omnidirectional motion graphics.

This work was born from ‘Beyond Motion Graphics’ (http://wowlab.net/research/beyond-motiongraphics) an internal company project at WOW aimed at exploring new possibilities for motion graphics. In using VR technology, consideration was given as to what kind of images to showcase and express as pieces of art.

作品について
夜のとばりに包まれた高層ビルの光、街頭のネオン、電車や道行く車のヘッドライトから家々が湛える暮らしの灯りまで、暗闇から現れる星屑のような街の断片を浮遊しながら、東京という都市の新たなビジョンへと至る、全天球型のモーショングラフィックス。

本作は、モーショングラフィックスの新たな可能性を問うWOWの社内プロジェクト「Beyond Motion Graphics」( http://wowlab.net/research/beyond-motiongraphics )から誕生。VRの技術的側面に加え、それによって切り拓かれるべき映像＝アートとしての表現性に注目した。

Staff
Art Director / Designer：Nakazi Takuma, Moriwaki Daisuke, Kudo Kaoru
Designer：Miyajima Tsutomu, Kojima Kazunori, Makino Shigeru, Sasaki Takuma, Kitabatake Ryo, Takagishi Hiroshi, Tanaka Kenji, Horai Misaki
Music：Marihiko Hara

# data[].attributes.body
360 audio is advancing to keep up with visual technology and it consists of arrays of microphones placed specifically to give a stereo experience that appears to be 360. Many of these microphones mimic natural ear shapes etc. so as to get the most accurate sounding audio.

AI can also be used to map audio to appear as if it is generated from certain parts of a space e.g. creating a immersive surround-sound experience from a simple stereo recording.

Study into generating spacial audio using neural networks:

http://www.svcl.ucsd.edu/~morgado/spatialaudiogen/spatialaudiogen.pdf

How 360 audio works:

https://endabates.wordpress.com/2016/03/29/360-audio-in-practice/

Types of microphones for 360 audio:

https://www.vrtonung.de/en/360-microphone-overview/

How AI is revolutionising spacial audio:

https://film-sound.berlin/blog/2023/06/09/adr-in-film-with-ai-revolutionizing-audio-post-production/

# data[].attributes.attachment_link.readable_text
Wenn man 360 Microphone erwähnt, denken die meisten Leute wohl an Ambisonics. Ich muss gestehen, das find ich erstmal gar nicht so verkehrt. Dieses Format ist unter realistischen Arbeitsbedingungen, etwa bei VR-Dreharbeiten mit einer 360° Kamera, durchaus praktisch.
Deswegen würde ich sie auch eher als 360 Microphone im Zusammenhang mit Virtual Reality und 360 Video bezeichnen, auch wenn 3D Mikrofone wohl treffender wäre. 360 Audio beschreibt historisch gesehen vermehrt Surround Sound, also kein Ton von oben oder unten, sondern nur planar.
Als logische Konsequenz gibt es auch eine ganze Reihe an Aufnahmetechniken. Um den Überblick zu behalten, möchte ich versuchen, sie zu kategorisieren, wie ich es bei meinem Artikel Virtual Reality Audio Formate getan habe.

Kanal-basiert (Mono, Stereo, Surround, immersiv)

Binaural (binaurales Stereo, Quad-Binaural)

Schallfeld-basiert (Ambisonic, Higher Order Ambisonic)

Objekt-basiert (das wichtigste zum Schluss)

Zunächst möchte ich drauf hinweisen, dass diese Übersicht nicht wertend zu verstehen ist, da bei der ganzen Diskussion über 360 Microphone und Tonaufnahmen vergessen wird: Jedes Setup hat Vor- und Nachteile und diese hängen meist nicht vom Mikro, sondern von den Umständen ab, unter denen die Aufnahme passiert. Darüber hinaus wurde ich von keinem der Hersteller gesponsort und will hier versuchen möglichst objektiv sein.

Update 2019/08:

Hinzugefügte Kunstkopf Mikrofone: DAVINCI HEAD MK2, Sennheiser MKE2002, Headacoustics HMS AACHENHEAD, Gras 45BB Kemar Head and Torso B&K type 4128-D

Hinzugefügte binaurale Inear Mikrofone: Soundman OKM, Aevox Audio IE, B&K type 4101-B

Kanalbasierte Mikrofone

Diese Mikrofone, bzw. Mikrofon-Arrays, wurden also meist für die spätere Wiedergabe auf Lautsprechern angedacht, was für VR Audio hingegen nur in Spezialfällen relevant ist. Trotzdem lohnt es sich die Herangehensweisen im Folgenden zu kennen.

Mono

Hierfür lässt sich eigentlich jedes Mikrofon nehmen, am besten eines mit kugelförmiger Richtcharakteristik, womit der Ton aus allen Richtungen aufgenommen wird. Folglich wird aber nur ein Kanal verwendet und die spätere Wiedergabe mit dynamischem Head-Tracking wird nicht funktionieren. Jedenfalls besser als der meiste Ton von 360 Kameras und folglich der Vollständigkeit halber erwähnt.

Stereo

Auch hier gilt, dass die Anwendung für VR auf den ersten Blick nicht ganz eindeutig erscheint. Insofern seien nur kurz die wichtigsten Vertrete, XY, AB, ORTF und Blumlein erwähnt. Trotzdem möchte ich an dieser Stelle auf einen Vortrag von Schoeps hinweisen, zum Thema Mikrofontechnik für VR. Folglich nutze ich auch gerne Stereo-Aufnahmen für 360 Videos, auch wenn sie erstmal nicht „die komplette Kugel mit Schall befüllen".

Surround Sound

Auch von bei Surround im klassichen Sinne, auch 360 Audio genannt, kein Schall über und unter dem Zuhörer wiedergegeben werden kann, gibt es einige interessante Anwendungen:

Zoom H2n

Ist wohl einer der bekanntesten Vertreter unter den hier genannten 360 Mikrofonen und hat immernoch seine Daseinsberechtigung. Dank eines Firmware Updates, kann er im Spatial Audio Modus direkt in Ambisonic B-Format aufnehmen, lediglich ohne Höheninformation, womit die dritte von vier Spuren im Ambix-Format leer bleibt.

IRT-Cross

Hierdurch wird eine Alternative vom Institut für Rundfunktechnik (IRT) zum vierkanligen Ambisonics (dazu später mehr) gezeigt. Durch die höhere Laufzeit zwischen den Mikrofonkapseln und der Ausrichtung auf einer Ebene, kann die Lokalisation genauer sein, als im Falle der Ambisonic A-Format Mikrofone.

5.1 Surround, Double MS

Auch wenn ich oft nach Dolby Surround 5.1, 7.1 oder ähnliches gefragt werde, sind diese Formate nur wenig relevant, da vor allem der Lautsprecher in der Mitte, also der Center eigentlich überflüssig wird. Trotzdme kann es in Spezialfällen angebracht sein, mehr Mikrofone in eine Richtung zu verstärken. Ähnliche Herangehensweisen sind etwa von Schoeps das Double MS System, Josephson c700s, Nevaton BPT oder Antelope Audio edge quadro. Mit eigenen Decodern können die Signale in komplexere Richtcharakterisitiken wie M/S, X/Y, Blumlein umgewandelt, oder gar mit einander kombiniert werden.

3D Audio

Endlich kommen wir zu den wirklich spannenden Vertretern der 360 Microphone, die meist, im Gegensatz zu Ambisonic, ihre Klangeigenschaften durch Laufzeitunterschiede erreichen.

ORTF-3D

Im Grunde genommen ist dieses Mikrofonarray ein Quader, dessen acht Mikrofone in je eine der Ecken zeigen. Dieses gibt es in verschiedenen Varianten, etwa mit Supernieren statt Nieren für die Höhenachse, oder zusätzlichen Laufzeiten. Vergleichbar ist das ESMA-3D (Equal Segment Microphone Array), welches damit auch interessant für das SPS Mach1 Format ist, auch wenn die Umwandlung etwas komplexer ausfällt.

Weitere 3D Mikrofonierung

Wie sich erahnen lässt, gibt es Mikrofone in den unterschiedlichsten Anordnungen, weswegen hier kurz folgende erwähnt sollen, aber keineswegs alle sind:

Binaurale Mikrofon Aufnahmen

Binaural beschreibt eine Wiedergabemethode, die ausschließlich für Kopfhörer geeignet ist. Infolgedessen also für Lautsprecher unbrauchbar, trotzdem ermöglicht es mit nur zwei Kanälen eine dreidimensionale Wiedergabe von Ton.

Stereo Binaural

Kunstkopf Mikrofon

Der bekannteste Vertreter ist wohl der Neumann KU100, welcher in Zeiten der 360 Microphone immer noch gerne als Referenz genommen wird.

Aber es gibt noch einige weitere, wie etwa DAVINCI HEAD MK2, Sennheiser MKE2002, Headacoustics HMS AACHENHEAD, Gras 45BB Kemar Head and Torso. Zu guter letzte ist B&K type 4128-D zwar nicht für binaurale Aufnahmen, dafür aufgrund der Bauweise interessant.

In-Ear Mikrofone

Falls man kein Fan von In-Ear Kopfhörern ist, kann man sich mit diesen Mikrofonen aber vielleicht doch noch dafür begeistern. Denn die hybriden Ohrhörer können Ton aufnehmen und wiedergeben. Damit wird ein sehr persönliches Klangbild geschaffen, was einfach zu handhaben ist, da die In-Ears lediglich in die Ohren gesteckt werden müssen.

Genannt seien hier der

Roland CS-10 EM, welcher etwa mit dem Zoom H2n verwendet werden kann, sobald die zusätzliche Speisung aktiviert wird.

Sennheiser Ambeo Smart Headset aufgrund der Kombination von acoustic noise-cancelling und transparent hearing, auch bekannt als „audio awareness". Damit können Außengeräusche je nach Wunsch entfernt oder hinzugefügt werden, was den Einsatz von 3D-Audio vor allem für Augmented Reality interessant macht.

Hooke Verse Audio Headset, welches kabellos entsprechend mit Bluetooth arbeitet und nicht nur für iOS Geräte geeignet ist.

Weitere, ähnliche Modelle: Soundman OKM, Aevox Audio IE, B&K type 4101-B

Omni-Binaural

In den meisten Fällen auch als Quad-Binaural bezeichnet wird, handelt es sich deswegen um vier Paaren von zwei binauralen Mikrofonen, also acht Ohren. Diese ermöglichen, im Vergleich zu den einfachen binauralen Mikrofonen, eine spätere Wiedergabe mit Head-Tracking in VR.

3Dio Omni-Binaural

Das wohl am skurrilsten aussehende 360 Mikrofon stammt von 3Dio und wurde zu Beginn des VR-Hypes rund um 360° Videos häufig genutzt, bevor es nun mittelfristig von Ambisonic-Mikrofonen abgelöst wurde. Trotzdem behält es seine Daseinsberechtigung.

8Ball Hear360

Vom Prinzip sehr ähnlich den obigen Ohren, hat aber ein ausgefeiltes System, was das Befestigen am Stativ unter der VR-Kamera sehr einfach macht. Außerdem wird durch die fehlende Ohrmuschel der Ton weniger zu einer HRTF (head related transfer function) gefiltert und ist damit brauchbarer für Lautsprecher Anwendungen.

Ambisonics

Kommen wir nun zum wohl bekanntesten Vertreter: Ambisonics. Diese Schallfeld-Technologie eignet sich hevorragend zur Aufnahme und Reproduktion von kugelförmigen Tonaufnahmen. Für mehr Infos dazu einfach hier klicken.

Ambisonic erster Ordnung

Für diese Art der Aufnahme hat sich vorallem eine tetraedische Anordnung der nierenförmigen Mikrofonkapseln etabliert, welche im Idealfall möglicht nah aneinander sitzen. Doch es gibt auch andere Möglichkeiten, wie ein Trippel-MS Setup, oder die Idee das A-Format zu überspringen und mit der richtigen Mikrofonanordnung nativ ins B-Format aufzunehmen mit einer Kugel und drei Achten.

Ambisonic A-Format Mikrofone

Weil es mittlerweile eine ganze Reihe von Audio Herstellern gibt, die ein sehr ähnliches 360 Microphone, manchmal auch als VR Mic oder 3D Mikrofon bezeichnet, auf den Markt für VR Sound gebracht haben, werden diese nur Stichpunktartig aufgeführt, in chronologischer Reihenfolge.
Es sollte aber erwähnt werden, dass alle vierkanalige sind, und meist in Kombination mit einem externen Aufnahmegerät genutzt werden. So haben der Zoom F8, Zoom F4, Sounddevices MixPre-6 und MixPre-10t, sowie der Tascam DR-701D eigene Firmware Updates bekommen, die ein eine einfache Aufnahme und Live-Preview für Kopfhörer ermöglichen.

Soundfield SPS200, Soundfield ST450 und weitere

Core Sound TetraMic

Brahma Ambisonic Standalone

Reynolds Microphones

Sennheiser AMBEO VR Mic 3D AUDIO Mikrofon

Røde NT-SF1 (Rode Soundfield)

Nevaton BPT Erweiterung um die Höhenachse (Nevaton VR)

Oktava MK-4012 4-D Ambisonic A-format Mikrofon

Einige der genannten 360 Microphone haben eigene Plug-Ins, wie etwa das AMBEO A-B-Konverter-Plugin oder dearVR Micro, sowie das SoundField by RØDE Plugin. Diese können die Konvertierung der Ambisonic A-Format Aufnahme in ein B-Format erster Ordnung in Ambix oder Fuma übernehmen. Darüber hinaus ist es möglich den Klang über verschiedene binaurale Decoder oder sogar Lautsprecher-Anordnungen direkt abzuhören.

Handliche Recorder

Mobile Aufnahmegeräte, die eine Ambisonic-Anordnung integriert haben, sind folgende:

Zoom H3-VR 360° VR Handy Recorder

Twirling720 VR Audio Recorder

Interessante Ambisonic Varianten

Ricoh TA-1 3D Mikrofon ist ein spezieller Mikrofon-Aufsatz, der ausschließlich mit der Ricoh Theta V 360° Kamera verwendet werden kann.

Audeze Planar Magnetic Microphones mit sehr großen, 100mm Membranen

Fraunhofer Uphear ist kein Mikrfon sondern eine Technologie, nutzt dabei aber die Mikrofone, die in einer 360-Kamera verbaut sind, um ein Schallfeld zu erzeugen.

Brahma Microphones (versatile Ambisonic recording systems) hat eine ganze Reihe von individuellen Lösungen wie Brahma-in-Zoom H2n, ein Großmembran Brahma Ambisonic Studio Microphone (siehe Bild unten) und einer Mischung dessen: einem tetraedischen Großmembram Array auf einem Zoom H2n

Für weitere Infos zu VR-Kameras, die für Spatial Audio geeignet (oder auch nicht) sind, wie die Sonicam, Nokia Ozo, Kandao Obsidian, GoPro Fusion oder Vuze Camera, bitte hier klicken.

Ambisonic zweiter Ordnung

Die nächste Generation an VR-Mikrofonen ist auch schon im Kommen, die acht statt vier Mikrofonkapseln nutzen. Vorteil ist unter anderem eine höhere Lokalisation und bessere Schallfeld Wiedergabe über Lautsprecher. Ob es sich nun wirklich um zweite Ordnung Ambisonic lasse ich außern vor, da Ambisonic im B-Format eigentlich neun Kanäle besitzt.

Brahma-8

Core Sound OctoMic (siehe Bild unten)

Mit beiden 360 Microphonen ist es möglich, im sogenannten SPS-8 Format aufzunehmen, sind aber genau genommen kein Mach1 Format. Man benötigt einen Zwischenschritt, um das A-Format in ein P-Format umzuwandeln, was mit einer Filtermatrix des Herstellers möglich ist.

Ambisonic höherer Ordnung

Sind in einem Mikrofon-Array, je nach Hersteller knappe 20 Mikrofone verbaut, spricht man von Higher Order Ambisonic. Auch wenn es noch wahnsinnig vielen Mikrofonen klingt, ist die eigentliche Anzahl der Produkte als 360 Microphone überschaubar.

Zylia ZM-1 hat 19 Kapseln und kann damit maximal in dritter Ordnung Ambisonic aufnehmen, was aktuell aber nur in Kombination mit einem Laptop oder Tablet über USB möglich ist. Mit dem 6DOF VR/AR Development Kit ist es darüber hinaus gibt es die Chance, sich von der Aufnahmeposition wegzubewegen und damit die sechs Freiheitsgrade nutzen kann

mhacoustics em32 Eigenmike microphone schafft sogar Ambsionic vierter Ordnung und ist damit wohl der aktuelle König unter den Ambisonic-Mikrofonen. Um die Anzahl der Mikrfone überhaupt handhaben zu können, ist auch hier eine digitale Singnalverarbeitung nötig.

Objektbasierte Aufnahme

Auch wenn ich eigentlich kein Fass aufmachen will und mit Audio-Objekten fachsimpeln will, wie sie etwa bei MPEG-H 3D Audio oder Dolby Atmos VR genutzt werden, sollte man das Wichtigste nicht vergessen:
Egal wie schön die Wahl des 360 Microphone ist, ob Ambisonic oder mit Laufzeit, in den allermeisten Fällen ist es wichtiger, sich auf Stützmikrofone zu konzentrieren. Seien es klassische Stützen bei 3D Orchester Aufnahmen oder Ansteckmikrofone bei Protagonisten, es bleiben meist Mono Mikrofone bei VR-Aufnahmen, denen ich meine größte Aufmerksamkeit schenke. Auch das hängt natürlich stark vom Fall ab und die Wahrheit liegt wohl irgendwo dazwischen, aber ich finde es wahnsinnig hilfreich, wenn man während der 3D Sound Aufnahme bereits an die immersive Audiomischung denkt.

Fazit

Es gibt also eine unheimlich große und immer weiter wachsende Auswahl and 360 Microphone, die man für VR-Aufnahmen oder 3D Audio nutzen kann. Doch man darf nicht vergessen, dass es nie das perfekte Mikrofon für jede Situation gibt, sondern lediglich ein oder mehrere beste Mikrofon für die jeweilige Situation. Bei der Suche nach der richtigen Wahl, helfe ich gerne.

Mehr über 360° Mikrofon Aufnahmen

# data[].attributes.attachment_link.title
360 Mikrophone für 3D Audio in VR Überblick - Virtual Reality Audio

# data[].attributes.attachment_link.description
360 Mikrophone sind für 3D Audio Aufnahmen bei Virtual Reality und 360 Videos geeignet um den Raumklang mit Spatial Sound Recording

# data[].attributes.body
This week covered 3D Cinema and stereoscopics. This was a useful insight into the parameters required to create a 3D experience and how video can be augmented to immerse the viewer.

There was also a talk given by industry professional, Jack Østergaard-Churchill. He detailed his lengthy experience with 360 video, XR technologies and AI which was useful in discerning different AI generative softwares and their uses.

# data[].attributes.body
Stimulating more senses helps immersion but footage must to be high resolution in-sync. As opposed to 2D film, depth of field and other cues can be used to separate foreground and background.

Developments in camera rigs aim to match the human visual system:

Camera separation (IA) = Human eye separation (IO)

Camera image magnification = human magnification

Distance to imaged subject = distance to viewed subject

4d exhibition:

https://www.gaudiexperiencia.com/en/4d-and-exhibition/

How it works:

https://stereoscopy.blog/2023/06/21/the-art-of-stereoscopy-andrew-brooks/

Underwater Stereoscopic Technology used in Avatar 2:

https://ymcinema.com/2022/10/24/the-underwater-cinematography-behind-avatar-2-3d-beam-splitter-and-nikonos-lenses/

# data[].attributes.body
Parameters of human eyesight:

Stereoscopic vision limited to 60°cone

Narrower 30° cone defines zone of focused detail

Wider 120° cone defines edges

Common AR fields of view:

Multiplex cinema FOV is about 54°

IMAX cinema average FOV is 70°

Samsung Gear VR horizontal FOV is 96°

Oculus Rift VR headset is 100°

# data[].attributes.attachment_link.readable_text
Depth perception - the ability to see our environment three-dimensionally - is based on "Stereoscopic vision", which requires two intact eyes that are aligned and move synchronously.

Try the following: place a round patterned object, e.g. a painted coffee cup, at arm's length on the table in front of you. Select a dot from the pattern and move a pointed pencil exactly to this dot. No problem. Now close one eye and try again - you move the pencil in direction of the dot and you notice that you have no idea how far away the dot is. To hit the dot you need to adjust your movement several times.

When you look straight into distance, your eyes are parallel to each other. The areas seen with the right and with the left eye overlap to a certain extent. With the left eye you see not only what happens at the left side of your body, but also what happens at the center and partially at the right side and vice versa.

The largest part of the visual field is seen binocularly, in other words with two eyes. Since our eyes are up to 2½ inches apart from each other, we receive two different pictures of our environment from the left and from the right eye. The brain "computes" the spatial information from the difference between the two pictures on the retina and creates a joint overall image, which provides extra information about distance to an object. This process is called stereoscopic vision.

You can get an impression of both individual images when you focus your eyes on a distant point and then hold a finger close to your face. You will see the finger with your right and with your left eye at the same time, but since it is not in the focus area, it cannot be calculated into a joint overall image, so you see two fingers. Now close first the left and then the right eye: the finger "jumps" first to the left, then to the right side. This is because you now see the two single images processed by each eye separately. In this special scenario, when viewed with both eyes, you are creating a double image.

The ability to focus with both eyes, that means to direct both eyes by control of the eye muscles in alignment to a certain point and fixate for further examination, is something learned in the early months after birth. This is why you can often observe babies squinting during these first months, which becomes less and less over time.

In order to process spatial image information faster - in splits seconds - our brain relies on our previous experience to save "computing power". This is why we get a lot of spatial information also when using only one eye, or why we see spatial depth also in two-dimensional images such as on a TV screen.

The following picture depicts several experiences:

1. We have learned that objects that are partially hidden are behind other objects. Therefore, you perceive that columns which obscuring other columns are closer to you.

2. The perceived object size is also a clue. A larger object is perceived to be closer to the viewer than a similar but smaller object.

3. The phenomenon of linear perspective: the columns converge at the end of the hall but are perceived as "parallel but further away".

4. The position of the shadow of an object, or objects located in the shadow, give additional spatial information.

These are all image impressions that can be processed with one eye only. They do not provide "real" spatial information, but can be used to create a depth impression in a two-dimensional picture.

These experiences can also be used for optical illusions, and can create an impression of an incorrect image. See also our blog on visual hallucinations and illusions.
Stereoscopic vision after stroke or brain injury
So we have learned that, even though we feel that our experiences allow us to grasp spatial depth and spatial relationships between objects with one eye only, "real" three-dimensional vision requires an overlapping visual image of two eyes.

Stereoscopic vision is often difficult for people who have a large visual field defect due to a stroke or other brain injury. They may have homonymous hemianopia and not be able to process the visual information in the complete left or right half of the visual field of both eyes. The section of the visual field that is needed for an overlapping image is missing, and therefore homonymous hemianopia can also result in loss of sterescopic vision and perception of spatial depth.

NovaVision's Vision Restoration Therapy can help to recover the visual field and help people regain their stereoscopic vision after stroke or brain injury. Find out more here or contact us to discover how NovaVision can help you or your loved one recover.

Written By: Sigrid Kenkel

References

http://www.psychologie.uni-heidelberg.de/ae/allg/lehre/wct/w/w6_raum/w640_binokulare_tk.htm

http://irtel.uni-mannheim.de/lehre/seminararbeiten/w96/Tiefe/binoc.html

# data[].attributes.attachment_link.description
Depth perception - the ability to see our environment three-dimensionally - is based on "Stereoscopic vision", which requires two intact eyes that are aligned and move synchronously. Try the following: place a round patterned object, e.g. a painted coffee cup, at arm's length on the table in front of you.

# data[].attributes.headline
Talk by Jack Østergaard-Churchill - Immersive Creative Technologist

# data[].attributes.subject
Talk by Jack Østergaard-Churchill - Immersive Creative Technologist

# data[].attributes.body
Jack gave many examples of industry-leading AI software, often used in conjunction with his medium of 360 video.

# data[].attributes.body
Augmented Reality (AR) - Augmenting the real world with additional content. Typically these are experienced through ‘windows’ such as mobile phones.

Mixed Reality (MR) - A blend of AR and VR where you use a headset or glasses to be shown AR content within your environment.

Virtual Reality (VR) - This can be computer generated or filmed, but is accessed through a VR headset or immersive environment that separates the viewer from the space they are in.

# data[].attributes.attachment_link.readable_text
More companies are entering the XR space

AR makes use of the existing environment by overlaying new information on top of it, whereas VR creates an artificial environment. In both AR and VR, the information or imagery is typically made available to the user using a headset. XR headset vendors include Meta, Pico, Sony, and HTC who offer VR devices, while Microsoft, Magic Leap, and Vuzix offer AR headsets and glasses. In June 2023, Apple introduced its first-ever immersive headset in the form of Apple Vision Pro. Designed primarily as a mixed-reality device, Apple Vision Pro allows users to move from AR to VR experiences through the turn of a dial.

XR applications continue to grow and develop

Experts suggest that improvements to XR hardware, such as the introduction of smaller and more comfortable devices, coupled with developments in software applications and connectivity, will support the growing adoption of XR across a range of sectors. Healthcare, manufacturing, and the automotive industry are expected to be among the most disrupted by XR technologies. As technology improves further, the use cases of XR will widen and strengthen, with the economic benefits of XR expected to be felt worldwide.

Is the metaverse the future or just hype?

Perceived to be the next iteration of the internet, the metaverse is where the physical and digital worlds come together. The evolution of social technologies allows digital representations of people, avatars, to interact with each other in a variety of settings. Whether it be at work, in an office, going to concerts or sports events, or even trying on clothes, the metaverse provides a space for endless, interconnected virtual communities. Already, several companies including Meta, Microsoft, Roblox, and Epic are investing in their versions of the metaverse. However, the development of the metaverse will require collaboration across companies, creators, and policymakers.

# data[].attributes.attachment_link.description
Find the most up-to-date statistics and facts about extended reality (XR): augmented reality (AR), virtual reality (VR), and mixed reality (MR)

# data[].attributes.body
This website details how to write prompts and which AI software is best suited to a format/task.

# data[].attributes.attachment_link.provider_name
Social Media Marketing & Management Dashboard

# data[].attributes.attachment_link.title
How to write AI art prompts [examples + templates]

# data[].attributes.attachment_link.description
Creating cool AI art prompts isn’t as complicated as it might seem. You can use simple formulas to generate high-quality AI art.

# data[].attributes.body
This program needs to be given prompts to understand the theme or style of the generated environment.

It provides the ability to turn a quick sketch into a skybox,

generating complete 3d environments in order to visualise scenes and lighting quickly.

# data[].attributes.attachment_link.readable_text
On a mission to

Bring out the world-building technowizard in everyone

We believe that the future belongs to the visionaries. At Blockade Labs, we're committed to breaking down the barriers between technology and creativity so that everyone can manifest their ideas into reality. Join us as we share our scientific spellcraft and keep an eye out for what's next!

# data[].attributes.attachment_link.description
Explore limitless AI-generated 3D worlds with Blockade Labs' Skybox. Immerse yourself in new landscapes and environments.

# data[].attributes.body
A user plugs a photo in and the AI can generate video from the image.

In future Gen-2 aims to generate video from text prompts.

# data[].attributes.attachment_link.readable_text
AI systems for image and video synthesis are quickly becoming more precise, realistic and controllable. Runway Research is at the forefront of these developments and is dedicated to ensuring the future of creativity is accessible, controllable and empowering for all.

# data[].attributes.attachment_link.description
A multimodal AI system that can generate novel videos with text, images or video clips.

# data[].attributes.body
"An AI tool that automatically animates, lights and composes CG characters into a live-action scene" - Wonder studio

Can generate a 3d object to replace and accurately track a subject in a video. It can also be used in order to generate a background.

Once the AI has produced the video, it gives 3d data for the model just created and this allows for fine tuning in a different 3d software e.g. Mya and Blender

# data[].attributes.body
A user can provide prompts to create a 3d space from a 2d image.

Can be useful for creatives requiring a quick mock-up to be able to quickly visualise a scene or 3d space.

# data[].attributes.attachment_link.readable_text
Trained on Adobe Stock images, openly licensed content, and public domain content, Firefly is designed to be safe for commercial use. To ensure that creators can benefit from generative AI, we've developed a compensation model for Adobe Stock contributors whose content is used in the dataset to retrain Firefly models.

# data[].attributes.attachment_link.title
Adobe Firefly - Generative AI for everyone

# data[].attributes.attachment_link.description
Use simple text prompts in 100+ languages to experiment and create extraordinary images, text styles, color schemes, and more with the Adobe Firefly web app.

# data[].attributes.body
This week we were given feedback on our work, I found this particularly helpful in deciding the question and creative direction for my essay.

I have chosen to focus on AI and its impact on the industry as many of the topics we have covered in the last weeks have had relevance to the emergence of AI and how its rapid advancement will have a large impact on the creative landscape.

We also had a talk by James Uren, the technical director at Mo-Sys. His company focuses in cutting-edge virtual production.

# data[].attributes.body
Virtual production is an umbrella term including a suite of techniques that combine live footage and graphics in real-time.

Virtual Production Uses:

Creates sets and environments quickly

Reduces the cost of traditional sets and props

Enables filmmakers to make instant adjustments

Allows for a more natural and controlled lighting setup

How does it work?

The main process behind virtual production is the integration of different technologies.

These include: CGI, motion capture, facial recognition, laser scanning, robotics, AR and AI. Most commonly, this is achieved through programs such as Unreal Engine and ILM’s Stagecraft.

Definition of VP:

https://www.studiobinder.com/blog/what-is-virtual-production-definition/#:~:text=Virtual%20production%20is%20a%20method,effects%20on%20a%20virtual%20set.

# data[].attributes.attachment_link.readable_text
Mo-Sys is an award-winning, world-renowned manufacturer of virtual production solutions & camera robotics for Film, HETV & Broadcast.

Mo-Sys robotic camera heads gave us the flexibility we needed. Their reliability and precision was essential for our production.

EMMANUEL LUBEZKI

3X ACADEMY AWARD WINNING CINEMATOGRAPHER

We are Mo-Sys

Specialising in virtual production solutions, Academy virtual production training, camera tracking, robotics and remote production systems, Mo-Sys has over 20 years of industry experience in the conception, production and installation of full turnkey and bespoke solutions. Everyone at Mo-Sys brings their own unique personality to the table and shares the same common vision: to deliver the highest quality innovations, products and service possible to our customers.

Broadcast Robotics

MoRail

High performance motorized rail

Camera Tracking

StarTracker Max

Mo-Sys' most accurate Virtual Production tracking solution in a miniat

Virtual Production

MultiViewXR

MultiViewXR brings a traditional multi-camera workflow to LED virtual

Broadcasting Giants

Our comprehensive range of broadcast robotics and patented optical camera tracking technology, StarTracker has been installed in more than 100+ TV stations around the world, powering virtual studios and augmented reality graphics.

Get in touch

Shot on Mo-Sys

Over the last 20 years we have established and maintained a strong reputation within the film community thanks to our high-quality remote camera heads. Mo-Sys film remote systems have been tailored and developed to the needs of filmmakers and are renowned for their strength, high-precision, responsiveness and ease of use.

Find Out More

# data[].attributes.attachment_link.description
Mo-Sys is a manufacturer of virtual production solutions, virtual production training, camera tracking, image robotics and remote production

# data[].attributes.headline
Popular CGI and Motion Graphics software

# data[].attributes.subject
Popular CGI and Motion Graphics software

# data[].attributes.body
Autodesk Flame:

https://www.autodesk.co.uk/products/flame/overview?term=1-YEAR&tab=subscription

Nuke:

https://www.foundry.com/products/nuke-family/nuke

# data[].attributes.attachment_link.title
Autodesk Maya | Get Prices & Buy Official Maya 2025 | Autodesk

# data[].attributes.attachment_link.description
Autodesk Maya is 3D visual effects software for character creation, rigging, animation and simulation. Buy a subscription from the official Autodesk store or a reseller.

# data[].attributes.body
This week we had a lecture directed by Lee McQueen, focusing on virtual production.

# data[].attributes.attachment_link.readable_text
Virtual production is a filmmaking method that involves creating digital sets and using them to shoot live-action scenes, combining virtual and physical worlds to create movies. This technique enables filmmakers to replace traditional physical sets and locations with virtual ones, which can be designed and modified in a much more flexible and cost-effective manner (Chanpun, 2023). In virtual production, filmmakers use visual effects technologies to create highly detailed and realistic digital environments, creating hyper-realistic environments.

The beginning of virtual production was back in 1930 rear projection techniques appear trying to recreate atmospheres with a projection, where the location and the actors were recorded at different times, as did Liliom, one of the first films to use this technique. Then, in 1968, Stanley Kubrick used front projection in the movie 2001: A Space Odyssey.

But, how does it work now? With the evolution of technology, nowadays actors perform against a LED wall -that is replacing the famous green screen- that displays the virtual environment, which is generated and manipulated in real-time by the production team. Do you notice how revolutionary this is? Basically, post-production is now pre-production, because all the visual effects are made even before shooting the scenes. This technique allows filmmakers to adjust lighting, camera angles, and other visual elements on the set and during the production, giving them more control over the creative process.

Virtual production has found applications in various movies, from high-budget sci-fi and fantasy movies to low-budget independent productions. Additionally, virtual production has also been utilized in other media forms, including video games and virtual reality experiences. As technology progresses, virtual production will become even more significant and impactful in the film industry.

What about Artificial Intelligence?

All of the above is possible thanks to Artificial Intelligence (AI).

AI plays an essential role in virtual film production by assisting in the creation of realistic and immersive digital environments (Hogg, 2021). Here are a few ways AI works in virtual film production:

AI is used to generate content, such as landscapes, architecture, and vegetation. This allows filmmakers to create digital environments quickly.

Machine learning is used to analyze and track the movement of objects and actors on a virtual set. This helps to create more realistic and natural-looking scenes by ensuring that objects and actors move and interact with each other realistically.

AI is used to capture the movements of actors and objects in real-time to record motion data, which can be used to animate digital characters and objects more accurately.

AI is used to generate complex special effects and lighting effects that would be challenging to create using traditional methods.

AI is used to render high-quality images and footage in real-time. This allows filmmakers to preview and adjust their digital environments and scenes quickly and easily.
By leveraging the power of AI, filmmakers can now design and construct digital sets that are incredibly detailed, convincing, and dynamic. AI is being used to enhance every stage of the filmmaking process. For instance, Filmmakers can employ AI tools to create 3D models of characters and props that can be used for virtual production. Furthermore, they can use machine learning algorithms to predict a movie's success or analyze feedback from the audience, enabling them to modify their productions to meet the preferences of their target audiences.

Benefits of Virtual Production

This technology offers significant cost and time savings, as filmmakers can avoid the expense of location shoots and create complex and elaborate sets that would be impossible to build in the real world. It also allows for more efficient and cost-effective filmmaking, as it eliminates the need for expensive location shoots and allows for quicker turnaround times (Kamat, 2022).

The more important benefits of film virtual production are:

Cost savings: Virtual production reduces the need for expensive physical sets, locations, and props, resulting in significant cost savings.

Flexibility: Filmmakers can create and modify virtual environments and scenes quickly and easily, providing them with greater creative control and flexibility.

Real-time visualization: Virtual production allows filmmakers to preview and adjust their scenes and environments in real time, enabling them to make informed decisions and save time during post-production.

Improved safety: Virtual production provides a safer alternative to traditional physical production, as it eliminates the need for actors and crew members to work in risky environments.

More realism: Virtual environments and characters can be made to look more realistic, resulting in more immersive and engaging films.

Increased efficiency: By using virtual production techniques, filmmakers can streamline their production processes, resulting in faster and more efficient productions.

# data[].attributes.attachment_link.title
AI and Virtual Production: The Past Meets the Future - AMT Lab @ CMU

# data[].attributes.attachment_link.description
Technology has played a critical role in the film industry since its inception. Advances in the film industry are always closely related to technological advances in society, integrating them quickly into the filmmaking process. Almost every aspect of filmmaking, from pre-production to post-producti

# data[].attributes.body
Case Study:

1899 was released in 2022 on Netflix and follows a group of European emigrants travelling from Southampton, UK on a steamship named Kerberos to start new lives in New York City, US. The series soon revealed itself to have more sci-fi elements embedded in the narrative.

This production enabled advances in:

Remote working

Carbon footprint

Atmospherics

Crowd scenes

Weather

Use of VR in location scouting

Photogrammetry

# data[].attributes.attachment_link.title
Into the Volume: A Behind-the-Scenes Look into the Virtual Production of 1899

# data[].attributes.body
Virtual Production is now also becoming affordable for smaller productions such as this example on Coronation street which enabled lead actors to be involved in complicated stunt work with no risk.

Considerations:

Lighting

Atmospherics (rain/smoke/dust)

Happy accidents

Actor and Crew experience

Actors not being present

The rise of experiential cinema (Gravity, Revenant, Fall)

# data[].attributes.attachment_link.title
Filming Coronation Street using virtual production

# data[].attributes.body
Some of the fundamentals of VP (VR scouting, camera tracking, match moving real to virtual) had already been established in productions such as Avatar and the Lion King remake.

LED screens have become more and more advanced and able to be constructed in large wall mainly used in festivals and gigs.

LED screen technology such as pixel pitch becomes more detailed and affordable mean that having 'Final Pixel' on screen will allow productions to have much more versatility in the set up of their shots.

The Silent Killer of Virtual Production:

https://www.youtube.com/watch?v=M-vtf2TpopY&t=211s

# data[].attributes.attachment_link.readable_text
Are we to witness the end of the green-screen era? A silent revolution is taking place on film and TV sets, virtual production is enabling productions to create immersive and dynamic sets and backgrounds, based on simple LED panels instead of elaborate and costly set designs.
LED panels prove to be the ideal solution and canvas to portray set and backgrounds created in virtual reality. Thereby creating maximum flexibility for the production teams to make last-minute set changes or slight alterations and comforting actors in a setting that is both real and immersive, enhancing their performance in comparison to performing against a green screen.

# data[].attributes.attachment_link.title
LED products for Virtual Production, XR and Film Studios | ROE Visual

# data[].attributes.attachment_link.description
Are we to witness the end of the green-screen era? With its standard-sized LED panels, the ROE Visual LED products are perfectly suited for Virtual Production or film sets.

# data[].attributes.body
The use of computer controlled movements in cameras for visual effects has been used for over 40 years. It is now often done by robotics, however many of the same considerations remain.

The movements used to be very simple and limited to dollying and jib or crane work but with cutting edge advancements in robotics mean that there is much more versatility in movement.

This website shows the advancements robotics have made on camera work:

https://medium.com/@adelstein/how-robotics-is-revolutionizing-film-production-eeb5e79cccd5#:~:text=By%20integrating%20robotics%20with%20Virtual,chances%20of%20post-production%20surprises.

Computer controlled camera movement allows adjustments for:

Multiple passes

Crowd

Scale

Atmospheres

Speed

# data[].attributes.attachment_link.title
Star Wars Visual Effects, from AT-ATs to Tauntauns

# data[].attributes.attachment_link.description
Learn the secrets behind the Star Wars saga's many innovative visual effects, from the Death Star's detailed interiors to the towering AT-AT walkers. This vintage featurette opens by detailing the chess game played between Chewbacca and C-3PO in Star Wars: Episode IV A New Hope.

# data[].attributes.body
This week I focused on finding relevant research to back up my essay, I even utilised AI myself in order to create a title image.

# data[].attributes.body
I experimented using this free AI text-to-image generator in order to produce a title image for my essay.

After trying various different prompts I settled on "AI in creative industry" which yielded the most accurate result for my topic.

# data[].attributes.attachment_link.readable_text
Create an image from text prompt

Choose a model

Standard

Choose a style

The selected style is only available to PRO users. Please upgrade or try a different style

This is an AI Image Generator. It creates an image from scratch from a text description.

Yes, this is the one you've been waiting for. This text to image generator uses AI to understand your words and convert them to a unique image each time. Like magic.

This can be used to generate AI art, or for general silliness.

Don't expect the quality to be photorealistic, however. You would need a really really big AI to do that, and have you priced those lately?

If you can't think of something, try "Balloon in the shape of X" where X is something you wouldn't find in balloon form.

API Docs

Recently generated images:

AI Image Generator API Documentation

Pricing: $5 per 100 API calls, or $5 per 500 for DeepAI Pro subscribers

grid_size
Pass a string, either "1" or "2"
"2" is the default, which returns a 2x2 grid with 4 images.
Pass "1" to only receive 1 image.

width, height
Pass a string, eg "256" or "768" (default 512)
Use values between 128 and 1536.
Note: values above approximately 700 or below 256 may produce strange outputs.

image_generator_version
Pass a string, either "standard" or "hd" (default standard)

negative_prompt
Pass a string to indicate what you want to be removed from the image
Can be used to enhance image quality and details.
Example negative prompts: bad anatomy,bad proportions, blurry, cloned face, cropped, deformed, dehydrated, disfigured, duplicate, error, extra arms, extra fingers, extra legs, extra limbs, fused fingers, gross proportions, jpeg artifacts, long neck, low quality, lowres, malformed limbs, missing arms, missing legs, morbid, mutated hands, mutation, mutilated, out of frame, poorly drawn face, poorly drawn hands, signature, text, too many fingers, ugly, username, watermark, worst quality.

Frequently Asked Questions

Is commercial use allowed?
Yes, all commercial use is allowed for the generated images.

You can use these images in general for any legal purpose you wish.

Please see our full terms of service here:Terms of Service

Can I use the generated images for NFT?
Yes.

Who owns the output?
The images are considered public domain, that is, they have no owner.

Copyright on output?
The images generated by the AI have no copyright.

Can I get higher resolution or higher quality images?
At this time we don't offer higher quality or higher resolution. What you see is what we have available, which will improve over time.

Is the quality of the images good enough for printing?
In general the quality is good enough for printing smaller images. Larger prints might become very blurry.

# data[].attributes.attachment_link.description
This is an AI Image Generator. It creates an image from scratch from a text description. Yes, this is the one you've been waiting for. This text to image generator uses AI to understand your words and convert them to a unique image each time. Like magic.

# data[].attributes.body
A large concern with AI is its threat to the authenticity of work in the creative industry.

This ability to potentially duplicate and therefore undermine the value of artistic expression has left many concerns for the future of original online art.

I applied some of the concepts mentioned in these references to highlight the importance of ethical considerations when using AI for profit and creative work:

How AI is affecting copyright issues:

https://thenewstack.io/the-power-and-ethical-dilemma-of-ai-image-generation-models/

An article from Harvard detailing the potential issues brought by AI's rapid advancement "Great promise but potential for peril":

https://news.harvard.edu/gazette/story/2020/10/ethical-concerns-mount-as-ai-takes-bigger-decision-making-role/

# data[].attributes.attachment_link.readable_text
← Media Ethics

Newsletter

Stay up to date
Notice: JavaScript is required for this content.

CASE STUDY: The Case of Dall-E 2

For thousands of years, art has been an endeavor of the human race. From Rembrandt to Basquiat, from the Benin Bronzes to the new wave cinema of Hong Kong, art has been recognized as creative expressions of human intelligence. With the public release of DALL-E 2, a neural network that generates images from phrases, the definition of art might be due for reevaluation to include media produced by artificial intelligence. Generative AI, like countless other technologies emerging in the cyber-physical realm, present numerous ethical challenges.

DALL-E 2, aptly named for the combinative surrealism of Salvador Dalí and the futurism of Pixar's WALL-E, is a machine learning model developed by research company OpenAI. The program model allows users to render images from a description in natural language. Because it relies on information databases from web servers as reference points, the illustrations are seemingly boundless. DALL-E 2 has the capability to learn and apply artistic styles, like "impressionist watercolor painting," too. Realistically, the limits of its potential depend on the creativity of its input phrases.

On September 28, 2022, DALL-E 2 became universally available. With its release, renewed debate about the merits of AI art came to the fore. Specifically, artists and graphic designers began to consider how this technology can facilitate their professions. Patrick Clair, Emmy-winning main title designer, said, "It's like working with a really willful concept artist," (Roose, 2022). Others in the visual arts scene worry that programs like DALL-E 2 might put them out of work in the same way that automated machinery shrunk the manufacturing workforce. For instance, it isn't hard to imagine the cover art for an electronic dance album to be designed by AI rather than a human. Why would a rock band commission Andy Warhol, say, when DALL-E 2 can generate " rotting banana in pop art style " for free?

Just as manufacturing companies defend the utility of automation, some artists don't denounce AI in the design sector. Many retail corporations argue that computerized self-checkout stations save money by allowing employees to do other work that can't be automated. In the same vein, AI art can maximize the efficiency of artists by sparking creativity or inspiring someone's first step in the final project. One interior designer said, "I think there's an element of good design that requires the empathetic touch of a human ... So I don't feel like it will take my job away," (Roose, 2022).

In addition to the ethics of employing AI art for commercial and professional use, the content DALL-E 2 and others produce is ripe for discussion. Dr. Eduardo Navas, an associate research professor at Pennsylvania State University who studies DALL-E 2, finds that it functions metaphorically "almost like God-all a person has to do is to state a prompt (the word) and it is." Aside from some restrictions in the algorithm, such as pornography and hate symbols, there are limited guardrails for what prompts can generate. While an earlier version of DALL-E filtered out all images of people, the current model allows users to render public figures in positions and settings that could be deemed offensive or simply implausible. Further, even if the average person can distinguish AI renderings from real images, some people might think a meme generated by DALL-E 2 actually happened. These artificial pictures can cause reputational damage for professionals and celebrities, and it can present national security implications for politicians and leaders.

According to Dr. Navas, the progression from DALL-E 2 static images to AI-generated video clips isn't a matter of "if, but when." This raises significant ethical concerns related to credibility and accountability. For one, falsified videos of public figures saying and doing outrageous things, or deepfakes, can erode trust in institutions. One study shows that rather than misleading people, AI deepfakes make them feel uncertain, which "reduces trust in news on social media," (Vaccari & Chadwick, 2020). It's entirely possible for bad actors to deceive the media with a false video, leading to widespread circulation even after it gets flagged for misinformation and retracted. Another AI researcher said, "If I got [an image] off the BBC website, the Guardian website, I hope they've done their homework and I could be a bit more trusting than if I got it off Twitter," (Taylor, 2022). Still, news websites regularly publish mistakes, and sometimes a redaction or correction isn't enough to curb the spread of it.

As a consequence of AI coding and algorithms, the next dilemma stems from bias and stereotypes. One journalist said, "Ask Dall-E for a nurse, and it will produce women. Ask it for a lawyer, it will produce men," (Hern, 2022). This is partly due to the web servers that provide the program with learning material. Some might argue that DALL-E 2 is still in its early phases, and the OpenAI programmers need time to work out the kinks. However, the algorithm likely can't eliminate stereotypes on the users' end. Dr. Navas, who is Latino, said that a colleague ran the prompt on DALL-E 2 to generate images of "the most beautiful woman in the world." DALL-E 2 then created images of white women (DALL-E 2 gives four initial results). Dr. Navas ran the same prompt a few days later and got a different set of images-in this case, of women who appeared ethnically ambiguous and who might be perceived as Latina. In all instances, the women in the images were portrayed in Western-style dress. It is not clear if user data was accessed for running each prompt, but problems with diversity and user privacy arise if such information played some role in the different results.

Then there's the complicated question of ownership: who should own the copyright to the generated images? One could make the case for the companies behind the coding, like OpenAI. After all, if it weren't for the labor of the programmers, there wouldn't be a final product. The terms and conditions for DALL-E 2 stipulate that OpenAI holds the copyright for images rendered, but the users retain ownership of the prompts they entered manually. This might be a fair compromise, but as Dr. Navas points out, the source material for the machine learning model isn't owned by OpenAI either. With "1.5 million users generating more than two million images every day," intellectual property laws will need to be reconsidered for AI art.

To conclude, DALL-E 2 and other text-to-image AI technology showcase myriad ethical challenges, and this article in no way intends to be exhaustive. As mentioned, AI art can pose risks to human artists and their opportunities to earn money. At the same time, the programs can aid designers by sparking creativity in the initial stages of a project. On the content side of AI art, the images can be offensive or harmful, and this danger is further exacerbated with the potential for AI deepfake videos. Then, because the code relies on human input and a vast repository of reference images, the generated content can be plagued with biases. Lastly, the realm of AI art is relatively new, so laws and policies related to ownership will require ethical reasoning to determine who or what owns the images. For better or worse, DALL-E 2 is now publicly available and gaining popularity. It's no longer a question of whether AI should generate art, but rather how ethics can guide the answers to these complicated and unique challenges.

1. Are there ethical problems when it comes to AI generating art? Which values are in conflict in this case study?

2. If you were on the team that helped create DALL-E 2, what kinds of content, if any, would you restrict from appearing in the results? Why?

3. In regard to copyright ownership, what would be an ethical way of determining which parties, if any, deserve the right of possession? What if the art is sold commercially?

4. Should we classify AI-generated images as art? What are the qualifications for something to be considered art?

Hern, Alex. "TechScape: This cutting edge AI creates art on demand-why is it so contentious?" The Guardian, May 4, 2022. Available at: https://www.theguardian.com/technology/2022/may/04/techscape-openai-dall-e-2

Robertson, Adi. "The US Copyright Office says an AI can't copyright its art." The Verge, February 21, 2022. Available at: https://www.theverge.com/2022/2/21/22944335/us-copyright-office-reject-ai-generated-art-recent-entrance-to-paradise

Roose, Kevin. "A.I.-Generated Art Is Already Transforming Creative Work." The New York Times, October 21, 2022. Available at: https://www.nytimes.com/2022/10/21/technology/ai-generated-art-jobs-dall-e-2.html

Taylor, Josh. "From Trump Nevermind babies to deep fakes: DALL-E and the ethics of AI art." The Guardian, June 18, 2022. Available at: https://www.theguardian.com/technology/2022/jun/19/from-trump-nevermind-babies-to-deep-fakes-dall-e-and-the-ethics-of-ai-art

Vaccari, C., & Chadwick, A. (2020). "Deepfakes and Disinformation: Exploring the Impact of Synthetic Political Video on Deception, Uncertainty, and Trust in News." Social Media + Society, 6(1). https://doi.org/10.1177/2056305120903408

Dex Parra & Scott R. Stroud, Ph.D.
Media Ethics Initiative
Center for Media Engagement
University of Texas at Austin
February 24, 2023

Image: "Vibrant portrait painting of Salvador Dalí with a robotic half face" / OpenAI
This case was supported by funding from the John S. and James L. Knight Foundation. It can be used in unmodified PDF form in classroom or educational settings. For use in publications such as textbooks, readers, and other works, please contact the Center for Media Engagement.
Ethics Case Study © 2023 by Center for Media Engagement is licensed under CC BY-NC-SA 4.0

# data[].attributes.attachment_link.description
← Media Ethics Stay up to date Notice: JavaScript is required for this content. For thousands of years, art has been an endeavor of the human race. From Rembrandt to Basquiat, from the Benin Bronzes to the new wave cinema of Hong Kong, art has been recognized as creative expressions of human intelligence.

# data[].attributes.body
Algorithms used in Spotify help tailor your music taste:

https://www.theverge.com/2024/4/8/24124070/spotify-ai-playlist-music-beta-launch-premium-uk-australia.

Netflix machine learning:

https://research.netflix.com/research-area/machine-learning

# data[].attributes.attachment_link.title
Generative Fill - AI Image Filler - Adobe Photoshop

# data[].attributes.attachment_link.description
With Generative Fill in Adobe Photoshop, add and remove contents from using simple texts prompts powered Adobe Firefly generative AI.

# data[].attributes.body
This link shows how some artists are working alongside AI, providing a silver lining to the idea that AI could be detrimental for creative work.

# data[].attributes.attachment_link.readable_text
AI: More than Human

AI: More than Human is an unprecedented survey of creative and scientific developments in Artificial Intelligence, exploring the evolution of the relationship between humans and technology.

AI: More than Human presents new commissions and projects by artists, scientists and researchers Joy Buolamwini, Mario Klingemann, Kode 9, Lawrence Lek, Massive Attack, Lauren McCarthy, Yoichi Ochiai, Neri Oxman, Anna Ridler, Chris Salter, Sam Twidale and Marija Avramovic, and Universal Everything.

The exhibition tells the rapidly developing story of AI, from its extraordinary ancient roots in Japanese Shintoism; Ada Lovelace and Charles Babbage's early experiments in computing; to AI's major developmental leaps from the 1940s to the present day to show how an age-old dream of creating intelligence has already become today's reality. Told through some of the most prominent and cutting-edge research projects, from Deepmind, Jigsaw, Massachusetts Institute of Technology (MIT) Computer Science Artificial Intelligence Laboratory and Sony Computer Science Laboratories,alongside the artists who are embracing its new possibilities in their work.

With digital media, immersive art installations and a chance for visitors to interact directly with exhibits to experience AI's capabilities first hand, this festival-style exhibition examines the subject from multiple global perspectives and give visitors the tools to decide for themselves how to navigate our evolving world. It will ask the big questions: What does it mean to be human? What is consciousness? Will machines ever outsmart humans? And how can humans and machines work collaboratively?

Co-produced in collaboration with Forum Groningen.

Meet the curators and artists

In collaboration with Google Arts & Culture, explore the AI: More than Human project page, featuring exhibition highlights, interviews with curators, artists and advisors, artist profiles.

Could the line between the artificial and human be different in Eastern and Western philosophies? Dr Beth Singler, a Research Fellow in Artificial Intelligence, draws on her experiences in Japan and her anthropological research to investigate.

Barcelona, Spain (17 Oct 2023 - 17 Mar 2024)
A Coruña, Spain (5 Sep 2022 - 5 Feb 2023)
Guangzhou, China (25 Jan 2022 - 10 Jun 2022)
Liverpool, UK (18 May 2021 - 31 Oct 2021)
Groningen, Netherlands (6 Dec 2019 - 16 Aug 2020)
London, UK (16 May 2019 - 26 Aug 2019)

Curatorial Team
Guest Curator: Suzanne Livingston and Maholo Uchida
Associate Curator: Luke Kemp
Assistant Curator: Anna Holsgrove and Marie-Charlotte Carrier
Advisory Team
Ramon Amaro
Paolo Antonelli
Natalia Fuchs
Hiroshi Ishiguro
Kenric McDowell
Anders Sanberg

Co-production partner
Forum Groningen, Netherlands

Hire fees
Upon application

Duration
Minimum three months

Availability
March 2023 - onwards

Dimensions
c. 800 - 1,200 m2
minimum ceiling height required: 3.5 m

Hire fee includes
Loan fee; exhibition structure and vitrines; crates; audio visual equipment; objects; rights clearance and licenses; texts in British English in a printable digital format; advice on layout and design; and advice on installation and removal.

Hire fee excludes
Barbican staff travel, accommodation and per diems; exhibition inbound transport (3 x 40ft sea container); exhibition spatial and graphic adaptation; text translation; gallery preparation and lighting; all operational costs including press and marketing.

Please note that Barbican touring exhibitions can only be reserved through formal contracts, letters of intent and payment of a deposit.

Explore the Barbican Immersive programme

AI: More than Human Exhibition Catalogue

In collaboration with Google Arts & Culture, explore the AI: More than Human project page, featuring exhibition highlights, interviews with curators, artists and advisors, artist profiles.

Could the line between the artificial and human be different in Eastern and Western philosophies? Dr Beth Singler, a Research Fellow in Artificial Intelligence, draws on her experiences in Japan and her anthropological research to investigate.

AI: More than Human Exhibition Catalogue

# data[].attributes.attachment_link.description
AI: More than Human is an unprecedented survey of creative and scientific developments in Artificial Intelligence, exploring the evolution of the relationship between humans and technology.