When you think of holograms, the first thing that comes to mind might be a glitchy Princess Leia telling Obi-Wan Kenobi that he’s her only hope. Or maybe it will be the Enterprise’s Holodeck or Voyager’s Emergency Medical Hologram (aka “The Doctor”). More recently, we’ve seen Shuri’s tangible (even drivable) holographic creations in Black Panther and a plethora of holographic tools in Tony Stark’s lab and Iron Man suit.
Holograms have captured our imaginations for many decades now, They’ve certainly become staples of popular culture like movies and television. So much so that holographic imagery has deeply influenced futuristic interface design, and elements like a blue glow and a low fidelity style have become iconic tropes for how future UIs will look. I spent a couple of years running this VFX studio and the talented team there was responsible for many entries in that FUI canon.
In fact, we’ve been so successful at conveying holograms in movies, and viewers have become so accustomed to seeing them, that most people believe that the kinds of 3D holograms commonly depicted onscreen are possible. But the truth is that in the real world they really aren’t. “But I have seen holograms!” some people protest, and indeed they have. But when you see the word “hologram” being used today, the images being referred to probably aren’t really holograms at all – they are other tricks of technology and light.
What is a hologram?
There are plenty of resources online to do a deep dive into what holograms are, like this one. I’ll just quote the first line, which sums it up pretty well: “Holography is a photographic technique that records the light scattered from an object, and then presents it in a way that appears three-dimensional.” The key to understanding this topic is that last part – we’ll come back to that in a minute.
We do have actual holograms that we can see in our everyday lives. But they don’t generally live up to our sci-fi expectations. One example is the kind used to prevent counterfeiting – they’re used on ID cards and currency and credit cards, and even some kinds of merchandise. You can buy hologram stickers to slap onto any old product. You can see the three-dimensionality in these holograms, but sadly they are no more impressive than the lenticular cards we had as toys when I was a kid.
Those holograms are static, like a still photo. To even begin to impress us, they will need to be dynamic representations, more like videos, and ideally real-time. Further from our everyday experience, researchers in labs are getting closer ever year to developing true holographic techniques. Last year researchers in Japan made some exciting progress using ‘metasurface’ materials. Earlier this year another team of researchers showed how deep neural networks can be used to generate 3D holograms in real-time. But such examples are both tiny and expensive – far from useful for satisfying our near term longing for holograms.
“Appears three-dimensional”
Here we can return to that definition from above. We don’t have to get hung up on the technicality of capturing the light field as a hologram is defined. What we really mean when we talk about the holograms of our popular imagination is volumetric display – any kind of visual display system that gives us a sense of 3D. In this definition, those moving lenticulars of the 1960s were not that far off. Creating a viable three-dimensional display system is not an easy problem to solve either – but it’s easier than true holography, and there have been no shortage of techniques on that front.
These days, one of the first technologies people might think of for three-dimensional imagery is an augmented reality system like Microsoft’s Hololens. That’s indeed an exciting technology and a great way to interact with 3D content. But since high-quality AR still requires a fairly bulky headset, it doesn’t really meet our expectations of a hologram. You might also remember the push several years ago for 3D televisions. These also required glasses with special lenses in order for the content to appear three-dimensional, so again we are left somewhat disappointed.
From here on out, let’s stipulate that we expect to be able to see a hologram with our naked eyes. No special eye-ware should be required. And let’s acknowledge again here that when we’re talking about holograms, we’re not really talking about “actual” holograms, but rather different kinds of display techniques that appear three-dimensional.
The ghosts of Pepper
Perhaps the best technique for implementing holograms is also the oldest. Back in the 1860s in London, spooked onlookers saw what appeared to be a ghost on stage, but in reality a sheet of glass that had been carefully hung across the stage was casting the reflection of someone down in the orchestra pit. This century-and-a-half-old parlor trick is called Pepper’s ghost. It’s been around so long that it vastly predates the invention of holography itself.
These days we can use powerful projectors to cast the images to be reflected, removing the need for the subject to be anywhere on site. Special foils can be used to better refract the light, resulting in a brighter, more opaque image. This mirror trick still gives the powerful impression that the resulting images are suspended in midair, and it still works well for stagecraft. The otherworldly performance of the late Tupac Shakur at 2012’s Coachella is one of the most oft-cited examples of holograms in action. They can be impressive, but creating holograms at that scale is complicated and expensive.
Creating a pyramid of reflecting panes can even allow viewers to move 360 degrees around the image, increasing the sense of dimensionality. I have a little pyramid like that in my office that supports an iPad and looks great with content made specially for that purpose. When scaled up this effect can come close to meeting our expectations for free-standing 3D holograms. But the apparatus needs to be properly sited and well constructed and at scale it takes a lot of floor space. The content must also be created carefully, and while you can move around the hologram you can only ever see the front of the object. It also requires a screen many times larger than the resulting image, which makes scaling up difficult.
No matter the form factor, any Pepper’s ghost experience requires near total darkness to look good. The more ambient light you have, the more washed out the imagery will appear. That’s why you’ll most often see these experiences in theme parks and entertainment venues, where environments can be tightly controlled and expenses can be amortized over a long period of ticket sales.
Dating back to the 1960s, Disneyland’s ballroom scene in the Haunted Mansion is easily the oldest example (and the closest to Pepper’s original illusion.) The Disney team has even developed a portable version that could be used to project ghostly images to individuals on a ride. You can see the illusion used everywhere from the presidential libraries of Abraham Lincoln and Ronald Reagan to The Hogwarts’ Express at Universal Studios in Florida, where Pepper’s ghost helps guests “disappear” behind the brick wall at Platform 93⁄4.
Because of the transparent imagery resulting from this trick, the effect was originally used to conjure “ghosts.” And our modern incarnations have been fairly ghostly as well – after the success of Tupac’s appearance, there has been something of a trend in bringing celebrities back from the dead, from Michael Jackson (2014) to Whitney Houston (2021). Perhaps that’s simply a lack of imagination for how holograms can be used. Or perhaps the cost and complexity of the “traditional” hologram is only worthwhile when’s there’s literally no other way to see the person.
Smoke and… spinning fans?
In our popular imagination, holograms are often some imagery that’s projected into the air. But we don’t yet have a technique where holograms can appear in thin air because light always needs something to bounce off of in order to reach your eye. That’s why Pepper’s ghost works so well (in the right situation.) But there are other ways to make imagery appear to float in the air or look three-dimensional.
When we talk about illusions, we often use the term “smoke and mirrors.” Pepper’s ghost is a mirror trick, and indeed we can use smoke (or mist) as well. Smoke is just particulate matter floating in the air, so it can be used to reflect back an image. Images can projected onto smoke, fog, or any fine mist in the air. We’ve seen commercially available and experimental versions of fog screens over the years. Like mirrors, smoke requires darkness, and it also only works as long as the mist remains in a smooth laminar flow – wind is as much an enemy as light in this case. Mist screens can be a nice element in an activation, such as the holographic catwalk from Dior’s 2017 “I Feel Blue” show in Shanghai, but they have found limited usefulness overall.
Of course, imagery can also be projected onto translucent fabrics. This can produce very high resolution imagery, since it’s quite similar to projecting onto a screen. Perhaps the most elaborate and well-known use of this technique is the German Circus Roncalli, which replaced all of its live animals with laser projections using 11 long-range projectors. Cirque du Soleil and other stage productions have also used projection onto fabric for years. While the image quality resulting from this approach is good, since fabric is not nearly as transparent as glass, some of the magical quality is lost.
Another technique that gained popularity a few years ago is using linear LED strips that rotate at high speed. Computer software controls each diode based on its location in order to display a video stored in the machine’s memory. Because the blades spin so fast, they become invisible and all your eye sees is the 3D visual content they’re programmed with, making the imagery look like it’s free floating. PUMA used this technique during the NBA All-Star weekend in Chicago in early 2020 to create a floating image of their Sky Dreamer sneaker. Adidas and Louis Vuitton have created activations using similar products. These devices create a nice bright image that works in normal light, and they can be combined to create a larger image surface. But the structure needed to support the devices somewhat defeats the transparency and you do have all the drawbacks of a wall of spinning fans.
A completely different approach to creating imagery that appears three-dimensional is the anamorphic use of LED screens, a technique that’s grown in popularity recently. This involves the creation of clever content for billboard-sized, multi-faceted LED screens that give their visuals the appearance of depth when seen from just the right perspective. One good example is the calico cat that recently became a tourist attraction in Tokyo’s Shinjuku district. Another is the famous WAVE cube in Seoul’s COEX Square. Some anamorphic LED screen activations give the effect of an object reaching out or breaking out of a billboard. While such installations can be engaging and fun, their narrow viewing angle is a big limitation. These are more like modern day trompe l’oeil – we generally wouldn’t consider them holograms.
3D volumetric displays
There have been some interesting new developments in the volumetric display world in recent years and some exciting new entrants into the space. One of those is the Brooklyn-based startup Looking Glass. Unusually true to the nature of holograms, they use light field photo capture to reproduce photos and videos, and create interactive applications with their proprietary 3D software. I have one of their new portrait displays on my desk, and the three-dimensionality of the image is great. Sadly the image quality feels low, because you can see the gradations that produce the dimensional effect. IMHO their previous generation, which had much thicker, heavier screens (and cost a lot more) looked better in that regard. The process to create content for it is cumbersome and proprietary, another potential drawback.
Then there is Sony’s Spatial Reality Display. I haven’t actually seen one of these in person, but from the reviews I’ve read they seem to perform similarly to the Looking Glass. This one talks specifically about the lenticular effect of the screen, similar to what I noted above in the Looking Glass. If anything, it looks to me like an inferior version, which might explain why I haven’t heard much buzz about them since they were released. But it looks like you could pick one up at B&H if you wanted to try it out.
A completely different approach has been taken by the Australian startup Voxon, and it’s one that speaks to my inner Star Wars fan. Their VX1 display, while small, very much feels like R2-D2’s little blue hologram of Leia. I haven’t seen this display firsthand either, but in pictures it looks good. The Voxon device evidently uses lasers to slice the imagery up into small segments so it appears different from every angle. But as we’ve discussed, you can’t project onto thin air. The company says these cross sections are “projected synchronously onto a specially designed high speed reciprocating screen.” So we can surmise that there is some kind of screen material that vibrates fast enough to be transparent – like those spinning fans. This one might be less impressive in person, and certainly could be challenging to scale.
All of these devices seem pretty cool, and they do display actual 3D content, getting us closer to true holograms than something like Pepper’s ghost. But they’re also all quite small (and not inexpensive.) Such displays are currently best suited for content creators and other professionals who need to work in 3D. I’m even part of that target demographic. I can certainly see using something like Looking Glass to model character animations or other 3D content. But I’ve found limited applications of these screens for public activations and the like. And their greatest strength is also one of their great weaknesses – because they display true 3D content, you must create 3D content in order to use them.
A better way
We’ve seen the limitations of Pepper’s ghost and spinning fans and even lenticular volumetric displays. But is there another way? Perhaps a better way? Los Angeles-based startup PORTL thinks there is. And I have to say that I agree. PORTL’s founder was one of many inspired by Tupac at Coachella. He was part of the company that bought those patents, and he worked on nearly all the subsequent hologram events for the next several years. So he certainly knows what works well as a hologram.
PORTL’s approach is simple but clever. They use a clear 4K LCD screen, which gives you the transparency we expect from a hologram. Behind that is a plain white lightbox, which gives you the depth. Voila, hologram! Any subject filmed against a solid background looks like it’s inside the box. The right lighting and shadows really convince the eye. The result is really impressive.
Many of the cleverest innovations seems obvious in retrospect. Such is the case for me with PORTL. I’ve been working with transparent screens and lightboxes for years. I’ve built many lightboxes with clear LCD screens on the front (fun fact: all LCD screens are clear – LCD televisions have a white backlight built into them.) I’ve built some lightboxes with clear OLEDs as well (OLEDs are transparent in the black, while LCDs are transparent in the white.) I’ve even used third-party products that are white lightboxes with clear LCD screens in front.
But in all that time I’d never thought to use a setup like that in this way. Such boxes are typically built to put some products inside and display some overlay content in front of them. Some grocery stores and pharmacies have installed clear screens in their drinks refrigerator doors to display advertising and other content. Google just opened their very first retail store (down the block from my office, actually) and it contains several rows of little lightboxes with clear screens used in precisely that way. That’s nifty, but definitely not the same. Seeing a PORTL in action, it’s hard to believe that someone isn’t actually standing inside it. It really feels like the holograms of our imaginations are finally here.
Production pipelines
The ultra-high-definition resolution of the PORTL provides an incredibly clear image for a hologram, even on a 7′ tall screen – a positive move away from the low-res Leia plea. And the brightness of the backlight makes it clearly visible in room light or daylight. It looks like the holographic equivalent of a regular television because, well, it is. The viewing angle is extremely wide – standing way off to one side does tend to defeat the illusion of depth, but the content itself still looks great. So overall this approach retains most of the benefits of Pepper’s ghost while solving some of its biggest drawbacks.
Human scale feels just right for a hologram, too. PORTL’s initial product, the Epic, is built for a person to be life-sized. Many of the techniques we’ve talked about here can’t scale up enough to be satisfying. Tupac-style holograms can be massive, but the practical applications of that beyond big stage productions are fairly limited. In contrast, seeing a person inside the PORTL at human scale helps to further the illusion. As humans, we’re social creatures, so we’re hardwired to connect with other people. Seeing 3D imagery of life-sized people generates an immediate sense of connection.
But the main benefit of this approach is the ease with which content can be created. It’s so simple that you can do it live. With a basic camera setup you can capture subjects and “beam” them directly onto the device, something PORTL calls “holoportation.” I have such a setup on one side of my office, from which you can beam into a PORTL on the other side. I’ve demonstrated this to dozens of people over the past few weeks and it never fails to amaze. And of course using modern telecommunications infrastructure you can beam across the globe just as easily as you would stream any other kind of video. Beaming to many PORTL Epics is as easy as beaming to one. This is a holographic display technology that fits nicely into our existing video and broadcast pipelines.
And while the content on a PORTL is not technically 3D, it works well for 3D animation also. Using the same simple principles of lighting and shadow, digital artists can easily render CG content that looks amazingly three-dimensional. The screen is driven by a computer so you could even create generative 3D content in an engine like Unity or Unreal that responds to user input, just like a video game would. The PORTL has a touchscreen and not one but two different cameras, so creators of engaging interactive experiences (like me) have all the tools we might want to interact with a user or an audience.
The last thing I love about the PORTL Epic is that it’s nicely productized as a turnkey solution. In theory, I could build something like this, as I’ve done in the past – but why would I? They’ve already done all the difficult engineering and testing and refinement. They’ve literally included everything I could think of adding. And they do own some patents on the visual techniques that produce the sense of depth.
We’ve already seen several exciting activations on this technology and I can’t wait to start seeing where it can have the most impact. It’s fairly early days for PORTL, which was founded in 2019. Once a lot of smart creative people start pushing the boundaries of what can be done with the tech, we’re going to see some really amazing things.
Featured Image: PORTL Hologram