'The Abyss', VFX Artist, Stephen Rosenbaum

0:09Jawad MirWelcome back, everyone, to another episode of Kingdom of Dreams Podcast, the making of stories. I'm your host, Jawad Mir. On today's episode, I have a guest who I've had the pleasure of speaking to last year for Masters of Air, VFX supervisor Stephen Rosenbaum. On today's episode, he's here to talk about one of the most underrated films, a classic, James Cameron's The Abyss. While there were several VFX shots, but one of the VFX shots that changed the course of history for the films that we see today, was the tentacle scene, the water tentacle scene, which interacts with the characters. And I also spoke to him about bunch of other films, which I'm gonna split into different parts. So in the next couple of weeks, there will be T2, Terminator 2, by James Cameron. Then after that couple of weeks, it'll be Jurassic Park, and then following that, it will be Robert Zemeckis', Forrest Gump. Without further ado, here is my conversation with VFX artist Stephen Rosenbaum.

1:10Jawad MirI'll tell you a little bit about The Abyss. It's probably one of the mo… for me personally, it's probably the most underrated James Cameron film, to date, in my opinion, because not many people talk about it. But it's, one of the most unique films that has come out on the subject that it is about. And, I remember watching it after I had seen T2, you know, with the whole, the water sequence and everything. And, still to-to date, it's one of my favorite... If you wanna call it an alien film or whatever that is, that it, that has been done, and very effectively and ambitiously.

1:50Stephen RosenbaumYeah, it's, it was f-for us, of course, we can get into, you know, how that came about. But…

1:57Jawad MirPlease

1:57Stephen Rosenbaum... It, you know, for a sci-fi movie, it was, it was really well done. I mean, typical of Jim, how he makes movies. He, you know, he knows how to grab you right from the beginning and hold you. But our work, interestingly, you know, there was quite a bit of effects work, but specifically in computer graphics, there was only about 90 seconds worth of footage that we produced. So... But it's funny because m… a lot of people who say they, you know, they, "Oh, I watched The Abyss. I loved that water creature." it was really just a minute and a half worth of our work. It had a lasting impact. It w… I think it was, it was really the beginning of, like, what you could do with animated characters, you know, albeit a water, a water creature. We call it the pseudopod, was the technical term, which I think Jim…

2:49Jawad MirPseudopod

2:49Stephen Rosenbaum... Jim made up. I, but yeah.

2:53Jawad MirBut that's so true that I... It's, it, I literally, 10 minutes ago, I pulled up that sequence on YouTube. And it's, I'm so glad somebody uploaded the anamorphic version because I have an anamorphic monitor, so j… you can just see it in all its glory.

3:08Stephen RosenbaumYeah.

3:08Jawad MirAnd the most am… it was the beginning of everything, Jurassic Park, T2, and all that stuff. Interestingly, considering it was, what, 40 years ago almost to date?

3:20Stephen RosenbaumYeah. Yeah.

3:20Jawad MirRight?

3:21Stephen Rosenbaum1989. Yeah. Yeah.

3:24Jawad MirIt seems almost like as it was done today. That's so amazing about it. It looks so real. There was probably only one part which you could tell had maybe, because it was a lot of movement…

3:37Stephen RosenbaumYeah

3:37Jawad Mir... You could tell it's a little bit, you could tell it's effects, is when it's about to go out, and out the door, and you can see a little bit of movement not happening organically. But everything else was just, you know, it's... Yeah, i… it's, it's amazing that how real it looks to date.

3:54Stephen RosenbaumThanks. It's, it's interesting. We… There's a lot of history. Let me, let me back up for a second.

4:00Jawad MirPlease.

4:00Stephen Rosenbaum'Cause I'll, I actually, you mentioned the door shot, and there's, there's history there that, most people, anybody, most nobody knows about, really. But the department itself, computer graphics department, part of ILM, was sort of this fledgling group. We were, I think at the time of The Abyss, we were maybe 10 people. And, we… It was a relatively new group. Just h… h… the history behind that department was originally it was formed as part of, George Lucas had the computer graphics department. And the guys, the previous guys who were there were Ed Catmull and Alvy Ray Smith and the likes of those, you know, basically the f… the f… our forefathers of, in computer graphics. And they were not doing a whole lot of production work per se. They were developing, you know, rendering software, which ultimately became RenderMan.

5:00Stephen RosenbaumIt was called ReYAS at the time. And, they were developing some, compositing tools and so forth. They had a paint program, you know, the very early paint program, where you could layer images, into a frame buffer and paint through from one to the other. And, the, Steve Jobs came along, and they, he struck some type of deal, where they literally, the entire department, the guys moved to the building next door and formed Pixar. And then they hi… they hired John Lasseter, and they, he st… they started h… principally to do, demo animations to show off their rendering software, you know, kind of amazingly. They weren't in business originally to do s… production work. They w… they wanted to, you know, they had software development ambitions. But they left all their tools behind and, you know, all the software that they had developed and some of the hardware and, including a scan a laser scanning device, which we would output, you know, our stuff to film.

6:08Stephen RosenbaumSo that was kinda interesting. So the… they were in the building next door. We still had a relationship with them. They left all the s… the tools behind. And, Doug Kay and George Soblov came in and ran the department and started hiring people over the next couple years. And I came in literally out of school. And, you know, we were kind of doing little bits of work here and there, but then Jim came along and said, "Let's do The Abyss." And w… we had no idea how to do that. And in particular, there was, there was no idea how to do facial animation. I mean, the concept of facial animation was, just nonexistent. But we had done… We had some really smart people. We had this guy, Doug Smythe, who actually is still at ILM, one of the original inventors of Morph. If you remember Morph, how popular it became in the '90s. Everybody's got a Morph something. We still use Morph to this day, but, you know, it's not d… it just doesn't have the, you know, amazement that people have really.

7:12Stephen RosenbaumIt's kind of you can do it now on your phone. But, we came up with this idea, of let's scan... We… There… Th… We had the ability to do cyber scans. Were they called cyber scans? They were just basically, essentially like a LIDAR, a laser scan, a depth map, essentially scan of your face. And we took, for example, Mary Elizabeth, Mary Elizabeth Mastrantonio, who was the actress in The Abyss, and we scanned her face and at various expressions. So we had her at a sort of a neutral pose and a smile and, you know, doing some simple basic expressions. And then what we would do is we would take the depth map of that, which was essentially represented as a two d… two-dimensional image, and morph between the two, the expressions. So we would say, "Okay, make Mary smile." So we'd go, you know, from a neutral pose to a smile.

8:15Stephen RosenbaumAnd you'd, you'd do it as a morph, and so you had control over timing, how fast the smile was, how big the smile was. You know, did you wanna make it a little more asymmetric or whatever. And then essentially you would project that onto the pseudopod, and you'd get, you know, which was basically just a simple water shader of a creature, a tube, a cylindrical tube. But posed in a position where we had essentially, you know, a three-d… you know, as the, as the depth map, it was three-dimensional, data, which we could then, perturb or, distort the water element, into that expression. So it was, it was kind of, you know, we were making it up as we went, but it was very effective at that point. But, there was a… You mentioned the door thing. I want… just wanted to touch on that because the… there was like these…

9:14Jawad MirSure

9:14Stephen Rosenbaum... All these little innovations that, you know, just showed the wealth of talent we had hiding in that department when Doug and George hired these people. They were really just unsung heroes of the b… early days of computer graphics. We had,.. One of the guys I worked with a lot was John Knoll, who's also still at ILM. And he, we… At… Back in the day, we, i… in The Abyss, we didn't have… We sort of had the ability to do our own composites, but everything was done traditionally at that point. We would, we would render our pseudopod element and then, and then, film it out as an el… as a, as an element. It's basically an element against black with its matte as a separate render. And then send those film elements over to optical, the optical department, where they would then bipack it against, a background plate and composite it in the, you know, the traditional means of compositing back in the day, optical compositing.

10:18Stephen RosenbaumAnd, it was sort of a curious thing because we h… the Pixar guys, and this... Sorry to give you this long-winded ex…

10:28Jawad MirNo, you're fine

10:28Stephen Rosenbaum... But it's, it's good background. So…

10:30Jawad MirAbsolutely

10:31Stephen Rosenbaum... The Pixar guys had left behind these, essentially they were frame buffers. They were, they were ability... It allowed you to load in images i… into a, into virtual memory, and then you could layer the images on top of each other. So you could do sort of rough composites, essentially. But we didn't have, we didn't have real control over matting, and that was a sort of the, sort of a big fundamental problem. So when you talk about like the door where the pseudopod moves in through the door, from behind the backside of the door, so it needs to be matted out behind that, or at least the backside of it, and then move out, and then it backs back out and it disappears. So that's all part of the matting. It needs to be behind the door and then in front of the door and then behind the door as it exits. So we didn't, we didn't have that ability to do mattes well. And John came in one day, and he s… s… he and I actually sat down and we did a test. We said, "Well, why… We have this paint program where we can l… l… load in the background plate, into the, into the back layer, and then, load in the rendered element into the middle layer and then start painting a matte over it."

11:48Stephen RosenbaumIt was a, you know, we had three layers and, we could just start articulating, frame by frame these mattes. And anyway, it's a sort of a long-winded story. It worked. We were able to composite that shot and then film it out, and then that was our first real f…... Official digital composite, which is like, wow, this is really cool. But the bigger story in that was that John, for those of you who don't know, John and his brother Tom, invented Photoshop, and that was the light bulb moment where John went, "Wait a minute, I can layer stuff. I can layer multiple images, and I can, I can control the, you know, the color information. I can control the opacity of things." all this, all these little ideas. And, his brother Tom is a programmer, and he, you know, he w… he basically wanted to fabricate it into, or, you know, create the program into something where you had sliders and dials that you could just, you know, the average user could just move.

12:53Stephen RosenbaumAt the time, we were, we were literally on the command line saying, "Okay, we want it to be darker." Okay, I had to literally write the command to tell the frame buffer how to darken this particular element and as a c… as a command. And so it was v… it was not exactly interactive. Anyway, so that was, that was sort of the sort of one interesting historical moment of the shot, of that shot that most nobody really knows about until I'm talking about it. But it never, I don't know why it was never revealed, but that was sort of the light bulb moment for John, where he went back to his office all excited, and he called his brother Tom, and he's like, "We need to do something like this. There needs to be like an easier way to do this." And they started constructing Photoshop, and that's, that was sort of the moment that inspired Photoshop. So it, you know, aside from the morph, you know, thing, which is sort of, you know, standard tool now in the, in the world of computer graphics that Photoshop has become that from, all from The Abyss really.

13:54Stephen RosenbaumSo kind of interesting stories, backstories on that.

13:58Jawad MirYeah, no, I appreciate you sharing that. And I was going to ask you when you talked about layers, movement, and everything, I was gonna ask you about Photoshop. I'm so glad that you mentioned that because that's literally what Photoshop, the basics of Photoshop is, like layers, masking…

14:12Stephen RosenbaumYeah

14:12Jawad Mir... A… and all that fun stuff. Going back to The Abyss, I've probably seen more behind the scenes stuff on every James Cameron film except The Abyss. Even though, even though I still have like a VHS, somewhere here, like the special edition.

14:29Stephen RosenbaumThere's a great... Yeah, there's, there's an a… awesome bit in there about the whole, oxygenated liquid, you know? Did you, did you

14:40Jawad MirI must have seen it a long time ago, but I don't remember it. So this is like…

14:45Stephen RosenbaumYeah

14:45Jawad Mir... The good old THX days and…

14:47Stephen RosenbaumThat's awesome.

14:49Jawad MirAnd it's called…

14:49Stephen RosenbaumDo you have, do you even have a working machine to play that anymore?

14:53Jawad MirNo, you know what? I've been meaning to get a VCR because I have so many VHS tapes. I even ha… I even have the letterbox edition of Jaws. But yeah.

15:02Stephen RosenbaumThat's a classic. You've gotta keep that.

15:05Jawad MirYeah, no, they, I still have them. I never get rid of any of my stuff. So, but yeah, no, this is like, it says here, contains 28 minutes of footage not currently available on video, director's special edition.

15:19Stephen RosenbaumYeah. No, there's a great... I, by the way, I too, much to my wife's dismay, I have a lot of old VHS stuff, and I still have w… two working VCRs.

15:31Jawad MirOh, you do? Good for you.

15:32Stephen RosenbaumShe's like, "Why do you have these?" Well, because I have these tapes, and maybe one day I might wanna pull one out, you know?

15:37Jawad MirYeah. I get it. I get it.

15:40Stephen RosenbaumBut, yeah, the, there's a, there's a great bit about the whole oxygenated, oxygen... How do you say that? Well, oxygen... How do you pron… how do you pronounce that word? Oxygenated? Oxygena… oxy... I can't.

15:56Jawad MirI don't, I don't know. This is the first time I'm hearing that word.

15:58Stephen RosenbaumThe, viscous material containing oxygen. How was that?

16:04Jawad MirOkay. Okay

16:04Stephen Rosenbaum... And, yeah, the moment where, you know, Ed Harris gets, you know, he goes down deep and he has to rely on this…

16:12Jawad MirOh, yes

16:13Stephen Rosenbaum... This substance in order to give you the pressure balance. And I think some people who follow Jim also know that he's a, he is an avid deep sea diver.

16:21Jawad MirOf course.

16:22Stephen RosenbaumYeah.

16:23Jawad MirYeah.

16:23Stephen RosenbaumSo he knows, he knows his stuff. Anyway, yeah, sorry. I digress.

16:27Jawad MirNo, it's fine. And, you know, one of the things I've spoken to John Rosengrant. I don't know if you've ever worked… ... With him or a few times.

16:35Stephen RosenbaumQuite a bit. Yeah.

16:36Jawad MirYeah. And, I, this is something nothing to do with effects. You know, I always wonder, you know, there's great filmmakers like Spielberg, Michael Bay, James Cameron, you know, even Martin Scorsese, Coppola, but I always wondered, and it dawned upon me a few years ago, and I, this could be just a lame theory, but it makes sense to me that why does James Cameron, why do James Cameron and Michael Bay films work? You know, they have great visual effects. They have great technology, action sequences and everything. But at a human level, why do they work? Why do people tap into them? And if you look at James Cameron films, you know, Linda Hamilton plays a waitress, oil, drillers in The Abyss, you know, a high school student in Terminator 2.

17:24Stephen RosenbaumYeah.

17:24Jawad MirDiCaprio plays like a, you know, like an average poor boy.

17:28Stephen RosenbaumYeah.

17:28Jawad MirAnd it's just that relatability that can tap into the average human being.

17:33Stephen RosenbaumYeah.

17:33Jawad MirAnd then it pulls you in through that realm, and then everything else is grand around.

17:38Stephen RosenbaumYeah. It's very true. He is very good with characters, and developing their personalities as likable characters.

17:46Jawad MirYeah.

17:46Stephen RosenbaumBut I think the other, the other variable with Jim is that, you know, most people associate Jim's movies as, you know, big effects movies.

17:55Jawad MirYeah.

17:55Stephen RosenbaumBut the thing about him is that they're, they're not gratuitous visual effects movies. They're, they're visuals that support his narrative. And whenever that strays, he, you know, he is about… You know, he has his moments where he's like, "I, you know, this has to be the big… biggest frigging effect you've ever seen in your life," and, you know, for that moment. But that, it's, it's appropriate as supporting the narrative. So he's, he's always very attentive to that. And his ability to choreograph action, I think, is al… is also really

18:28Jawad MirYeah

18:29Stephen Rosenbaum... You know, underrated. He's, he's exceptionally good at that. I've, I've, I've worked with him on a few shows and, you know, each time, particularly the last show I worked with him on, you know, you get into... You start thinking, "Okay, well, how would I do this? I'd do da." And then you start imagining the, you know, visualizing it in your head, and then he comes up with something that's, like, 10 times better, and you go, "Oh, wow, that's just genius. It's really..." He's j… He just sees it very clearly. He... Very smart guy.

19:00Jawad MirYeah. Yeah, he is. For The Abyss, the,.. You know, there was that 90-second sequence that we just talked about, but then that next set of, if my memory serves me right, the next set of effects do not come until, like, the last, you know, 15, 20 minutes of the film.

19:18Stephen RosenbaumYeah.

19:18Jawad MirDid you, did you and your team had a lot to do with that, or a little, or...?

19:24Stephen RosenbaumNo, we only focused on the pseudopod, so it was just…

19:26Jawad MirOkay

19:26Stephen Rosenbaum... Just that time there. I can connect you with other people that did work on it. Actually, my neighbor…

19:34Jawad MirYeah, that's fine.

19:35Stephen RosenbaumYeah.

19:35Jawad MirAbsolutely, we can do that, but I was just, well, wondering if you did so we could talk about it. But let's, let's move from that to Terminator 2, because there's…

19:43Stephen RosenbaumWait, I have a s… I... One story I have to relay…

19:45Jawad MirSure

19:46Stephen Rosenbaum... On this that is sort of like a, I think your viewers will laugh at, 'cause this is... We... Just sort of, it really dates the show in terms of w… what we were capable of. First of all, for those computer geeks out there, we were using VAX 850s, I think. And so the, if... It, there were basically micro... There were super computers, or not even super computers. They were just... I mean, what were they? They were just…

20:12Jawad MirLike, for, ma… 486? What was the

20:14Stephen RosenbaumThe regular mainframes computers…

20:16Jawad MirYeah

20:16Stephen Rosenbaum... That had, like, 850 megabytes of disk on them. Giant washing machines, and you would spin it down, and you'd lift it out, and you'd do it. Anyway, w… it was, it was pretty crude, back in those days, but, you know, w… we also were really young and naive on how to interface with, like, Jim Cameron, and so we would, we would do these, animation tests. Steve Williams was our first animator, and he did the animation for the pseudopod. And, we would send these tests to Jim on a three-quarter inch tape, you know, so we'd, you know, record it…

20:55Jawad MirYeah

20:55Stephen Rosenbaum... And send it FedEx, and then it would get... He w… he would fax back a response. And, you know, it, he would literally scribble some notes down, and then he'd send it through, and he, we'd all hover around the fax when... 'Cause it would make a sound, you know, it would beep, you know, it's gonna come. Oh, the fax is, Jim's, Jim's responding. And we'd get the notes. But this one time, we sent it, we sent, an animation test of the pseudopod, and this is Jim's sense of humor. So he faxed back the note, and it's coming through, and if, for those of us that are old enough to remember faxes, you know, it's painfully slow, right? It's like…

21:34Jawad MirOh, yeah

21:34Stephen Rosenbaum... Paper coming out. It's like... You know, that type of thing. And so it's coming out, where there's, like, you know, five of us hovered around the fax machine excitedly waiting for this, for his notes, and it's coming through, and it says, "Looks..." And he just wrote on one piece of paper in big, pen. "Looks like," coming through. "A big dick." So that was his notes to us, basically. Okay, I guess he doesn't like that, so we should work on that. But anyway.

22:16Jawad MirWell…

22:16Stephen RosenbaumOne of those notes…

22:17Jawad MirNo, that…

22:17Stephen RosenbaumOne of those notes to, relay that, you know, Jim Cameron infamously, responded to.

22:25Jawad MirWell, he's a very upfront and straightforward guy, and he would say things that, as he feels he wants to, you know, say that. He's actually originally from where I am from. He used to be a truck driver in,.. I know in Ontario. I don't know if he was in Toronto…

22:38Stephen RosenbaumYeah

22:38Jawad Mir... But probably, yeah.

22:39Stephen RosenbaumYeah.

22:40Jawad MirAnd…

22:40Stephen RosenbaumYeah, from…

22:40Jawad Mir... Look where he is now.

22:42Stephen RosenbaumYeah. Yeah. Yeah, I, he still, he still talks about life in Canada and near the Falls, and I think he grew up around there, right?

22:51Jawad MirNiagara Falls, yeah.

22:52Stephen RosenbaumYeah.

22:52Jawad MirProbably, like, Fort Erie, Niagara Falls, or St. Catharines could be that.

22:56Stephen RosenbaumYeah.

22:57Jawad MirBut now he's in all the way New Zealand, the other part of the world.

23:00Stephen RosenbaumYeah. Yeah.

23:02Jawad MirBut, no, th… thank you so much for…... Literally, like, sharing all these stories, because Abyss, like I said, is, like, one of the most underrated Cameron movie, and I think it was the, one of the reasons Titanic happened is because of The Abyss. Because of his ex…

23:18Jawad Mir... Because it... I think Abyss was more of a difficult shoot, I don't know, I'm just guessing, because of its time and because a giant water tank that they had to build and everything…

23:26Stephen RosenbaumYeah

23:27Jawad Mir... Than Titanic probably was. But obviously…

23:31Stephen RosenbaumYeah

23:31Jawad Mir... Titanic was more expensive. But yeah, no…

23:34Stephen RosenbaumThey had a, they had a... From stories I heard about that, just to digress a little bit more on that…

23:39Jawad MirYeah, sure

23:39Stephen Rosenbaum... Is they, you know, they shot in a, old, nuclear reactor cooling tank, right?

23:47Jawad MirYes.

23:48Stephen RosenbaumAnd, you know, basically shot underwater, 'cause Jim felt like you can't recreate that, you know, you can't do dry for wet, well. And so he, everything. But I remember talking to one camera guy, and he was like, "Yeah, you just, the hours are so long underwater." And he would literally, like, and the water temperature was warm, of course, while they were shooting, so he said a, there were occasionally he would just drift off to sleep underwater. So it's just like…

24:16Jawad MirWow

24:16Stephen Rosenbaum... Soothing him. You know, this water is waiting. 'Cause, you know, those who're not aware, you know, shooting, you know, a movie can be incredibly boring and dull. You know, you're waiting often, you know, half an hour for somebody to set something up, and you're just kinda…

24:32Jawad MirYeah

24:32Stephen Rosenbaum... You can't go anywhere, especially when you're underwater, so you just sit there. Anyway, go on. Sorry.

24:38Jawad MirNo, that makes, that makes... I'm glad you said that because nowadays you can probably set up 10 different cameras in a dif… 10 different corners of a water tank or a swimming pool without them being visible to each other, and you don't... I mean, you can, you need a DP, but you don't necessarily need a DP unless, you know, unless you're doing some, a lot of movement shots. And now the technology has advanced so much that I think those days, to me they're much more appreciated because, you know, you put s… I'm not comparing k… today versus yesterday as to being more better, much better than the other, but it's just the effort and the time that used to go into those things w… especially something like that. Like, it's similar... I think The Abyss had a similar, arc like Jaws did, like how Jaws was the first film to be shot…

25:21Stephen RosenbaumYeah

25:21Jawad Mir... On water, you know, and it had random…

25:23Stephen RosenbaumYeah

25:23Jawad Mir... Ridiculous problems. And sim… Because shooting in water or on water is not easy.

25:29Stephen RosenbaumNo.

25:30Jawad MirNot easy at all.

25:30Stephen RosenbaumWe have a saying in the industry, "Shoot on water, lose a day," basically.

25:33Jawad MirYeah. And it's the same thing with VFX world, like m… now, I mean, technology has advanced so much, but... Because I remember in the '90s and '80s, and even early 2000, animating water was probably one of the most challenging things. Like, the…

25:44Stephen RosenbaumYeah

25:44Jawad Mir... Finding Nemo was not an easy film to make from a…

25:47Stephen RosenbaumNo

25:47Jawad Mir... A budgetary standpoint.

25:50Stephen RosenbaumNow the digital water is so good. They, you know, Weta in particular has that nailed.