It can be said that Spider-Man: Into the Spider-Verse is right now a hot topic in animated features since its theatrical release in December 2018 with its great story and innovative visuals.

At FMX 2019, Danny Dimian (VFX Supervisor, Sony Pictures Imageworks) and Bret St. Clair (Look Development Supervisor, Sony Pictures Imageworks) from the visual effects team for Spider-Man: Into the Spider-Verse were invited as special guest speakers. Foundry a leading software company for VFX and 3D visual creation, returned to FMX this year as a main sponsor and organised 10 insightful sessions.

We talked to Danny and Bret, as well as Jordan Thistlewood (Product Manager, Foundry) about how they developed the innovative visuals of the film Spider-Man: Into the Spider-Verse using three of Foundry’s products: “Katana”(look dev and lighting toolset), “Mari” (3D painting toolset) and “Nuke”(Node-based compositing, editorial and review toolkit).

Katana and Arnold were used as the primary lighting and rendering tools for the project. Katana was useful to test ideas, textures and patterns and to blend all those in to create an effect that could ripple and yet feel like invisibility (as opposed to say refraction, or cut paper fluttering). It had to have depth and yet the effect had to somehow convey something that told you it was rooted in the graphic style (source: Foundry’s case study).
Mari was the main painting and surfacing package used on the film. They used it to paint and texture the majority of characters, environments, and props. It made the difficult challenge of surfacing hundreds of assets far more manageable and helped the team easily manage details in layers within each of the channels. Other paint packages were also used to allow us to take advantage of their particular features, but then the team would ingest those back into Mari to complete the final look of the textures (source: Foundry’s case study).

Interview with Danny Dimian, Bret St. Clair (Sony Pictures Imageworks) and Jordan Thistlewood (Foundry)

Animationweek (AW): Could you let us know which scenes and images in Spider-Man: Into the Spider-Verse reaped most of the benefits from using Foundry’s products “Katana”, “Mari” and “Nuke”?

Bret St. Clair: I think the scenes that really benefited the most were obviously the end sequence scenes, where we were jamming together the geometry of all of the props and things within an entire city. Plus, we were building these very complex, multi-layered collider beams that we were coming across and interacting with everything, so we got tons and tons of layers.

As for the characters, each of them has their own universe that they live in, so we carry with it all of this look development. It’s very specific, as well as all the render passes that are related to each of their looks. So, between Katana and Nuke, we were able to manage a ton of complexity and do it pretty effortlessly. Just the ability to have one character with an entirely different look, like Spider-Man Noir sitting next to Peni Parker in a scene, and be able to manage that and hand it off to multiple lighters, and knowing that the results are going to be dependable and what you expect, was a huge benefit for us.

Danny Dimian: Like Bret said, the sequences at the end of the movie were not only the most complex, but they were also the ones where we have the least amount of planning for. So, we had a lot of artists and a lot of departments working all at the same time. The ability to bring everything together through our pipeline, especially with the artists having the flexibility in Nuke to actually get everything to look the way they wanted to, and bring in all of these assets as everybody was working on them at the same time, was a real strength for us.

AW: Could you elaborate how those scenes were actually developed?

Danny Dimian: One of the interesting things about these sequences, because they were at the end of the movie, is that they represented a climax for us and they had to be bigger than anything we’d ever done in the movie. Leading up to it, we had been pushing ourselves to be as impressive as possible, so when we got to this point, we were feeling like we’d already got our movie to a 10 out of 10, so where do we go to really be impressive?

One of the challenges was that in order to exceed that, we were so crazy with effects and colors and action, that to bring all of that together required a lot of artistry. The flexibility of all of our toolsets really allowed for all of those changes and tweaks to be very quick, and to really find our way at the end.

Bret St. Clair: Because the tools worked together really well, we were able to push a lot of what we were doing inside of Katana into Nuke, so that meant iteration times, that would’ve been a day or half a day to turn something around, went down into things that we could dial interactively with a slider. We can see numerous iterations, which was incredibly important to us as we were delivering the shots because there were so many and had such complexity.

Danny Dimian: There was not as much of a plan for where everything would go, so if we made a change all of the sudden, it meant that a bunch of other shots had to change very quickly. I’d say the versioning and the structuring, accounting for what everybody was doing with the files, was really helpful to us as well.

AW: What was your impression of these products through the development of Spider-Man: Into the Spider-Verse?

Danny Dimian: They’re incredibly powerful and flexible, because in order for an artist to be able to turn around all the crazy things I asked for as quickly as they do, that is proof of that.

Bret St. Clair: We’ve worked with the tools for a long time, so I had just come to expect that they were going to work well. Admittedly in this film, I typically work more on the 3D side, so I was pleasantly surprised when I needed to get in and port some things over into Nuke, to see how powerful it was and how easily we can build in some of this crazy stuff we were doing inside of the renderer: Just write it inside Nuke and make it interactive.

Danny Dimian: Actually, related to that, because it was a different looking movie, it didn’t go through our regular pipeline, so we wrote a lot of different tools and we broke our pipeline in terms of how we do things. The flexibility of Katana allows us to write all the tools around it and use them as structure, as our hub, that we would build everything onto. That was really helpful to us.

AW: Which features of “Katana”, “Mari” and “Nuke” helped you realize the images that you wanted to have?

Bret St. Clair: I think Edwin Fabros (Texture Paint Supervisor, Sony Pictures Imageworks) has mentioned the ability to paint across multiple surfaces; A lot of the tools can quickly get textures onto surfaces so that we can prototype looks and get them. I know we were leaning heavily on the texture department to get those textures to us as quickly as possible, and so that was never a problem. Plus, Mari is really the gateway through which textures come. We used a number of packages, but it all comes through Mari in the end.

Danny Dimian: Everything is organized, and that and its versioning goes through Mari. Again, when we’re working this fast, and there’s so many people doing everything at the same time, and you’re sharing across artists, the way that all these tools fit into our versioning and publishing of the work is really great too.

Bret St. Clair: Of course, Katana is fantastic because all of this prototyping we were doing would be incredibly hard to, first of all, prototype it and work out all the looks. I was using Katana just as a rendering machine, where I would line up 50 tests side by side, and Katana lets you do that. It lets you manage complexity incredibly easily, but in addition to that, you can just hand something off as a macro that anyone can pick up and we knew that it was just going to work. We were going to be able to juggle all the inputs and do crazy stuff with it, but at the end of the day, it was solid. That was something great to depend on.

And then everything that we did, in the end, went through Nuke, and it was what I relied heavily on, but once I discovered BlinkScripts, it allowed me to prototype things, and it really maps the same kind of interface I was building on the Katana side.

Danny Dimian: In some ways, our workflow was maybe a little bit more live-action than people think: The flexibility, the control, and the turnaround that we needed. A lot of the tools that we started in rendering got ported to Nuke, and so the compositing in this movie… I’m not sure if people realize just how big a role compositing played in this movie.

We certainly had a soft spot for Katana. It was born in Sony Pictures Imageworks, and we’ve had a very long relationship with Foundry, So Katana’s been in the cornerstone for us for a very long time.

Jordan Thistlewood: And to echo a lot of what Bret and Danny has said, our mandate on how we build the tools is to make sure that they may not have hard conduits between each other, but we build them with APIs and extensibility so that groups like Sony can build these very custom and high efficiency, high-creative-gear pipeline.

So, it’s the sum of the parts together that gives its value. Each one has it’s own particular strengths, but the fact that we have them all and we have the teams in the same building able to talk about different client needs, and then try and make sure that there’s some consistency going through them, is what really tries to pave the way.

Danny Dimian: And the flexibility of those APIs is wonderful, from my point of view.

AW: Could you please briefly summarize the benefits of using “Katana” and “Mari” and “Nuke”, in general?

Jordan Thistlewood: One of them is the extensibility. If programs don’t come out of the box ready to do the thing that somebody needs, it’s very easy to extend them.

I would also then say that it’s the scalability. There are many tools in the industry that can achieve or create a result. When we talk to clients about Nuke, they talk about the terabytes of data that they push through for a single frame that still has an interactive element for the artist.

Katana is really about the scalability, the volume of production that you can achieve, and the fact that you can group shots together. A single artist can work on a whole packet of shots, so instead of Bret and Danny having to spend their time going between multiple artists saying: “make this tweak”, “match those lights”, “go match his or her work”, that person just does all the work together, so it creates a global efficiency.

Mari is the best blend of an enterprise tool system, mixed with a creative painting package. Altogether, it’s this theme of artist freedom, scalability, and then the enterprise pipeline. Those are the core elements to the Foundry’s suite of tools.

error: Content is protected !!

By continuing to use the site, you agree to the use of cookies. more information

This site uses cookies to ensure that we give you the best possible experience. By continuing to use the website, you consent to the receiving, storage and accessing of all cookies. To find out more, please visit our Cookie Policy, which is accessible here.