ken museth
Ken Museth

Interview: Ken Museth on OpenVDB

OpenVDB is not in Blender yet, but there are plans to include it in future releases. You might wonder - what is this OpenVDB all about? BlenderDiplom talked with Ken Museth, lead developer of OpenVDB at Dreamworks, about the project and what users can expect from a software that uses OpenVDB.

BD: Hello Ken, at the moment you are working at Dreamworks. What exactly are you doing on there, what are your projects and what is your involvement with OpenVDB?

Ken: I work at Dreamworks Animation in the Los Angeles area and have been there since 2009. I'm the R&D VFX supervisor and a senior principal engineer. I lead a team of researchers that develop tools for production. At the same time I am a developer myself. My involvement in VDB? Well, I invented VDB and published it in 2009. It has been gradually improved and in 2011 it was released at SIGGRAPH. The main focus of my group is to maintain, extend and release OpenVDB.

openvdb mesher

OpenVDB comes with a meshing system

 

BD: As an ordinary user, what can you expect from OpenVDB support in an application?

Ken: Let's put in the context of the kind of volumes people have been using for many years and also in many third-party applications. Those are usually so-called dense grids where the user has a kind of box and the effects are restricted to the enclosure of the box. The size of the box has a huge impact on the amount of memory the computer has to store. As you increase the resolution of the box, there are more and more voxels in it. A voxel is analogous to a three dimensional pixel. So the problem is that these dense grids require a lot of memory. What VDB allows you is to throw the box away. It allows you to store information in a way that is very compact.
For the end user this allows for more detailed simulations like for example smoke, fire or water. For many volumetric applications, especially if they are sparse, VDB is a very good option. Since 2012 OpenVDB has been integrated into Houdini. So if you are using Houdini version 12.5 or higher, you are using OpenVDB internally.
Most of the effects that are produced in the effects department involve volumes. We typically have volumetric effects like animated dust, smoke, water, fire and even fracturing. Many effects that do not appear to be volumetric in nature, like rigid body dynamics and fracturing, can in fact involve volumes. You can use volumes to avoid penetration of objects, for instance using techniques like volumetric collision detection and resolution. Another example is the use of volumes for various types of rendering applications. So all in all the VFX department relies heavily on volumes.

openvdb mesh cleaning

It can also be used to clean up mesh intersections.

 

BD: OpenVDB is supported by various third party tools and renderers.

Ken: Arnold has added support via plugins, and there is also support in Pixar's Renderman. Realflow is also using OpenVDB internally. Many applications use OpenVDB. In fact this was one of the main motivations for Dreamworks to Open Source it, so these companies would integrate it for us, freeing up more time for other development. One might not realize that we are spending a majority of our time integrating techniques into our existing pipelines, like developing user interfaces, scripting and other support. Those tasks take a lot of time. By having other companies picking up this work, we are offloaded and free to pursue other development like improving OpenVDB. We found out that by having SideFX integrating and maintaining it we have a lot more time to actually develop the technology further, which is really what we want to do.

BD: This is one of the pros of Open Sourcing your product, but what about the cons?

Ken: One of the cons is that it is time consuming. A lot of time is spent on maintaining repositories, build systems, writing documentation and stuff like that. There is also a lot of pressure from the Open Source community. Many times we get requests for new features, a common question for example is "Why don't you support Windows". You always have to balance such requests against what Dreamworks wants. Dreamworks for example uses Linux machines and thus does not have a particular interest in supporting Windows. In this case we were actually lucky because SideFX supports Windows so they are dealing with this task. Dealing with pressure and the fact that it is time consuming are some of the negative aspects of Open Sourcing. But these cons are more than balanced out by the pros. As I already mentioned, the first and most important benefit is really adoption. The fact that we don't have to support the integration into all those tools and thus have more time for other development.
Another advantage we found is that Open Source projects are a great way of collaborating with other movie studios and also with people in academia. The fact that we have a piece of software with no issues regarding intellectual property, that everyone can use and contribute to, is great for collaboration and we have done that a lot of times. Sometimes academic groups will use it and actually base their research on it and some of their ideas might propagate back into the product. We have also done this with other studios.

BD: The lead engineer of another popular Open Source project in VFX said at FMX two years ago that Open Source is not really cheaper than developing in-house but it is also not more expensive. What is your gut feeling about that?

Ken: I would agree with him, it is not more expensive. But more importantly: It raises the bar! The code quality is a lot better than when you develop just for yourself. You share it with the entire world, everyone can see your work. That is a tremendous incentive to increase the quality of both code and documentation. It is a lot easier to maintain a project that is both well written and well documented and has a clean API.

BD: This is exactly what Larry Gritz told us about the Open Sourcing of OSL. He also mentioned that Open Sourcing is exposing the developers to an audience. Is it good or not-so-good to get a lot of e-mails etc.?

Ken: Exactly! To be honest, we did not really know this when the started the project, it was sort of a surprise to us, and a very pleasant surprise. We have developers at Dreamworks who looked at the OpenVDB code and told us that it was the cleanest code they had seen at the studio. And that's exactly what we wanted to achieve.

BD: Are you getting a lof of praise or more criticism from the outside world?

Ken: I think one of the problems with OpenVDB specifically is that it uses a lot of template metaprogramming. It is well documented but if you are not familiar with this kind of programming technique, it is a little hard to follow when looking at the source code. Given the fact that VDB itself, the data structure, is a little complicated, we have few people that have contributed to the guts of the library. And I think that is because it is a little intimidating. It is not easy to pick up and start messing with the data structure. Most contributions we got were tools building on top of the data structure which of course is also super useful.

openvdb attributes

OpenVDB can store a lot of different attributes for volume data.
From left to right: Bouyant density, velocity and vorticity, the latter two represented as vector fields in a 2D slice.

 

BD: Was it difficult to Open Source?

Ken: Certainly! OpenVDB is the first and only Open Source project from Dreamworks Animation. It took almost two years to get it approved. People were supportive all the time, we never hit a brick wall, but it took the legal department a lot of time to approve it. The third-party adoption was the main selling point. The fact that SideFX would support us helped a lot.

BD: SideFX was on board with OpenVDB from the very beginning?

Ken: Yes, we worked closely with the SideFX developers, they would look at the code and make suggestions for changes. It was a very close cooperation.

BD: What third-party programs would you like to see adopt OpenVDB?

Ken: We would like to see it in for example Maya, and we would also love to see Blender pick it up. Those are the main two third-party projects where we would like to see adoption of OpenVDB. We have a rudimentary support for Maya, we actually released some nodes but we don't use Maya that much in-house for volumetric effects, so we would like someone from the Open Source community to pick it up.

BD: Are you planning to port OpenVDB to the GPU?

Ken: It's a great idea. We don't have it yet and it is not entirely obvious how to do that. OpenVDB was highly optimized for the CPU and the GPU is a completely different architecture. It's not an easy task but it is certainly a very interesting path to take. We have been in touch with nVidia, they told us that they had some initial success porting it but I guess we would have to make a lot of changes. In-house we have a GPU based renderer that sort of supports it but it only converts the OpenVDB data structure into a tiled grid before loading it into the video memory.
The short answer is: GPU support is something we want to look into.

openvdb gpu rendering

Realtime Preview of OpenVDB data using OpenGL on the GPU (converted to a grid).

 

BD: What other plans do you have for OpenVDB? Maybe a structure to store particle data?

Ken: Yes, particle data is a huge project that I am pushing right now. During the production of the upcoming "How to train a dragon 2" we faced the problem of converting huge amounts of particle data, for example 300 million particles, into VDBs, and we realized that the existing tools just didn't cut it. So we started looking into ways to store particle information in the same tree structure and we had quite a lot of success. We hope to release it as part of OpenVDB 3.0 at SIGGRAPH this summer. This is also an interesting project because it is the first time we have a formal cooperation with another studio. A researcher from Double Negative will be coming over to collaborate with us.
That is probably the biggest project we are having right now but we are also looking into level of detail. So we are trying to find out how to store different resolutions of the volume data so that the renderer can pick the right resolution depending on the distance to the camera.
We also want tools for cloud modeling and we are researching out-of-core loading. Instead of loading the full volume, VDB will manage the delayed loading so when the rays from the render hit different regions of space, it only loads that parts.

BD: Do you think rendering on the GPU will actually be feasible in regards to memory consumption?

Ken: Typically the volumes we are dealing with are rarely more than a couple of gigabytes so as long as your card has that much video ram it should be possible.

BD: How about distributed network rendering?

Ken: We thought about it but it is not something we usually do at Dreamworks. We have a render farm but every core gets a full copy of the scene. I think OpenVDB could deal with this kind of rendering really well because it is a tile grid, the leaf nodes could be distributed among many different computers. I have heard that there is a studio that actually explores this idea for liquid solvers and they reported that it worked.

BD: Are there any other projects at Dreamworks where you think Open Sourcing would be beneficial?

Ken: We talked about a few, but I don't think anything is in the pipeline. There was talks about taking a project before OpenVDB as a test balloon. I don't know of any upcoming Open Source projects, but I would not be surprised if something comes up given the success of OpenVDB.

This interview was first released in German for the magazine Digital Production. BlenderDiplom is now presenting the interview in its original, non-translated form.

All images in this interview are properties of Dreamworks Animation


Related Items:


Interview: Andrew Kaufman from Image Engine
Interview: Alicevision's photogrammetry solution Meshroom
Interview: François Beaune on Appleseed Renderer
Interview: Aidy Burrows on Blender in AAA Games