Checksums in 3ds Max (part 2 of 2)

In Part 1 I showed how to calculate checksums inside 3ds Max. Here's how to do something useful with them.

Any TA that's crossed paths with 3ds Max can tell you it doesn't do the best job of managing scene materials. Due to scene object merges/imports and other typical operations, it's common for a given material to be copied several times in one Max scene. Meaning, it's not instanced across several objects, but actually copied several times in memory. This can lead to increased memory usage and potentially inefficiencies in your game engine (depending how your exporter deals with this).

What's worse, you usually can't rely on similar material names to find duplicates by hand. To do a thorough search with MaxScript, you would need to loop through every material in the scene and compare every property in it to every other material's property. This would be a slow process in C, and a complete horror-show with MaxScript.

Enough grim talk. Here's a walkthrough of a MaxScript that uses checksums to make short work of this. To summarize, the script loops through the materials in your scene, creating a checksum for each as it goes. It uses that checksum to do a quick compare on previous material checksums it found, to see if they're actually property-identical. If it finds a dupe, that object is given the original material instead, effectively deleting the duplicated material.

The script is divided into three functions and a short bit of main code.

getChecksum() is the first function, taken from my previous post. It calls the Python COM object we registered, which returns a checksum to the MaxScript. If you can't (or don't want to) set-up the COM object, you can use the MaxScript implementation I listed in that blog post instead... it's just less robust than the MD5 checksums used by the Python method.

Next is the getPropsString() and getMaterialChecksum() functions:

------------------------------------------------------------
-- (str)getPropsString (material)mat
--
-- Description:
-- Builds a string representing the property names/values
-- of the supplied Max material.
------------------------------------------------------------
fn getPropsString mat = (
   myStr = "" as stringStream
   if (mat == undefined) then (
      format "undefined" to:myStr
   ) else (
      -- Start our string w/the classname
      format (classOf mat as string) to:myStr
      if (classof mat == ArrayParameter) then (
         -- Array, so recursively add strings for each element
         for element in mat do (
            format (getPropsString element) to:myStr
         )
      ) else (
         -- Not an array, so see if it has properties
         propNames = undefined
         try (
            propNames = getPropNames mat
         ) catch ()
         if (classOf mat == BitMap) then (
            try (  -- Add bitmap's filename
               format mat.filename to:myStr
            ) catch ()
         ) else if (propNames == undefined) then (
            format (mat as string) to:myStr
         ) else (
            format (propNames as string) to:myStr
            -- Loop through properties, adding their names
            -- and values to our string to be checksummed
            for i in 1 to propNames.count do (
               format (i as string) to:myStr
               p = propNames[i]
               val = getProperty mat p
               format (i as string) to:myStr
               format (getPropsString val) to:myStr
            )
         )
      )
   )
   (myStr as string)
)

------------------------------------------------------------
-- (str)getMaterialChecksum (material)mat
--
-- Description:
-- Takes a Max material (or multi-sub material) and
-- calculates a checksum value from it, for use as a
-- hashtable key, or whatever you like.
------------------------------------------------------------
fn getMaterialChecksum mat = (
   str = ""
   if (classof mat == Multimaterial) then (
      for id in mat.materialIDList do (
         -- Add material IDs as factors
         str += id as string
      )
      for subMat in mat.materialList do (
         -- Get string representing each submaterial
         str += (getPropsString subMat)
      )
      ) else (
         -- Get string representing this material
         str += getPropsString mat
      )
   -- Add string length as a factor
   str += str.count as string

   -- Get checksum from our base string
   -- 99991 = largest prime number under 10k
   (getChecksum str)
)

The above functions work together to generate a string of data representing the supplied 3ds Max material (or Multi-Sub material). Once it has that string, it's passed to getChecksum().

The main code block loops through the entire 3ds Max scene, doing the above for every material found on geometry objects:

----------
-- MAIN
----------
-- Set up a few things first.

timeStart = timestamp()  -- Time we started process
removedCount = 0  -- Counters for printing info below
uniqueCount = 0

-- Array of two synced arrays, first with the material
-- checksums, second with materials themselves.
-- Basically a poor-man's hashtable.
csMatArr = #(#(), #())

format "Scanning scene materials...\n"

-- Loop through all geometry
for obj in geometry do (
   mat = obj.material

   alreadyDone = (findItem csMatArr[2] mat) != 0

   if (not alreadyDone) and (mat != undefined) then (
      -- First get this material's checksum
      csum = getMaterialChecksum mat

      idx = findItem csMatArr[1] csum

      if (idx != 0) then (
         -- Dupe material found, so remove it by
         -- assigning the first mat to this object
         format "Replacing material '%' with '%'\n" mat.name csMatArr[2][idx].name
         obj.material = csMatArr[2][idx]
         removedCount += 1
      ) else (
         -- New checksum, so add it to our table,
         -- along with the material itself.
         append csMatArr[1] csum
         append csMatArr[2] mat
         uniqueCount += 1
      )
   )
)

gc()  -- Remind Max to take out the trash

-- Done, print some results
format "-- DONE in % secs --\n" ((timestamp() - timeStart) / 1000.0)
format "Old material count = %\n" (uniqueCount + removedCount)
format "New material count = %\n" uniqueCount
format "Duplicates removed = %\n" removedCount

That's it. At the end a summary is printed to the MaxScript Listener.

In the Max scene I was working with today this script cut the root material count from 533 to 261. That's 51% fewer materials! It also reduced the file load time from 136 seconds to 102 seconds.

You can download the complete script above here: RemoveDupeMaterials.zip (4 KB)
It includes the Python script to register the COM server, and the alternate MaxScript checksum method.

Update 7/25/08: I modified getPropsString() to better handle bitmap values, and generally run faster. The ZIP file above is updated as well. Thanks to MoonDoggie/Colin on CGTalk for the feedback!