Monday, December 5, 2011

py2exe, Windows 7 & Vista

I don't use py2exe very often, but it can be a useful tool for environments that may not have an existing Python installation.

I recently used py2exe on my Windows 7 PC to build a small tasktray tool. The resulting executable ran fine on my PC (doesn't it always?), but threw an exception on any Vista PC it was run on.

File "win32com\__init__.pyo", line 5, in File "win32api.pyo", line 12, in 
File "win32api.pyo", line 10, in __load
ImportError: DLL load failed: The specified module could not be found.
After more online searching than I'd like to admit, I found a post that said py2exe may be including W7-specific DLLs, when instead it should be leaving those out, forcing Vista to go find its native builds of those DLLs.

I was able to fix the problem by adding two DLLs to the "dll_excludes" list in my py2exe setup script:
options = {
   "bundle_files": 3,
   "compressed": 1,
   "optimize": 1,
   "excludes": excludes,
   "packages": packages,
   'dll_excludes': [ 'mswsock.dll', 'powrprof.dll' ]
}
The tool now runs on both Vista and Windows 7.

Sunday, February 27, 2011

GDC 2011 Wrap-up, download

The Technical Artist Boot Camp went really well today. I personally learned a great deal from both the attendees and fellow presenters. Thanks to everyone that turned out, and thanks for all the great questions!

Here is the sample script file I promised in my talk on databases. It's a simple, working illustration of how to use SQLAlchemy ORM to map a Python class to a database table. Please let me know if you have any questions.

GDC2011_AdamPletcher_PythonSamples.zip (2 KB)

Sunday, February 20, 2011

GDC 2011 - Technical Artist Boot Camp


Game Developers Conference 2011 is a week away. I'm co-presenting at the Technical Artist Boot Camp, a special all-day Tutorial session on Tuesday March 1.

My portion of the Boot Camp is called "Embrace the Database." Here's my summary:

They may have a mystical aura about them, but databases are far easier to use than you may think. They can be a Technical Artist’s greatest ally in game development, powering your most important tools, gathering usage and error data you’ve never had access to, enabling new workflows and revealing hidden weaknesses (and strengths) in your content pipelines.

This session will also explore how to use Python to unlock the power of databases at your studio. We will look at what databases do best, using practical examples to get you started. We'll discover how Object-Relational Mapping lets you interact with a database in a simple manner that any Python user will already understand
.

I'm really excited to be presenting alongside so many talented TAs from our industry. It's going to be packed with great ideas and techniques. If you plan to be in SF for GDC, definitely stop by for our session.

Saturday, September 25, 2010

Using Sharepoint Lists with Python

Continuing my tradition of shoving Python into new and unusual places, I recently worked out how to use Python to post items to a Sharepoint List.

We use Sharepoint for some of our intranet needs, and I was experimenting with error-reporting workflows. While I ultimately didn't stick with Sharepoint for this purpose, the Python code worked fine and I wanted to share.

First, a few things to note. Sharepoint uses several SOAP-based webservices as a means of exposing functionality to other tools/languages. One of these is for manipulating Lists, which are Sharepoint's basic storehouse for items that hold arbitrary columns/fields of data.

I was unfamiliar with SOAP prior to this, and ended up using the "suds" extension for Python to help with the formatting. Suds is necessary to run the examples below, and can be downloaded on the Suds SourceForge page.

The "sharepoint" module I'm posting below has basic usage like this:

import sharepoint

item_data = {
   'item_id'   : 32,
   'Log Time'  : datetime.datetime.now( ),
   'Message'   : 'Sharepoint is rather obtuse',
   'User Name' : 'adam.pletcher',
}

sp_list = sharepoint.Sharepoint_List( 'http://sp_server/some_site/', 'Sharepoint List Name', 'domain_name', 'user_name', 'rot13_encoded_password' )
result = sp_list.add_item( item_data )
Walking through the above, first we create a simple dictionary of key/value pairs. The keys are the Sharepoint names for the columns in your list, and the values are the values you wish to submit for your new list item. Then, we construct a Sharepoint_List instance, passing it the site URL, List Name, domain, user and password. Finally, call our add_item method on that object, passing it the data dictionary we made.

The add_item method accepts most data types, including Python datetime objects, as shown. You can easily extend add_item to do something more elaborate.

One thing to watch out for is the list's column/field names used in your dictionary. These must be strings matching the true Sharepoint names of your fields. Even if you rename a column later, it's true name will not change once the list has been created. The only flexibility I currently provide in add_item is the ability to leave spaces in the field names.

I experimented with NTLM as a means of using the current user's Windows Authentication instead of requiring the username/pass each time. I was not successful in that, however... if anyone has success there I'd love to hear about it. As it stands, the Sharepoint_List class requires the username and a ROT-13 encoded password. If you've never encoded anything, here's how to do that from a Python prompt:
>>> 'my_password'.encode( 'rot-13' )
'zl_cnffjbeq'
The complete "sharepoint" module described here can be downloaded below. Adding an item isn't the only thing the Sharepoint lists webservice offers. You can use the framework in this module to do more including edits, deletes and other actions.

Python_Sharepoint.zip (3 KB)

Sunday, December 6, 2009

MaxScript DotNet Sockets with Python

I create and work with several Python tools that manipulate scenes in 3ds Max. These are usually floating dialogs linked to the main 3ds Max window that send MaxScript commands via COM. This works well until I need the tool's UI to update when something happens in 3ds Max. Like refresh an object list when the selection in Max changes.

You would think doing a COM connection the other way would work. However, since Python-registered COM servers run separately in their own instance of the interpreter, there's no native connection to the original tool.

Nathaniel Albright, a fellow TA at Volition, recently created a Python COM server that communicated to his Python tool via TCP/IP socket. So it went 3ds Max -> Python COM server -> TCP/IP -> Python tool. This works well, but I wondered if the DotNet facilities in MaxScript offered a direct way to use sockets.

I had yet to touch DotNet in MaxScript, so this seemed like a good opportunity to learn a few things. After a lot of searching online I only turned up a few scraps of info, no complete recipe. However, I did find enough to get MXS DotNet sockets working, and assemble a comprehensive example.

I created a little Python tool that displays the names of all selected objects in the 3ds Max scene. As the scene selection changes, the list of names automatically updates. I won't go over all the code in this post, but the full working example tool is included in the zipfile below.

There's three main points of interest in the example:

1. Using DotNet in MaxScript to communicate via TCP/IP socket
2. Listening on a socket in a background thread in Python
3. Creating and posting custom wxPython events

The MaxScript Client

I made a MaxScript struct called "mxs_socket". The code follows, and also included in the zipfile below.

struct mxs_socket (
   ip_address = "127.0.0.1", -- "localhost" also valid
   port       = 2323,        -- default port

   -- <dotnet>connect <string>ip_string <int>port_int
   --
   -- Description:
   -- Takes IP address, port and connects to socket listener at that
   -- address
   fn connect ip_string port_int = (
      socket = dotNetObject "System.Net.Sockets.Socket" ( dotnetclass "System.Net.Sockets.AddressFamily" ).InterNetwork ( dotnetclass "System.Net.Sockets.SocketType" ).Stream ( dotnetclass "System.Net.Sockets.ProtocolType" ).Tcp
      socket.Connect ip_string port_int

      socket   -- return
   ),

   -- <int>send <string>data
   --
   -- Description:
   -- Converts a string (or any object that can be converted
   -- to a string) to dotnet ASCII-encoded byte sequence and
   -- sends it via socket. Uses ip_address and port defined
   -- in struct above, or set by client.
   -- Returns integer of how many bytes were sent.
   fn send data = (
      -- Convert string to bytes
      ascii_encoder = dotNetObject "System.Text.ASCIIEncoding"
      bytes = ascii_encoder.GetBytes ( data as string )

      -- Connect, send bytes, then close
      socket = connect ip_address port
      -- result is # of bytes sent
      result = socket.Send bytes
      socket.Close()

      result  -- return # of bytes sent
   )
)
Using this, I can send bytes to any socket listener on port 5432 by doing the following:
socket = mxs_socket port:5432
socket.send "Hello, World!"
The connect method was pretty simple in the end. The only twist turned out to be converting the socket integer into a DotNet socket object.

The send method converts the string into an ASCII-encoded DotNet bytes object, connects to the listener, sends the bytes, then closes the connection. The value returned is the number of bytes sent.

The last lines of the above code sets up a MaxScript callback that fires when the object selection changes in the scene. That uses mxs_socket to send a string containing the names of all the selected objects to any tool that's listening on that port.

Now I just need to make my Python tool listen.

The Python Server

My Python server/listener (also in the zipfile below) is a typical wxPython frame, but with two added qualities... It uses a background thread to listen on a socket, and posts a custom wx.Event when data is received. I had never used either of these techniques before, but it was fun getting it working.

Since a typical wxPython app sits in its main loop waiting for user input, I created a Socket_Listen_Thread class, a subclass of threading.Thread. This does the listening in a background thread while the main UI thread waits on the user. The run method here does the real work:
def run( self ):
   self.running = True

   while ( self.running ):
      # Starting server...
      # Listen for connection.  We're in non-blocking mode so it can
      # check for the signal to shut down from the main thread.
      try:
         client_socket, clientaddr = self.socket.accept( )
         data_received = True
      except socket.error:
         data_received = False

      if ( data_received ):
         # Set new client socket to block.  Otherwise it will
         # inherit the non-blocking mode of the server socket.
         client_socket.setblocking( True )

         # Connection found, read its data then close
         data = client_socket.recv( self.buffer_size )
         client_socket.close( )

         # Create wx event and post it to our app window
         event = self.event_class( data = data )
         wx.PostEvent( self.window, event )
This listening thread runs quietly in the background until it receives data. At that point I need a way to break into the main UI thread. There's other ways to do this, but using a custom wx.Event seemed to be the best fit here.

First, when the wx.Frame is opened, I create a custom wxPython event.
(Max_Update_Event, EVT_3DS_MAX_UPDATE) = wx.lib.newevent.NewEvent()
Calling NewEvent returns both a new Event class and an object to bind the event handler to. I pass the event class to the listener thread, bind an event handler to it, and that's all.

When data comes in over that TCP/IP port from our MaxScript tool, the listening thread receives it and posts our custom event to the main wx.Frame. That in turn fires the event handler to update the UI.

My example MaxScript client and Python listener described above can be found in the following ZIP file. Drop me a line if you do something useful with them, I'd love to hear about it.

MaxScript_DotNet_Sockets_Python.zip (4 KB)

Thanks to Nate Albright and everyone contributing to the "dotNet + MXS" and "Python + MXS" threads on the CGSociety forums. Those long-running threads have been very inspiring, and contain several tips that were key in getting the MaxScript DotNet socket stuff hammered out.