This entry was going to cover thinking about your .Net
classes from the perspective of Python consumption… but actually it's
not, we'll cover that another time.
What this post is really
going to be about is how to spelunk a little into our managed classes
via the IronPython interactive console - and to see how some common
practices in .Net have interesting results in python.
This
entry is going to be a little slow paced, I’m making the assumption
you’re a .Net developer, and haven’t cut Python code before, if you
know your way around Python probably best to pass this one over.
Also worth noting at this point that I'm really just a Python hacker, and certainly no guru...
So
first off, we have a class with a bunch of overloads for various
methods, here is the interface… I’ve left the code for the class itself
out, but the class is called MyFileStore.
public interface IMyFileStore
{
Guid AddFile(string fileName);
Guid AddFile(string fileName, string mimeType);
Guid AddFile(byte[] contents, string mimeType);
Guid[] AddFiles(params string[] fileNames);
Guid AddFile(byte[] contents);
byte[] ReadFile(Guid id, out string mimeType);
void ReadFile(Guid id, string fileName);
void ExtractFiles(ref IList<byte[]> myList);
}
Pretty simple, now lets look at using this class with Iron python, I’ll be running with the ipy
(the interactive interpreter) so we can investigate our types as we go
along (this is an executable which comes with the Iron Python
binaries). The ipy console has a prompt consisting of three
greater then signs “>>>” so you’ll know what I’m typing into
the console by looking for those.
First off, we need to load our type… this is pretty trivial…
>>> import clr
>>> clr.AddReferenceToFileAndPath("D:\\Demo\\SampleIronPythonLib.dll")
>>> from IronPythonLib import *
All
done, lets make sure our class is in there, using the inbuilt function
dir() – a list of built-in functions can be found here: http://www.python.net/quick-ref1_52.html#BuiltIn
>>> dir()
['IMyFileStore', 'MyFileStore', '_', '__builtins__', '__doc__', '__name__', 'clr', 'results', 'site', 'store', 'sys']
MyFileStore, sweet.
First off, lets create an instance of my type (not yours ;o)
>>> store = new MyFileStore()
Traceback (most recent call last):
SyntaxError: unexpected token MyFileStore (<stdin>, line 1)
>>>
Arse, python doesn’t have a new operator… forgot that, lets try it again.
>>> store = MyFileStore
Right,
looks good, interpreter didn’t return any errors… but wait, lets make
sure we got what we want, which is an instance of MyFileStore….
>>> store
<type ‘MyFileStore’>
>>>
Woops, that’s not what we wanted, we want an instance, not the type… lets try again:
>>> store = MyFileStore()
>>> store
<MyFileStore object at 0x0000000000000035>
>>>
That’s better, lets find out what our store is capable off, using the dir command.
>>> dir(store)
['AddFile',
'AddFiles', 'Equals', 'ExtractFiles', 'Finalize', 'GetHashCode',
'GetType', 'MakeDynamicType', 'MemberwiseClone', 'ReadFile', 'Reduce',
'ReferenceEquals', 'ToString', '__class__', '__doc__', '__init__',
'__module__', '__new__','__reduce__', '__reduce_ex__', '__repr__']
>>>
We
see our AddFile, AddFiles,ReadFile & ExtractFiles methods –
everything looks to be in order… but what are the parameters for those
methods?
>>> dir(store.AddFile)
['Call', 'CallInstance', 'Equals', 'Finalize', 'GetHashCode', 'GetTargetType', '
GetType',
'Make', 'MakeDelegate', 'MakeDynamicType', 'MemberwiseClone',
'Overloads', 'PrependInstance', 'Reduce', 'ReferenceEquals',
'ToString', '__call__', '__class__', '__doc__', '__eq__',
'__getitem__', '__hash__', '__init__', '__module__', '__name__',
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__self__',
'__str__']
>>>
Hmm… what’s the “Overloads” thing, lets find out…
>>> dir(store.AddFile.Overloads)
['Equals',
'Finalize', 'Function', 'GetHashCode', 'GetOverload',
'GetTargetFunction', 'GetType', 'MakeDynamicType', 'MemberwiseClone',
'Reduce', 'ReferenceEquals', 'Targets', 'ToString', '__class__',
'__doc__', '__getitem__', '__init__', '__module__', '__new__',
'__reduce__', '__reduce_ex__', '__repr__', '__str__']
>>>
That doesn’t seem very useful, though what if we check it’s string representation?
>>> store.AddFile.Overloads
<IronPython.Runtime.Calls.BuiltinFunctionOverloadMapper
object at 0x0000000000000036 [{'Guid AddFile(self, str fileName)':
<built-in function AddFile>, 'Guid AddFile(self, str fileName,
str mimeType)': <built-in function AddFile>, 'Guid AddFile(self,
Array[Byte] contents, str mimeType)': <built-in function
AddFile>, 'Guid AddFile(self, Array[Byte] contents)': <built-in
function AddFile>}]>
>>>
Look at that, there are our 4 overloads – but it’s a bit jumbled, what am I looking at it….
Well
at first glance it looks a bit like a python dictionary… notice the
colon separating the signature for the method and <built-in function
AddFile> … what if we try to grab one for use later, maybe for some
functional programming, why don’t we get the one which takes the
fileName and mimeType parameters eh?
>>> myFunc = store.AddFile.Overloads[str,str]
Traceback (most recent call last):
File , line 0, in <stdin>##100
TypeError: __getitem__() takes exactly 1 argument (1 given)
>>>
Bugger,
that didn’t work… still, not a bad guess - one parameter eh? Lets have
another stab, maybe it just wants us to group the types together
somehow?
>>> myFunc = store.AddFile.Overloads[(str,str)]
>>> myFunc
<built-in method AddFile of MyFileStore object at 0x0000000000000035>
>>>
Looks better, of course if we wanted to we could use our function right now… let’s give it a whirl:
>>> myFunc("d:\\testinput.3gp", "video/3gpp")
<System.Guid object at 0x0000000000000037 [7b6d6dc5-bbe6-4831-bff7-21984031684f]>
>>>
Looks
to have worked, the method has successfully returned us a Guid
identifying the new file… shame we didn’t save it into a variable for
future reference, cest la vie.
Luckily I can just copy and paste it – lets see if I can get the file back using this overload…
byte[] ReadFile(Guid id, out string mimeType);
Simple as.. lets try…
>>> id = Guid("7b6d6dc5-bbe6-4831-bff7-21984031684f")
Traceback (most recent call last):
File , line 0, in <stdin>##116
NameError: name 'Guid' not defined
>>>
Woops, we need to import the System namespace…
>>> import System
Now lets try again…
>>> id = Guid("7b6d6dc5-bbe6-4831-bff7-21984031684f")
Traceback (most recent call last):
File , line 0, in <stdin>##116
NameError: name 'Guid' not defined
>>>
Huh… oh wait, let’s check our local symbol table:
>>> dir()
['MyFileStore',
'System', '_', '__builtins__', '__doc__', '__name__', 'clr', 'myArray',
'myFunc', 'results', 'site', 'store', 'sys']
System eh? Lets dig into it…
>>> dir(System)
['AccessViolationException', 'Action', 'ActivationContext', 'Activator', 'AppDom
ain', 'AppDomainInitializer', 'AppDomainManager', 'AppDomainManagerInitializatio
nOptions', 'AppDomainSetup', 'AppDomainUnloadedException', 'ApplicationException… and all the rest.
>>
Well,
what can we do about that… we could import them all using “from System
import *” – but we only need Guid, so why don’t we just grab that eh?
>>> from System import Guid
>>> dir()
['Guid',
'MyFileStore', 'System', '_', '__builtins__', '__doc__', '__name__',
'clr', 'myArray', 'myFunc', 'results', 'site', 'store', 'sys']
>>>
Sweet, now lets try it again… finally!
>>> id = Guid("7b6d6dc5-bbe6-4831-bff7-21984031684f")
>>> mimeType = ""
>>> contents = store.ReadFile(id, mimeType)
Traceback (most recent call last):
File , line 0, in <stdin>##136
File , line 0, in ReadFile##66
File D:\dev\Projects\IronPythonDemo\IronPythonLib\MyClass.cs, line 70, in Read
File
File mscorlib, line unknown, in WriteAllBytes
File mscorlib, line unknown, in .ctor
File mscorlib, line unknown, in Init
ValueError: Empty path name is not legal.
>>>
Oh
no! what went wrong??… what’s this about an empty path name, I was
expecting to get some bytes back…. Hmmm, don’t panic, lets just do some
digging…
>>> store.ReadFile.Overloads
<IronPython.Runtime.Calls.BuiltinFunctionOverloadMapper
object at 0x0000000000000039 [{'(Array[Byte], str) ReadFile(self, Guid
id)': <built-in function ReadFile>, 'ReadFile(self, Guid id, str
fileName)': <built-in function ReadFile>}]>
>>>
Well
there’s the problem, I’ve called the wrong version, but… that’s odd,
there’s a ReadFile there I haven’t defined in my class… and one that’s
missing… hmmm, time to file a bug report.
Or not, this is just where the Python and .Net world don’t see eye to eye – Python doesn’t know about out
parameters as such, so it simulates the effect by altering the method’s
signature – lets have a go at playing by IronPython's rules:
>>> contentsAndMimeType = store.ReadFile(id)
>>> len(contentsAndMimeType)
2
>>> type(contentsAndMimeType[0])
<type 'Array[Byte]'>
>>> type(contentsAndMimeType[1])
<type 'str'>
>>> contentsAndMimeType[1]
'video/3gpp'
>>>
So our “out string mimeType” parameter was returned by the method instead, as the second item in an array… hmm… interesting.
Well
if python does this to out parameters, what does it do to ref
parameters – lets try the extract files method to see what happens…
this method is implemented to create a new list if we don’t supply a
valid instance, but lets pass in a valid one first – I wont be caught
out this time, so I’ll import the types I need first (List<T>):
>>> from System.Collections.Generic import *
>>> bytes = List<byte[]>()
Traceback (most recent call last):
SyntaxError: unexpected token ] (<stdin>, line 1)
>>>
Oh dear, looks like Python doesn’t use the <T> syntax… lets start digging again…
>>> dir(List)
['Add', 'AddRange', 'AsReadOnly', 'BinarySearch', 'Capacity', 'Clear', 'Contains
', 'ConvertAll', 'CopyTo', 'Count', 'Enumerator', 'Equals', 'Exists', 'Finalize'
, 'Find', 'FindAll', 'FindIndex', 'FindLast', 'FindLastIndex', 'ForEach', 'GetEn
umerator',
'GetHashCode', 'GetRange', 'GetType', 'IndexOf', 'Insert',
'InsertRange', 'LastIndexOf', 'MakeDynamicType', 'MemberwiseClone',
'Reduce', 'ReferenceEquals', 'Remove', 'RemoveAll', 'RemoveAt',
'RemoveRange', 'Reverse', 'Sort', 'ToArray', 'ToString', 'TrimExcess',
'TrueForAll', '__class__', '__doc__', '__getitem__', '__init__',
'__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__setitem__']
>>> List
<type 'List[T]'>
>>>
Hmm… nothing obvious there, but we could hazard a guess…
>>> List[str]
<type 'List[str]'>
>>>
Hmmm… so lets give our List of byte array another go…
>>> bytes = List[byte[]]()
Traceback (most recent call last):
SyntaxError: unexpected token ] (<stdin>, line 1)
Hmm… it doesn’t like byte, lets have a look…
>>> byte
Traceback (most recent call last):
File , line 0, in <stdin>##56
NameError: name 'byte' not defined
>>> Byte
<type 'Byte'>
>>>
Ahhh…
I forgot that “byte” (lowercase) isn’t actually what the Clr calls it…
hmm… we could try it again, but it wont work – simply because Byte[]
doesn’t make sense (if we’re using square brackets to index into a
generic type… ) so having another dig in the IronPython docs we see
that we need to index to the type “Array” with the type of our array,
given this knowledge we can get what we want…
>>> bytes = List[Array[Byte]]()
>>> bytes
<List[Array[Byte]] object at 0x000000000000002C>
>>>
Sweet, now to call that ExtractFiles method…
>>> store.ExtractFiles(bytes)
<List[Array[Byte]] object at 0x000000000000002C>
>>> bytes
<List[Array[Byte]] object at 0x000000000000002C>
>>> len(bytes)
8
Well
it returned a list… but it did accept our reference parameter, and it
has updated the collection we passed to it… but what if we had passed
it a null instead?
>>> bytes = None
>>> store.ExtractFiles(bytes)
<List[Array[Byte]] object at 0x0000000000000030>
>>> len(bytes)
Traceback (most recent call last):
File , line 0, in <stdin>##83
File , line 0, in Length##75
TypeError: len() of unsized object of type <type 'NoneType'>
>>>
Hmmm… it returned a list, but it didn’t set the value in our variable "bytes" – this isn’t really ref like behavior… yet, if you think about it we can get almost the same thing by doing this:
>>> bytes = store.ExtractFiles(bytes)
>>> len(bytes)
8
>>>
Right, our journey has almost come to an end… looking at the interface we have a method declared like so:
Guid[] AddFiles(params string[] fileNames);
params
is handy in .Net code – anything to save tedious typing of array
declarations, lets see if we can do the same thing in python?
>>> store.AddFiles("d:\\testinput1.3gp", "d:\\testinput2.3gp")
System.Guid[](<System.Guid
object at 0x0000000000000031
[d3e8f099-3005-461c-a63a-fdc69b2091ee]>, <System.Guid object at
0x0000000000000032 [d9d1fb25-fc7a-47a2-89ac-c11264bfa47f]>)
>>>
Sweet, that’s a pleasant surprise after that whole out and ref debacle, I was beginning to loosing hope!
But wait, why not revive our faith in Python a little more by looking at a few tricks…
>>> paramsDict = { "fileName" : "d:\\testinput.3gp", "mimeType" : "video/3gpp"}
>>> store.AddFile(**paramsDict)
<System.Guid object at 0x0000000000000033 [e07680d9-f544-4847-9a1e-d04a0ef137f7]>
>>>
What
did we just do? Well… given a dictionary, where the keys are the
parameter names, we used them as the parameters for one of our .Net
methods using the double asterisk syntax – think of what you’d
have to do to code this in .Net… reflection city ;o)
It doesn’t just have to be dictionaries… we can use arrays too…
>>> paramsArray = [ "d:\\testinput.3gp", "video/3gpp" ]
>>> store.AddFile(*paramsArray)
<System.Guid object at 0x0000000000000034 [85006c43-e6f5-4e90-b837-0868788cf453]>
>>>
Neat,
of course, being a dynamic language and all, the selection of the
appropriate overload is based on the types in the array or dictionary…
it would be a nightmare to do both of these in .Net with reflection
yourself.
Faith still not restored eh, what about creating a
python class that wraps an existing (possibly sealed) .Net class, that
forwards calls on unless we want to override the behavior…
>>> class MyFileStoreWrapper:
... def __init__(self, realStore):
... self.realStore = realStore
... def ExtractFiles(self, myList):
... raise Exception, "this method isn't allowed"
... def __getattr__(self, name):
... if name == "ExtractFiles": return ExtractFiles
... return eval("self.realStore."+name)
...
>>> store = MyFileStoreWrapper(MyFileStore())
>>> store.AddFile("d:\\testinput.3gp")
<System.Guid object at 0x000000000000002C [a2bf8747-cc71-4b79-8c02-0b61a87ff67f]
>
>>> store.ExtractFiles(None)
Traceback (most recent call last):
File , line 0, in <stdin>##52
File , line 0, in ExtractFiles
Exception: this method isn't allowed
>>>
ExtractFiles
can’t be invoked on our wrapper, however our wrapper automagically
responds for other methods like AddFile – sadly I’ve used an eval (which,
rhymes with Evil - coincidence?) here because MyFileStore doesn’t
expose a __getattr__ method (because it's a .Net classes, not a
native Python one).
The nice thing now, is that given a native
python class, we can do some things we aren’t allowed to do to the
underlying .Net class – like adding new methods at run time.
>>> def ExtractMoreFiles(self, myList):
... raise Exception, 'not implemented yet'
...
>>> MyFileStoreWrapper.ExtractMoreFiles = ExtractMoreFiles
>>> store.ExtractMoreFiles(None)
Traceback (most recent call last):
File , line 0, in <stdin>##60
File , line 0, in ExtractMoreFiles
Exception: not implemented yet
>>>
I
think it’s pretty cool, and though python's metaprogramming model isn’t
quite up to ruby standards, it’s still pretty easy to dig down and
create some pleasantly surprising results :) and if you have your
thinking caps on you can probably see how this stuff would really help
to bring a DSL to life for your applications special needs.
And
that’s where I’m going to conclude this rambling post…
;o) Hopefully it'll spark some thoughts about what's possible with
IronPython and allow you avoid some simple mistakes when using .Net
classes in iron python.