PyRevit CPython-WPF support saga

In the WIP version of pyRevit, we updated python.net library to version 3. It solves some things, but it doesn’t support WPF yet.
These are my findings on the process of getting something working.

NOTE: I’ll use the terms CPython and python.net interchangeably, since the latter is the library that let us run the former inside .NET applications.

Refactoring pyrevit.forms

The first thing I did wasn’t specific to CPython, but it was something I wanted to do for a while: split down the pyrevit\forms\__init__.py module into separate, more manageable, modules.
I’ve got to one module for each class, while I grouped the functions logically (file dialogs, list selection, alerts, input prompts, and so on).
To maintain backward compatibility, everything is imported back into the original __init__ module, but it’s all there it is in that file now.
Besides making the modules shorter, this helps gaining clarity on which parts need which imported library.
For example, the Autodesk objects are only used by the balloon (more on that later).

Turn the “Forms Module Test” button to CPython

In the spirit of Test Driven Development, I needed to start with some tests; fortunately the PyRevitDevTools extension already has a button to test most of the things provided by pyrevit.forms, so instead of writing my own tests, I just added the magic line to run them on CPython:

#! python3
"""Unit Tests for pyrevit.forms module."""
...

Obviously running the button didn’t work; as many of you already know, trying to use something from pyrevit.forms in a CPython script results in the following exception:

pyrevit.forms is not currently supported under CPython

Remove the IronPython check

I like to play with (digital) fire, so I went ahead and removed the lines that rasies the exception from the __init__ module:

if not IRONPY:
    raise PyRevitCPythonNotSupported('pyrevit.forms')

This changed the Exception to

could not import Autodesk.Windows.ComponentManager

Ignoring Autodesk stuff right now

As I mentioned before, that object is only used for the forms.show_balloon function, and since I’m not that interested in it, I just moved the call at the end of the test module, so that I could focus on WPF stuff.
Of course We’ll have to solve that portion of code at some point…
The error now becomes something like this:

AttributeError: 'NoneType' has no attribute 'LoadComponent'

The first function called by my updated test button is forms.ask_for_string, that creates a GetValueWindow object; this class inherits from TemplateUserInputWindow which in turn inherits from WPFWindow; in this class the __init__ method calls load_xml, which finally calls wpf.LoadComponent.
wpf is declared in pyrevit.framework:

wpf = None
if IRONPY:
    wpf_assmname = '{prefix}IronPython.Wpf'.format(prefix=eng.EnginePrefix)
    wpf_dllpath = op.join(eng.EnginePath, wpf_assmname + ASSEMBLY_FILE_EXT)
    clr.AddReferenceToFileAndPath(wpf_dllpath)
    import wpf

There it is, wpf is initialized only under IronPython, and it is None when using CPython! And this is expected, since wpf is an IronPython component that is incompatible with python.net.
To keep the current code as it is, I needed to come up with some compatibility layer to be imported when running CPython scripts.
Since LoadComponent is the only method of the wpf module used throughout the pyRevit code base, it is “just” a matter of implementing a compatible method.

Some fiddling, googling and stackoverflowing later I come up with the pyrevit.wpf module

import os.path

from System.IO import StreamReader
from System.Windows.Markup import XamlReader


def LoadComponent(root, xaml):
    root.window = _load_xaml(xaml)


def _load_xaml(xaml):
    if not isinstance(xaml, str):
        return XamlReader.Parse(xaml)
    xaml_stream = StreamReader(xaml).BaseStream if os.path.isfile(xaml) else xaml
    return XamlReader.Load(xaml_stream)

And after changing the portion of pyrevit.framework to:

if IRONPY:
    wpf_assmname = '{prefix}IronPython.Wpf'.format(prefix=eng.EnginePrefix)
    wpf_dllpath = op.join(eng.EnginePath, wpf_assmname + ASSEMBLY_FILE_EXT)
    clr.AddReferenceToFileAndPath(wpf_dllpath)
    import wpf
else:
    from . import wpf

And running the tests, I got welcomed by this error:

CPython Traceback:
'Failed to create a 'TextChanged' from the text 'string_value_changed'.' Line number '189' and line position '22'.

Had I read more thoroughly this SO answer, I would already know that python.net doesn’t support defining the event handlers inside the XAML file!

The short term, but very cumbersome solution would be to rewrite all the XAML files and move the bindings inside the python code. But this is not something we want to do, also because it breaks one of the coolest things in WPF, that is, being able to separate the presentation (UI) from the logic.

Another solution would be to wait until python.net developers solve this issue, but I cannot see any ETA on that; there’s an issue open since September 2018 that didn’t get much traction, and the project is pretty much in the same conditions as ours, very few people actively involved.

So what’s left is to try to understand what IronPython.wpf.LoadComponent does and replicate it in CPython… What could go wrong? :sweat_smile:

Undestanding IronPython.wpf

Searching the IronPython source code for LoadComponent, I saw that the Wpf class has various methods called like that (one for each type of object that we can pass to it, like the file path, a stream, an xml or xaml reader). This is the one accepting the xaml file path:

/// <summary>
/// Loads XAML from the specified XmlReader and returns the deserialized object.  Any event handlers
/// are bound to methods defined in the provided module.  Any named objects are assigned to the object.
/// 
/// The provided object is expected to be the same type as the root of the XAML element.
/// </summary>
public static object LoadComponent(CodeContext context, object self, string filename) {
    // some checks left out for brevity...
    return DynamicXamlReader.LoadComponent(self, context.LanguageContext.Operations, filename, XamlReader.GetWpfSchemaContext());
}

This is interesting, on the python side we only specify the self and filename arguments, what is the CodeContext object of which this method asks for the LanguageContext.Operations attribute? and what are those DynamicOperations? Let’s worry about it later.

Following the trail of method calls, the magic begins in DynamicXamlReader.LoadComponent(dynamic scope, DynamicOperations operations, XamlXmlReader reader):

  • the XAML file gets parsed and a custom XamlObjectWriter is used to keep track of the various x:Named objects
  • for each name found, the corresponding value is read and, if not null, the operations.SetMember((object)scope, name, value) is called (that is, the Window class will have an attribute of the specified name and value dinamically set)

Can we use the DynamicXamlReader in python.net?
The problem is that python.net doesn’t use CodeContext or DynamicOperations, hence we need to come up with something homemade…

Porting to Python

That SetMember thing… could it be translated to a setattr in python?
Let’s try to port the parts we need from DynamicXamlReader.cs:

import clr
from collections import deque

from System.IO import StreamReader
from System.Reflection import MemberTypes
from System.Windows.Markup import XamlReader
clr.AddReference("System.Xaml")
from System.Xaml import XamlMember
from System.Xaml import XamlXmlReader
from System.Xaml import XamlObjectWriter
from System.Xaml import XamlObjectWriterSettings
from System.Xaml.Schema import XamlMemberInvoker


def LoadComponent(root, filename):
    reader = XamlXmlReader(StreamReader(filename), XamlReader.GetWpfSchemaContext())
    settings = XamlObjectWriterSettings()
    settings.RootObjectInstance = root
    writer = _DynamicWriter(root, reader.SchemaContext, settings)
    while reader.Read():
        writer.WriteNode(reader)
    for name in writer.names:
        value = writer.RootNameScope.FindName(name)
        if value is not None:
            setattr(root, name, value)
    return writer.Result


class _DynamicWriter(XamlObjectWriter):
    def __init__(self, scope, context, settings):
        super(_DynamicWriter, self).__init__(context, settings)
        self._scope = scope
        self._names = set()
        self._name_stack = deque()

    @property
    def names(self):
        return self._names

    def WriteValue(self, value):
        if self._name_stack[-1] and isinstance(value, str) and value:
            self._names.add(value)
        super(_DynamicWriter, self).WriteValue(value)

    def WriteEndMember(self):
        self._name_stack.pop()
        super(_DynamicWriter, self).WriteEndMember()

    def WriteStartMember(self, property):
        self._name_stack.append(
            property.Name == "Name" and property.Type.UnderlyingType == str
        )

        if property.UnderlyingMember and property.UnderlyingMember.MemberType == MemberTypes.Event:
            super(_DynamicWriter, self).WriteStartMember(
                _DynamicEventMember(self, property.UnderlyingMember, self.SchemaContext)
            )
        else:
            super(_DynamicWriter, self).WriteStartMember(property)


class _DynamicEventMember(XamlMember):
    def __init__(self, writer, event_info, context):
        super(_DynamicEventMember, self).__init__(
            event_info.Name,
            lambda: None,
            context,
            _DynamicEventInvoker(event_info, writer)
        )

class _DynamicEventInvoker(XamlMemberInvoker):
    def __init__(self, event_info, writer):
        super(_DynamicEventInvoker, self).__init__()
        self._writer = writer
        self._info = event_info

    def SetValue(self, instance, value):
        target = getattr(self._writer._scope, str(value))
        self._info.AddEventHandler(instance, target)

And…

Set property "System.Windows.Window.ShowInTaskbar" threw an exception

ShowInTaskbar is the first attribute in the GetValueWindow.xaml file, so it seems that it’s trying to set attributes that shouldn’t be set.
I clearly don’t know what am I doing here, and it’s getting late so I’m stopping here for the moment.

If anybody wants to chime in, here’s my WIP branch with all the edits I made.
Feel free to suggest anything, even the dumbest question could help moving this forward!

4 Likes