Just an open question, how do you (or would you) implement any form of unit & integration testing in your pyRevit extension?
The reason I’m asking is as follows:
I’m maintaining a pretty big list of tools, and testing them all when upgrading to revit 2024 is becoming more and more of a pain. I’ve dabbled with unit testing, and I am starting to set up some tests on my library extension using the unittest wrapper in pyrevit. However this is only the library part, i wonder if anyone has managed to also find a way to test their scripts themselves? If I understood correctly, testing a script would fall under the category 'Integration Testing? (or End to end?)
I’m starting to kind of fear that I’ll have to move every single function i’ve written in a script into the library and write unit tests for those… and that can’t be right… right?
I always struggle when developing plugins for autodesk products, and most of the time avoid unit testing and let my colleagues do the test for me
Obviusly this could work for small scale plugins (and pyrevit scripts) and companies, but I’m not endorsing it in any way!
A good software design is one that abstract the interfaces, so that the core logic can be unit tested without the need for the service/software that it calls at production time;
This allows you to quickly run the tests without running revit, but you have to emulate (mock) every single call made to the revit API. And you don’t have the certainty that the code will run with the actual API, especially when there are changes (exactly your case!)
So to answer your question:
yes, move what you can (code that doesn’t require revit) to a library and unit test it;
use pyrevit library as much as you can instead of calling revit api directly, and then use mocks in the unit tests to emulate/check pyrevit calls
for the integration tests you can try to use pyrevit cli and automate the tests (I’m interested to see if anyone already did this!)
if you need to emulare the user interaction, then end to end test are the way to go (but only after the other test pass)… I never implemented these kind of tests myself, but I understand they are really useful! Again, id somebody thit that, please share!
I am guessing you already checked the unit tests in the pyRevit Dev Extension.
I test all tools myself. Check for new breaking changes in the Revit API every year. Keeping up to date. This is actually the best way to work things out when upgrading a set of tools to the latest Revit API.
Not optimum, but with the time I have to actually maintain many tools, it is the best I can do.
I tend to build most of my tools with a UI based on the forms module of pyRevit, so the UI part stays robust and simple.
As @sanzoghenzo mentioned, it can be pretty tedious to split the frontend part from the backend on and not necessarily practical as many of the tools require interaction with Revit.
I also rely on my colleagues a lot. Deploying my extensions and tools from GitHub, it is pretty easy to quickly fix and deploy fixes. I can also fix it from my cell phone on the go.
Regarding your last question… it is kind of right. I am not a big fan of this approach though, as it requires a reload of the library and modules every time you make a change, which makes writing code a bit more tedious. For End to End testing you could use something similar to AutoHotKey. Very low tech approach but when requiring to simulate user interaction, it does make sense.
Concerning Unit Tests, I think I can eventually get some decent code coverage.
I think that one of the best options for me now might be to boy scout myself through my code. Any time I work on a script for whatever reason, I could move generic functionality to my library extension and write some unit tests for those. As time goes on I’d have myself a decent chunk of testable code.
I’ve also thought about doing integration tests using the pyrevit cli.
My first idea would be to create some testfiles in revit. I reckon making a testfile for each discipline would be a minimum since a lot of my functionality also revolves around the various kinds of templates we have in our company. I could then do a whole bunch of tests on these test.rvt files. It might take some time but they could run overnight ofcourse. I might be willing to try that. At the very least I could try that for any new kind of script I’m building.
As for the end-to-end testing.
I’m willing to look into the auto-hotkey approach, but I’m a bit worried that it wouldn’t be able to respond to any kind of errors popping up… then again I could check the logs instead of the pyRevit output window.
I’ve also found an Autodesk lecture somewhere about the ‘Revit Testing Framework’ which would allow you to test your code without the need of an active revit session. The joke in the lecture was that though the audience acknowledged that they did some form of test-driven development, nobody knew about this framework autodesk provides.t
From what I’ve gathered (from a glance) it seems to be a nuget package that allows you to run tests on your addin dll. Though pyrevit runs python scripts through an interpreter, if I understand correctly pyrevit DOES provide a dll as some sort of middleman layer between python and revit right? Would there be any benefit in exploring this?
There are other tools that do a better job at UI automation for testing; After a quick search I found this article that compares 3 open source tools to do automation testing for windows apps, one of which is developed by microsoft (WinAppDriver). Pick your poison and let us know!
A google search returned me this repository of the Dynamo folks; I found it before but never tried myself!
It is composed of a CLI and GUI app, they released it as a nuget package but it doesn’t need visual studio to run.
Unfortunately the latest release is dated 2018, and they halted the development with revit 2020, so you need to update the references and compile the tool yourself!
If you look at the implementation of the preflight checks tool, it resembles a unit testing: a main script calling and running a script or a set of scritps
