Import time of custom library module

Hello pyRevit friends!

After noticing pretty long loading times, even for scripts where nothing should happen, I started investigating the issue.

I found out it is the importing of a custom module i have created in my lib folder.
It takes 3-6 seconds to load in my scripts, i dont know why it varies that much.

This is a very short code, so I started to test loading time of other custom modules. A pretty large one with nearly 1000 lines of code takes only 0,2-0,3 seconds to load. They are in the same library folder.

A test setup that only loads the “problem module” gives me a time of only 0,3 seconds.

from pyrevit import script
from datetime import datetime
start_time = datetime.now()

import Log

print datetime.now() - start_time

But importing this module in my other scripts gives me importing times of 3-6 seconds.
I´m clueless why the loading time can be that different.

Is there anything I can do to improve that? For example, how to avoid double imports like:

uidoc = __revit__.ActiveUIDocument
doc = __revit__.ActiveUIDocument.Document
app = __revit__.Application

Because I import that in my script and in the module.
This is the mentioned module, happy about any advice and thoughts on this topic, thanks in advance!

# -*- coding: utf-8 -*-

import io
import os
import csv
import time
from datetime import datetime
from pyrevit import script

uidoc = __revit__.ActiveUIDocument
doc = __revit__.ActiveUIDocument.Document
app = __revit__.Application

FileName = script.get_bundle_name()
RevitFileName = doc.Title
User = app.Username

PROJECT_PATH = os.path.normpath(os.path.join(__file__, '../../../../'))
LogFilePath = os.path.join(PROJECT_PATH, 'UserLog\LOG.csv')

def LOG_Entry():
    now = datetime.now()
    dateformat = "%d.%m.%Y"
    timeformat = "%X"
    Date = now.strftime(dateformat)
    Time = now.strftime(timeformat)
    
    LOG_Entry = Time+"\t"+Date+"\t"+User+"\t"+FileName+"\t"+RevitFileName
    return LOG_Entry
    
def Read_LOG():
    retry = 0
    max_retries=5
    while True:
        try:
            with io.open(LogFilePath,"r", encoding = "utf-8") as LOGfile:
                LOG = LOGfile.read().splitlines()
        except OSError:
            time.sleep(0.5)
            retry += 1
            if retry > max_retries:
                raise
        else:
            return LOG
        
def Write_LOG():
    if os.path.exists(LogFilePath):
        LOG = Read_LOG()
        LOG_New_Entry = LOG_Entry()
        LOG.append(LOG_New_Entry)
        retry = 0
        max_retries=5
        while True:
            try:
                with io.open(LogFilePath,"w", encoding = "UTF8", newline='') as LOGfile:    
                    for L in LOG:
                        writer = csv.writer(LOGfile,delimiter="'")
                        writer.writerow([L])
            except OSError:
                time.sleep(0.5)
                retry += 1
                if retry > max_retries:
                    raise
            else:
                return LOG_New_Entry

I tried to improve the library file code and it really seems that it did work, down to 0,2-0,3 seconds for importing.

What I did:

  • I put all functions in an outer function, instead of calling the single functions one after another I now call the whole thing.
  • I reduced double imports by passing doc, uidoc and app as function variables

It seems that these improvements reduced the import time.

# -*- coding: utf-8 -*-

import io
import os
import csv
import time
from datetime import datetime

def create_log_entry(doc,uidoc,app):

    PROJECT_PATH = os.path.normpath(os.path.join(__file__, '../../../../'))
    LogFilePath = os.path.join(PROJECT_PATH, 'UserLog\LOG.csv')

    def LOG_Entry():
        FileName = __commandname__
        RevitFileName = doc.Title
        User = app.Username
        now = datetime.now()
        dateformat = "%d.%m.%Y"
        timeformat = "%X"
        Date = now.strftime(dateformat)
        Time = now.strftime(timeformat)
        
        LOG_Entry = Time+"\t"+Date+"\t"+User+"\t"+FileName+"\t"+RevitFileName
        return LOG_Entry
        
    def Read_LOG():
        retry = 0
        max_retries=5
        while True:
            try:
                with io.open(LogFilePath,"r", encoding = "utf-8") as LOGfile:
                    LOG = LOGfile.read().splitlines()
            except OSError:
                time.sleep(0.5)
                retry += 1
                if retry > max_retries:
                    raise
            else:
                return LOG
            
    def Write_LOG():
        if os.path.exists(LogFilePath):
            LOG = Read_LOG()
            LOG_New_Entry = LOG_Entry()
            LOG.append(LOG_New_Entry)
            retry = 0
            max_retries=5
            while True:
                try:
                    with io.open(LogFilePath,"w", encoding = "UTF8", newline='') as LOGfile:    
                        for L in LOG:
                            writer = csv.writer(LOGfile,delimiter="'")
                            writer.writerow([L])
                except OSError:
                    time.sleep(0.5)
                    retry += 1
                    if retry > max_retries:
                        raise
                else:
                    return LOG_New_Entry

    LOG_Entry()
    Read_LOG()
    Write_LOG()

Calling it like that:

Log.create_log_entry(doc,uidoc,app)

Hi @Gerhard.P, I’m glad you solved you problem.

May I ask you why you keep reading from and writing to the file instead of just append to the csv (using the “a” mode)?

Also, for this kind of task you may want to check out pyRevit’s telemetry system

Some random suggestions that don’t have anything to do with your question :sweat_smile: :

  • you don’t use the uidoc in the functions, so you can remove it from the main function arguments;
  • to avoid confusion (and possibly bugs), avoid naming variables with the same name as the function that contains them (I’m talking about LOG_Entry);
  • Instead of string concatenation, you can just pass a list to the csv writer: return [Time, Date, User, FileName, RevitFileName];
  • Since you already know the maximum retries, you can use a for loop instead of the while True loop. You can also turn the logic into a reusable function:
    def retry_fn(function, *args, **kwargs):
        max_retries = kwargs.pop("max_retries", 5)
        for _ in range(max_retries):
            try:
                return function()
            except OSError:
                time.sleep(0.5)
        raise OSError
    
  • To avoid to deep indentation, exit early when checking if the file exists:
    if not os.path.exists(LogFilePath):
        return
    
  • the “code-nazi” in me can’t help telling you to use lower_snake_case format for the variables and function names :wink:

All of these can be written as a single, untested, function:

def create_log_entry(doc, app):
    project_path = os.path.normpath(os.path.join(__file__, '../../../../'))
    log_path = os.path.join(project_path, 'UserLog\LOG.csv')
    now = datetime.now()
    log_entry = [
        now.strftime("%X"),
        now.strftime("%d.%m.%Y"),
        app.Username,
        __commandname__,
        doc.Title,
    ]
    # leaving the retry part out...
    with io.open(log_path, "a", encoding = "UTF8", newline='\n') as log_file:
        writer_object = writer(log_file)
        writer_object.writerow(log_entry)
1 Like

Hello @sanzoghenzo,

Thanks for your reply!

You come along just at the right time. I had a very very hard time setting up this code as you can see here:

So as I got it running I just never wanted to touch it again :smiley:

But now is the right time for me to improve that code because i will start to deploy the pyRevit toolbar to my coworkers in the next week. So i really appreciate your advice!

The telemtry system is a liitle overkill for me right now, i will get back to that when i have time in the future, error logging would be great.