Testing CAD applications automatically: checking AutoCAD and BricsCAD add-ins

Test automation

One add-in for AutoCAD and BricsCAD, two program versions each, a new one every year: if you check that by hand, sooner or later you only test what you changed last. The defects then come from where nobody looked. CAD add-ins can be tested automatically – just not with a single tool.

Three levels, three tools

A new car goes onto the test bench before it reaches the customer. A machine measures the brakes, not the engineer, and does so the same way for every car. For an add-in this testing is split across three levels:

Each level is faster and cheaper than the next. So test as much as possible at the bottom. The lowest level has a side effect: a test project that only includes the CAD-free source files stops building as soon as an AutoCAD type slips into the logic. It guards your layers.

The middle level: AutoCAD without a window

The AutoCAD console is AutoCAD without a user interface: database, command line, LISP and .NET are all there, and it starts in a few seconds. It works through a script and then exits. A test command in the add-in creates a fresh drawing in memory for each case, checks it and finally writes a line that a script can search for:

[CommandMethod("TestStandard")]
public void TestStandard()
{
    var editor = Application.DocumentManager.MdiActiveDocument.Editor;
    var errors = 0;
    try
    {
        // Fresh drawing in memory: default contents yes, document no
        using (var db = new Database(true, true))
        using (var tr = db.TransactionManager.StartTransaction())
        {
            CreateLayer(tr, db, "A-WALL");
            CreateLayer(tr, db, "Furniture");
            var wrong = Standard.FindWrongLayers(tr, db);
            if (wrong.Count != 1 || !wrong[0].StartsWith("Furniture"))
            {
                editor.WriteMessage("\nERROR  Standard: " + string.Join(", ", wrong));
                errors++;
            }
            tr.Commit();
        }
    }
    catch (System.Exception ex)
    {
        // An unhandled exception would end the console
        editor.WriteMessage("\nERROR  Exception: " + ex.Message);
        errors++;
    }
    // The test script searches the log for this line
    editor.WriteMessage($"\nTESTRESULT: {errors} ERRORS");
}

The console is called with a drawing and a script:

"C:\Program Files\Autodesk\AutoCAD 2027\accoreconsole.exe" /i "C:\Tests\Empty.dwg" /s "C:\Tests\test.scr"

test.scr (ANSI, CRLF):
(setvar "SECURELOAD" 0)
_NETLOAD
"C:\Tests\MyAddin.dll"
TestStandard
(setvar "SECURELOAD" 1)

What we learned while testing the examples for the book “AutoCAD .NET API”:

• AutoCAD reads scripts without a BOM as ANSI. In UTF-8, accented characters turn into gibberish.
• An unknown command, even a typo, ends the whole script without an error message.
• An unhandled exception ends the console. Test commands therefore catch everything themselves.
• If an input is missing from the script, the console waits forever. The calling script needs a timeout.
• SECURELOAD is stored in the registry profile and then also applies to normal AutoCAD. If the script aborts early, the value is never reset. A separate profile or a separate Windows user for tests is cleaner.

With the /readonly switch, the same console checks whole folders of customer drawings without changing them. If the “TESTRESULT” line is missing from the log, something worse than a rule violation has happened: a corrupt file or a timeout. For BricsCAD, the Bricsys help names the startup switches /b for a script and /automation for BricsCAD with a hidden main window. The script then has to close the program itself.

Learning tests: what your code expects from AutoCAD

Some tests do not check your code but the behaviour of the CAD program your code relies on. Does a method throw an exception for a certain value? Does a new drawing come with or without the Defpoints layer? If that changes with the next version, a test should report it, not a user. And such tests belong in the environment where the behaviour occurs: a test in a drawing in memory does not behave in every respect like the same code in the active drawing. A learning test showed us exactly that – by turning red.

The top level: user interface and drawing area

Dialogs and the ribbon do not exist in the console. This is where our test tool TesteriX comes in. For the drawing area it deliberately does not record mouse clicks, because that would influence the add-ins too much. Instead it controls the drawing area with CAD commands from its catalogue, organised by object, layer and drawing. A command can return a value, such as the ID of a new circle or the number of layers. The value goes into a variable, and a comparison checks it against the expected value. If a CAD program starts a further process, as OEM variants of AutoCAD and BricsCAD do, TesteriX distinguishes the started program from the process actually running. Our recommendation for the workstation: two monitors, CAD on one, TesteriX on the other.

Typical mistakes

• Testing everything through the user interface. Whatever can be checked in the console belongs there.
• Mixing logic and CAD access. Then nothing can be tested without CAD any more.
• Test commands without their own try/catch. The first exception ends the entire run.
• Test scripts without a timeout. A waiting command blocks the build server.
• Switching off SECURELOAD in the test script and leaving normal AutoCAD unprotected.
• Testing original drawings instead of copies.
• Running only your own tests after a version change, without any learning tests.

Sources: Ebi F. Eilber, AutoCAD .NET API (book, expected 2027), chapter “Test”; TesteriX help, chapters “CAD specifics” and “Catalogue”; Bricsys Help, “Startup options”.