LISP Benchmark

LISP Benchmarking Utility

Initially developed for private use, the Lisp Benchmark has grown over time to become an assisting tool even for developing the BricsCAD LISP engine. The Lisp Benchmark is still under development, and all major work, new functionality and improved Lisp engine implementations are covered by the benchmark system.

This Lisp Developer Support Package now includes the Lisp Benchmark for all Lisp developers.
New, extended and improved versions of this Lisp Benchmark will be provided with updated packages in the future.

We provide LISP Benchmarking Utility as FreeWare,
as long as all copyright notes remain unchanged throughout all its files.

Installation

Installation is very simple :

unzip and copy the folder "LispBenchmark" to any disk location of your choice (write-access needed !)
add a "Support Path" for "LispBenchmark" folder to CAD system's Search/Support Path configuration

Benchmark Concepts and Features

The Lisp Benchmark is designed to run on any LISP enabled CAD system, providing an AutoLISP compatible language and environment, on Windows. Linux and Macintosh platforms. It contains more than 360 individual benchmarks, grouped in 8 categories, covering the following areas :

Standard AutoLISP functions	standard functions A...Z as defined by AutoCAD R12
VisualLISP functions	(vl-xxx) functions provided by VisualLISP since AutoCAD 2000
COM utility functions	(vlax-xxx) functions
COM interface functions	(vla-xxx), (vla-get-xxx) and (vla-put-xxx) functions
VisualLISP Extension functions	verifying BricsCAD's (vle-xxx) functions - emulated on other systems
Fast-COM utility + interface functions	verifying BricsCAD's "Fast-COM" mode - emulated on other systems
LISP-Optimiser verification benchmarks	verifying BricsCAD's "Lisp-Optimiser" - emulated on other systems
User-defined (complex) benchmarks	complex scenarios based on user feedback & demand

Each individual benchmark file examines a particular, single LISP function in typical usage (different arguments etc.); this verifies the low-level, implementation-specific performance of the target function.

For all categories a specific command is provided to run all the benchmarks of that category; these commands also create a CAD-System + Platform + Command specific log file to store benchmarking results.

Besides predefined benchmark commands, each individual benchmark Lisp file defines a C:Run command to run separately - the easiest way to benchmark the particular function.

Running Benchmarks

There are 2 possible ways to run the benchmark(s) :

using main "Bench.lsp" Lisp file - simply load that Lisp file (Drag & Drop, APPLOAD etc.), a number of commands are reported in commandline :

BENCH
BENCHS
BENCHSV
BENCHV
BENCHVL
BENCHVLA
BENCHVLAX
BENCHVLAF
BENCHVLE
BENCHOPT
BENCHUSER

all functions
ST
ST + VL
VL + VLA + VLAX
VL
VLA
VLAX
VLA (Fast COM)
VLE Lisp
Optimizer
User & Application Benchmarks

using individual benchmark Lisp files - located in sub-folder ./Benches, there are benchmark Lisp files sorted by category; each benchmark file tests 1 dedicated Lisp API function;
-> simply drag & drop the desired benchmark file into the CAD system, or use APPLOAD or use (load "xxx.lsp")
-> each benchmark Lisp file contains a C:RUN command, enter RUN to start

Note :

all the category-specific commands defined by Bench.lsp also save a log file for the benchmark, located under "./BenchLogs" folder; the log filename also contains the platform and benchmark command, which allows to keep and compare all the benchmark results.