How does AVIX perform?

For performance testing use is made of the open source Thread-Metric benchmark test-suite produced by Express Logic Inc. More information on this test suite is provided later. Including AVIX, a total of six RTOSes are benchmarked using this test suite. Results of the tests are shown in the figure and table below. A blue entry in the table marks the fastest RTOS for that test.

AVIX is the winner in six of the seven tests contained in the Thread-Metric test suite.
Especially preemption handling, one of the key aspects of any RTOS is the fastest available.
 

If AVIX is that fast why then is it not the best on all tests?

AVIX does not produce top scores on the Memory Processing test. The reason for this is that AVIX allows memory blocks to be allocated and freed by Interrupt Service Routines where most competing products only allow threads to do so. Being allowed to use memory blocks both from Interrupt Service Routines and threads has a large advantage in that it aids high speed Interrupt based communication. With AVIX an Interrupt Service Routine is allowed to allocate a memory block, fill it with data coming in to the related device and once the data is ready to be processed by a thread, just send the id of the memory block to the thread through a pipe or a message. The thread can subsequently process the data and free the memory block. It shall be obvious this mechanism too comes without ever disabling interrupts but the consequence of this is that the implementation is more elaborate which is reflected by the result of the performance test. Still, AVIX-RT is convinced this very valuable functionality is essential to offer.

In absolute figures the AVIX memory performance is still quite good. Allocating a memory block takes less than 1.5µs on a 40MIPS part. Likewise does freeing a memory block take less than 1.3µs.

How to measure performance

Performance is generally considered one of the more discriminating aspects when choosing between different RTOS products. Many vendors provide figures about their products where every product looks fast and attractive. The question one should always ask is: “How is it measured?”. Saying a product is fast actually provides no information at all. The only objective way to say something about a products performance is to base the figures on results from an independent test suite. Such a test suite exists in the form of Thread-Metric. Thread-Metric is an open source test suite produced by Express Logic Inc. More information about this test suite can be found here...

How is the test performed

Initial test results were published by Mr. David Thedens, a US based embedded systems developer and RTOS expert. The test results were published on the Microchip web site in the Online Discussion Groups. You can find these results here. After conducting these initial tests upgrades have been made to a number of products. Also additional RTOS vendors decided to participate in the testing while others explicitly decided not to for reasons which are not clear. Results presented here are kept up to date as much as possible and are based on the initial tests and new test results as provided by a number of RTOS vendors on their own website.

AVIX-RT tries to keep the numbers actual as much as possible. We can however not guarantee that for certain products new versions are available for which no performance figures are presented by the specific vendor. When you think the figures presented here are outdated, please inform us and we will update the figures as soon as possible.

In able to validate the test results for yourself, AVIX-RT provides a Free Demo Distribution containing the test code. This Distribution can be downloaded from the download area on this site. Some other RTOS vendors provide the same service. If you want to receive the Thread-Metric sources for a specific RTOS, please send a message through the contact page and AVIX-RT will take care of forwarding the request to Mr. Thedens.

Which RTOSes are tested?

Even using the Thread-Metric test suite on a standalone product does not provide much information. Given a standard test, the real interesting question is “How do different products compare to each other?”. For the performance test, AVIX is compared against Q-Kernel, ThreadX, uc/OS-II. TNKernel and AVA. These are all products for the 16-bit controllers. For PIC32 no representative test has been performed.

Details

All tests are conducted for a 24HJ256GP610 controller running 40MIPS in the MPLAB™ development environment. Code is compiled using the Microchip C30 compiler version 3.02 or later. For each test the highest possible optimization that could be used for a certain product was used.