Home

AVIX is among the best performing RTOS’s on the market as proven by the results of the Thread-Metric open source benchmark!

AVIX is compared against five competing products, representative for today’s market. The comparison is made using the open source Thread-Metric benchmark test-suite produced by Express Logic Inc. AVIX wins in five of the seven tests contained in this test-suite. Especially preemption handling, one of the key aspects of any RTOS, is the fastest available.

AVIX for PIC24-dsPIC 4.0

18,730,514

11,460,380

17,125,013

6,023,870

8,151,857

27,878,435

12,618,419

Q-Kernel 3.0 (1418)

15,986,417

11,205,261

15,986,102 (3)

5,848,623

12,489,734 (3)

24,978,625 (3)

24,978,541

ThreadX

11,847,800

4,870,885

6,918,050

3,052,151

6,928,383

15,337,354

12,863,624

uc/OS-II

(1)

3,909,085

5,259,998

(1)

(2)

10,293,318

6,814,817

TNKernel

(1)

4,138,692

7,784,052

3,180,224

5,722,266

13,623,702

9,745,907

AVA

(1)

1,724,948

5,207,762

1,260,190

2,761,154

7,514,799

10,235,182

(1) For a reason why this specific test is not conducted, please refer the Microchip Online Discussion Group where the original results are published. You can find this information here.

(2) For the Thread-Metric test messages of 16 bytes are used. uc/OS-II transfers a pointer to a 16 byte area and does not meet the test criteria.

(3) Read below to learn the background behind this figure and the meaning of the horizontal line in the bar.

What do the performance figures tell us?

Performance figures only tell a part of the story. One of the key aspects of any RTOS is that it helps in managing the applications resources. This is done by offering potentially blocking functions. When an application attempts to lock a semaphore while this is not available, the thread is blocked and another thread will become active. Only after the semaphore is released by another thread, the blocked thread will start running again. For the application, locking the semaphore just looks like a regular function call. Whether or not the calling thread is blocked in the process can not be seen in the code. The alternative would be to poll the semaphore. Polling however does consume unnecessary CPU cycles, makes the code harder to read and preventing polling is why an RTOS is used in the first place. For this reason it is correct to assume the RTOS functions used in the Thread-Metric test should be potentially blocking functions, since only then the figures reflect the performance that may be expected in the application using the RTOS. A few of the originally published figures however are based on using non-blocking functions. In those cases, the tests are also executed using the blocking version of the function used so the figures better reflect the performance that may be expected in the application. For tests where this is applicable, the bar in the figure above contains a horizontal separation line. The bar as a whole reflects the performance based on the non-blocking function while the part of the bar below the separation line reflects performance when using the blocking version of the function.

 

 

How to measure performance

Performance is generally considered one of the more discriminating aspects when choosing between different RTOS products. Many vendors provide figures about their products where every product looks fast and attractive. The question one should always ask is: “How is it measured?”. Saying a product is fast actually provides no information at all. The only objective way to say something about a products performance is to base the figures on results from an independent test suite. Such a test suite exists in the form of Thread-Metric. Thread-Metric is an open source test suite produced by Express Logic Inc. More information about this test suite can be found here...

 

 

How is the test performed

Initial test results were published by Mr. David Thedens, a US based embedded systems developer and RTOS expert. The test results were published on the Microchip web site in the Online Discussion Groups. You can find these results here. After conducting these initial tests upgrades have been made to a number of products. Also additional RTOS vendors decided to participate in the testing while others explicitly decided not to for reasons which are not clear. Results presented here are kept up to date as much as possible and are based on the initial tests and new test results as provided by a number of RTOS vendors on their own website.

AVIX-RT tries to keep the numbers actual as much as possible. We can however not guarantee that for certain products new versions are available for which no performance figures are presented by the specific vendor. When you think the figures presented here are outdated, please inform us and we will update the figures as soon as possible.

In able to validate the test results for yourself, AVIX-RT provides a Free Demo Distribution containing the test code. This Distribution can be downloaded from the download area on this site. Some other RTOS vendors provide the same service. If you want to receive the Thread-Metric sources for a specific RTOS, please send a message through the contact page and AVIX-RT will take care of forwarding the request to Mr. Thedens.

 

 

Which RTOSes are tested?

Even using the Thread-Metric test suite on a standalone product does not provide much information. Given a standard test, the real interesting question is “How do different products compare to each other?”. For the performance test, AVIX is compared against Q-Kernel, ThreadX, uc/OS-II. TNKernel and AVA. These are all products for the 16-bit controllers. For PIC32 no representative test has been performed.

 

 

Details

All tests are conducted for a 24HJ256GP610 controller running 40MIPS in the MPLAB™ development environment. Code is compiled using the Microchip C30 compiler version 3.02 or later. For each test the highest possible optimization that could be used for a certain product was used.

AVIX-RT © 2006-2012, All Rights Reserved

Legal Disclaimer

Privacy Policy