Showing posts with label test. Show all posts
Showing posts with label test. Show all posts

Sunday, March 25, 2012

Test Metric Redefined

As programming of our test programs goes on, some of our goals have changed.

The "Fastest Bounce" program works through the CLD, intermittently. We seemed to be getting a bounce time of roughly 1 to 2 milliseconds, though this varied greatly. Also, the program only gets through a few bounces before crashing. We believe that this is due to the CLD front-end, so we are currently trying to implement this test with a stripped-down front-end that will avoid tampering with test results.
Also, as a side note, our last entry didn't include that accuracy will also be tested, even though we haven't seen anything to even remotely suggest that accuracy has been an issue.

The "Largest Message" program will be a moot point initially, since BlueMesh has a maximum message size of 1024 bytes. This will be added to the API. The "Fastest Bounce" test will be repeated for max size, however, to try and obtain a max throughput. As a later goal, we may want to try editing this maximum, but that would require serious code rewrites and is being tabled.

Friday, March 23, 2012

Testing Metrics and BlueMesh

We have been working on developing a number of metric test programs to try and find the technical limitations and performance ability of BlueMesh. Hopefully, we'll be able to get some useful and important data concerning performance and limitations for developers.

Right now, the tests we're trying to create and the questions we have are:
- Fastest Bounce : How quickly can BlueMesh send a small message from device A to device B and back? How much does this time change with N intermediate devices?

- Largest Message: What's the largest message that can reliably be sent through BlueMesh? Does this result in a significant slowdown? What's the greatest data throughput we can achieve with BlueMesh, and what affects it?

I'm considering adding a "Proliferation" test, to see how quickly a message can be sent to a large number of devices, but due to a lack of testing devices, this will likely have to wait.

Fastest Bounce has already been written and is on the git's development branch. It has not yet been shown to work, nor has it been run for numbers. Since I need two phones at least to run a test, achieving this has been difficult, but should be accomplished soon.