/
Connexion Peformance Test in PT

Connexion Peformance Test in PT

Owned by Jonathan Reis (Unlicensed)
Sept 02, 2014

Test Scenario - Feb 2014

 

Load Generator Setup

The Load Generator is configured to run 10 channels, each of which is generating patient episodes. There are 8 messages per episode including

  • ADT^A04
  • ADT^A08
  • 2 x ORM^O01
  • 2 x ORU^R01

The generated messages are sent to the Application server using the Outbound HL7 Device. Each Outbound HL7 Device points to a specific port on the Application Server.

The Connexion config file for the Load Generator channels can be found here: Connexion Peformance Test in PT. A screen-shot of the setup can be found below.

 

 

Application Server Setup

The Application Server is set up with 10 channels to each receiving HL7 on a separate HL7 Inbound Device running on a distinct port.

The received HL7 is run through an HL7 Transform device which has 6 simple transform steps, then it is sent to an HL7 Outbound device which sends the data to an HL7 Endpoint on the Application Server which simple receives and then throws away the message.

The Connexion config file for the Application Server channels can be found here: Connexion Peformance Test in PT. A screen-shot of the setup can be found below.

NOTE: Since the drive space on the Database Server is undersized, the test could only be run for 20 hrs before the database would run out of disk space. To prevent fill-up, an additional channel was added to periodically purge messages older than 20 hours old.

Observations

A screen shot of the performance characteristics of Connexion during this test can be found below. As can be seen from the image:

  1. the peak processing speed is approximately 4.2 million messages per hour
  2. the peak queuing speed is approximately 3 million messages per hour
  3. the memory utilization is quite low for this test. Connexion is using less than 200MB for the 10 channels. Note, however; that the messages being processed are very small
  4. the sinusoidal wave characteristic occurs because the processing happens much more quickly than the queuing (queuing is single threaded, whereas some aspects of the processing pipeline are threaded). The queue gets drained of all "queued" messages, and then goes to sleep for a period of time.
  5. The Application Server's CPU utilization ranges from 35%-95% in the same sinusoidal pattern as the message processing. During pure queuing the CPU is the lowest, during peak processing, the CPU reaches its peak utilization.

 

Conclusions

  1. Under the current PT setup, a sustained throughput of 650 messages/second seems reasonable. However; given the load on the application server CPUs there is not much head-room to perform more complex message processing operations.
  2. The lack of disk space in the database server prevents any meaningful conclusions on the long-term performance characteristics of the storage infrastructure. The database runs out of space after 20 hours of operation and needs to be purged constantly to prevent this condition from occurring.
  3. Based on the CPU utilization of the Application Server, we believe the hardware setup is too under-powered to come to any real conclusions about the message processing capacity of Connexion. In a production system we would have many more channels (albeit processing at a low rate) and they would perform more complex message processing, such is the case when processing messages using the Clinical Data Store filing device. We recommend the Application Server's hardware be increased to a 16-Core (dedicated, instead of shared Cores as was tested on) machine. Similarly we believe the Database Server needs to be more powerful, have faster drives, and less network latency. It to should have dedicated Cores rather than shared ones.
  4. Even with under powered hardware, we have comparable performance Mirth's published tests (Connexion Peformance Test in PT) on their highest end server ($40,000 dollars) which utilizes 24 striped SSD drives.
  5. The sizes of messages we tested with were too small. The tests need to be adjusted to meet the message size estimates found in the Mercy production system (Connexion Peformance Test in PT).

UPDATE 1 - June 30, 2014

After PT environment upgrade, the results are worse than before. See graph below. I am not sure if this is due to changes in the application. We went from using synchronous methods, to asynchronous ones which seem to require more CPU cycles.

Application Server is more CPU bound than the tests performed in February 2014.