Compress vs Throughput
- Jonathan Reis (Unlicensed)
Analytics
Test Scenario 1
- Cooper Messages - 9K->10MB, 26892 messages, 1.427GB uncompressed size
- Measure time/throughput of Inbound HL7 Device -> QueueDevice (I->Q)
- Measure time/throughput of QueueDevice -> Null Device (Q->N)
- Clear Queues after each test
| Date | Version | Rev | Simultaneous Streams | Compression Type | Compressed Size MB* | Ratio | I->Q (sec) | I-Q m/sec | Q->I (sec)** | Q-> m/sec | Notes |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Nov 14, 2013 | 13.02.07R15 | d7299b191ce1 | 1 | IONIC ZIP | 108 MB | 92% | 130 | 206 | 71 | 379 | Heavy CPU |
| Nov 15, 2013 | 13.02.07R15 | d7299b191ce1 | 2 | IONIC ZIP | 216 MB | 92% | 130 | 413 | 147 | 366 | Heavy CPU |
| Nov 14, 2013 | 14.01.010 | 3607f2d38292 | 1 | .NET ZIP MAX Compression | 101 MB | 93% | 126 | 213 | 23 | 1169 | Heavy CPU |
| Nov 14, 2013 | 14.01.010 | 3607f2d38292 | 2 | .NET ZIP MAX Compression | 202 MB | 93% | 110 | 488 | 45 | 1195 | Heavy CPU |
| Nov 14, 2013 | 14.01.010 | 3607f2d38292 | 1 | LZ4 | 152 MB | 89% | 81 | 332 | 17 | 1582 | Lower CPU |
| Nov 14, 2013 | 14.01.010 | 3607f2d38292 | 2 | LZ4 | 304 MB | 89% | 74 | 726 | 33 | 1629 | Lower CPU |
| Nov 18, 2013 | 14.01.010 | 3607f2d38292 | 1 | LZ4 (64-bit) | 152 MB | 89% | 84 | 320 | 18 | 1494 | Lower CPU |
| Nov 18, 2013 | 14.01.010 | 3607f2d38292 | 2 | LZ4 (64-bit) | 304 MB | 89% | 82 | 655 | 39 | 1379 | Lower CPU |
Note strange behavior where 1 stream is much slower than 2 streams running at the same time
* Calculated from: select sum(LEN(MessageDataBytes)) from Message
** Calculated from: select DATEDIFF(ss,(select top 1 ModifiedDateTime from Message order by MessageId), (select top 1 ModifiedDateTime from Message order by MessageId desc))
Test Scenario 2
- Chicago Messages - 533 bytes->35782 bytes, 100000 messages, 927MB uncompressed size
- Measure time/throughput of Inbound HL7 Device -> QueueDevice (I->Q)
- Measure time/throughput of QueueDevice -> Null Device (Q->N)
- Clear Queues after each test
| Date | Version | Rev | Simultaneous Streams | Compression Type | Compressed Size MB* | Ratio | I->Q (sec) | I-Q m/sec | Q->I (sec)** | Q-I m/sec | Notes |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Nov 14, 2016 | 13.02.07R15 | d7299b191ce1 | 1 | IONIC ZIP | 49 MB | 46% | 151 | 662 | 76 | 1315 | Heavy CPU |
| Nov 15, 2013 | 13.02.07R15 | d7299b191ce1 | 2 | IONIC ZIP | 99 MB | 46% | 180 | 1111 | 156 | 1282 | Heavy CPU |
| Nov 14, 2013 | 14.01.010 | 3607f2d38292 | 1 | .NET ZIP MAX Compression | 46 MB | 50% | 156 | 641 | 32 | 3125 | Heavy CPU |
| Nov 14, 2013 | 14.01.010 | 3607f2d38292 | 2 | .NET ZIP MAX Compression | 92 MB | 50% | 108 | 1852 | 70 | 2857 | Heavy CPU |
| Nov 14, 2013 | 14.01.010 | 3607f2d38292 | 1 | LZ4 | 63 MB | 32% | 130 | 769 | 32 | 3125 | Lower CPU |
| Nov 14, 2013 | 14.01.010 | 3607f2d38292 | 2 | LZ4 | 126 MB | 32% | 99 | 2020 | 65 | 3076 | Lower CPU |
Note strange behavior where 1 stream is much slower than 2 streams running at the same time
* Calculated from: select sum(LEN(MessageDataBytes)) from Message
** Calculated from: select DATEDIFF(ss,(select top 1 ModifiedDateTime from Message order by MessageId), (select top 1 ModifiedDateTime from Message order by MessageId desc))