Skip to content

Commit 7a010b3

Browse files
author
Mark Cavage
committed
silly microbenchmark
1 parent 0a6a8dc commit 7a010b3

7 files changed

Lines changed: 203 additions & 0 deletions

File tree

‎examples/bench/README.md‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
# tl;dr
2+
3+
I get asked *all the time* to compare restify and express. Here's a trivial
4+
"microbenchmark" of restify 1.4.4, restify 2.0.0 and express 3.0.5. In all
5+
honesty, I think this is not tremendously useful, as `ab` is notoriously
6+
unreliable, and in particular, both restify and express will let you build a
7+
fast application, and in the real world you are far more likely to be I/O bound.
8+
9+
That said, we can clearly see a massive throughput gain in restify 2.0 over 1.4.
10+
The latency profile is very similar in all three tests cases, which is expected
11+
(i.e., it's driven by node).
12+
13+
# Sample code
14+
15+
Simple ability to switch between restify/express:
16+
17+
var server = require('restify').createServer();
18+
//var server = require('express')();
19+
20+
server.get('/echo/:name', function (req, res, next) {
21+
res.setHeader('content-type', 'text/plain');
22+
res.send(200, req.params.name);
23+
});
24+
25+
server.listen(8080, function () {
26+
console.log('ready');
27+
});
28+
29+
Note there is no use of the `cluster` api, so in all cases we expect
30+
to (easily) overwhelm the V8 event thread.
31+
32+
# Command
33+
34+
All tests were run using node 0.8.15 on [SmartOS](http://smartos.org/) in a
35+
VMWare VM, and each set was run at least 3 times; outliers were discarded,
36+
and the "average" of each test run was recorded below.
37+
38+
Just using `ab` on localhost, the results of the following command
39+
were tracked at varying concurrency levels:
40+
41+
$ ab -k -c N -n 50000 http://127.0.0.1:8080/echo/mark
42+
43+
Only RPS and P99 latency was tracked (mean, median and average are mostly
44+
useless in the real world).
45+
46+
# Results
47+
48+
## Restify 1.4.4
49+
50+
<table>
51+
<tr>
52+
<td>Concurrency</td>
53+
<td>Requests/Second</td>
54+
<td>P99 Latency (ms)</td>
55+
</tr>
56+
<tr>
57+
<td>10</td>
58+
<td>3806.33</td>
59+
<td>5</td>
60+
</tr>
61+
<tr>
62+
<td>50</td>
63+
<td>3842.84</td>
64+
<td>22</td>
65+
</tr>
66+
<tr>
67+
<td>100</td>
68+
<td>3850.95</td>
69+
<td>34</td>
70+
</tr>
71+
<tr>
72+
<td>500</td>
73+
<td>3707.68</td>
74+
<td>158</td>
75+
</tr>
76+
<tr>
77+
<td>1000</td>
78+
<td>3560.86</td>
79+
<td>3564</td>
80+
</tr>
81+
</table>
82+
83+
## Restify 2.0.0
84+
85+
<table>
86+
<tr>
87+
<td>Concurrency</td>
88+
<td>Requests/Second</td>
89+
<td>P99 Latency (ms)</td>
90+
</tr>
91+
<tr>
92+
<td>10</td>
93+
<td>6293.61</td>
94+
<td>2</td>
95+
</tr>
96+
<tr>
97+
<td>50</td>
98+
<td>6249.21</td>
99+
<td>12</td>
100+
</tr>
101+
<tr>
102+
<td>100</td>
103+
<td>6073.08</td>
104+
<td>20</td>
105+
</tr>
106+
<tr>
107+
<td>500</td>
108+
<td>5653.52</td>
109+
<td>105</td>
110+
</tr>
111+
<tr>
112+
<td>1000</td>
113+
<td>5657.44</td>
114+
<td>3490</td>
115+
</tr>
116+
</table>
117+
118+
119+
## Express 3.0.5
120+
121+
<table>
122+
<tr>
123+
<td>Concurrency</td>
124+
<td>Requests/Second</td>
125+
<td>P99 Latency (ms)</td>
126+
</tr>
127+
<tr>
128+
<td>10</td>
129+
<td>6057.91</td>
130+
<td>3</td>
131+
</tr>
132+
<tr>
133+
<td>50</td>
134+
<td>5781.69</td>
135+
<td>11</td>
136+
</tr>
137+
<tr>
138+
<td>100</td>
139+
<td>5533.35</td>
140+
<td>24</td>
141+
</tr>
142+
<tr>
143+
<td>500</td>
144+
<td>5300.36</td>
145+
<td>114</td>
146+
</tr>
147+
<tr>
148+
<td>1000</td>
149+
<td>5261.26</td>
150+
<td>3512</td>
151+
</tr>
152+
</table>

‎examples/bench/bench.js‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
var server = require('restify').createServer();
2+
//var server = require('express')();
3+
4+
server.get('/echo/:name', function (req, res, next) {
5+
res.setHeader('content-type', 'text/plain');
6+
res.send(200, req.params.name);
7+
});
8+
9+
server.listen(8080, function () {
10+
console.log('ready');
11+
});

‎examples/bench/dat/e305.dat‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
10 6057.91 3
2+
50 5781.69 11
3+
100 5533.35 24
4+
500 5300.36 114
5+
1000 5261.26 3512

‎examples/bench/dat/r144.dat‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
10 3806.33 5
2+
50 3842.84 22
3+
100 3850.95 34
4+
500 3707.68 158
5+
1000 3560.86 3564

‎examples/bench/dat/r200.dat‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
10 6293.61 2
2+
50 6249.21 12
3+
100 6073.08 20
4+
500 5653.52 105
5+
1000 5657.44 3490

‎examples/bench/plot.txt‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
set terminal png
2+
set output "req_and_latency.png"
3+
4+
set grid
5+
6+
set xlabel "concurrency"
7+
8+
set ylabel "requests/second"
9+
set y2label "response time p99 (ms)"
10+
11+
set xrange [10 : 1000]
12+
set yrange [3500 : 6500]
13+
set y2range [0 : 3600]
14+
15+
set ytics
16+
set y2tics
17+
18+
plot \
19+
"dat/r144.dat" using 1:2 axes x1y1 smooth sbezier with lines title "restify 1.4 r/s", \
20+
"dat/r144.dat" using 1:3 axes x1y2 smooth sbezier with lines title "restify 1.4 p99", \
21+
"dat/r200.dat" using 1:2 axes x1y1 smooth sbezier with lines title "restify 2.0 r/s", \
22+
"dat/r200.dat" using 1:3 axes x1y2 smooth sbezier with lines title "restify 2.0 p99", \
23+
"dat/e305.dat" using 1:2 axes x1y1 smooth sbezier with lines title "express 3.0 r/s", \
24+
"dat/e305.dat" using 1:3 axes x1y2 smooth sbezier with lines title "express 3.0 p99"
25+

‎examples/bench/req_and_latency.png‎

7.22 KB
Loading

0 commit comments

Comments
 (0)