Overall
Derived
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput index | 32.4 | — | — | 12 / 15 |
| CPU efficiency index | 28.9 | — | — | 13 / 15 |
| Peak memory | 156.6 MiB | — | — | 10 / 15 |
Plug router on the Bandit HTTP server (the server Phoenix uses), Postgrex with prepared statements reused like Ecto does.
Every scenario and metric measured for this implementation. Primary metrics first; the rest are one click away. Ranks count all 15 implementations of the run.
Derived
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput index | 32.4 | — | — | 12 / 15 |
| CPU efficiency index | 28.9 | — | — | 13 / 15 |
| Peak memory | 156.6 MiB | — | — | 10 / 15 |
Start-up
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Cold start | 1,203 ms 95 % CI 1,190–1,242 | 1,190–1,242 | 5 | 12 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Memory | ||||
| Memory at idle | 125.6 MiB 95 % CI 122.8–133.7 | 122.8–133.7 | 5 | 10 / 15 |
| Anonymous memory at idle | 90.2 MiB 95 % CI 87.5–98.4 | 87.5–98.4 | 5 | 11 / 15 |
| Start-up | ||||
| Cold start (fastest) | 1,177 ms 95 % CI 1,125–1,198 | 1,125–1,198 | 5 | 12 / 15 |
| Cold start (slowest) | 1,270 ms 95 % CI 1,215–1,328 | 1,215–1,328 | 5 | 12 / 15 |
| Footprint | ||||
| Image size | 249 MB | — | 5 | 6 / 15 |
Cold load
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | 20,981 req/s 95 % CI 19,528–21,222 | 19,528–21,222 | 5 | 10 / 15 |
| Latency p99 | 7.86 ms 95 % CI 7.58–9.76 | 7.58–9.76 | 5 | 5 / 15 |
| CPU per request | 95.2 µs 95 % CI 94.1–102.1 | 94.1–102.1 | 5 | 10 / 15 |
| Memory under load (PSS) | 131.1 MiB 95 % CI 129.5–141.8 | 129.5–141.8 | 5 | 10 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | ||||
| Errors | 0 requests | — | 5 | 1 / 15 |
| Error rate | 0.0000 | — | 5 | 1 / 15 |
| Latency | ||||
| Latency p50 | 6.00 ms 95 % CI 5.89–6.29 | 5.89–6.29 | 5 | 10 / 15 |
| Latency p90 | 6.50 ms 95 % CI 6.41–7.65 | 6.41–7.65 | 5 | 8 / 15 |
| Latency p99.9 | 32.94 ms 95 % CI 19.24–33.26 | 19.24–33.26 | 5 | 3 / 15 |
| Latency p75 | 6.25 ms 95 % CI 6.11–6.77 | 6.11–6.77 | 5 | 9 / 15 |
| Latency p99.99 | 52.01 ms 95 % CI 32.72–58.20 | 32.72–58.20 | 5 | 4 / 15 |
| Latency mean | 6.11 ms 95 % CI 6.03–6.55 | 6.03–6.55 | 5 | 8 / 15 |
| Latency std. dev. | 1.48 ms 95 % CI 0.84–1.76 | 0.84–1.76 | 5 | 3 / 15 |
| Latency max | 58.13 ms 95 % CI 38.52–68.47 | 38.52–68.47 | 5 | 4 / 15 |
| CPU | ||||
| CPU used | 1.99 cores 95 % CI 1.99–1.99 | 1.99–1.99 | 5 | — |
| User-space CPU share | 0.801 95 % CI 0.800–0.806 | 0.800–0.806 | 5 | — |
| CPU throttling | 14.6 ms 95 % CI 7.6–22.0 | 7.6–22.0 | 5 | 13 / 15 |
| Memory | ||||
| Anonymous memory peak | 95.8 MiB 95 % CI 94.3–106.6 | 94.3–106.6 | 5 | 9 / 15 |
| Database | ||||
| Database CPU per request | 66.4 µs 95 % CI 53.8–82.8 | 53.8–82.8 | 5 | 7 / 15 |
| SELECTs per request | 1.00 queries 95 % CI 1.00–1.00 | 1.00–1.00 | 5 | — |
| UPDATEs per request | 0.00 queries | — | 5 | — |
| Start-up | ||||
| Cold vs. warm throughput | 1.04 95 % CI 0.98–1.05 | 0.98–1.05 | 5 | 3 / 15 |
| Energy | ||||
| CPU package energy | 504.9 J 95 % CI 498.7–519.0 | 498.7–519.0 | 5 | — |
| Package energy per request | 2.372 mJ 95 % CI 2.356–2.632 | 2.356–2.632 | 5 | 10 / 15 |
| Network | ||||
| Bytes per response | 229 B 95 % CI 229–229 | 229–229 | 5 | — |
| Disk I/O | ||||
| Disk writes | 0 KiB | — | 5 | 1 / 15 |
| Measurement quality | ||||
| Hypervisor steal | 0.0 % | — | 5 | 1 / 15 |
| Host load | 5.07 runnable tasks 95 % CI 3.07–6.12 | 3.07–6.12 | 5 | 12 / 15 |
| CPU clock | 3,455 MHz 95 % CI 3,400–3,563 | 3,400–3,563 | 5 | — |
| Load generator CPU | 0.23 95 % CI 0.19–0.24 | 0.19–0.24 | 5 | 6 / 15 |
Closed loop
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | 45,480 req/s 95 % CI 42,683–47,276 | 42,683–47,276 | 5 | 12 / 15 |
| Latency p99 | 4.12 ms 95 % CI 3.29–4.51 | 3.29–4.51 | 5 | 7 / 15 |
| CPU per request | 44.0 µs 95 % CI 42.3–46.8 | 42.3–46.8 | 5 | 12 / 15 |
| Memory under load (PSS) | 150.7 MiB 95 % CI 143.5–160.5 | 143.5–160.5 | 5 | 11 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | ||||
| Errors | 0 requests | — | 5 | 1 / 15 |
| Error rate | 0.0000 | — | 5 | 1 / 15 |
| Latency | ||||
| Latency p50 | 2.78 ms 95 % CI 2.67–2.89 | 2.67–2.89 | 5 | 12 / 15 |
| Latency p90 | 3.03 ms 95 % CI 2.84–3.46 | 2.84–3.46 | 5 | 12 / 15 |
| Latency p99.9 | 4.86 ms 95 % CI 3.90–5.02 | 3.90–5.02 | 5 | 6 / 15 |
| Latency p75 | 2.87 ms 95 % CI 2.76–3.06 | 2.76–3.06 | 5 | 12 / 15 |
| Latency p99.99 | 6.96 ms 95 % CI 6.05–11.05 | 6.05–11.05 | 5 | 6 / 15 |
| Latency mean | 2.79 ms 95 % CI 2.68–2.97 | 2.68–2.97 | 5 | 12 / 15 |
| Latency std. dev. | 0.33 ms 95 % CI 0.23–0.43 | 0.23–0.43 | 5 | 5 / 15 |
| Latency max | 14.47 ms 95 % CI 10.46–18.68 | 10.46–18.68 | 5 | 7 / 15 |
| CPU | ||||
| CPU used | 2.00 cores 95 % CI 2.00–2.00 | 2.00–2.00 | 5 | — |
| User-space CPU share | 0.803 95 % CI 0.800–0.805 | 0.800–0.805 | 5 | — |
| CPU throttling | 20.6 ms 95 % CI 11.0–27.3 | 11.0–27.3 | 5 | 12 / 15 |
| Memory | ||||
| Anonymous memory peak | 115.3 MiB 95 % CI 108.1–125.3 | 108.1–125.3 | 5 | 11 / 15 |
| Database | ||||
| Database CPU per request | < 0.1 µs 95 % CI < 0.1–< 0.1 | < 0.1–< 0.1 | 5 | 10 / 15 |
| Energy | ||||
| CPU package energy | 709.4 J 95 % CI 696.5–760.1 | 696.5–760.1 | 5 | — |
| Package energy per request | 1.032 mJ 95 % CI 0.994–1.179 | 0.994–1.179 | 5 | 12 / 15 |
| Network | ||||
| Bytes per response | 225 B | — | 5 | — |
| Disk I/O | ||||
| Disk writes | 0 KiB | — | 5 | 1 / 15 |
| Measurement quality | ||||
| Hypervisor steal | 0.0 % | — | 5 | 1 / 15 |
| Host load | 4.80 runnable tasks 95 % CI 3.57–5.96 | 3.57–5.96 | 5 | 2 / 15 |
| CPU clock | 3,717 MHz 95 % CI 3,614–3,800 | 3,614–3,800 | 5 | — |
| Load generator CPU | 0.50 95 % CI 0.44–0.51 | 0.44–0.51 | 5 | 4 / 15 |
Closed loop
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | 20,218 req/s 95 % CI 19,968–20,765 | 19,968–20,765 | 5 | 12 / 15 |
| Latency p99 | 7.91 ms 95 % CI 7.58–9.26 | 7.58–9.26 | 5 | 10 / 15 |
| CPU per request | 98.7 µs 95 % CI 96.2–100.1 | 96.2–100.1 | 5 | 12 / 15 |
| Memory under load (PSS) | 145.9 MiB 95 % CI 141.6–154.3 | 141.6–154.3 | 5 | 10 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | ||||
| Errors | 0 requests | — | 5 | 1 / 15 |
| Error rate | 0.0000 | — | 5 | 1 / 15 |
| Latency | ||||
| Latency p50 | 6.23 ms 95 % CI 6.08–6.32 | 6.08–6.32 | 5 | 12 / 15 |
| Latency p90 | 6.80 ms 95 % CI 6.65–7.11 | 6.65–7.11 | 5 | 10 / 15 |
| Latency p99.9 | 9.51 ms 95 % CI 9.20–10.42 | 9.20–10.42 | 5 | 7 / 15 |
| Latency p75 | 6.50 ms 95 % CI 6.34–6.57 | 6.34–6.57 | 5 | 12 / 15 |
| Latency p99.99 | 12.56 ms 95 % CI 11.53–20.05 | 11.53–20.05 | 5 | 6 / 15 |
| Latency mean | 6.31 ms 95 % CI 6.12–6.37 | 6.12–6.37 | 5 | 12 / 15 |
| Latency std. dev. | 0.55 ms 95 % CI 0.47–0.79 | 0.47–0.79 | 5 | 5 / 15 |
| Latency max | 14.13 ms 95 % CI 13.02–26.00 | 13.02–26.00 | 5 | 6 / 15 |
| CPU | ||||
| CPU used | 1.99 cores 95 % CI 1.99–2.00 | 1.99–2.00 | 5 | — |
| User-space CPU share | 0.794 95 % CI 0.789–0.806 | 0.789–0.806 | 5 | — |
| CPU throttling | 15.0 ms 95 % CI 12.1–34.0 | 12.1–34.0 | 5 | 11 / 15 |
| Memory | ||||
| Anonymous memory peak | 110.5 MiB 95 % CI 106.3–119.1 | 106.3–119.1 | 5 | 9 / 15 |
| Database | ||||
| Database CPU per request | 99.6 µs 95 % CI 69.1–108.6 | 69.1–108.6 | 5 | 13 / 15 |
| SELECTs per request | 1.00 queries 95 % CI 1.00–1.00 | 1.00–1.00 | 5 | — |
| UPDATEs per request | 0.00 queries | — | 5 | — |
| Energy | ||||
| CPU package energy | 724.9 J 95 % CI 707.3–778.5 | 707.3–778.5 | 5 | — |
| Package energy per request | 2.360 mJ 95 % CI 2.325–2.582 | 2.325–2.582 | 5 | 12 / 15 |
| Network | ||||
| Bytes per response | 229 B 95 % CI 229–229 | 229–229 | 5 | — |
| Disk I/O | ||||
| Disk writes | 0 KiB | — | 5 | 1 / 15 |
| Measurement quality | ||||
| Hypervisor steal | 0.0 % | — | 5 | 1 / 15 |
| Host load | 4.72 runnable tasks 95 % CI 3.77–5.63 | 3.77–5.63 | 5 | 2 / 15 |
| CPU clock | 3,520 MHz 95 % CI 3,467–3,693 | 3,467–3,693 | 5 | — |
| Load generator CPU | 0.22 95 % CI 0.20–0.23 | 0.20–0.23 | 5 | 4 / 15 |
Closed loop
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | 3,474 req/s 95 % CI 3,331–3,503 | 3,331–3,503 | 5 | 12 / 15 |
| Latency p99 | 45.04 ms 95 % CI 43.68–51.12 | 43.68–51.12 | 5 | 7 / 15 |
| CPU per request | 570.4 µs 95 % CI 566.1–595.3 | 566.1–595.3 | 5 | 13 / 15 |
| Memory under load (PSS) | 145.6 MiB 95 % CI 141.5–154.1 | 141.5–154.1 | 5 | 10 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | ||||
| Errors | 0 requests | — | 5 | 1 / 15 |
| Error rate | 0.0000 | — | 5 | 1 / 15 |
| Latency | ||||
| Latency p50 | 36.40 ms 95 % CI 36.10–37.65 | 36.10–37.65 | 5 | 13 / 15 |
| Latency p90 | 38.57 ms 95 % CI 38.06–41.02 | 38.06–41.02 | 5 | 8 / 15 |
| Latency p99.9 | 58.24 ms 95 % CI 48.05–66.66 | 48.05–66.66 | 5 | 8 / 15 |
| Latency p75 | 37.41 ms 95 % CI 37.03–38.98 | 37.03–38.98 | 5 | 9 / 15 |
| Latency p99.99 | 62.22 ms 95 % CI 49.42–71.92 | 49.42–71.92 | 5 | 9 / 15 |
| Latency mean | 36.70 ms 95 % CI 36.44–38.30 | 36.44–38.30 | 5 | 12 / 15 |
| Latency std. dev. | 2.20 ms 95 % CI 1.73–3.06 | 1.73–3.06 | 5 | 5 / 15 |
| Latency max | 63.14 ms 95 % CI 50.40–75.86 | 50.40–75.86 | 5 | 9 / 15 |
| CPU | ||||
| CPU used | 1.98 cores 95 % CI 1.98–1.98 | 1.98–1.98 | 5 | — |
| User-space CPU share | 0.690 95 % CI 0.687–0.700 | 0.687–0.700 | 5 | — |
| CPU throttling | 8.5 ms 95 % CI 2.4–11.8 | 2.4–11.8 | 5 | 10 / 15 |
| Memory | ||||
| Anonymous memory peak | 110.3 MiB 95 % CI 106.1–118.9 | 106.1–118.9 | 5 | 9 / 15 |
| Database | ||||
| Database CPU per request | 803.0 µs 95 % CI 800.1–805.1 | 800.1–805.1 | 5 | 5 / 15 |
| SELECTs per request | 20.05 queries 95 % CI 20.05–20.05 | 20.05–20.05 | 5 | — |
| UPDATEs per request | 0.00 queries | — | 5 | — |
| Energy | ||||
| CPU package energy | 928.7 J 95 % CI 915.7–932.6 | 915.7–932.6 | 5 | — |
| Package energy per request | 17.716 mJ 95 % CI 17.653–18.469 | 17.653–18.469 | 5 | 13 / 15 |
| Network | ||||
| Bytes per response | 836 B 95 % CI 836–836 | 836–836 | 5 | — |
| Disk I/O | ||||
| Disk writes | 0 KiB | — | 5 | 1 / 15 |
| Measurement quality | ||||
| Hypervisor steal | 0.0 % | — | 5 | 1 / 15 |
| Host load | 4.88 runnable tasks 95 % CI 4.31–6.23 | 4.31–6.23 | 5 | 3 / 15 |
| CPU clock | 3,600 MHz 95 % CI 3,467–3,627 | 3,467–3,627 | 5 | — |
| Load generator CPU | 0.04 95 % CI 0.04–0.04 | 0.04–0.04 | 5 | 3 / 15 |
Closed loop
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | 5,709 req/s 95 % CI 5,625–5,763 | 5,625–5,763 | 5 | 13 / 15 |
| Latency p99 | 28.88 ms 95 % CI 27.51–29.89 | 27.51–29.89 | 5 | 10 / 15 |
| CPU per request | 345.9 µs 95 % CI 344.1–353.0 | 344.1–353.0 | 5 | 13 / 15 |
| Memory under load (PSS) | 141.4 MiB 95 % CI 136.7–150.5 | 136.7–150.5 | 5 | 10 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | ||||
| Errors | 0 requests | — | 5 | 1 / 15 |
| Error rate | 0.0000 | — | 5 | 1 / 15 |
| Latency | ||||
| Latency p50 | 22.11 ms 95 % CI 21.96–22.46 | 21.96–22.46 | 5 | 13 / 15 |
| Latency p90 | 23.44 ms 95 % CI 23.22–23.84 | 23.22–23.84 | 5 | 8 / 15 |
| Latency p99.9 | 35.71 ms 95 % CI 33.65–39.03 | 33.65–39.03 | 5 | 10 / 15 |
| Latency p75 | 22.72 ms 95 % CI 22.55–23.10 | 22.55–23.10 | 5 | 11 / 15 |
| Latency p99.99 | 40.39 ms 95 % CI 37.73–46.23 | 37.73–46.23 | 5 | 9 / 15 |
| Latency mean | 22.37 ms 95 % CI 22.15–22.69 | 22.15–22.69 | 5 | 13 / 15 |
| Latency std. dev. | 1.56 ms 95 % CI 1.25–1.78 | 1.25–1.78 | 5 | 5 / 15 |
| Latency max | 42.98 ms 95 % CI 38.26–51.73 | 38.26–51.73 | 5 | 9 / 15 |
| CPU | ||||
| CPU used | 1.98 cores 95 % CI 1.97–1.98 | 1.97–1.98 | 5 | — |
| User-space CPU share | 0.717 95 % CI 0.713–0.721 | 0.713–0.721 | 5 | — |
| CPU throttling | 6.9 ms 95 % CI 4.3–8.5 | 4.3–8.5 | 5 | 10 / 15 |
| Memory | ||||
| Anonymous memory peak | 106.0 MiB 95 % CI 101.4–115.3 | 101.4–115.3 | 5 | 9 / 15 |
| Database | ||||
| Database CPU per request | 489.0 µs 95 % CI 486.1–508.9 | 486.1–508.9 | 5 | 6 / 15 |
| SELECTs per request | 5.01 queries 95 % CI 5.01–5.01 | 5.01–5.01 | 5 | — |
| UPDATEs per request | 5.01 queries 95 % CI 5.01–5.01 | 5.01–5.01 | 5 | — |
| Energy | ||||
| CPU package energy | 929.0 J 95 % CI 915.7–937.0 | 915.7–937.0 | 5 | — |
| Package energy per request | 10.819 mJ 95 % CI 10.677–10.855 | 10.677–10.855 | 5 | 13 / 15 |
| Network | ||||
| Bytes per response | 359 B 95 % CI 359–359 | 359–359 | 5 | — |
| Disk I/O | ||||
| Disk writes | 0 KiB | — | 5 | 1 / 15 |
| Measurement quality | ||||
| Hypervisor steal | 0.0 % | — | 5 | 1 / 15 |
| Host load | 4.90 runnable tasks 95 % CI 4.59–6.36 | 4.59–6.36 | 5 | 2 / 15 |
| CPU clock | 3,560 MHz 95 % CI 3,423–3,627 | 3,423–3,627 | 5 | — |
| Load generator CPU | 0.06 95 % CI 0.06–0.07 | 0.06–0.07 | 5 | 3 / 15 |
Closed loop
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | 9,219 req/s 95 % CI 8,907–9,244 | 8,907–9,244 | 5 | 12 / 15 |
| Latency p99 | 17.35 ms 95 % CI 16.75–19.70 | 16.75–19.70 | 5 | 12 / 15 |
| CPU per request | 216.8 µs 95 % CI 216.1–224.7 | 216.1–224.7 | 5 | 12 / 15 |
| Memory under load (PSS) | 152.0 MiB 95 % CI 151.5–167.4 | 151.5–167.4 | 5 | 10 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | ||||
| Errors | 0 requests | — | 5 | 1 / 15 |
| Error rate | 0.0000 | — | 5 | 1 / 15 |
| Latency | ||||
| Latency p50 | 13.75 ms 95 % CI 13.61–14.23 | 13.61–14.23 | 5 | 12 / 15 |
| Latency p90 | 15.01 ms 95 % CI 14.92–15.43 | 14.92–15.43 | 5 | 12 / 15 |
| Latency p99.9 | 20.17 ms 95 % CI 18.72–23.17 | 18.72–23.17 | 5 | 8 / 15 |
| Latency p75 | 14.37 ms 95 % CI 14.18–14.77 | 14.18–14.77 | 5 | 12 / 15 |
| Latency p99.99 | 22.41 ms 95 % CI 21.35–25.06 | 21.35–25.06 | 5 | 8 / 15 |
| Latency mean | 13.85 ms 95 % CI 13.80–14.30 | 13.80–14.30 | 5 | 12 / 15 |
| Latency std. dev. | 1.03 ms 95 % CI 0.89–1.24 | 0.89–1.24 | 5 | 6 / 15 |
| Latency max | 25.39 ms 95 % CI 21.94–29.48 | 21.94–29.48 | 5 | 8 / 15 |
| CPU | ||||
| CPU used | 2.00 cores 95 % CI 2.00–2.00 | 2.00–2.00 | 5 | — |
| User-space CPU share | 0.879 95 % CI 0.875–0.883 | 0.875–0.883 | 5 | — |
| CPU throttling | 19.0 ms 95 % CI 11.4–31.6 | 11.4–31.6 | 5 | 12 / 15 |
| Memory | ||||
| Anonymous memory peak | 116.6 MiB 95 % CI 116.2–132.2 | 116.2–132.2 | 5 | 10 / 15 |
| Database | ||||
| Database CPU per request | 246.7 µs 95 % CI 242.0–252.2 | 242.0–252.2 | 5 | 14 / 15 |
| SELECTs per request | 1.00 queries 95 % CI 1.00–1.00 | 1.00–1.00 | 5 | — |
| UPDATEs per request | 0.00 queries | — | 5 | — |
| Energy | ||||
| CPU package energy | 739.7 J 95 % CI 722.1–756.2 | 722.1–756.2 | 5 | — |
| Package energy per request | 5.369 mJ 95 % CI 5.334–5.458 | 5.334–5.458 | 5 | 13 / 15 |
| Network | ||||
| Bytes per response | 4,830 B 95 % CI 4,830–4,830 | 4,830–4,830 | 5 | — |
| Disk I/O | ||||
| Disk writes | 0 KiB | — | 5 | 1 / 15 |
| Measurement quality | ||||
| Hypervisor steal | 0.0 % | — | 5 | 1 / 15 |
| Host load | 4.84 runnable tasks 95 % CI 4.00–6.24 | 4.00–6.24 | 5 | 2 / 15 |
| CPU clock | 3,520 MHz 95 % CI 3,413–3,600 | 3,413–3,600 | 5 | — |
| Load generator CPU | 0.11 95 % CI 0.10–0.11 | 0.10–0.11 | 5 | 4 / 15 |
Closed loop
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | 15,377 req/s 95 % CI 14,996–15,681 | 14,996–15,681 | 5 | 12 / 15 |
| Latency p99 | 11.45 ms 95 % CI 11.02–13.16 | 11.02–13.16 | 5 | 11 / 15 |
| CPU per request | 130.0 µs 95 % CI 127.5–133.3 | 127.5–133.3 | 5 | 13 / 15 |
| Memory under load (PSS) | 145.4 MiB 95 % CI 142.4–163.1 | 142.4–163.1 | 5 | 11 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | ||||
| Errors | 0 requests | — | 5 | 1 / 15 |
| Error rate | 0.0000 | — | 5 | 1 / 15 |
| Latency | ||||
| Latency p50 | 8.08 ms 95 % CI 7.97–8.28 | 7.97–8.28 | 5 | 13 / 15 |
| Latency p90 | 9.82 ms 95 % CI 9.35–10.20 | 9.35–10.20 | 5 | 12 / 15 |
| Latency p99.9 | 13.96 ms 95 % CI 12.99–18.58 | 12.99–18.58 | 5 | 9 / 15 |
| Latency p75 | 8.46 ms 95 % CI 8.18–8.58 | 8.18–8.58 | 5 | 12 / 15 |
| Latency p99.99 | 17.18 ms 95 % CI 15.83–26.40 | 15.83–26.40 | 5 | 9 / 15 |
| Latency mean | 8.28 ms 95 % CI 8.14–8.52 | 8.14–8.52 | 5 | 12 / 15 |
| Latency std. dev. | 1.05 ms 95 % CI 0.88–1.30 | 0.88–1.30 | 5 | 6 / 15 |
| Latency max | 24.25 ms 95 % CI 21.87–46.49 | 21.87–46.49 | 5 | 8 / 15 |
| CPU | ||||
| CPU used | 2.00 cores 95 % CI 1.99–2.00 | 1.99–2.00 | 5 | — |
| User-space CPU share | 0.906 95 % CI 0.906–0.909 | 0.906–0.909 | 5 | — |
| CPU throttling | 16.6 ms 95 % CI 12.3–34.4 | 12.3–34.4 | 5 | 11 / 15 |
| Memory | ||||
| Anonymous memory peak | 110.2 MiB 95 % CI 107.1–127.9 | 107.1–127.9 | 5 | 10 / 15 |
| Database | ||||
| Database CPU per request | 0.1 µs 95 % CI < 0.1–0.1 | < 0.1–0.1 | 5 | 11 / 15 |
| Energy | ||||
| CPU package energy | 705.9 J 95 % CI 700.0–731.5 | 700.0–731.5 | 5 | — |
| Package energy per request | 3.064 mJ 95 % CI 3.018–3.167 | 3.018–3.167 | 5 | 13 / 15 |
| Network | ||||
| Bytes per response | 268 B | — | 5 | — |
| Disk I/O | ||||
| Disk writes | 0 KiB | — | 5 | 1 / 15 |
| Measurement quality | ||||
| Hypervisor steal | 0.0 % | — | 5 | 1 / 15 |
| Host load | 4.77 runnable tasks 95 % CI 4.39–5.59 | 4.39–5.59 | 5 | 3 / 15 |
| CPU clock | 3,627 MHz 95 % CI 3,580–3,733 | 3,580–3,733 | 5 | — |
| Load generator CPU | 0.17 95 % CI 0.17–0.17 | 0.17–0.17 | 5 | 4 / 15 |
Closed loop
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | 19,034 req/s 95 % CI 18,825–19,823 | 18,825–19,823 | 5 | 12 / 15 |
| Latency p99 | 2.07 ms 95 % CI 1.94–2.63 | 1.94–2.63 | 5 | 9 / 15 |
| CPU per request | 103.2 µs 95 % CI 99.3–104.1 | 99.3–104.1 | 5 | 12 / 15 |
| Memory under load (PSS) | 144.5 MiB 95 % CI 141.5–162.0 | 141.5–162.0 | 5 | 11 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | ||||
| Errors | 0 requests | — | 5 | 1 / 15 |
| Error rate | 0.0000 | — | 5 | 1 / 15 |
| Latency | ||||
| Latency p50 | 0.81 ms 95 % CI 0.78–0.81 | 0.78–0.81 | 5 | 12 / 15 |
| Latency p90 | 1.06 ms 95 % CI 1.01–1.08 | 1.01–1.08 | 5 | 11 / 15 |
| Latency p99.9 | 5.89 ms 95 % CI 5.28–6.51 | 5.28–6.51 | 5 | 10 / 15 |
| Latency p75 | 0.93 ms 95 % CI 0.89–0.94 | 0.89–0.94 | 5 | 12 / 15 |
| Latency p99.99 | 9.36 ms 95 % CI 8.84–9.95 | 8.84–9.95 | 5 | 12 / 15 |
| Latency mean | 0.85 ms 95 % CI 0.82–0.87 | 0.82–0.87 | 5 | 12 / 15 |
| Latency std. dev. | 0.38 ms 95 % CI 0.35–0.43 | 0.35–0.43 | 5 | 10 / 15 |
| Latency max | 12.03 ms 95 % CI 10.50–12.94 | 10.50–12.94 | 5 | 9 / 15 |
| CPU | ||||
| CPU used | 1.96 cores 95 % CI 1.95–1.97 | 1.95–1.97 | 5 | — |
| User-space CPU share | 0.787 95 % CI 0.784–0.793 | 0.784–0.793 | 5 | — |
| CPU throttling | 3.5 ms 95 % CI 1.6–6.5 | 1.6–6.5 | 5 | 8 / 15 |
| Memory | ||||
| Anonymous memory peak | 109.3 MiB 95 % CI 106.1–126.8 | 106.1–126.8 | 5 | 10 / 15 |
| Database | ||||
| Database CPU per request | 111.5 µs 95 % CI 100.1–111.7 | 100.1–111.7 | 5 | 13 / 15 |
| SELECTs per request | 1.00 queries 95 % CI 1.00–1.00 | 1.00–1.00 | 5 | — |
| UPDATEs per request | 0.00 queries | — | 5 | — |
| Energy | ||||
| CPU package energy | 721.4 J 95 % CI 708.1–729.0 | 708.1–729.0 | 5 | — |
| Package energy per request | 2.486 mJ 95 % CI 2.413–2.565 | 2.413–2.565 | 5 | 12 / 15 |
| Network | ||||
| Bytes per response | 229 B 95 % CI 229–229 | 229–229 | 5 | — |
| Disk I/O | ||||
| Disk writes | 0 KiB | — | 5 | 1 / 15 |
| Measurement quality | ||||
| Hypervisor steal | 0.0 % | — | 5 | 1 / 15 |
| Host load | 4.41 runnable tasks 95 % CI 4.01–5.23 | 4.01–5.23 | 5 | 3 / 15 |
| CPU clock | 3,558 MHz 95 % CI 3,479–3,647 | 3,479–3,647 | 5 | — |
| Load generator CPU | 0.21 95 % CI 0.20–0.22 | 0.20–0.22 | 5 | 4 / 15 |
Closed loop
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | 19,792 req/s 95 % CI 19,277–20,114 | 19,277–20,114 | 5 | 12 / 15 |
| Latency p99 | 30.10 ms 95 % CI 28.76–35.88 | 28.76–35.88 | 5 | 10 / 15 |
| CPU per request | 100.8 µs 95 % CI 99.3–103.6 | 99.3–103.6 | 5 | 12 / 15 |
| Memory under load (PSS) | 156.6 MiB 95 % CI 148.5–169.9 | 148.5–169.9 | 5 | 10 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | ||||
| Errors | 0 requests | — | 5 | 1 / 15 |
| Error rate | 0.0000 | — | 5 | 1 / 15 |
| Latency | ||||
| Latency p50 | 25.52 ms 95 % CI 24.64–26.28 | 24.64–26.28 | 5 | 12 / 15 |
| Latency p90 | 27.27 ms 95 % CI 26.81–27.73 | 26.81–27.73 | 5 | 10 / 15 |
| Latency p99.9 | 32.75 ms 95 % CI 32.17–40.60 | 32.17–40.60 | 5 | 7 / 15 |
| Latency p75 | 26.18 ms 95 % CI 25.81–27.01 | 25.81–27.01 | 5 | 10 / 15 |
| Latency p99.99 | 42.11 ms 95 % CI 34.18–48.00 | 34.18–48.00 | 5 | 8 / 15 |
| Latency mean | 25.75 ms 95 % CI 25.32–26.44 | 25.32–26.44 | 5 | 12 / 15 |
| Latency std. dev. | 1.14 ms 95 % CI 1.11–2.49 | 1.11–2.49 | 5 | 5 / 15 |
| Latency max | 49.32 ms 95 % CI 42.62–51.81 | 42.62–51.81 | 5 | 8 / 15 |
| CPU | ||||
| CPU used | 1.99 cores 95 % CI 1.99–2.00 | 1.99–2.00 | 5 | — |
| User-space CPU share | 0.788 95 % CI 0.787–0.799 | 0.787–0.799 | 5 | — |
| CPU throttling | 18.0 ms 95 % CI 8.2–32.0 | 8.2–32.0 | 5 | 12 / 15 |
| Memory | ||||
| Anonymous memory peak | 121.2 MiB 95 % CI 113.3–134.7 | 113.3–134.7 | 5 | 9 / 15 |
| Database | ||||
| Database CPU per request | 109.1 µs 95 % CI 49.4–111.9 | 49.4–111.9 | 5 | 13 / 15 |
| SELECTs per request | 1.00 queries 95 % CI 1.00–1.00 | 1.00–1.00 | 5 | — |
| UPDATEs per request | 0.00 queries | — | 5 | — |
| Energy | ||||
| CPU package energy | 721.6 J 95 % CI 704.8–818.3 | 704.8–818.3 | 5 | — |
| Package energy per request | 2.447 mJ 95 % CI 2.398–2.701 | 2.398–2.701 | 5 | 12 / 15 |
| Network | ||||
| Bytes per response | 229 B 95 % CI 229–229 | 229–229 | 5 | — |
| Disk I/O | ||||
| Disk writes | 0 KiB | — | 5 | 1 / 15 |
| Measurement quality | ||||
| Hypervisor steal | 0.0 % | — | 5 | 1 / 15 |
| Host load | 4.90 runnable tasks 95 % CI 4.34–5.62 | 4.34–5.62 | 5 | 3 / 15 |
| CPU clock | 3,560 MHz 95 % CI 3,427–3,623 | 3,427–3,623 | 5 | — |
| Load generator CPU | 0.22 95 % CI 0.22–0.23 | 0.22–0.23 | 5 | 4 / 15 |
Fixed rate
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | 4,999 req/s 95 % CI 4,999–4,999 | 4,999–4,999 | 5 | 13 / 15 |
| Latency p99 | 4.87 ms 95 % CI 4.77–5.80 | 4.77–5.80 | 5 | 11 / 15 |
| CPU per request | 265.4 µs 95 % CI 257.9–266.6 | 257.9–266.6 | 5 | 13 / 15 |
| Memory under load (PSS) | 151.3 MiB 95 % CI 142.0–165.7 | 142.0–165.7 | 5 | 10 / 15 |
| Metric | Median | 95 % CI | n | Rank |
|---|---|---|---|---|
| Throughput | ||||
| Errors | 0 requests | — | 5 | 1 / 15 |
| Error rate | 0.0000 | — | 5 | 1 / 15 |
| Latency | ||||
| Latency p50 | 1.08 ms 95 % CI 1.03–1.13 | 1.03–1.13 | 5 | 14 / 15 |
| Latency p90 | 2.24 ms 95 % CI 2.22–2.56 | 2.22–2.56 | 5 | 12 / 15 |
| Latency p99.9 | 8.72 ms 95 % CI 8.20–11.26 | 8.20–11.26 | 5 | 10 / 15 |
| Latency p75 | 1.54 ms 95 % CI 1.48–1.67 | 1.48–1.67 | 5 | 13 / 15 |
| Latency p99.99 | 15.91 ms 95 % CI 15.02–16.50 | 15.02–16.50 | 5 | 9 / 15 |
| Latency mean | 1.32 ms 95 % CI 1.32–1.45 | 1.32–1.45 | 5 | 12 / 15 |
| Latency max | 17.84 ms 95 % CI 16.10–19.13 | 16.10–19.13 | 5 | 8 / 15 |
| CPU | ||||
| CPU used | 1.32 cores 95 % CI 1.29–1.33 | 1.29–1.33 | 5 | — |
| User-space CPU share | 0.798 95 % CI 0.737–0.813 | 0.737–0.813 | 5 | — |
| CPU throttling | 0.0 ms | — | 5 | 1 / 15 |
| Memory | ||||
| Anonymous memory peak | 116.0 MiB 95 % CI 106.8–130.5 | 106.8–130.5 | 5 | 10 / 15 |
| Database | ||||
| Database CPU per request | 130.4 µs 95 % CI 121.8–130.9 | 121.8–130.9 | 5 | 3 / 15 |
| SELECTs per request | 1.00 queries 95 % CI 1.00–1.00 | 1.00–1.00 | 5 | — |
| UPDATEs per request | 0.00 queries | — | 5 | — |
| Energy | ||||
| CPU package energy | 514.5 J 95 % CI 502.6–590.8 | 502.6–590.8 | 5 | — |
| Package energy per request | 5.145 mJ 95 % CI 5.027–5.908 | 5.027–5.908 | 5 | 14 / 15 |
| Network | ||||
| Bytes per response | 30 B | — | 5 | — |
| Disk I/O | ||||
| Disk writes | 0 KiB | — | 5 | 1 / 15 |
| Measurement quality | ||||
| Hypervisor steal | 0.0 % | — | 5 | 1 / 15 |
| Host load | 5.07 runnable tasks 95 % CI 4.45–5.33 | 4.45–5.33 | 5 | 4 / 15 |
| CPU clock | 2,419 MHz 95 % CI 2,348–2,552 | 2,348–2,552 | 5 | — |
| Load generator CPU | 0.13 95 % CI 0.12–0.13 | 0.12–0.13 | 5 | 1 / 15 |