rtcstats-server: Requirements & Operations Guide
The production hub for rtcstats-server: what to check before go-live, how to pick a deployment shape, security and privacy, monitoring, upgrades, symptom routing, and what to send when you ask for help.
Last updated Applies tortcstats-server
Open Source12 articles
On this page21 sections
How to use this guide
If you are here, then what you are looking for is to set up the open source rtcStats WebRTC observability solution: rtcstats-server. Here's how to approach it:
- Start at the before-production checklist
- Keep the rest as the operating reference for the running service
The two quickstart guides (AWS App Runner, DigitalOcean Apps Platform) get you to a server that boots. They say plainly that what they produce is not production ready. This page will get you to production ready.
If you are reviewing this deployment rather than performing it, start instead with IT specifications and requirements. It answers an infrastructure or security review directly: ports and firewall rules, external dependencies, high availability, serviceability, data protection, and the known limitations.
Scope:
- Covered:
rtcstats-serveronly. - Requires: a PostgreSQL-compatible database for session metadata, and S3-compatible object storage for dump files.
- Out of scope: provisioning, sizing, backup, and high availability of those two services, plus
rtcstats-features(offline dump processing).
Before-production checklist
Follow this checklist. Be sure you've got the first four nailed down or else they'll bite you later on.
| # | Check | Why it matters |
|---|---|---|
| 1 | NODE_ENV is set to production |
This isn't cosmetic. See below. |
| 2 | storage.s3 is configured and a real dump file has landed in the bucket |
Without storage, dump files stay on the instance disk and are deleted on the next restart. |
| 3 | database.postgres.connectionString is configured |
Without it, nothing points at the stored dumps. |
| 4 | authorization.jwtSecret is set and clients send tokens |
Unset means no authorization is performed at all. |
| 5 | TLS terminates at the load balancer, and the leg behind it is a private network | The server does not handle TLS itself. |
| 6 | The load balancer health check is HTTP on /healthcheck, not TCP |
TCP only proves the port is open. |
| 7 | Session affinity is on at the load balancer | One session holds one WebSocket to one instance for its whole life. |
| 8 | Disk is sized for in-flight sessions, and deleteAfterUpload is left at its default |
Both default to cleaning up. Turning either off is a development convenience that fills a production disk. |
| 9 | You have decided what, if anything, forwards to rtcstats.com | Forwarding is disabled by default. |
Why NODE_ENV=production is item 1
Both quickstart guides deliberately set NODE_ENV to not-yet-production, so promoting a quickstart deploy means changing it. Two behaviors are gated on the literal string production:
- Multi-process operation. In production the primary process forks one worker per available CPU. Outside production it runs a single process, so a 4 vCPU instance uses one core.
server.numberOfProcessescaps the worker count when you want fewer than the machine has;0(the default) means use them all. - Top-level error handling. The
uncaughtExceptionandunhandledRejectionhandlers are only installed in production. Async event handlers return promises the emitter never awaits, so without those handlers a single rejection can terminate the process, and every session it was collecting with it.
A production deploy that still says not-yet-production is a single-core server one unhandled rejection away from dropping its connections.
Choose your deployment and scaling shape
Two decisions, in this order.
Managed platform or your own machines
| Managed platform | Your own VMs or containers | |
|---|---|---|
| Examples | AWS App Runner, DigitalOcean Apps Platform | EC2, Droplets, ECS, Kubernetes, anything that runs the Dockerfile |
| Validated guides exist | Yes, see AWS App Runner and DigitalOcean | No platform-specific guide |
| TLS and load balancing | Handled by the platform | You provision it |
| Watch out for | Platform request idle timeouts. App Runner enforces one, and long WebRTC sessions can hit it and reconnect mid-call. Validate against your real session lengths. | Nothing platform-specific, but the health check, affinity, and private-network work in the checklist above is yours to do |
Managed platforms are the shorter path to a first production deployment. Neither one removes any checklist item.
One instance or a cluster
Start at two instances behind a load balancer if you want high availability at all. A single instance is a single point of failure and there is no way to make it otherwise: when an instance goes down, the sessions it was collecting are lost.
Scale out, not up. The preference is more, smaller machines, limiting blast radius. Plan to add instances when an instance approaches 1,000+ concurrent connections, or when CPU or memory utilization sits consistently above 70 to 80 percent.
Treat ~1,000 concurrent sessions per production node as a planning ceiling and confirm the real number with a load test on your own hardware and traffic. Session weight varies widely and there is no universal sessions-per-vCPU constant.
The full requirements are in the horizontal scaling guide. The database and the object storage must scale along the same dimension: every instance in the cluster writes to the same two services.
Machine requirements
Operating system. Linux (x86_64 or arm64), any distribution able to run a current LTS Node.js runtime. Container deployment is supported.
Node.js. Node.js 22 is the baseline. The project Dockerfile builds on node:22-alpine, and both validated managed deployments use the platform's Node.js 22 runtime. Keep your instances current with the Node.js release lifecycle.
Baseline sizing.
| Tier | vCPU | Memory | Local disk | Intended use |
|---|---|---|---|---|
| Pilot / staging | 1 | 2 GB | 20 GB | Validation, low traffic. |
| Production node | 2 | 4 GB | 50 GB | Standard production instance in a cluster of 2 or more. |
The 1 vCPU / 2 GB tier is the documented reference configuration used in both quickstart guides. The 2 vCPU / 4 GB tier is a recommended production starting point. Validate both against your own traffic.
Sizing local disk. The server uses two directories on the instance disk: server.workDirectory for sessions still in flight, and server.uploadDirectory for finished dumps waiting to upload. Size for the sum of both: concurrent sessions x average dump size x safety factor. Example with assumptions to adjust: 1,000 concurrent sessions x 5 MB average in-flight dump x 4 = 20 GB. Compressed dumps are often much smaller, hundreds of KB for short calls. Long sessions with many peer connections are larger.
Network. One inbound port (default 8080) receiving plain WebSocket traffic from the load balancer. Instances sit in a private subnet, only the load balancer is internet-facing. Outbound access is required to the object storage endpoint and the database, plus rtcstats.com if you enable forwarding.
Capacity drivers. Per instance, capacity is bounded by CPU (message parsing, delta decompression of getStats payloads, dump compression), disk (temporary files for every active session), and concurrent WebSocket connections (file descriptors and memory per socket).
Architecture
(browser / app)"] LB["Load balancer
(TLS terminates here, session affinity on)"] Server["rtcstats-server
1..N instances, N processes each"] Disk["Local disk
work/ during the session
upload/ after it"] Storage[("S3-compatible storage
dump files, on session end")] DB[("PostgreSQL
session metadata + storage pointer")] Cloud["rtcstats.com
(optional, off by default)"] JS -->|"WSS (TLS)"| LB LB -->|"WS (plain, private network)"| Server Server --> Disk Disk --> Storage Server --> DB Server -.-> Cloud
Each rtcstats-js client holds one persistent WebSocket connection to one instance for the duration of its session. Events are written to a temporary file on local disk. When the connection ends, the file is uploaded to storage and the database is updated with a pointer to the storage URL. Instances are stateless between sessions: all durable state lives in the database and the object storage, so a new instance can start at any time.
Security and privacy checklist
Transport
rtcstats-jsconnects over WSS to the load balancer.- The load balancer terminates TLS.
rtcstats-serverdoes not handle TLS or certificates itself. - The leg from load balancer to instances is plain WebSocket on port 8080 and must stay inside a private network: VPC or private subnet, with firewall rules restricting ingress to the load balancer.
- Certificate issuance and rotation therefore live entirely at the load balancer layer.
Authorization
- Set
authorization.jwtSecret. If it is not set, no authorization is performed and anyone who finds your URL can post data to it. - Each client presents a JWT carrying the claims
user,session, andconference, passed as thertcstats-tokenquery parameter during the WebSocket connect phase. - The token is signed with an HMAC shared secret and validated by the server. An invalid token closes the WebSocket with a
policy-violation(code 1008). - Use at least 256 bits of entropy for the secret, have a rotation process that does not require downtime, and set expiration long enough for a session but not indefinite. Six to 24 hours is a good range.
- Putting
session,conference, anduserin the claims is also what stops a client impersonating another session, and it is how those identifiers reach the database at all. - Full setup: How to authenticate clients with rtcstats-server. For ad-hoc testing,
bin/generate-token.jsin the repo signs a token from the configured secret.
The HTTP upload path
server.httpUploadPath enables a POST upload as an alternative to WebSockets. It is disabled by default and only active when you set the path. It goes through the same JWT authorization as the WebSocket path. If you are not using it, leave it unset rather than set to something obscure.
Privacy
server.obfuscateIpAddressesdefaults to true. All IP addresses the server encounters are masked before dumps are written to storage, so stored files carry no IP-level PII. Files forwarded to rtcstats.com are anonymized for the same reason.- Do not put PII in the user ID. No names, no email addresses. Hash it, or better, use a random UUID you can associate back to the user yourself. The same applies to conference and session IDs, at lower risk.
- Geolocation, if you enable GeoIP enrichment, runs before anonymization and keeps coarse granularity by design.
- Because
rtcstats-serverruns in your infrastructure, you are the data controller and the self-hosted mediation layer is what satisfies data residency requirements. - Verify rather than assume: inspect a stored dump for IP addresses and URLs, then upload a test session and confirm no PII appears in rtcstats.com.
- Details: How to configure rtcstats-server for privacy and Data privacy and compliance.
Secrets handling
The JWT secret, the S3 credentials, the database connection string, and the rtcstats.com token all arrive through NODE_CONFIG or a production.yaml. Inject them as encrypted environment variables rather than committing them. On AWS, an instance role removes the S3 credentials from the config entirely, which is one fewer secret to rotate.
Monitoring and failure handling
What the server exposes
GET /healthcheck returns 200 OK, with no authentication required. Every other GET path returns 404.
That is the entire monitoring surface. There are no metrics, no stats endpoint, and no structured telemetry. Monitor the rest from the outside:
| Signal | Where it comes from |
|---|---|
| Instance health | /healthcheck via the load balancer, HTTP not TCP |
| CPU, memory, free disk | Your cloud provider's standard instance monitoring |
| Concurrent connections | Load balancer connection counts |
| Upload failures, S3 and database errors | The runtime logs. The server logs to stdout and stderr |
| Dumps actually landing | Object counts in the bucket, row counts in the database |
Run --host-identifier <string> on each instance. The value is stored with every dump, which is the only way to attribute a dump back to the instance that received it once you are running a cluster.
Failure handling
- An instance goes down. The sessions it was collecting are lost. There is no recovery path for the in-flight data. A cluster limits the blast radius to one instance's worth of sessions.
- A WebSocket is severed mid-call. The rest of that session is not collected. The client does not resume into a different instance.
- Planned maintenance. Remove the instance from the load balancer targets, wait for its existing WebSocket connections to drain, then stop it. The server installs no
SIGTERMhandler, so it does not drain on its own. A hard stop is the same event as an instance going down. - Storage or database unreachable. Uploads and inserts fail and are logged. Check the runtime logs before assuming the collection side is at fault.
Upgrades
rtcstats-server lives in the open source monorepo. There are no numbered server releases: the current source on main is the current version, and bugs are fixed there.
- Cadence. Update at least every 6 months, and never less than once a year.
- Turn off autodeploy. Both quickstart guides set the deployment trigger to manual on purpose. You do not want a push to
mainto silently redeploy production. - Rolling upgrade. With two or more instances, upgrade the way you do maintenance: drain one instance out of the load balancer, replace it, return it, then move to the next. Sessions in flight on the drained instance are still lost, so drain rather than cut.
- Dump file format. The current format is version 3. Versions 1 and 2 came from the legacy server and are not supported. Breaking format changes require an upgrade plan and are not made silently.
- Node.js. Track the Node.js release lifecycle alongside the server.
- GeoIP data. If you use it, download a fresh GeoLite2 database on every deployment. IP blocks get reassigned.
Symptom routing
Start here when something is wrong with the service. For symptoms in the calls themselves rather than the collector, go to WebRTC troubleshooting.
| Symptom | Most likely cause | Where to go |
|---|---|---|
| Health check is green, but the bucket is empty | Storage not configured at all, or wrong region / bucket / endpoint, or the instance role does not grant s3:PutObject on that bucket ARN |
AWS or DigitalOcean Stage 2 |
| Dumps existed, then vanished after a restart | Storage not configured. Files are kept locally and removed on the next restart | Checklist item 2 |
| Clients cannot connect at all | JWT. jwt malformed, rtcstats-token is missing, or TokenExpiredError, all visible in the server logs. The close code is 1008 |
Authenticate clients |
| Anyone can post data to your server | authorization.jwtSecret is unset, so no authorization runs |
Checklist item 4 |
| Sessions are truncated or cut mid-call | Load balancer is not sticky, an instance restarted, or a platform idle timeout fired | Horizontal scaling, and the App Runner idle timeout note above |
| Disk fills up | deleteAtStart or deleteAfterUpload turned off, or upload failures leaving files in uploadDirectory |
Sizing above, then the runtime logs |
| One core busy, others idle | NODE_ENV is not production, so the server is running a single process |
Checklist item 1 |
| The process dies under load | Same cause. The error handlers are only installed in production | Checklist item 1 |
| No country or city on sessions | maxmind.path unset, or the .mmdb file is not where the config says |
GeoIP enrichment |
| Sessions have no user, conference, or session ID | Those fields come from the JWT claims. No token means no identifiers | Session, conference and user identifiers |
| Nothing appears in rtcstats.com | Forwarding is off by default and needs a token, or randomPercentage is sampling below 1.0 |
Integration guide |
| Database rows appear but dumps do not, or the reverse | Two different services, two different failures. Check the runtime logs for which one errored | Monitoring above |
Requesting support
Send these with the first message. Every one of them is something we would otherwise have to ask for, and each round trip costs a day.
About the deployment
- Commit SHA or approximate date of the
rtcstats/rtcstatscheckout you are running - Node.js version and how you deploy: managed platform and which one, container, or VM
- Instance count and size, and whether the problem shows on one instance or all of them
About the configuration
- Your
NODE_CONFIGor config file, with the JWT secret, S3 credentials, database connection string, and rtcstats.com token removed. Send the shape, not the secrets - The value of
NODE_ENV - Load balancer type, and whether session affinity and the HTTP health check are on
About the symptom
- What you expected, what happened, and when it started
- Relevant lines from the runtime logs, including the surrounding lines and not only the error
- Whether it is intermittent or constant, and roughly what fraction of sessions it affects
- Concurrent session counts and CPU, memory, and disk at the time
Evidence
- A dump file that shows the problem, anonymized first
- The session, conference, and user identifiers for an affected session
- The
--host-identifierof the instance, if you run a cluster
Where to send it depends on your arrangement. Open source users without a support contract use the GitHub repository. If you want a ticket system, an SLA, or a shared Slack channel, see premium support for rtcStats open source. Anything else, contact us.
Sources
- rtcstats repository and the rtcstats-server README
- rtcstats-server: IT Specifications and Requirements
- Horizontally scale rtcstats-server in a cluster
- Authenticate clients with rtcstats-server
- Configure rtcstats-server for privacy
- Data privacy and compliance
- Deploy rtcstats-server on AWS App Runner
- Deploy rtcstats-server on DigitalOcean Apps Platform
- Enrich rtcstats-server with GeoIP data
- Getting premium support for rtcStats open source
Was this page helpful?