RTSP Streaming over VPN with Auto-Scaling Restreaming, HLS Delivery & Recording

We built a VPN-tunneled RTSP streaming platform with a VPN hub for secure camera connectivity, auto-scaling RTSP restreaming servers for multi-protocol delivery, HLS packaging for browser playback, and continuous recording with retention management.

Architecture

• VPN Hub: Centralized VPN server establishing secure tunnels to remote camera sites
• RTSP Ingest: VPN-connected cameras publish RTSP streams through encrypted tunnels
• Restreaming Cluster: Auto-scaling MediaMTX servers for RTSP relay, HLS conversion, and stream distribution
• HLS Packaging: Real-time RTSP-to-HLS conversion for web browser delivery
• Recording Service: Continuous stream recording with segment-based storage and retention policies
• Load Balancer: Distributes viewer and AI worker connections across restreaming servers
• Orchestrator: Monitors demand and scales the restreaming cluster up or down
• Storage: Object storage for recorded segments with lifecycle management

VPN Tunnel Architecture

Secure Camera Connectivity

• VPN hub deployed in the cloud establishes encrypted tunnels to each remote site
• Cameras at remote locations stream RTSP within the VPN tunnel — no public internet exposure
• Each site gets a VPN client that connects to the hub, creating a private network overlay
• Camera RTSP streams are accessible by cloud infrastructure via private VPN IP addresses
• Multiple cameras per site share a single VPN tunnel

NAT/Firewall Traversal

• VPN clients initiate outbound connections from the camera site (no inbound firewall rules needed)
• Hub accepts incoming connections, creating bidirectional tunnels
• Cloud services access camera RTSP feeds via the VPN network as if they were local

Reliability

• Automatic tunnel reconnection on network interruptions
• Keepalive probes detect and recover from silent failures
• Multiple tunnel protocols supported for compatibility with restrictive networks
• Health monitoring per tunnel with alerting on prolonged disconnections

Auto-Scaling Restreaming

RTSP Relay

Restreaming servers pull camera RTSP feeds from the VPN network and make them available for downstream consumers:

• AI workers connect via RTSP for low-latency, frame-accurate processing
• Multiple consumers can access the same camera stream without additional load on the camera
• Stream multiplexing reduces bandwidth from the camera site (one pull, many consumers)

HLS Conversion

For web-based viewing, restreaming servers convert RTSP to HLS in real-time:

• Segment-based packaging for HTTP delivery via standard web infrastructure
• Adaptive bitrate support for varying viewer network conditions
• CDN-compatible output for edge-cached delivery
• Low-latency HLS configuration for near-real-time viewing

Auto-Scaling

The restreaming cluster scales based on demand:

• Scale Up — When viewer/AI connections per server exceed threshold or CPU utilization is high
• Scale Down — When utilization drops below threshold for a sustained period
• New servers pull camera feeds from the VPN network and register with the load balancer
• Graceful shutdown drains connections before removing servers (viewers reconnect automatically via stable URLs)

Recording Pipeline

Continuous Recording

• Restreaming servers or dedicated recording workers capture RTSP streams as segmented files
• Segments written at configurable intervals for manageable file sizes
• Each segment tagged with camera ID, timestamp, and duration metadata

Storage Management

• Segments uploaded to object storage for durable archival
• Retention policies automatically delete segments older than the configured retention period
• Storage lifecycle rules move older segments to cheaper storage tiers before deletion
• Per-camera and per-site storage quotas prevent runaway costs

Playback

• Recorded segments accessible via time-based queries (camera + time range)
• HLS manifest generated on-the-fly for browser-based playback of recorded footage
• Seek and scrub through recorded timeline with segment-level granularity

Multi-Protocol Delivery

| Consumer | Protocol | Latency | Use Case |

|----------|----------|---------|----------|

| AI Workers | RTSP | Low (~500ms) | Real-time object detection, analytics |

| Web Viewers | HLS | Medium (2-6s) | Browser-based live monitoring |

| Mobile Apps | HLS | Medium (2-6s) | Remote monitoring on mobile devices |

| Archival | Recorded Segments | N/A | Incident review, compliance, forensics |

Monitoring & Reliability

Stream Health

• Per-camera stream status monitoring (connected, buffering, disconnected)
• VPN tunnel health per site (latency, packet loss, uptime)
• Restreaming server metrics (CPU, bandwidth, connection count)
• Recording pipeline health (segment write rate, storage usage, upload status)

Alerting

• Camera disconnection alerts with site identification
• VPN tunnel failure notifications
• Restreaming cluster capacity warnings
• Storage quota and retention alerts
• Recording gaps detected via segment continuity checks

Key Features

1. VPN-Secured Ingest — Encrypted tunnels protect RTSP streams from remote cameras
2. NAT/Firewall Traversal — Outbound VPN connections bypass restrictive network configurations
3. Auto-Scaling Restreaming — Cluster scales with viewer and AI demand
4. HLS Conversion — Real-time RTSP-to-HLS for browser-based viewing
5. RTSP Relay — Low-latency stream access for AI processing workers
6. Continuous Recording — Segment-based recording with retention management
7. Multi-Protocol — RTSP, HLS, and recorded segments from a single ingest
8. Automatic Reconnection — VPN tunnels and stream connections recover from interruptions
9. Storage Lifecycle — Tiered storage with automatic retention enforcement
10. Stream Multiplexing — One camera pull serves multiple consumers without additional camera load