Latest Announcemnent
New thread

Need Advice: Serving 10k+ Concurrent HLS Viewers on a HDD VDS (No CDN)

W

warez1233

New Member
4
2025
3
70
Hello everyone,
I’m running a streaming website for my TV series and movie archive. All videos are stored on a 64TB HDD-based VDS server configured in RAID 0.

Current setup
  • VDS with ~64 TB HDD storage in RAID0
  • Linux + Nginx serving HLS
  • Workload is read-heavy (segments requested by many users at once)
  • Network/CPU are fine; issues begin when the disk queue grows and I/O wait spikes

Constraints
  • I want to avoid paid CDN for now (too expensive at my scale).
  • I’m open to adding a few SSD/NVMe boxes if that’s the most cost-effective path.

The problem:
When I reach around 10,000+ concurrent viewers, the HDD RAID 0’s IOPS usage becomes extremely high, causing playback stuttering and buffering issues for users. I want to ensure smooth, uninterrupted playback even during peak traffic.


I’m looking for advice on the best server architecture and optimization setup for HLS streaming that can handle this kind of load without using a CDN (too expensive for my use case).


Here are some solutions I’ve considered but haven’t implemented yet:


  1. Using LVM Cache to put an SSD cache layer in front of the HDD array
  2. Using Bcache for SSD caching
  3. Enabling NGINX Proxy Cache
  4. Adding 3+ additional SSD-based servers to serve the most-watched videos
  5. CDN – Not an option for me due to high cost

My goal is to design a future-proof setup that can handle heavy traffic and deliver videos quickly without buffering.


If you were in my situation, how would you build and optimize such a system?
Any real-world advice, configuration tips, or proven architectures would be highly appreciated.
 
9 comments
I would recommend you to get 3 servers with SSD hard drives, and shift your most watchable content to them, as no matter what you do, HDD has its limits, and will come in effect, a solid and reliable solution is only to buy new servers with SSD hard drives, make sure those servers have a good minimum of 1gbps uplink and fast network connections for good streaming.
 
Serving 10,000+ concurrent HLS viewers from a HDD-based VDS without a CDN is extremely difficult and in most cases not feasible unless the setup is engineered very carefully — and even then, HDD is usually the bottleneck.
 
Both bcache/bcachefs and Nginx proxy (look into chunked/range caching) work great. Nginx you can even make horizontally scalable (think of raid-0, instead of a cache mirror) - but you need haproxy with predictable loadbalancing on uri.

Either way though, it only really works great if you have _really_ a few highly streamed movies. So whatever is being streamed at least 2+ times should fit in your hot storage.

Certain sites have such a long-tail on traffic, that it becomes less feasible due to high randomness. Then you just need better drives.

Also 64TB of spinning disks on Raid-0? Exciting..
 
Best solution is to use nginx cache , to store hot video on SSDs to offload the HDD's as its supports cache expires , enable also the memory store it should work perfect if you have lot of memory and your videos are in HLS
 
  • VDS with ~64 TB HDD storage in RAID0

The Office Lol GIF
 
Back
Top