During last couple of months, we have been playing with varnish for
files.myopera.com, the My Opera main files server.
I'm not sure this is a typical use case for varnish, maybe it is, but it has a few unique challenges which I'll try to explain here:
- really high number of connections (in the 10k range)
- large file set, ~100 millions
- the longer the TTL, the better (10 days it's the default)
- really simple or no VCL logic
In other Varnish installations we're maintaining here at Opera, the real challenge is to seamlessly interface with backend application servers, but in this case the "backend" is just another http file server with little more logic.
Searching around, and using the resources I mentioned some blog posts ago, we have found a few critical settings that need to be tuned to achieve consistent performance levels. Some are obvious, some others are not so obvious:
- threads related parameters
- hash bucket size
- Session related parameters
- Grace config and health checking
I'll explain all the settings we had to change and how they affected us in a later post.