I've been using varnish mostly on Debian systems. I found the reload-vcl
script included in Debian to be useful.
The reload-vcl
script
It's part of the standard varnish debian package. It uses the system defaults in /etc/defaults/varnish
, so it knows how to correctly invoke the varnishadm
utility to perform administrative commands. As the name implies, it reloads the default VCL file using the vcl.load
and vcl.use
commands, checking that every step succeeds properly before continuing so it's safe to use. It loads the new VCL file and labels it automatically with a unique id.
Something analogous but regarding the purge functionality could have been useful, so I looked at the source code for reload-vcl
. Most of it deals with loading of /etc/defaults/varnish
and various sanity checks. I reused that bit to make another script, to control cache purging.
The purge-cache script
Here's the full source code. Below there's a link to download the latest version from github.
#!/bin/sh
# purge-cache: Script to purge varnish cache. Defaults are defined in
# /etc/default/varnish.
#
# Cosimo <cosimo@cpan.org>
# Based on reload-vcl, by Stig Sandbeck Mathisen <ssm at debian dot org>
# Settings
defaults=/etc/default/varnish
# Paths
varnishadm=/usr/bin/varnishadm
date=/bin/date
tempfile=/bin/tempfile
# Messages
# msg_no_varnishadm: varnishadm
msg_no_varnishadm="Error: Cannot execute %sn"
msg_no_management="Error: $DAEMON_OPTS must contain '-T hostname:port'n"
# msg_defaults_not_readable: defaults
msg_defaults_not_readable="Error: %s is not readablen"
# msg_defaults_not_there: defaults
msg_defaults_not_there="Error: %s does not existn"
msg_usage="Usage: $0 [-h][-q][-u <url>|-r <regex>|-a]nt-htdisplay helpnt-qtbe quietnt-utpurge by exact (relative) url (ex.: /en/products/)nt-rtpurge objects with URL matching a regex (ex.: ^/blogs/)nt-atpurge all objects from cachen"
msg_purge_failed="Error: purge command failedn"
# msg_purge_url: url
msg_purge_url="Purging objects by exact url: %sn"
# msg_purge_regex: regex
msg_purge_regex="Purging objects with URL matching regex: %sn"
msg_purge_all="Purging all cachen"
msg_purge_ok="Purge command successfuln"
# Load defaults file
if [ -f "$defaults" ]
then
if [ -r "$defaults" ]
then
. "$defaults"
else
printf >&2 "$msg_defaults_not_readable" $defaults
exit 1
fi
else
printf >&2 "$msg_defaults_not_there" $defaults
exit 1
fi
# parse command line arguments
while getopts hqu:r:a flag
do
case $flag in
h)
printf >&2 "$msg_usage"
exit 0
;;
u)
purge_method=url
url="$OPTARG"
;;
r)
purge_method=regex
regex="$OPTARG"
;;
a)
purge_method=all
;;
q)
quiet=1
;;
*)
printf >&2 "$msg_usagen"
exit 1
;;
esac
done
# Parse $DAEMON_OPTS (options must be kept in sync with varnishd).
# Extract the -f and the -T option, and (try to) ensure that the
# management interface is on the form hostname:address
OPTIND=1
while getopts a:b:dFf:g:h:l:n:P:p:s:T:t:u:Vw: flag $DAEMON_OPTS
do
case $flag in
f)
if [ -f "$OPTARG" ]; then
vcl_file="$OPTARG"
fi
;;
T)
if [ -n "$OPTARG" -a "$OPTARG" != "${OPTARG%%:*}" ]
then
mgmt_interface="$OPTARG"
fi
;;
esac
done
# Sanity checks
if [ ! -x "$varnishadm" ]
then
printf >&2 "$msg_no_varnishadm" $varnishadm
exit 1
fi
if [ -z "$mgmt_interface" ]
then
printf >&2 "$msg_no_management"
exit 1
fi
logfile=$($tempfile)
purge_command="vcl.list"
# Now run the purge command against the admin interface
if [[ $purge_method = "url" ]]
then
purge_command="purge req.url == $url"
printf >&2 "$msg_purge_url" $url | grep -v "^$" > $logfile
else
if [[ $purge_method = "regex" ]]
then
purge_command="purge.url $regex"
printf >&2 "$msg_purge_regex" $regex | grep -v "^$" > $logfile
else
if [[ $purge_method = "all" ]]
then
purge_command="purge.url ."
printf >&2 "$msg_purge_all" | grep -v "^$" > $logfile
fi
fi
fi
# For some reason, using:
#
# fi | grep -v "^$" > $logfile
#
# results in purge_command assignment being wiped out
# at the end of the block??
if [ -z "$purge_command" ]
then
printf >&2 "$msg_usagen"
exit 1
fi
# echo "cmd: $varnishadm -T $mgmt_interface $purge_command"
if $varnishadm -T $mgmt_interface $purge_command
then
printf >&2 "$msg_purge_ok"
else
printf >&2 "$msg_purge_failed"
exitstatus=1
fi | grep -v "^$" > $logfile
# Blather
if [ -z "${quiet}" -o -n "$exitstatus" ]
then
cat >&2 $logfile
fi
# Cleanup
rm -f $logfile
exit $exitstatus
You can control how objects are purged from the cache with 3 options:
-a
: purges all objects-u <url>
: purges an exact url-r <regexp>
: purges objects matching a regular expression
Examples
# Purges all objects
purge-cache -a
# Purges all objects starting with "/products"
purge-cache -r '^/products'
# Purges objects with exact URL
purge-cache -u '/en/homepage'
Goal: no downtime
Both reload-vcl
and purge-cache
can be combined together in a single script to be triggered when deploying new VCL code or new backend applications. Instead of restarting varnish, which I really don't like, and it's not very reliable either (on Debian sometimes it won't come back up), I use purge-cache -a
to purge all objects and then reload-vcl
to load and use the newly deployed VCL code.
This procedure has no downtime at all. The effect of purging all objects can potentially be hard on the backends, but we're not at that point yet. Usually in the busiest applications we have, it takes around 10-20 seconds to reach 70%-75% of hit rate, so I would say that's not really a problem right now.
Download!
You can download the purge-cache script from github. I contacted the maintainer of the reload-vcl
script. Maybe he will include purge-cache in the next release of the varnish debian package… or maybe I could package it as a Perl CPAN module.