{"id":459,"date":"2011-01-06T16:43:12","date_gmt":"2011-01-06T15:43:12","guid":{"rendered":"http:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/"},"modified":"2011-01-06T16:43:12","modified_gmt":"2011-01-06T15:43:12","slug":"upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content","status":"publish","type":"post","link":"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/","title":{"rendered":"Upgrade of puppet from 0.24.5 to 2.6.2: “Got nil value for content”"},"content":{"rendered":"

Something I should have probably done a while ago. Upgrading our puppet clients and master from 0.24.5 (current Debian stable) to 2.6.2 (current backports). I've been advised to go all the way to 2.6.4, that contains an important security fix. I'll probably do that soon.<\/p>\n

So, first error I did was to upgrade one client before the puppetmaster. In my mental model, it's always ok to upgrade the server first (puppetmaster), but I remember I was kind of surprised when I learned that puppet needs upgrading the clients first, so that is something I seemed to remember well: first upgrade clients, then the puppetmaster.<\/p>\n

So I did, and that didn't work at all. The client couldn't retrieve the catalog. I discovered I had to update the puppetmaster first<\/strong>. Another very common thing with puppet: if something doesn't work, wipe out the SSL certificates. Almost every puppet user, maybe unexperienced like me, will tell you to remove the entire \/var\/lib\/puppet\/ssl<\/code> directory, and restart your client. Good. So I did.<\/p>\n

After fiddling for a while with client and server-side SSL certificates, removing, --waitforcert<\/code> and puppetca<\/code> invocations, I was able to finally connect the client with the puppetmaster and have them talk to each other correctly, downloading the manifests and applying the changes. However, one problem remained…<\/p>\n

Digression: custom facter plugins<\/h3>\n

I wrote a specific facter plugin<\/a> that provides a datacenter<\/strong> fact. Just as an example, it works like this:<\/p>\n

<\/p>\n

#\r\n# Provide an additional 'datacenter' fact\r\n# to use in generic modules to provide datacenter\r\n# specific settings, such as resolv.conf\r\n#\r\n# Cosimo, 03\/Aug\/2010\r\n#\r\n# $Id: datacenter.rb 15240 2011-01-03 14:27:44Z cosimo $\r\n\r\nFacter.add("datacenter") do\r\n    setcode do\r\n\r\n        datacenter = "unknown"\r\n\r\n        # Get current ip address from Facter's own database\r\n        ipaddr = Facter.value(:ipaddress)\r\n\r\n        if ipaddr.match("^12.34.56.")\r\n            datacenter = "dc1"\r\n        elsif ipaddr.match("^34.56.78.")\r\n            datacenter = "dc2"\r\n        elsif ipaddr.match("^56.78.12.")\r\n            datacenter = "dc3"\r\n        # etc ...\r\n        # etc ...\r\n        end\r\n\r\n        datacenter\r\n    end\r\nend\r\n<\/pre>\n
<\/code><\/p>\n
This allows for very cool things, like specific DNS resolver settings by datacenter:<\/p>\n
# Data-center based settings\r\ncase $datacenter {\r\n    "dc1" :  { include opera::datacenters::dc1 }\r\n    "dc2" :  { include opera::datacenters::dc2 }\r\n    "dc3" :  { include opera::datacenters::dc3 }\r\n    # ...\r\n    default: { include opera::datacenters::dc1 }\r\n}\r\n<\/code><\/pre>\n
where each opera::datacenter::dc<\/code> class contains all the datacenter-dependent settings. In my case, just the resolver for now.<\/p>\n
class opera::datacenters::dc1 {\r\n    resolver::dns { "dc1-ns":\r\n        domain => "opera.com",\r\n        nameservers => [ "1.2.3.4", "5.6.7.8" ],\r\n    }\r\n}\r\n<\/code><\/pre>\n
resolver::dns<\/code> is in turn a small define in a resolver<\/code> module I wrote to generate resolv.conf<\/code> contents from a small template. It's so small I can copy\/paste it here:<\/p>\n
# "resolver\/manifests\/init.pp"\r\ndefine resolver::dns ($nameservers, $search="", $domain="") {\r\n    file { "\/etc\/resolv.conf":\r\n        ensure => "present",\r\n        owner  => "root",\r\n        group  => "root",\r\n        mode   => 644,\r\n        content => template("resolver\/resolv.conf.erb"),\r\n    }\r\n}\r\n\r\n# "resolver\/templates\/resolv.conf.erb"\r\n# File managed by puppet <%= puppetversion %> on <%= fqdn %>\r\n# Data center: <%= name %>\r\n<% if domain != "" %>domain <%= domain %>\r\n<% end %>\r\n<% if search != "" %>search <%= search %>\r\n<% end %>\r\n<% nameservers.each do |ns| %>nameserver <%= ns %>\r\n<% end %>\r\n<\/code><\/pre>\n
One problem remained…<\/h3>\n
Let's get back to the remaining problem. Every client run was spitting this error string:<\/p>\n
info: Retrieving plugin\r\nerr: \/File[\/var\/lib\/puppet\/lib]: Could not evaluate: Got nil value for content\r\n<\/code><\/pre>\n
After a lot of searching and reading through similar (but not quite the same) messages, I found this thread<\/a> where one guy was having a very similar problem, but with a different filename, \/root\/.gitconfig<\/code> that he obviously specified in his manifests.<\/p>\n
My problem happened with \/var\/lib\/puppet\/lib<\/code>, which is never specified in any of my manifests. But the symlink bit got me thinking. At some point, to have the specific facter plugin work, and having read about differences between older and 2.6.x versions of puppet, I had put it into my module's lib\/facter<\/code> folder, but creating also a symlink called plugins<\/code> (required by the new puppet). Doing so, I thought, would probably prevent problems, as the new puppet could read the file from the new folder, while the older version could read it from "lib".<\/p>\n
Well, turns out that puppet will not be able to read custom facter plugins from a plugins<\/code> folder that is a symlink. Removing the symlink and making {module}\/plugins\/facter<\/code> a real directory solved that problem too.<\/p>\n
I really hope to save someone else the time I spent on this.<\/strong> Cheers :)<\/p>\n","protected":false},"excerpt":{"rendered":"
Something I should have probably done a while ago. Upgrading our puppet clients and master from 0.24.5 (current Debian stable) to 2.6.2 (current backports). I've been advised to go all the way to 2.6.4, that contains an important security fix. I'll probably do that soon. So, first error I did was to upgrade one client […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[91,347,148,104,260,348,76,78],"yoast_head":"\nUpgrade of puppet from 0.24.5 to 2.6.2: "Got nil value for content" - Random hacking<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Upgrade of puppet from 0.24.5 to 2.6.2: "Got nil value for content" - Random hacking\" \/>\n<meta property=\"og:description\" content=\"Something I should have probably done a while ago. Upgrading our puppet clients and master from 0.24.5 (current Debian stable) to 2.6.2 (current backports). I've been advised to go all the way to 2.6.4, that contains an important security fix. I'll probably do that soon. So, first error I did was to upgrade one client […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/\" \/>\n<meta property=\"og:site_name\" content=\"Random hacking\" \/>\n<meta property=\"article:published_time\" content=\"2011-01-06T15:43:12+00:00\" \/>\n<meta name=\"author\" content=\"cosimo\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"cosimo\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/\"},\"author\":{\"name\":\"cosimo\",\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/c443bedbf6ecf99550d6395620801df1\"},\"headline\":\"Upgrade of puppet from 0.24.5 to 2.6.2: “Got nil value for content”\",\"datePublished\":\"2011-01-06T15:43:12+00:00\",\"dateModified\":\"2011-01-06T15:43:12+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/\"},\"wordCount\":503,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/c443bedbf6ecf99550d6395620801df1\"},\"keywords\":[\"development\",\"facter\",\"fun\",\"plugin\",\"puppet\",\"puppetmaster\",\"servers\",\"upgrade\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/\",\"url\":\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/\",\"name\":\"Upgrade of puppet from 0.24.5 to 2.6.2: \\\"Got nil value for content\\\" - Random hacking\",\"isPartOf\":{\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/#website\"},\"datePublished\":\"2011-01-06T15:43:12+00:00\",\"dateModified\":\"2011-01-06T15:43:12+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.streppone.it\/cosimo\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Upgrade of puppet from 0.24.5 to 2.6.2: “Got nil value for content”\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/#website\",\"url\":\"https:\/\/www.streppone.it\/cosimo\/blog\/\",\"name\":\"Random hacking\",\"description\":\"Assume nothing. Code defensively. Keep it simple, stupid!\",\"publisher\":{\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/c443bedbf6ecf99550d6395620801df1\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.streppone.it\/cosimo\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/c443bedbf6ecf99550d6395620801df1\",\"name\":\"cosimo\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/cb1d938720df45a2720724aae99e3bfc?s=96&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/cb1d938720df45a2720724aae99e3bfc?s=96&r=g\",\"caption\":\"cosimo\"},\"logo\":{\"@id\":\"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/image\/\"},\"url\":\"https:\/\/www.streppone.it\/cosimo\/blog\/author\/cosimo\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Upgrade of puppet from 0.24.5 to 2.6.2: \"Got nil value for content\" - Random hacking","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/","og_locale":"en_US","og_type":"article","og_title":"Upgrade of puppet from 0.24.5 to 2.6.2: \"Got nil value for content\" - Random hacking","og_description":"Something I should have probably done a while ago. Upgrading our puppet clients and master from 0.24.5 (current Debian stable) to 2.6.2 (current backports). I've been advised to go all the way to 2.6.4, that contains an important security fix. I'll probably do that soon. So, first error I did was to upgrade one client […]","og_url":"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/","og_site_name":"Random hacking","article_published_time":"2011-01-06T15:43:12+00:00","author":"cosimo","twitter_card":"summary_large_image","twitter_misc":{"Written by":"cosimo","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/#article","isPartOf":{"@id":"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/"},"author":{"name":"cosimo","@id":"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/c443bedbf6ecf99550d6395620801df1"},"headline":"Upgrade of puppet from 0.24.5 to 2.6.2: “Got nil value for content”","datePublished":"2011-01-06T15:43:12+00:00","dateModified":"2011-01-06T15:43:12+00:00","mainEntityOfPage":{"@id":"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/"},"wordCount":503,"commentCount":0,"publisher":{"@id":"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/c443bedbf6ecf99550d6395620801df1"},"keywords":["development","facter","fun","plugin","puppet","puppetmaster","servers","upgrade"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/","url":"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/","name":"Upgrade of puppet from 0.24.5 to 2.6.2: \"Got nil value for content\" - Random hacking","isPartOf":{"@id":"https:\/\/www.streppone.it\/cosimo\/blog\/#website"},"datePublished":"2011-01-06T15:43:12+00:00","dateModified":"2011-01-06T15:43:12+00:00","breadcrumb":{"@id":"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.streppone.it\/cosimo\/blog\/2011\/01\/upgrade-of-puppet-from-0-24-5-to-2-6-2-got-nil-value-for-content\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.streppone.it\/cosimo\/blog\/"},{"@type":"ListItem","position":2,"name":"Upgrade of puppet from 0.24.5 to 2.6.2: “Got nil value for content”"}]},{"@type":"WebSite","@id":"https:\/\/www.streppone.it\/cosimo\/blog\/#website","url":"https:\/\/www.streppone.it\/cosimo\/blog\/","name":"Random hacking","description":"Assume nothing. Code defensively. Keep it simple, stupid!","publisher":{"@id":"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/c443bedbf6ecf99550d6395620801df1"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.streppone.it\/cosimo\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/c443bedbf6ecf99550d6395620801df1","name":"cosimo","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/cb1d938720df45a2720724aae99e3bfc?s=96&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/cb1d938720df45a2720724aae99e3bfc?s=96&r=g","caption":"cosimo"},"logo":{"@id":"https:\/\/www.streppone.it\/cosimo\/blog\/#\/schema\/person\/image\/"},"url":"https:\/\/www.streppone.it\/cosimo\/blog\/author\/cosimo\/"}]}},"_links":{"self":[{"href":"https:\/\/www.streppone.it\/cosimo\/blog\/wp-json\/wp\/v2\/posts\/459"}],"collection":[{"href":"https:\/\/www.streppone.it\/cosimo\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.streppone.it\/cosimo\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.streppone.it\/cosimo\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.streppone.it\/cosimo\/blog\/wp-json\/wp\/v2\/comments?post=459"}],"version-history":[{"count":0,"href":"https:\/\/www.streppone.it\/cosimo\/blog\/wp-json\/wp\/v2\/posts\/459\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.streppone.it\/cosimo\/blog\/wp-json\/wp\/v2\/media?parent=459"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.streppone.it\/cosimo\/blog\/wp-json\/wp\/v2\/categories?post=459"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.streppone.it\/cosimo\/blog\/wp-json\/wp\/v2\/tags?post=459"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}