Skip to content

How I got Let’s Encrypt setup and operating on CentOS

Dear Reader,

WARNING: Here there be techy-stuff. Mere mortals, tread lightly.
Translation: I am not responsible if you hose your website trying to implement what I describe here.

letsencrypt

 

The web has been given a wonderful gift by the name of Let’s Encrypt. A free way to get an SSL certificate for your website. Many hosting services are already incorporating Let’s Encrypt in their control panels. Soon there will be no good reason for every website not to be encrypted.

Encryption vs. Verification

Before I go any further, let’s talk about the two things that an SSL certificate can provide for your website.

Encryption

The first thing that an SSL certificate gives you is encryption. For the uninitiated, this means that before your server sends a single byte of content, it negotiates a secure channel with the client. THEN it sends the content. This is what Let’s Encrypt is all about, securing the web.

Previously, you had to purchase an SSL certificate to get encryption. Granted, the price has come way down in the past few years. The last one I actually purchased was for Voices of the ElePHPant and it was only $8 a year! Still, that is a barrier for some people. So Let’s Encrypt provides a way for everyone to get encrypted websites for free.

Verification

The other thing that some commercial SSL certificate do is provide verification. To get one of the more expensive SSL certificates – e.g. not the $8/year ones – you have to prove who you or your business are. The company issuing the SSL certificate will ask you a lot of questions, you may even have to provide a DUNS to be verified. This helps prevent against fraud on the web and is very important to e-commerce sites, banking sites, and any site where you want to ensure the user that they are seeing the site for the company they THINK they are.

Let’s Encrypt does not provide Verification. If verification is important to your users, don’t use Let’s Encrypt. On the other hand, verification is not real important to sites like this blog, so Let’s Encrypt is a great option.

Doing the Deed

Let’s Encrypt provides wonderful documentation and a lot of automation if you are running Debian. If you aren’t – like me – or if you have a non-standard Apache setup, then you have to do a few extra steps.

So I am going to describe what I do when I set up a new SSL certificate from Let’s Encrypt, and how I keep them renewed automatically.

The following assumes you have installed Let’s Encrypt on your server. I simply cloned the Let’s Encrypt GitHub repo so that I can keep up to date. Twitter follower Nick Le Mouton also pointed out that Let’s Encrypt is also EPEL and that you can use yum to install it as well.

Step 1: Get the certificate

I have a script to manage this for me. I manually edit it before I run it each time, changing the domain name. Since I don’t purchase more than one domain per month these days – I’m on a domain name diet – this is easier for me than remembering parameters.

 

The Certificate Script

<code>
#!/bin/bash
#
# makeCert.sh
# This script requests a cert the first time. This is not the auto renew script.
#
# Set DOMAIN before running
#
DOMAIN=example.com
SITE=$DOMAIN
 
#
# Make sure this is set correctly
#
ENCRYPTDIR=/opt/letsencrypt
 
#
# Once it is working, you shoudn't have to change anything below here.
#
cd $ENCRYPTDIR
 
 
#
# How many dots are in the $SITE. If more than 1 then it is a subdomain 
# so don't register www as well.
#
FILESEPS=$(echo $SITE | awk 'BEGIN { FS="."} {print NF}')
 
if [ $FILESEPS -le 2 ]; then
       DOMAIN="$SITE -d www.$SITE "
else
        DOMAIN="$SITE"
fi
 
#
# Requst the cert
#
./letsencrypt-auto certonly \
          --agree-tos \
          -d $DOMAIN \
          -a webroot \
          --webroot-path /var/www/$SITE/public_html \
          --server https://acme-v01.api.letsencrypt.org/directory
</code>

 

All of my websites sit in /var/www on my servers. If you have a different scheme, you will have to adjust.

Update Apache

Ok, that gets my initial SSL certificate. Since I can’t use any of the automation, I have to manually update my apache conf files. How you do this is up to you. Since you are reading this because you have a non-standard setup, it is not possible for me to tell you exactly what you need to do. Honestly, if you don’t know, you probably shouldn’t be reading this in the first place. The important part is that you add these lines to your site’s apache conf file, where ever that may be.

The important part is that you add these lines to your site’s apache conf file, where ever that may be.

<code>
   SSLEngine on
   SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
   SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
   SSLCACertificateFile /etc/letsencrypt/live/example.com/chain.pem 
</code>

Once you’ve done that, restart apache.

Step 2: Get the renewals

Let’s Encrypt SSL certificates are only good for 90 days. So you will have to renew them at least once a quarter. Thankfully, renewal is not difficult. I have a bash script that I run once a morning. I won’t tell you when but pick a non-standard hour and minute so we aren’t all hitting it at say 1:15 AM.

You run this once a day for a couple of reasons.

First, we are not renewing the certs every day, we are only checking to see if they need renewing. It is a much lighter touch and doesn’t even involve hitting their API unless you actually do need to renew.

Second, if you run it once every 90 days and it fails, your certs will expire before your next run. Even every 30 days is a risk. If you run it every day and you have a cert nearing expiration, if for some reason you can’t reach the API on day 1, you get more chances.

The Renewal Script

<code>
#!/bin/bash
#
# renewCerts.sh
# This script checks every cert on this server and if necessary, renews it.
#
 
#
# Configure
#
ROOTDIR=/etc/letsencrypt/live
WEBROOT=/var/www
ENCRYPTDIR=/opt/letsencrypt
 
#
# Loop and renew
#
cd $ENCRYPTDIR
 
for SITE in $(ls -1 $ROOTDIR); do
 
        FILESEPS=$(echo $SITE | awk 'BEGIN { FS="."} {print NF}')
 
        if [ $FILESEPS -le 2 ]; then
                DOMAIN="$SITE -d www.$SITE "
        else
                DOMAIN="$SITE"
        fi
 
        ./letsencrypt-auto certonly \
          --agree-tos \
          --keep-until-expiring \
          -d $DOMAIN \
          -a webroot \
          --webroot-path $WEBROOT/$SITE/public_html \
          --server https://acme-v01.api.letsencrypt.org/directory
done
 
 
#
# Restart Apache
#
systemctl restart httpd
 
</code>

The script above will most likely be run either as root or by using sudo as it has to restart apache automatically.

The big change is --keep-until-expiring. This tells the Let’s Encrypt script to check the cert and if it’s expiration date is greater than 30 days out, don’t bother with it. If it’s less than 30 days, the script will go ahead and request a new SSL certificate.

Gotchas

There is only one gotcha I have found in doing things this way. If you have more than 3 subdomains, and you have certs for all of them individually – like I do – you can only request 3 SSL certificates per 24 hour period for any given domain. The answer to this is actually very easy, stagger the dates on your certificates. You can have as many as you want as long as more than 3 of them don’t expire on the same day.

Wrap-up

I developed these scripts right when the Let’s Encrypt public beta opened. As the project matures, all of this may not be necessary. Before implementing these scripts, check the Let’s Encrypt docs to see if things have changed.

Until next time,
I <3 |<
=C=

p.s. I hate to say this but I am going to. Change example.com in all of the above scripts to your domain name. An SSL certificate for example.com won’t do you much good. :)

set_include_path() failing

Dear Reader,

The title of this post will confuse most of you but I chose it on purpose. I hit a wall last night and couldn’t find anything on the web to help me over it.

The Problem

<?PHP
var_dump(set_include_path('./'));

Drop this little piece of code into a file and place it on your web server. What you SHOULD get back is a string describing your old include path. If you get false, you have hit the problem I hit last night. I went so far as to recompile 2 difference versions of Apache thinking (almost correctly) that it was something that had changed in my Apache config files.

The Solution

Like most Apache installs, my development server has a single config file for each domain. (In my case they are all *.dev domains, I don’t host any live domains on my cable modem) In each config file, I set not only Apache settings but some defaults for things like php’s include path for that domain. Here’s an example for my development are for calevans.com

<virtualhost 192.168.0.99:80>
    ServerAdmin webmaster@dummy-host.example.com
    DocumentRoot /home/calevans.com/public_html
    ServerName www.calevans.dev
    ServerAlias www.calevans.dev calevans.dev
    ErrorLog logs/dummy-host.example.com-error_log
    CustomLog logs/dummy-host.example.com-access_log common
    php_value include_path /usr/local/lib/php:/home/phpinc/:/home/calevans.com/phpinc
</virtualhost>

As you can see, I use php_value to set the default include path. However, for some reason, I had started using php_admin_value to set it and other settings. After 4 long hours of banging my head on this, my friend Davey Shafik popped into #phpwomen (yes, I hang out there…its like taking “Home Economics” in high school, you do it because it’s where all the girls are) and said:

[22:54] Davey: I know what it is ;)
[22:54] Davey: php_admin_value include_path
[22:54] Davey: you have that somewhere
[22:55] Davey: you *cannot* override php_admin_value settings

Son-of-a-gun if he wasn’t right. For some reason I had started using php_admin_value instead of just php_value. php_value can be overridden in your code, however, php_admin_value tells PHP not to allow the value to be overridden.

Why is this important

If you are working with Zend Framework, then this is VERY important. Zend_Loader temporarily modifies the include_path when including things like Controllers. If it can’t do so, your apps will fail in a most spectacular way.

Wife 1.24 (The lovely and talented Kathy) says that now I know how everyone else feels when I walk over and solve problems. (I have a tendency to do that and am usually pretty snarky about it when I do)

Until Next Time,
(l)(k)(bunny)
=C=

Battling with Apache and PassEnv

Dear Reader,

I’ve been managing Apache servers since 1999 and while I consider myself competent, I never processed to be an expert. So it really didn’t surprise me yesterday when I learned something new.

A buddy of mine was discussing a problem he was having with his corporate web site. It’s hosted on multiple web servers behind a load balancer. The problem is, they are having problems with one of the web servers but they can’t figure out which. I remember that back at Jupiter, we had a similar problem and that one of my guys there injected a header that allowed us to track which server a specific request was coming from. Unfortunately for my friend, that was about as far as I remembered. (I was management then, I didn’t pay attention to details) However, he’s a bright guy and had already thought of this…the difference was, he knew basically how to do it.

I started playing with ideas on my development server and got about 75% of the way there when he IMed me that he had it working. (So much for speed) I fired up FireFox and TamperData and sure enough, he had a custom header in there. (BTW, TamperData is invaluable for debugging things like this.) However, when I followed his direction, it just was not happening for me. Googling around found me a LOT of copies of the Apache manual, but no concrete examples. So, since I couldn’t find the answer on the web, I decided to post how I did this in case some other Apache noob comes looking for it.

First, an important detail, my development environment is a customized version of CentOS 5. I use yum for just about everything, except Apache, PHP, MySQL and their support programs. I use a script that comes with DirectAdmin, my production control panel of choice, to maintain those pieces. The important thing to note here is that CentOS does NOT use apachectrl to start and stop apache.

The answer to inserting a header, of course, revolves around PassEnv. If you have mod_env installed in your Apache 1.3.7 or Apache 2.x, this will work.

I’m working with Virtual Hosts and each development site has it’s on conf file that is included into the main httpd.conf. The great thing about PassEvn though is you can put it in an .htaccess file as well. This means for testing, you don’t have to constantly be bouncing the service.

To identify my server, I decided I wanted to add a header that displayed the host name that the page was being served from. This means in my conf file or .htaccess, I need the following:

PassEnv HOSTNAME
Header set X-MyHeader "%{HOSTNAME}e"

That gets us about 75% there. The PassEnv makes the HOSTNAME environment variable available to APache and the Header command actually sets the new header. “%{VARAIABLENAME}e” is the syntax for displaying the variable in the header.

All that is fine and good and if you drop that in your .htaccess file and then hit a page on the site, you will get:

X-MyHeader (null)

Now, my friend said he added:

export HOSTNAME=`hostname`

to his apachectrl script and all was good. I tried this and (if you read the note above, you see this coming) it did not fix the problem.

Digging deeper, I found that apachectrl sources a file /usr/sbin/envvars. Looking it, it’s obvious that this is where they expect you to put these commands so you don’t have to have a customized version of apachectrl. So I took my export out of apachectrl and put it in envvars. Still no dice, yeah, I know, you expected this.

The problem, as I explained above, is that if you are using “service httpd start” on CentOS then apachectrl is being ignored. Lickily for us, /etc/init.d/httpd is another bash script. So I grabbed the lines out of apachectrl that sourced /usr/sbin/envvars, inserted them in /etc/init.d/httpd and BINGO, we have a header.

if test -f /usr/sbin/envvars; then
  . /usr/sbin/envvars
fi

I put them high up in the file, near where it sources the functions file.

So, it is possible, and even easy to add custom headers into Apache that include environment variables. You need to make sure that mod_env and mod_headers are both installed and the rest is just figuring out where to put things.

UPDATE: Since I was asked, yes you can access this new information in PHP via the $_ENV super global.

A var_dump of the $_ENV on my development server now looks like this:

array(8) {
  ["HOSTNAME"]=>
  string(5) "david"
  ["TERM"]=>
  string(5) "xterm"
  ["LD_LIBRARY_PATH"]=>
  string(15) "/etc/httpd/lib:"
  ["PATH"]=>
  string(29) "/sbin:/usr/sbin:/bin:/usr/bin"
  ["PWD"]=>
  string(1) "/"
  ["LANG"]=>
  string(11) "en_US.UTF-8"
  ["SHLVL"]=>
  string(1) "2"
  ["_"]=>
  string(15) "/usr/sbin/httpd"
}

So you can see, you can see that while the Header X-MyHeader is not exposed, the variable it contains, HOSTNAME, is.

For the record, hostname was probably not the best example since it’s usually in the $_SERVER array. However, the point of the experiment was not to expose this information to PHP, it was to put it in the response form the server.

Until next time,
(l)(k)(bunny)
=C=

Forks!=Threads!

Dear Reader,

Yes, another programming blog…get over it. For you non-techies, go get some candy or something, let the grown-ups talk. BTW, TinaT! (This is not a Tutorial!) This is just a monologue of what I went through to get a program to fork.

Ok, so over the holidays I spent some time deep into code. I wrote apacheLogSplit a few weeks ago so I could import a customer’s log files into MySQL so I could root around in them. (Hey, I like rooting around in data!) Anyhow, program worked like a charm but it was a bit slow. So after I released it, I decided to see if I could speed it up. The first thing that came to mind was “boy it would be nice if this were a multi-threaded application.” Unfortunately, PHP is not a multi-threaded environment. The best we can do is fork it. I like working on new concepts with existing projects because I’ve already worked out the details of the logic so I can concentrate on the new concepts I’m learning. So like a fat-man doing a swan-dive, I took the plunge with apacheLogSplit.

Having never worked with forking before, I did what any lazy programmer does, I googled around to see what others had done that I could steal. As luck would have it, there’s a class in pear called PHP_Fork. It’s specifically designed to encapsulate the forking process and make it easier to work with in an OOP environment. (and if you are not writing OOP then just hang your head in shame and walk away slowly.) since apacheLogSplit was built using OO, it seemed the perfect solution. That is until I tried to install it. PHP_Fork requires PHP be compiled with –enable-cli –with-pcntl –enable-shmop. Even after recompiling it to meet the specs (I didn’t have shmop compiled originally) it still wouldn’t *&^%!!! install using pear. Well, when the tools fail you, reach for the sledgehammer; I resorted to downloading it and installing it manually. Once I placed it manually in the proper pear directory I was on my way.

I knew my program was overly complex. Version 1.0 was a learning experience for a different lesson. I was experimenting with PHP5’s new OOP features. While I got everything to work the way I wanted to, if you’ve looked at the code you know there are a bunch of abstract classes, even an interface that gets implemented. So before I could do much with it, I had to simplify it. (Much like I need to do with this blog.) I got rid of all the abstract classes and such and pared it down to 3 main classes (the main controller class, the line class and the SQL writer class) and a few lines of procedural code in the main program to handle parameters and such. (Yes, I KNOW I could have moved that into an object!)

So, we now have code we can work with and our Fork.php installed. We are now ready to get down to some serious forking around. The first thing I learned is that I had a mis-conception of how forking worked. I knew that it was not the same as a thread but I assumed that when you forked code that the child started again form the beginning. Not so, the child process begins running with the statement after the fork. Very important distinction and it’s probably a mis-conception that only I had.

Once I crossed that hurdle, I was able to make quick progress. I had already designed my code so that the main body of processing took place inside an object; I made this object my thread marshaler. It read in a line from the file and then handed it to the first idle thread.

My threads have 2 subordinate objects, a line parser and a SQL writer. As I said earlier, for simplicity’s sake I did away with all the abstract classes etc. This version of the program can not be easily extended to write to XML, etc. I did that so as to simplify the code for readability and because I question whether anyone will actually take the time to do it. Once simplified, the object interaction is simple and easy to follow.

This brings me to the final hurdle I had to overcome, parent-child communication. This is where I REALLY miss threads. Threads make this a might easier because you can directly fire methods on the child form the parent. And because they are threads, they will sit in a sleep state until you do fire a method on them. In this case, each child sits in a loop waiting for a message. Once it receives a message, it processes the line and then goes back into its loop.

The only way to send the child a message is to use shared memory. PHP_Fork handles this nicely by setting up the semaphores and defining the 2 methods setVariable() and getVariable(). Using them you can set a variable in the parent and retrieve it in the child. So in my loop I set the variable lineValue. The child loops until it finds a value for lineValue and then goes to work processing it. Easy as pie, interprocess communication. Note that it is possible but a bit more complicated to fire a method on the child thread with a parameter. I chose not to bother with it in this case but I could have just as easily done that.

That’s it. My little PHP program is now a multi-process program. Here’s the kicker. It didn’t make any difference in speed. The main problem I have is that I’m running the program on the same box as MySQL. Thus they are competing for processor resources. 2 threads ran about the same speed 10 threads actually ran slower. I think the results would have been different if I were running them on separate boxes on a LAN.

Until next time, come back home safely.

(l)(k)(bunny)
=C=