Postcards From My Life

Dear Reader,

The title of this post will confuse most of you but I chose it on purpose. I hit a wall last night and couldn’t find anything on the web to help me over it.

The Problem

<?PHP
var_dump(set_include_path('./'));

Drop this little piece of code into a file and place it on your web server. What you SHOULD get back is a string describing your old include path. If you get false, you have hit the problem I hit last night. I went so far as to recompile 2 difference versions of Apache thinking (almost correctly) that it was something that had changed in my Apache config files.

The Solution

Like most Apache installs, my development server has a single config file for each domain. (In my case they are all *.dev domains, I don’t host any live domains on my cable modem) In each config file, I set not only Apache settings but some defaults for things like php’s include path for that domain. Here’s an example for my development are for calevans.com

<virtualhost 192.168.0.99:80>
    ServerAdmin [email protected]
    DocumentRoot /home/calevans.com/public_html
    ServerName www.calevans.dev
    ServerAlias www.calevans.dev calevans.dev
    ErrorLog logs/dummy-host.example.com-error_log
    CustomLog logs/dummy-host.example.com-access_log common
    php_value include_path /usr/local/lib/php:/home/phpinc/:/home/calevans.com/phpinc
</virtualhost>

As you can see, I use php_value to set the default include path. However, for some reason, I had started using php_admin_value to set it and other settings. After 4 long hours of banging my head on this, my friend Davey Shafik popped into #phpwomen (yes, I hang out there…its like taking “Home Economics” in high school, you do it because it’s where all the girls are) and said:

[22:54] Davey: I know what it is ;)
[22:54] Davey: php_admin_value include_path
[22:54] Davey: you have that somewhere
[22:55] Davey: you *cannot* override php_admin_value settings

Son-of-a-gun if he wasn’t right. For some reason I had started using php_admin_value instead of just php_value. php_value can be overridden in your code, however, php_admin_value tells PHP not to allow the value to be overridden.

Why is this important

If you are working with Zend Framework, then this is VERY important. Zend_Loader temporarily modifies the include_path when including things like Controllers. If it can’t do so, your apps will fail in a most spectacular way.

Wife 1.24 (The lovely and talented Kathy) says that now I know how everyone else feels when I walk over and solve problems. (I have a tendency to do that and am usually pretty snarky about it when I do)

Until Next Time,
(l)(k)(bunny)
=C=

Dear Reader,

I’ve been managing Apache servers since 1999 and while I consider myself competent, I never processed to be an expert. So it really didn’t surprise me yesterday when I learned something new.

A buddy of mine was discussing a problem he was having with his corporate web site. It’s hosted on multiple web servers behind a load balancer. The problem is, they are having problems with one of the web servers but they can’t figure out which. I remember that back at Jupiter, we had a similar problem and that one of my guys there injected a header that allowed us to track which server a specific request was coming from. Unfortunately for my friend, that was about as far as I remembered. (I was management then, I didn’t pay attention to details) However, he’s a bright guy and had already thought of this…the difference was, he knew basically how to do it.

I started playing with ideas on my development server and got about 75% of the way there when he IMed me that he had it working. (So much for speed) I fired up FireFox and TamperData and sure enough, he had a custom header in there. (BTW, TamperData is invaluable for debugging things like this.) However, when I followed his direction, it just was not happening for me. Googling around found me a LOT of copies of the Apache manual, but no concrete examples. So, since I couldn’t find the answer on the web, I decided to post how I did this in case some other Apache noob comes looking for it.

First, an important detail, my development environment is a customized version of CentOS 5. I use yum for just about everything, except Apache, PHP, MySQL and their support programs. I use a script that comes with DirectAdmin, my production control panel of choice, to maintain those pieces. The important thing to note here is that CentOS does NOT use apachectrl to start and stop apache.

The answer to inserting a header, of course, revolves around PassEnv. If you have mod_env installed in your Apache 1.3.7 or Apache 2.x, this will work.

I’m working with Virtual Hosts and each development site has it’s on conf file that is included into the main httpd.conf. The great thing about PassEvn though is you can put it in an .htaccess file as well. This means for testing, you don’t have to constantly be bouncing the service.

To identify my server, I decided I wanted to add a header that displayed the host name that the page was being served from. This means in my conf file or .htaccess, I need the following:

PassEnv HOSTNAME
Header set X-MyHeader "%{HOSTNAME}e"

That gets us about 75% there. The PassEnv makes the HOSTNAME environment variable available to APache and the Header command actually sets the new header. “%{VARAIABLENAME}e” is the syntax for displaying the variable in the header.

All that is fine and good and if you drop that in your .htaccess file and then hit a page on the site, you will get:

X-MyHeader (null)

Now, my friend said he added:

export HOSTNAME=`hostname`

to his apachectrl script and all was good. I tried this and (if you read the note above, you see this coming) it did not fix the problem.

Digging deeper, I found that apachectrl sources a file /usr/sbin/envvars. Looking it, it’s obvious that this is where they expect you to put these commands so you don’t have to have a customized version of apachectrl. So I took my export out of apachectrl and put it in envvars. Still no dice, yeah, I know, you expected this.

The problem, as I explained above, is that if you are using “service httpd start” on CentOS then apachectrl is being ignored. Lickily for us, /etc/init.d/httpd is another bash script. So I grabbed the lines out of apachectrl that sourced /usr/sbin/envvars, inserted them in /etc/init.d/httpd and BINGO, we have a header.

if test -f /usr/sbin/envvars; then
  . /usr/sbin/envvars
fi

I put them high up in the file, near where it sources the functions file.

So, it is possible, and even easy to add custom headers into Apache that include environment variables. You need to make sure that mod_env and mod_headers are both installed and the rest is just figuring out where to put things.

UPDATE: Since I was asked, yes you can access this new information in PHP via the $_ENV super global.

A var_dump of the $_ENV on my development server now looks like this:

array(8) {
  ["HOSTNAME"]=>
  string(5) "david"
  ["TERM"]=>
  string(5) "xterm"
  ["LD_LIBRARY_PATH"]=>
  string(15) "/etc/httpd/lib:"
  ["PATH"]=>
  string(29) "/sbin:/usr/sbin:/bin:/usr/bin"
  ["PWD"]=>
  string(1) "/"
  ["LANG"]=>
  string(11) "en_US.UTF-8"
  ["SHLVL"]=>
  string(1) "2"
  ["_"]=>
  string(15) "/usr/sbin/httpd"
}

So you can see, you can see that while the Header X-MyHeader is not exposed, the variable it contains, HOSTNAME, is.

For the record, hostname was probably not the best example since it’s usually in the $_SERVER array. However, the point of the experiment was not to expose this information to PHP, it was to put it in the response form the server.

Until next time,
(l)(k)(bunny)
=C=

Dear Reader,

Yes, another programming blog…get over it. For you non-techies, go get some candy or something, let the grown-ups talk. BTW, TinaT! (This is not a Tutorial!) This is just a monologue of what I went through to get a program to fork.

Ok, so over the holidays I spent some time deep into code. I wrote apacheLogSplit a few weeks ago so I could import a customer’s log files into MySQL so I could root around in them. (Hey, I like rooting around in data!) Anyhow, program worked like a charm but it was a bit slow. So after I released it, I decided to see if I could speed it up. The first thing that came to mind was “boy it would be nice if this were a multi-threaded application.” Unfortunately, PHP is not a multi-threaded environment. The best we can do is fork it. I like working on new concepts with existing projects because I’ve already worked out the details of the logic so I can concentrate on the new concepts I’m learning. So like a fat-man doing a swan-dive, I took the plunge with apacheLogSplit.

Having never worked with forking before, I did what any lazy programmer does, I googled around to see what others had done that I could steal. As luck would have it, there’s a class in pear called PHP_Fork. It’s specifically designed to encapsulate the forking process and make it easier to work with in an OOP environment. (and if you are not writing OOP then just hang your head in shame and walk away slowly.) since apacheLogSplit was built using OO, it seemed the perfect solution. That is until I tried to install it. PHP_Fork requires PHP be compiled with –enable-cli –with-pcntl –enable-shmop. Even after recompiling it to meet the specs (I didn’t have shmop compiled originally) it still wouldn’t *&^%!!! install using pear. Well, when the tools fail you, reach for the sledgehammer; I resorted to downloading it and installing it manually. Once I placed it manually in the proper pear directory I was on my way.

I knew my program was overly complex. Version 1.0 was a learning experience for a different lesson. I was experimenting with PHP5′s new OOP features. While I got everything to work the way I wanted to, if you’ve looked at the code you know there are a bunch of abstract classes, even an interface that gets implemented. So before I could do much with it, I had to simplify it. (Much like I need to do with this blog.) I got rid of all the abstract classes and such and pared it down to 3 main classes (the main controller class, the line class and the SQL writer class) and a few lines of procedural code in the main program to handle parameters and such. (Yes, I KNOW I could have moved that into an object!)

So, we now have code we can work with and our Fork.php installed. We are now ready to get down to some serious forking around. The first thing I learned is that I had a mis-conception of how forking worked. I knew that it was not the same as a thread but I assumed that when you forked code that the child started again form the beginning. Not so, the child process begins running with the statement after the fork. Very important distinction and it’s probably a mis-conception that only I had.

Once I crossed that hurdle, I was able to make quick progress. I had already designed my code so that the main body of processing took place inside an object; I made this object my thread marshaler. It read in a line from the file and then handed it to the first idle thread.

My threads have 2 subordinate objects, a line parser and a SQL writer. As I said earlier, for simplicity’s sake I did away with all the abstract classes etc. This version of the program can not be easily extended to write to XML, etc. I did that so as to simplify the code for readability and because I question whether anyone will actually take the time to do it. Once simplified, the object interaction is simple and easy to follow.

This brings me to the final hurdle I had to overcome, parent-child communication. This is where I REALLY miss threads. Threads make this a might easier because you can directly fire methods on the child form the parent. And because they are threads, they will sit in a sleep state until you do fire a method on them. In this case, each child sits in a loop waiting for a message. Once it receives a message, it processes the line and then goes back into its loop.

The only way to send the child a message is to use shared memory. PHP_Fork handles this nicely by setting up the semaphores and defining the 2 methods setVariable() and getVariable(). Using them you can set a variable in the parent and retrieve it in the child. So in my loop I set the variable lineValue. The child loops until it finds a value for lineValue and then goes to work processing it. Easy as pie, interprocess communication. Note that it is possible but a bit more complicated to fire a method on the child thread with a parameter. I chose not to bother with it in this case but I could have just as easily done that.

That’s it. My little PHP program is now a multi-process program. Here’s the kicker. It didn’t make any difference in speed. The main problem I have is that I’m running the program on the same box as MySQL. Thus they are competing for processor resources. 2 threads ran about the same speed 10 threads actually ran slower. I think the results would have been different if I were running them on separate boxes on a LAN.

Until next time, come back home safely.

(l)(k)(bunny)
=C=