Skip to content

When Giants Battle. Google, Twitter, Apple, and encrypting the web. A podcaster’s story

Dear Reader,

This is a story about my fight to change the feed on my little podcast, Voices of the ElePHPant.

 

Google

Google's attitude towards encryption on the web

Twitter

Twitter's attitude towards encryption on the web

Apple

Apple's attitude towards encryption on the web

As a podcaster, I’m in a quandary. I know that encrypting the web is a good thing. Most of my sites now sport a Let’s Encrypt cert. (See this post for how I automated let’s Encrypt on CentOS) However, several giants of the web have different views on encryption.

  • Google – Encrypt everything.
    That’s good, right? I mean encryption is good and Google will reward encrypted sites with better ranking.
  • Twitter – Encrypt Player-cards and all the assets.
    Ok, Since I’m encrypting everything, encrypting my player cards and all their assets are encrypted. I even wrote a plugin to do thePlayer-cards and part of it makes sure that everything is served over https.
  • Apple – Don’t encrypt your feed or anything in it.
    Ok, Apple doesn’t come right out and say this, they simply give you a “Can’t read your feed” error if you submit a https URL and you are left to try and figure out why. Turns out that with a little experimentation you will find that not only that your feed can’t be encrypted, nothing in your feed can be encrypted.

Welcome to the modern web.

Trying to play nice

iTunes is still the most popular destination for podcast discovery. So you’ve got to have your feed there. iTunes has a lot of rules though. Early on, we had tools like FeedBurner that would “wash” our feeds and make them iTunes friendly. Feedburner WOULD accept an encrypted URL and spit out unencrypted.

As with all free services on the web, priorities change and services die. FeedBurner hasn’t had an update in many years and if you listen closely, you can hear the rumblings about Google shuttering it soon. That’s fine, long ago, I moved my entire podcast infrastructure to using the Bluberry Powerpress plugin for WordPress and it puts out an iTunes friendly feed by itself. So you would think that it’s just a matter of telling iTunes you have a new feed right? Well, um…no.

The Fix

The problem is that all three of these giants, Google, Twitter, and Apple, have influence over my little podcast.  After hours of searching for a solution, and switching to a paid cert simply because someone suggested that Apple simply didn’t like Lets Encrypt (not true) I finally decided to try an unencrypted feed. I turned off the automatic redirect to https that I had in my Apache configuration file for my podcast site. This almost worked.

I resubmitted the regular HTTP URL.  I could see from tailing my log files that iTunes accepted it and actually read it this time. I was elated for about half a second until Apple came back with an error message telling me that the podcast image couldn’t be encrypted either. <sad trombone />

Making Progress

Ok, I was making progress. This was further than I had gotten in several weeks of research. I jumped to the conclusion that Apple didn’t want to see any https inside the feed. This presented an interesting challenge because of the way that WordPress and thus PowerPress handle feeds.

The feed is generated in feed-podcast.php inside the PowerPress plugin directory. Looking it over, this is just a big while loop that pulls in the necessary information and then echoes out XML, one post at a time. Since it uses WordPress’s native functions to get urls and such, everything says https. I didn’t want to change WordPress because then my entire site would be back to unencrypted. I also didn’t want to check each echo statement for https and remove the s if it was found. This could get messy in a hurry.

It became obvious that I needed to grep the entire feed and replace https with HTTP. I’m a programmer, this should be easy. Except that there is no point in the WordPress flow where I can intercept the entire feed before it is sent out. WordPress has a complex system of hooks and filters but none of them were “RSS_FEED_BEFORE_IT_IS_SENT_OUT”. <sad trombone />

Since WordPress doesn’t gather the entire feed into a variable and then spit it out, a different solution is needed. WordPress treats the feed like any other page, it has a template that is executed that directly outputs XML. This was the absolute worst case scenario. Since Powerpress controlled the feed for my podcast, it looked like I was going to have to hack the core of the PowerPress plugin itself.

Old-School PHP to the rescue

Digging around for solutions, I came across a snippet of code that suggested using PHP’s Output buffer. Something I’ve not done in a very long time. The code in the snippet was not helpful, but the idea it sparked was what worked.

function my_callback($buffer) 
{
    return str_replace('https://','http://',$buffer);
}
 
ob_start("my_callback");

The heart of the solution is the function my_callback. In it, I simply replace HTTPS:// with HTTP://. Not terribly difficult to do in PHP. If I wanted to get fancy, I could probably have used an anonymous function int he ob_start() command, this solution is easier to read.

Next, we put in the ob_start(). In it we put the optional callback parameter and specify the name of the function we created just above it. For those not familiar with the callback parameter, here is an excerpt from the manual.

An optional output_callback function may be specified. This function takes a string as a parameter and should return a string. The function will be called when the output buffer is flushed (sent) or cleaned (with ob_flush(), ob_clean() or similar function) or when the output buffer is flushed to the browser at the end of the request. When output_callback is called, it will receive the contents of the output buffer as its parameter and is expected to return a new output buffer as a result, which will be sent to the browser. If the output_callback is not a callable function, this function will return FALSE.

tl;dr my_callback is called and passed everything that output buffering collected. I can make any modifications to it and whatever I return is what is actually sent out. That is exactly what I did.

At the bottom of the feed template, I simply put

<?php ob_end_flush(); ?>

This triggers the callback that is the secret sauce to the solution.

That’s all it took. I was able to exactly what I needed to do by going old-school on it’s butt and using output buffering. This gathered everything into a single variable that I was able to wash before outputting.

Once this was in place, I resubmitted the newly cleansed feed to Apple and not only did it accept it, the change was completed and visible in iTunes within about 30 minutes. (See the pretty new logo that the lovely and talented Kathy did for us two years ago? Until yesterday we still had our old logo there)

 

Screen Shot 2016-04-09 at 12.01.09 PM
 

fClose()

This blog post is not about the horrors of using a free service like FeedBurner. FeedBurner served me well for many years. It was my fault to begin with for submitting the FeedBurner url itself and not one hosted on my site as a 301 Redirect. The only lesson there is to make sure you own all the important pieces of your project, like feed URLs. :)

Here are the takeaways though.

  • iTunes does not hate Let’s Encrypt, iTunes hates encryption.
  • My site now runs both https and HTTP but I only advertise https
  • My feed is now exclusively HTTP, everything else on the site is HTTPS
  • I had to hack the core, which sucks but sometimes is necessary.

With regards to that last point. I believe that because PowerPress is awesome, there is a way to do this by specifying my own feed template. Reading the code, it looks like it is possible. I’ve written them to get clarification and will update this post when they get back to me.

Honestly, I don’t know how muggles deal with this. I was able to solve this because I am a programmer.

Until next time,
I <3 |<
=C=

The EICC Report for PHP User Group Leaders

Dear Reader,

Hi all!

Ok, I’ve got new news, old news, and freebies.

[I originally tried to post this to the UG Amins list but because of the restrictions put on any list hosted at php.net, I could not. I tried 4 times, the last time with nothing that looked like a domain name. So instead of fighting with it anymore, I’m just posting it here. We REALLY need to move the UG Admins list off of that domain.]

 

Nomad PHP (https://nomadphp.com ) FREEBIE
As a PUG leader, you are always welcome to attend any Nomad PHP meeting for free. Go to the website, scroll to the bottom, there is a link just for you. :) Just pick the meeting you want to attend, and submit the form. Want to attend more than one, submit more than once. Want to attend them all, submit once for each time. We won’t turn you down. it is our small way of saying thank you for the hard work you put into Nomad PHP.
Day Camp 4 Developers (http://daycamp4developers.com ) STRINGS ATTACHED
The next DC4D is now public and we need your help getting the word out. If you would please send an email to your list letting them know that tickets are now available for “Day Camp 4 Developers:Public Speaking for Developers II” we would appreciate it. Once you have sent the email to your list, forward me a copy of the email and I’ll send you a coupon for a free ticket to the event. You can use it yourself or raffle it off to your group.
Raffle Items FREEBIE
Do you need things to raffle off to your user group? Let me know. I will give you coupons to any of my e-books (http://leanpub.com/u/calevans) if it will help. Also, I’m happy to give you a ticket or two to Nomad PHP to give away as well.
The CFP Report (https://thecfpreport.com) I NEED HELP
Ok, this request is in two parts.

PUG leaders
Would you please include a notice about The CFP Report in your next newsletter? (thecfpreport [dot] com ) Let your members who are interested in speaking know about it. There is no charge for The CFP Report and I don’t ever intend for there to be one. it is just my way of helping to increase the speaker pool.

Conf Organisers
If you run a conference, I would appreciate it if you would include a short paragraph in your rejection letters. Nobody likes to get rejection letters but if you offer people a ray of hope, they dislike it a bit less. This is something I’ve tried to do for the past 10 years. So if you could include something like this sample paragraph in your rejection letters, you could help them and me. (win-win-win) :)

Sample Paragraph
There are several things you can do to increase the odds of you being selected as a speaker, one of them is to keep applying and not get discouraged. To help you do this, our friend Cal Evans has setup a free service called “The CFP Report” (https://thecfpreport.com). Each week you will get an email with all the CFPs that are open, along with important information about the conference to help you decide if you should apply or not. We encourage you to subscribe to The CFP Report, find other CFPs to submit to, and next year when you see ours pop up, submit again. Remember, this email isn’t a “NO”, it’s a “not now”.

 

 

Ok, that’s all I’ve got for you this time. As always, if there is anything I can do to help you or your PUG, please don’t hesitate to ask. :)

If you haven’t yet, make sure you register your group with http://php.ug! Every PUG should be on that list.

Cheers!

Until next time
I <3 |<
=C=

Mailchimp API v3 and Address fields

Freddie_OGDear Reader,

I love MailChimp. I mean I seriously love it. I’m not a big fan of SaaS vendors. I think that most of them are just obvious ideas that someone found a way to shove behind a paywall. But I love MailChimp.

One of the reasons I love it is that it seems to have been built for developers. Everything about it seems to be geared towards developers. This includes their API. I’ve been using v2 of their API in scrips for a few years now and it was just so easy to work with. Their PHP wrapper for it was one of the easiest to use API wrappers I’ve used.

Enter API V3

Recently, (March is recently, right?) they announced v3 of their API. It was “RESTful”. What used to be an absolute breeze to use in v2 has now become a laborious chore to use. It doesn’t help that they don’t bother to release sample code in PHP, if you aren’t a Python or Ruby developer, you have to figure it out on your own.

So that is what I did. I sat down with Guzzle and beat it into submission. While you can still do everything you could do before, to blindly adhere to the principals of REST, they have made retrieving something as simple as all of the information about a single email address 3-4 calls. Complexity for the sake of “doing it right”. (Because there is no way this is actually easier to use, but REST purists will love it.)

Address the issue

One area of change was how the merge fields work. Now this isn’t actually harder than v2, because this was the weak point in v2. However, using their ADDRESS type merge field, I found something very interesting.

The MailChimp Address field is actually made up of 6 different properties.

  • Address Line 1 (addr1)
  • Address Line 2 (addr2)
  • City (city)
  • State (state)
  • Zip Code (zipcode)
  • Country (country)

It’s not terribly difficult to work with from the API…unless. I found an interesting anomaly in the API. Your address lines cannot contain 2 consecutive spaces. Dang near drove me up the wall because it doesn’t tell you that this is a problem, or even which line in the address field has a problem. The error message that comes back simply tells you to enter a valid address.

Conclusion

Ok, if you are using the MailChimp API v3 and you are using PHP, Guzzle is your friend. Also, make sure you strip out any double spaces. Here is how I did it.


$addr1 = filter_var($payload["addr1"],FILTER_SANITIZE_STRING);
$mcRecord->merge_fields->ADDRESS->addr1 = strtr($addr1,['  '=>' ']);

Personally, I believe that this behavior is an artifact of how Mailchimp’s importer works. If you read the docs, they use double spaces to split apart the pieces of an address. Since I was assigning a string with double spaces to a single field. I think the API was choking on that and that is why it spit it back out at me. Of course a helpful error message or even a blog post mentioning this rule would have saved me some frustration.

Still, Mailchimp beats everything else, hands down. Still a fan, even if a bit frustrated.

UPDATE: See the comment below, Pete addresses the issues. (Thank you Pete!) :)

Until next time,
I<3|<
=C=

Celebrating 20 years of PHP

elephpants-512

Dear Reader,

Here is my PHP story.

In 1997, I built my first web based application. When completed it was running on 2 Microsoft Windows servers and powered by ASP (Classic) and SQL Server 6.5. It was around this time that Microsoft decided that if you wanted to use SQL Server on the web, you have to purchase a special web license that was very expensive. This actually was a concession for them because originally they stated that you have to have a license for each person who used your website.  I’m pretty sure they didn’t think that one through, thus the back-peddling and the new Microsoft tax on the web.

I had already purchased 2 licenses for Windows Server and one license for SQL Server and the little company I was building this for could not afford the $15,000 for the new licenses and upgrades. No new hardware and no new functionality, just Microsoft sticking a spigot into our bank account and opening it.

So in 1999, I started looking around for alternatives. I had heard of this new language called PHP and that it was open source. I wasn’t really sure what that meant at the time but figured that it couldn’t cost any MORE than Microsoft, so I started investigating. It turns out that it was gonna cost me a LOT less. No, things weren’t free, but with a little creativity and with the help of a lot of open source software, I was able to rebuild the entire application using PHP.

Not counting what they paid me as a programmer, the new system cost $4,500, that was the price of a shiny new server. All of the software I used was open source.

  • RedHat Linux (Remember RPM hell?) :)
  • Apache
  • PHP
  • MySQL (back when it WAS MySQL)
  • Shoutcast (I has streaming audio way back then)
  •  and a few dozen other packages I can’t recall now

PHP was the catalyst. Since I built that system, I’ve toyed with other languages. I’ve even become proficient enough with JavaScript to get web pages working. However, I’ve never seen the appeal of JavaScript on the server. PHP does the tasks I need done without an issue. (FTR, I felt the same way about Server Side JS the first time I saw it back in ’97ish? What? You thought this was a NEW idea? :) )

PHP is a solid language, it has a great manual, and an awesome community. Those three things raise the bar for switching pretty high.

Thank you to everyone who has contributed to PHP

  • Thank you to all of the core contributors
  • Thank you to all the people who work on the manual
  • Thank you to everyone who runs a PHP User Group
  •  Thank you to everyone who contributes to the PHP community

I owe so many people so much that I can’t even begin to start naming names. In the spirit of the PHP community, I am now, and will continue to, pay it forward each and every day.

Until next time,
I <3 |<
=C=