Skip to content

Mautic Step 2 – Cron Jobs

Dear Reader,

This is the third post on “My Journey into Mautic”. I’m getting Mautic – a Marketing Automation Tool – setup and running on my own server. Along the way I’m sharing what I’ve learned in hopes that it helps someone, or at least helps me the next time I go down the road.

Maintaining Mautic from the command line

This time we are talking about the cron jobs necessary to make Mautic run. Mautic has several commands that are necessary to execute that are not web based. They are run from the command line manually (dumb idea) or using a scheduler like cron on Linux. As with my “Installing Mautic” post, this post is only interesting to those of you self-hosting Mautic.

There is a great manual page on this titles “Cron Jobs”. It tells you a lot of what I’ll tell you here. If you haven’t read it yet, I highly recommend you start there.

The jobs

There are 4 main cron jobs that need to run. All of them are commands build on top of the symfony/console package. Symfony/console is my favorite package to install from packagist. Most of my projects are run form the cli and I use symfony/console for all of them. The heart of the Mautic console commands is app/console. So all of the commands look like this:

$ php /path/to/mautic/app/console mautic:command

You need to locate the app directory on your installation and adjust the examples accordingly.

The 4 jobs that you need to run regularly are:

  • mautic:segments:update
  • mautic:campaigns:rebuild
  • mautic:campaigns:trigger
  • mautic:messages:send

The last one is necessary if you are using Mautic to send out marketing campaigns. Currently, I am not, I’m still fighting with the MailChimp integration in hopes of making that work instead. However, I have this one enabled anyhow.

My current install for Mautic is very low volume. I think there are currently 10 contacts in the DB. Even so, the manual recommends that you stagger the jobs so that they don’t all run at once. They recommend the following.

0,15,30,45 <— mautic:segments:update
5,20,35,50 <— mautic:campaigns:update
10,25,40,55 <— mautic:campaigns:trigger

The problem I ran into

All of this is pretty standard stuff if you are familiar with Linux and crontab. There is one thing that bit me in the butt, and it’s the reason I am blogging this, permissions.

Originally, I had all of those setup in my crontab. My “my crontab” I mean the cron tab for my personal account on the server. Since I am a member of the group that the web server runs under – mine is not a shared hosting environment, my sites are the only sites on the server – so I should have been able to do everything, right? Nope. The thing I forgot is that most everything is created with 0755 permissions. This means I can see the files created by the web server, but I can’t modify them. This hits you pretty quick when running any of these commands form any accounts that doesn’t have write permissions, because they all write to the app/logs/*.log files.

The obvious solution was to simply move them to the crontab of the account that my web server runs under, right? Yep, expect that that account has it’s shell set to nologin. This means that I can’t easily test them, or run them as a one-off for example clearing the cache. All system accounts are setup this way to adhere to the Best Practice Principal of Least Privilege.

The answer is yes, to add them into the account that the web server runs under, but you have to add the following to the beginning of the crontab.

SHELL=/bin/bash

NOW at least the cron jobs can run without a problem. I still have to jump through a couple of hoops to clear the cache.

Conclusion

Permissions will bite you in the butt every time on *nix systems. Be careful with them. The answer is almost never :

$ chmod -R 777 *

There is usually a way to get things done that don’t compromise your application’s security.

If you a reading this thinking “Well D’uh!” Again, this is more to remind me for next time than anything else. :) If it helps you along the way, I’m happy.

Until next time,
I <3 |<
=C=

Mautic Step 1 – Configuring an Email Service Provider

Dear Reader,

NOTE: If you are running the hosted version of Mautic, this post probably doesn’t apply to you. You are welcome to anything you find useful here, but my entire series is for those hosting their own Mautic instance.

This is the second post in a series titled “My Journey into Mautic”. If you are starting here, you might get an incomplete picture, you may want to check out the previous articles.

There are two things that really confuse me about Mautic, properly configuring an Email Service Provider (ESP), and segmenting & tagging. We’ll tackle the latter one in a later post, but the former is an important topic. It is also one that I do not fully understand. What is presented here is what I have learned through trial and error. it my be partially or wholly incorrect. If you find something that I’ve gotten wrong, please, by all means, correct me in the comments.

What is an ESP?

An ESP is an Email Service Provider. What once was just a service running on a server, is now a service you can pay for instead of having to manage yourself. Trust me, it’s worth it to pay for this one. :)

I ran my own mail server for around 15 years. I understand exactly how difficult it is to manage a secure and well maintained email server. I finally decided I had had enough and moved all of my personal email over to Google Apps. Life has been so much simpler since then. :) The only problem that I have run into is that many of my programs – or other websites running WordPress – need to be able to send email. It also helps f they can receive email as well, but that’s not imperative to me.

After testing out several good solutions, I settled on MailGun by RackSpace for all of my “transnational emails”.

Transactional vs. Broadcast emails

Transactional emails are any emails that your system generates that are not “broadcast” emails. Broadcast emails are things like mailing lists, etc where you send the same message out to a bunch of email addresses all at once. MailGun is not a good solution for that. For those types of emails, I use MailChimp. There are other good providers out there but I’ve settled on MailChimp.

Trasnactional emails are everything from receipts sent from an eCommerce shop to lost password emails. In the case of Mautic, transactional emails are marketing emails, but they are not “broadcast” emails as described above. In a traditional drip campaign, the emails are sent one at a time. Depending on the popularity of your campaign, you might be sending a lot of them one at a time, but in the end they are still one at a time. These are still transactional emails.

Sending and Receiving

Since I already had MailGun setup and operational I wanted to stick with it. Unfortunately, Mautic does not support MailGun‘s API out of the box. It does however support “Other SMTP” which allows me to still use MailGun. I would prefer to use the API over SMTP because with SMTP, I have to open another port in my firewall. However, it is still better than trying to setup ANOTHER ESP, just for Mautic. Hopefully, Mautic will support MailGun‘s API in the near future.

Out of the box, Mautic supports the following ESP vendors:

  • Amazon SES
  • Elastic Email
  • GMail *
  • Mandril
  • Mailjet
  • Other SMTP Server **
  • PHPMail ***
  • PostMark
  • Sendgrid
  • Sendmail
  • SparkPost

* Gmail is not an API, you give it your Gmail’s account credentials and it will send from that.
** Same as Gmail, except that you have to define all the different parameters yourself.
*** PHP’s Built in mail() function.

If you use one of those ESPs for sending, Mautic is good. If you don’t, you will need to configure everything. The good news, is that it is not terribly difficult to guess the right values to use. The bad news is, yes, there is a little guess work involved.

Step 1 – Firewall

Since I am running Mautic self hosted, and using a non-standard ESP, I have to open up the proper ports on my firewall before anything can be done. This of course did not occur to me until after I had properly configured Mautic and then tinkered with it for half an hour to try and get it to work. :) So if you are self hosting, and you are using a non-standard ESP, make sure you configure your firewall first. In my case, this mean

Step 2 – Configure Send

Now it’s time to tell Mautic to use MailGun to send things. Here is what the screen looks like.

Mautic email sending options screen shot

  • Name to send mail as – This is pretty obvious but in case it is not, this is the default
  • Email address to send mail from – Similar to name, this can be overridden if you want, but this is the default.
  • Custom return path (bounce) address – I believe this is the email address you want bounces to go to. I am not entirely sure and since I want bounces to be processed normally, I left it blank.
  • Mailer is Owner – This is where you override the first to, I think. Since my setup only has a single user, it is easy for me to just leave this off and use the defaults.
  • Service to send mail through – Pick from the list above. Since mine isn’t in the list, I chose other.
  • SMTP host – You get this from your ESP. Note, if you are using a supported ESP and they have an API, your choices will be different. Usually for those, you’ll just enter in your API credentials.
  • SMTP encryption – None, SSL or TLS. Check with your ESP. HINT: None should never be your option. If it is,
    find a better ESP.
  • SMTP port – This will vary based on your encryption choice.
  • SMTP authentication mode – This determines how your let your SMTP server know who you are so it can determine whether you are allowed to send mail through it. Your choices are “None”, “Plain”, “Login”, or “Cram-MD5”
  • User name and password – Since I am using SMTP and I told it to use Login, these fields allow me to provide the username and password for MailGun.
  • How should email be handled – This one I guessed at. Since I almost always want my emails to be sent immediately when a customer fills out a form, I set it to immediately. I’ll be honest in saying that I don’t know the ramifications – if any – of this decision. The other option is “Queue”. My assumption is that if Queue is selected, then the cron jobs (another post) will deal with it.

Step 3 – Configure Receive

Mautic email receiving options screen shot

Here is where I get confused. Also where I think using an ESP with a supported API might make things easier. Then again, maybe not since most transactional ESPs don’t support receiving. Still, this part confuses the heck out of me. Apparently, here we have to configure an incoming ESP. MailGun handles this for me by use of aliases, but here, Mautic wants an actual email address and access to it. Then – if I understand it correctly – you have to configure folders for bounces and unsubscribes.

I have a couple of questions that I can’t seem to find the answers to.

  1. Do I have to set the email address of this box up in Custom return path (bounce) address?
  2. Do I have to setup the separate folders?
  3. Is it considered a best practice to set them up?
  4. Is there a size threshold (e.g. 2+ users where it becomes important to set the folders up?
  5. Exactly what are the folders for?

In the end, I used a GMAIL I setup just for this purpose. I left the folder set to INBOX on both of them. It seems that you can also setup separate email addresses for bounces and unsubscribes. I used the single default settings for this. However, I am sure on larger installations, it would be a good idea to have different email addresses for each.

Conclusion

As I said at the start, I’m sure I got some of these wrong. I look forward to those with a lot more knowledge correcting me so I can correct this post…and my Mautic installation. In setting up Mautic, setting up an ESP was the most confusing and time consuming.

Part of my confusion was of my own making. Since MailGun wasn’t supported, I decided to spend some time looking at other ESPs. Of all the ones I looked at PostMarkstood out as exemplary. Honestly, the only think that kept me from implementing them is I am cheap and MailGun gives me ten thousand sends/receives a month. PostMark only gave me twenty five thousand, total free. By my calculation, that’s about a years worth of free service for me, so it’s not a bad deal at all. Also, the pain of switching everything over or running two separate ESPs was enough to keep me where I was. If you haven’t selected an ESP, give PostMark a look.

If you can make it through this, the rest of the configuration is a breeze. :)

Until next time,
I <3 |<
=C=

Mautic Step 0 – Installing Mautic

Dear Reader,

Last time, in the post titled “My Journey into Mautic”, I explained that I was starting to play with the marketing automation package Mautic. Today’s post is about installing Mautic.

This post is not necessary for everyone. If you are using a hosted version of Mautic, it is probably safe to ignore this post. Even if you are hosting it with a provider like SiteGround or GoDaddy, the first part of these instructions won’t apply. You can skip the “Foundation” if you installed Mautic from a control panel and go straight to “Running the Mautic Installer”.

The Foundation

I use Digital Ocean as my hosting platform. For small businesses, Mautic WILL run on a $5/month droplet. However, I strongly suggest that if you are going this route to invest a little more and get at the minimum a $10/month droplet.

You will need to install:

  • Apache (or the web server of your choice)
  • PHP
  • MySQL/MariaDB

Additionally, Mautic makes this recommendation.

Mautic recommended configuration options.

You can also run Mautic in a sub-directory of your existing website. If you already have a website that is based on the LAMP stack then Mautic will either run without problems or it will tell you what the problem is and let you correct it before you install.

If you do not have a website, Mautic will not do you any good. Start with my friends over at SiteGround and get a WordPress installation up, running, and completely fleshed out before you even think the word Mautic.

Installing Mautic

Once you have your hosting platform up and ready to go, download and unzip Mautic into the proper directory. The proper directory will depend on how you decided to install it. If you decided to install it as a sub-domain, you will unzip Mautic into the root directory of the sub-domain. If you decided to install it in a sub-directory of your main domain, you will need to unzip it into that sub-directory.

Database

Before you begin, you will need a blank database for Mautic to use. Connect to your MySQL/MariaDB service with your favorite tool and create a database, and a user. Make sure you give the user access to the newly created database but ONLY access to that one database.

DO NOT us an existing database. Create a new database. if your hosting provider won’t let you create another database, get a better hosting provider. Start with a clean and empty database.

DO NOT user your root db or a root equivalent user account. Create a new account just for this database. Give it all the permissions except for GRANT.

If you are using a hosting provider with a tool like cPanel, you can use cPanel to create the database, and the user account.

Running the Mautic Installer

Installing Mautic is very straight forward. If you have your web server setup properly then you just point a browser to wherever you unzipped it and BOOM, it starts the install process.

Mautic install screen number one. This screen tells you if your platform is sane. it also makes optional recomendations.

This first screen tells you if your platform is sane. You can see from the green “Ready to Install” that Mautic likes my setup. However, the yellow section makes a few recommendations for how it could be better. Since I manage my own server, it was easy enough to fix the issues. If you are running on a platform where you do not have root access, it might not be as easy for you to fix them. In that case, I would recommend working with your hosting provider to see if they can resolve any or all of the recommendations.

Clicking “Next Step” takes you to the second screen.

This is the second Mautic install screen. This is where you enter all the information about your database.

On this screen, you enter all the information that Mautic needs to connect to your database server. If you have installed applications on a web server before (e.g. WordPress) then this will be familiar to you.

 

The only two things that might not ring a bell on this screen are:

  • Backup Existing Tables/Prefix for backup tables.
    This is set by default to YES. Since you are installing into a new database, it really makes no difference at all. I turn it to no as there should not be any existing tables. Either way, since no backup tables will be created, the prefix for the backup tables is irrelevant. It is safe to ignore this.
  • Database Table Prefix
    This is blank but I highly recommend you putting something here. three random letters and an underscore is plenty. This is a security feature. Anyone who reads the code for Mautic knows what the table names are. This gives an attacker a piece of information they can use in their favor. If you put a random prefix on the tables, it just makes it a little harder for an attacker to compromise your system. (Security s like an onion, it is made of of layers)

Clicking “Next Step” takes you to the screen where you will create the admin user for Mautic.

 

This screen will let you enter in the Mautic admin user name and password. Since this account will have God like powers in your application, do not simply reuse a long and password that you use for other websites. Make the user name something meaningful and use a password manager to create a strong password.

Once you have filled this form out, click “Next Step” to take you to the “Email Configuration” screen.

This is the Mautic email configuration screen. This is where you setup your Email Server provider information.

Enter your name and email address here. Don’t worry about the rest, we are going to properly configure all of this in a future post. Honestly, I’m not really sure why this information is asked for here. I’m sure there is a good reason though.

Success!

Mautic log in screen

If all goes well – and honestly, the installer is designed so well that if something is not going to go well, it won’t let you get to this point – then the next thing you see is the Mautic log in screen. Go ahead, enter your email address and password, you’ve earned it.

As you can see, even from scratch, Mautic is very easy to install. It rivals the WordPress “5 Minute Install” for ease of use and completeness.

Next, we will talk about Email Service Providers.

Until next time,
I <3 |<
=C=

My Journey Into Mautic

Mautic logoDear Reader,

Those that know me know that I have an obsession with marketing. I mean I’m no good at it, but the topic fascinates me. Almost all of the podcasts I listen to on a regular basis are marketing related. One topic in particular that interests me is “Marketing Automation”. Marketing Automation covers a huge swath of topics and since I am not an expert at the, I won’t attempt to explain them. However, three things that are covered by MA that I understand reasonably well are:

  • Lead Generators
  • Landing Pages
  • Email Marketing Campaigns

Even here we have topics so broad that entire books have been written on each of them. Still, these thee topics represent the heart of what is called “Inbound Marketing”.

Inbound Marketing is you trying to convince people to come to your site and buy/join your mailing list. This is as opposed to Outbound Marketing where you contact potential customers directly and try to convince them to buy.

Introducing Mautic

Because I am interested in Marketing Automation and want to start applying the techniques in the projects I run.

I started looking around for vendors who could provide these services. What I found is that most SaaS vendors assume that everybody who wants to use their software has deep pockets.

Side Note: I had a real interesting interview with someone form PostMark this past week after I tweeted that I did not choose them but chose Mail Gun. The subject of price came up and my words to him were “Yes, it’s only $15/month. However, right now I’ve got 7-8 companies wanting just $15-$25 per month. It all ads up quick.

During my research into solutions that may or may not work but I couldn’t afford to try, I cam across a project called Mautic. Mautic had three major things going for it right away.

  1. It is Open Source
  2. It is written in PHP
  3. One of the leads at Mautic is a friend of mine, Don Gilbert

Wow! To me, a long time PHP developer, this was a home run. I began digging deeper into it.

  • It integrates into WordPress, my tool of choice for building websites.
  • I can host it myself. (This is probably more important to me than others. My reasons are partly technical and partly political.)
  • I can contribute back to the project.

So we have a winner and I was able to give a big raspberry to all the other SaaS vendors who wanted me to pony up each month. Well, that’s what I thought at least.

As it turns out Mautic – while it it is most of the things I said – is still open source software. This means that development is at the whim of contributors that have other priorities. This meas that there are problems with Mautic that will get fixed when they are a problem for someone with the knowledge and time to fix them. While this is ok for me because it is possible for me to dive in and fix things if they reach a level of importance to me, it’s probably a downside to most non-developer users.

Where to go from here?

Despite some obvious flaws and at least one huge show stopping bug, I see a bright future for Mautic. So I’m going to invest my time in getting it setup and running. I’ve already run the install twice and I’m happy with the results the second time.

Along the way, I am going to blog what I learn. This is both for me so I can reference it later, and to help anyone else who is working with Mautic.

My setup will be:

Most of what I do can be done without having to worry about hosting your own copy. I am doing it this way so that I can integrate Mautic into my existing infrastructure. So don’t worry if you aren’t a programming, you can still learn from my mistakes. :)

Along the way, I will get things wrong – my definitions above may already be wrong. Leave me a comment and correct me. I’m not claiming that I know what I’m doing. I’m just saying that I’ll tell you what I’ve done, and what I’ve learned.

Posts

Until next time,
I <3 |<
=C=