We previously discussed how to get everything you needed to start configuring a high performance LAMP stack with caching

Today we will describe how to configure each of the elements so they work together for the ultimate in performance and security in a LAMP stack.  In particular, we discuss the following technologies:

This tutorial will show you how to take what you previously learned about installing Varnish, Pound, PHP-FPM, Acmetool, Pound, and Apache and how to make them all work together nicely.  Please understand, while this will work for most, you may need to adapt it to work for you.  Never copy and paste verbatim from a howto guide, as you will end up with unexpected results and no idea how it happened.  You should understand what you are copying and pasting into your configuration before doing so.  Blind copy and paste is bad!

Today, getting all these technologies to work together so you can get a coherent and operational system is the goal.  This howto concludes you have everything installed on your system already, and only need to configure each individual component to work.  If you have yet to install the required software on your server, you can read our previous post on how to install the components, then come back here after to finish the configuration.

Quick note on sudo:  The commands we run do not have "sudo" attached to them because we are in fact running our commands as root.  This is bad in almost every case out there.  While outside the scope of this tutorial, you should always run sudo and never use root.  (For those really curious, we run as root because we have all Virtual Private Servers on our own dedicated server, and each virtualized server runs under root as default)

Quick note on editing files:  When we edit files, we use Vim.  You are welcome to use any editor you want, but if you choose to follow along with Vim, here is a quick Vim Crash Course:

Press Insert on your keyboard to enter edit mode
Ctrl-C to exit edit mode
:w (collon w, no space) will save
:q (collon q, no space) will quit
:wq (collon wq, no spaces) will write and quit)

Setting up IPTables

Let's start by making sure our firewall is not going to block any of our attempts to get things working.

Start by issuing the command:

cat /etc/sysconfig/iptables

This should render output similar to the following:

# sample configuration for iptables service
# you can edit this manually or use system-config-firewall
# please do not ask us to add additional ports/services to this default configuration
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited

This is the default settings for most basic installs, where port 22 is left open (ssh) and everything else is allowed only if it's established first (e.g. outgoing traffic makes a connection first, the incoming traffic related is allowed)  We want to change these settings a little bit so our web server will be accessible.  To do this, open the file:

vim /etc/sysconfig/iptables

and add add the following lines:

-A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT

Place them either right before or right after the rule for port 22 SSH.  What these two lines do is open port 80 (default web traffic) and port 443 (secure / https traffic) on the built in firewall.  Without these two lines, we won't be able to get any traffic to our webserver.  Once we have made these changes, it's important to restart your firewall to make sure the new rules apply.

Issue the following command to reload your firewall:

systemctl reload iptables

We also want to make sure our new rules apply even if the server gets restarted for whatever reason.  To do this, we need to make sure the iptables service is enabled.  This is done by issuing the following command:

systemctl enable iptables.service

To check that the ports are in fact open in the firewall, you can look at your current rules like this:

iptables -L -v

And the end result should look similar to this:

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
  254  278K ACCEPT     all  --  any    any     anywhere             anywhere             state RELATED,ESTABLISHED
    0     0 ACCEPT     icmp --  any    any     anywhere             anywhere
   16   960 ACCEPT     all  --  lo     any     anywhere             anywhere
    1    52 ACCEPT     tcp  --  any    any     anywhere             anywhere             state NEW tcp dpt:http
    4   232 ACCEPT     tcp  --  any    any     anywhere             anywhere             state NEW tcp dpt:https
    0     0 ACCEPT     tcp  --  any    any     anywhere             anywhere             state NEW tcp dpt:ssh
   11   452 REJECT     all  --  any    any     anywhere             anywhere             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 REJECT     all  --  any    any     anywhere             anywhere             reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT 312 packets, 333K bytes)
 pkts bytes target     prot opt in     out     source               destination

What you are looking for is the rule in your INPUT Chain for port 80 and 443.  Since these are known services to the server, it automatically makes port 80 show at http, port 443 at https, port 22 as ssh, etc.  So we can see that the following two lines are now showing:

    1    52 ACCEPT     tcp  --  any    any     anywhere             anywhere             state NEW tcp dpt:http
    4   232 ACCEPT     tcp  --  any    any     anywhere             anywhere             state NEW tcp dpt:https

This means that http and https are allowed in, and your web server is now able to receive traffic.  Time to move on to the web server and get that working.

Setting up Apache 2.4 Web Server

Now that we have the ports open to let traffic get to our web server, we might want to actually configure it to turn on and be functional.  A few quick steps we need to do after installing (previous post) is enable the server, and make sure it turns on after reboots.  Start with enabling the service like this:

systemctl enable httpd.service

Now that the Web Server is enabled, let's turn it on to start receiving requests like this:

systemctl start httpd.service

If there are no errors, you should see a wonderful testing page by navigating in a web browser to your server IP address.  The page should be similar to the following image:

Default Apache Testing Page in CentOS

If you see this image, that means the web server is working (as the message indicates, obviously).  If you are having issues locating your server ip address, you can try issuing the following command to get your IP Address:

ip addr show eth0 | grep inet | awk '{ print $2; }' | sed 's/\/.*$//'

You should get something back that looks similar to this (note, the x was added to the IP address for security reasons, your results will show a complete ip address):

Anything that doesn't say is your IP address.  If for any reason you type this command and see an error saying that "eth0" does not exist, you are likely on a VPS that is configured with a different network card.  You can still locate your ip address by typing the same command without the show eth0 piece.  Try this:

ip addr | grep inet | awk '{ print $2; }' | sed 's/\/.*$//'

This command will give you a few more numbers, and maybe even one that looks like ::1 (which is IPv6), again you are looking for the ip address that is not

Once you've found your ip address, you can test your web server setup by opening your browser and navigating to:


Setting up Percona Server

In our previous post we installed Percona Server, and now we need to get it set up completely.  By default since MySQL 5.5 Percona has automatically been assigning a temporary root password instead of leaving it blank for you to setup later.  In order to configure Percona, we need that password so we can change it, as well as administer all of the databases.  To find the password that was assigned during install, try issuing the following command:

grep -r 'temporary password' /var/log/mysqld.log

This should output something very similar to this:

2017-08-21T14:49:59.572001Z 1 [Note] A temporary password is generated for root@localhost: md7cvAue;4/N

While we would never remember that password, you need it to change the password.  The easiest way to set up Percona and a new root password is using mysql_secure_installation.  In order to use this tool, Percona needs to be running.  Start by issuing the following command:

systemctl start mysqld.service

Using this command starts mysqld (Percona) and then we can issue the mysql_secure_installation command, allowing us to change our password, remove anonymous tables, and other cleanup tasks that we should do before using a production server anyway.  The answer to every question is yes (capital Y), but please make sure you read what it's asking you before blinding saying yes or no.  There may be use cases where you want to say no, or future releases will have other questions which you want to make sure you are answering properly.

[root@binarycpu ~]# mysql_secure_installation

Securing the MySQL server deployment.

Enter password for user root:  <ENTER TEMPORARY PASSWORD HERE>
The 'validate_password' plugin is installed on the server.
The subsequent steps will run with the existing configuration
of the plugin.
Using existing password for root.

Estimated strength of the password: 100
Change the password for root ? ((Press y|Y for Yes, any other key for No) : Y


Re-enter new password: <ENTER NEW PASSWORD HERE>

Estimated strength of the password: 100
Do you wish to continue with the password provided?(Press y|Y for Yes, any other key for No) : Y
By default, a MySQL installation has an anonymous user,
allowing anyone to log into MySQL without having to have
a user account created for them. This is intended only for
testing, and to make the installation go a bit smoother.
You should remove them before moving into a production

Remove anonymous users? (Press y|Y for Yes, any other key for No) : Y

Normally, root should only be allowed to connect from
'localhost'. This ensures that someone cannot guess at
the root password from the network.

Disallow root login remotely? (Press y|Y for Yes, any other key for No) : Y

By default, MySQL comes with a database named 'test' that
anyone can access. This is also intended only for testing,
and should be removed before moving into a production

Remove test database and access to it? (Press y|Y for Yes, any other key for No) : Y
 - Dropping test database...

 - Removing privileges on test database...

Reloading the privilege tables will ensure that all changes
made so far will take effect immediately.

Reload privilege tables now? (Press y|Y for Yes, any other key for No) : Y

All done!

The above changes our root password from that really awful and hard to remember password to our own root password that we assign, gets rid of anonymous users, disallows root to login anywhere besides locally, gets rid of any testing databases, and reloads privileges so any changes we did happen immediately.  After these changes, a stock Percona Database Server is ready to go.  Let's get it enabled and set up to turn on at boot time.

systemctl enable mysqld

Configure PHP-FPM

PHP-FPM is a great replacement for the stock Apache PHP handler.  It allows for better handling in general of events that could otherwise potentially cause your webserver (Apache) to crash and not recover without intervention.  Per their own website, they state:

PHP-FPM (FastCGI Process Manager) is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites.
-  php-fpm.org

Notice the emphasis on "busier sites" or if you want to read into it, "better performance".  So let's get PHP-FPM working with Apache.

First we need to edit the /etc/php-fpm.d/www.conf:

vim /etc/php-fpm.d/www.conf

You are going to look for the line that says "listen =" and make sure it's not commented out with a semicolon in front of it.  You also want to make sure that the user and group are set to apache, as well as the listen.owner and listen.group being set to apache, with listen.mode set to 0660.  The end result should look similar to this:

; Unix user/group of processes
; Note: The user is mandatory. If the group is not set, the default user's group
;       will be used.
; RPM: apache user chosen to provide access to the same directories as httpd
user = apache
; RPM: Keep a group allowed to write in log dir.
group = apache

; The address on which to accept FastCGI requests.
; Valid syntaxes are:
;   'ip.add.re.ss:port'    - to listen on a TCP socket to a specific IPv4 address on
;                            a specific port;
;   '[ip:6:addr:ess]:port' - to listen on a TCP socket to a specific IPv6 address on
;                            a specific port;
;   'port'                 - to listen on a TCP socket to all IPv4 addresses on a
;                            specific port;
;   '[::]:port'            - to listen on a TCP socket to all addresses
;                            (IPv6 and IPv4-mapped) on a specific port;
;   '/path/to/unix/socket' - to listen on a unix socket.
; Note: This value is mandatory.
listen =

; Set listen(2) backlog.
; Default Value: 65535
;listen.backlog = 65535

; Set permissions for unix socket, if one is used. In Linux, read/write
; permissions must be set in order to allow connections from a web server.
; Default Values: user and group are set as the running user
;                 mode is set to 0660
listen.owner = apache
listen.group = apache
listen.mode = 0660

Make sure to save this file and then start and enable PHP-FPM like this:

systemctl enable php-fpm.service
systemctl start php-fpm.service

In order for PHP-FPM to work with Apache, we need to make a few changes over on the Apache side.

Let's first enable our PHP-FPM proxy in Apache.  The best method is to make a custom.conf file for anything we are adding, so we can clearly see where we make changes and don't have to hunt through lines of code and guess what we may have done.  Start by making the new file:

vim /etc/httpd/conf.d/custom.conf

And then inside we add our ProxyPassMatch rule:

<IfModule proxy_module>
ProxyPassMatch ^(.*\.php)$ fcgi://$1

The first line with ProxyPassMatch passes any request to a .php file to the proxy running (in our case PHP-FPM)

Important: If you have changed the location of your web root, please be sure to change the above from /var/www/html/$1 to /your/web/root/$1

Save the file and restart Apache:

systemctl restart httpd.service

Apache is now configured to use PHP-FPM for PHP processing.  We should see immediate performance gains by doing this simple step alone.

Configure Varnish

Since you have already visited the website, and we know Apache works (right?) we can now place Varnish in front of it so we can get some high performance caching going on!  By default, it listens on port 6081, which is fine if we need to do some crazy forwarding rules for cached SSL (yes, you can serve cached content over SSL so it's not rediculously slow).  We aren't going to dive into that quite yet.  For now, let's get Varnish setup to be the default listening server instead of Apache.  To do that, edit your varnish configuration at /etc/varnish/varnish.params like this:

vim /etc/varnish/varnish.params

and change your listening port to 80, so you have a final configuration looking similar to this:

# Varnish environment configuration description. This was derived from
# the old style sysconfig/defaults settings

# Set this to 1 to make systemd reload try to switch VCL without restart.

# Set WARMUP_TIME to force a delay in reload-vcl between vcl.load and vcl.use
# This is useful when backend probe definitions need some time before declaring
# configured backends healthy, to avoid routing traffic to a non-healthy backend.

# Main configuration file. You probably want to change it.

# Default address and port to bind to. Blank address means all IPv4
# and IPv6 interfaces, otherwise specify a host name, an IPv4 dotted
# quad, or an IPv6 address in brackets.

# Admin interface listen address and port

# Shared secret file for admin interface

# Backend storage specification, see Storage Types in the varnishd(5)
# man page for details.

# User and group for the varnishd worker processes

Then we need to change the default.vcl file to point to the correct backend.  To do this, first open /etc/varnish/default.vcl like this:

vim /etc/varnish/default.vcl

Now locate the section that talks about the backend, specifically the port setting.  You should have something looking similar to this:

backend default {
    .host = "";
    .port = "8080";

While normally this would be correct and work, we need to make sure we have a way to validate our SSL Certificate without restarting our server every time.  To do this, we need to add directors so we can setup multiple backends.  This will allow Varnish to listen and push traffic to our Apache server, as well as listen and respond to Verification Requests on a seperate internal backend within Varnish.  Start by checking the following 3 lines exist at the top of your default.vcl file:

vcl 4.0;

import directors;
import std;

After these lines are added if they do not already exist, we need to change our backend setup a little bit.  Your new backend setup should look similar to this:

backend sitea {
  .host = "";
  .port = "8080";

backend acmetool {
   .host = "";
   .port = "402";

We are using the name sitea for our regular website backend, and acmetool is a good name to remember what we are redirecting to port 402.

Make sure to change the port to 8080 for "sitea" if it does not currently say 8080.  This is important, because it tells Varnish where to look for Apache.

Now we want to add another few lines to handle the new sites we created.  To do this, locate (or create) sub vlc_init and make sure it looks similar to this:

# Define the director that determines how to distribute incoming requests.
sub vcl_init {
   new bar = directors.fallback();

#EXAMPLE:  If you wanted to add multiple backends, use the following logic:
#    bar.add_backend(siteb);
# If you have multiple backends, you can add multiple sites here for load balancing
# Be sure to define a backend before attempting to call the backend in this section

You can remove the comments in the actual VCL file if they do not apply to you.  Keeping them in the VCL will not impact performance.

Then at the very beginning of sub vcl_recv add the following:

if (req.url ~ "^/.well-known/acme-challenge/") {
        set req.backend_hint = acmetool;



This line will intercept any request to /.well-known/acme-challenge/ which is where we will configure acmetool to listen to proxy requests.  This passes the request directly to port 402 which we configured above in our backends.  In other words, no down time and instant verification of your site to get certificates issued.  You can even set a cron task to automate the entire renewal process so you never have to think about SSL again.  That is outside the scope of this tutorial for the moment.

Now we want Apache to listen on port 8080, or things will start to break.  To do that, open /etc/httpd/conf/httpd.conf like this:

vim /etc/httpd/conf/httpd.conf

Locate the line toward the top that says Listen 80 and change it to Listen 8080.  The end result will look like this:

# Listen: Allows you to bind Apache to specific IP addresses and/or
# ports, instead of the default. See also the <VirtualHost>
# directive.
# Change this to Listen on specific IP addresses as shown below to
# prevent Apache from glomming onto all bound IP addresses.
Listen 8080

Save the file and exit.  Now we will restart Apache and start Varnish and see if everything still works.  To do this, issue the following commands:

systemctl restart httpd.service
systemctl enable varnish.service
systemctl start varnish.service

If you didn't get any errors, you should now be up and running with Varnish sitting in front of your Apache Web Server.  You won't be able to visit your website until you get Pound setup and pointing traffic where it needs to go.

Configure Pound

Pound is an amazing reverse proxy for HTTP and HTTPS traffic.  Since the proxy is able to handle both regular and secure traffic, we can do redirects at the proxy level to force HTTPS everywhere with no performance loss or added overhead of hitting the web server, then back to the proxy again.  Pound is a popular, if not misunderstood piece of software that simply works.  It can handle thousands of connections without an issue, and performance is outstanding.  In the years we have used Pound, we found the lack of documentation to be the major downfall to using Pound.  BUT, when you do find the right answers (trial and error for us) you'll find it works wonders.  Lucky for you, we are giving you some of that documentation.

Start off by opening the Pound configuration file location at /etc/pound.cfg like this:

vim /etc/pound.cfg

We want to change two important sections:

         Port    80
                       Port    6081

This first directive makes it so any incoming connections on port 80 will be redirected to our Varnish Cache listening at port 6081.  Make sure you set the Address to your server's external IP address.  The second change we need to make is to HTTPS traffic.

         Port    443
         Cert    "/etc/ssl/local.server.pem"
                      Port    6081

What this does is listens on port 443, and then directs any of the secure traffic to our Varnish Cache listening on 6081 as well.  In other words, our SSL traffic is going to our Varnish Cache and getting all the speed improvements while on SSL.  Save and exit, then enable and start Pound like this:

systemctl enable pound.service
systemctl start pound.service

With any luck, you can now visit your website.  Check that both HTTP and HTTPS works.  Note:  HTTPS will give you a warning about the SSL Certificate not being valid.  This is okay, we are only testing that HTTPS works, not that the certificate is valid.  Once we switch out the self signed SSL Certificate for our free LetsEncrypt certificate, that error will go away.

Configure LetsEncrypt SSL

LetsEncrypt is a no excuses method of getting free SSL for almost any application.  In our case, we need to get SSL working with Pound.  Please make sure you have your DNS properly setup prior to attempting any of the SSL configuration, or it will likely fail.  The quickest way is to run Acmetool with the quickstart method selected like this:

acmetool quickstart

This will want to have you follow a few steps, including providing a contact email address and accepting the terms and conditions.  After you have provided an email and accepted the terms and conditions, it will ask you if this is Live or Staging, and we want a Live Certificate.  it will look similar to this:

------------------------- Select ACME Server -----------------------
Please choose an ACME server from which to request certificates. Your
principal choices are the Let's Encrypt Live Server, and the Let's
Encrypt Staging Server.

You can use the Let's Encrypt Live Server to get real certificates.

The Let's Encrypt Staging Server does not issue publicly trusted
certificates. It is useful for development purposes, as it has far
higher rate limits than the live server.

  1) Let's Encrypt (Live) - I want live certificates
  2) Let's Encrypt (Staging) - I want test certificates
  3) Enter an ACME server URL
> 1

After you select 1 for Live, you are then asked how you want the Challenge to be delivered.  Since we are running a web server on port 80 and 443, we can't use the default method of listening for challenges.  The easiest method would be to setup the Proxy method, as we already setup the web server prior to running this tool.  For us, Option 2 was PROXY.  Select this option.  You should see output similar to this:

----------------- Select Challenge Conveyance Method ---------------
acmetool needs to be able to convey challenge responses to the ACME
server in order to prove its control of the domains for which you
issue certificates. These authorizations expire rapidly, as do
ACME-issued certificates (Let's Encrypt certificates have a 90 day
lifetime), thus it is essential that the completion of these
challenges is a) automated and b) functioning properly. There are
several options by which challenges can be facilitated:

WEBROOT: The webroot option installs challenge files to a given
directory. You must configure your web server so that the files will
be available at <http://[HOST]/.well-known/acme-challenge/>. For
example, if your webroot is "/var/www", specifying a webroot of
"/var/www/.well-known/acme-challenge" is likely to work well. The
directory will be created automatically if it does not already exist.

PROXY: The proxy option requires you to configure your web server to
proxy requests for paths under /.well-known/acme-challenge/ to a
special web server running on port 402, which will serve challenges

REDIRECTOR: The redirector option runs a special web server daemon on
port 80. This means that you cannot run your own web server on port
80. The redirector redirects all HTTP requests to the equivalent HTTPS
URL, so this is useful if you want to enforce use of HTTPS. You will
need to configure your web server to not listen on port 80, and you
will need to configure your system to run "acmetool redirector" as a
daemon. If your system uses systemd, an appropriate unit file can
automatically be installed.

LISTEN: Directly listen on port 80 or 443, whichever is available, in
order to complete challenges. This is useful only for development

STATELESS: Some web servers can be configured to respond to challenges
themselves. This removes any need for interaction with acmetool. See
documentation for web server support.

HOOK: Programmatic challenge provisioning. Advanced users only. Please
see documentation.

  1) WEBROOT - Place challenges in a directory
  2) PROXY - I'll proxy challenge requests to an HTTP server
  3) REDIRECTOR - I want to use acmetool's redirect-to-HTTPS functionality
  4) LISTEN - Listen on port 80 or 443 (only useful for development purposes)
  5) STATELESS - I will configure my web server with my account key
  6) HOOK - I will write scripts to provision challenges
> 2
[=======================================================] 100.00% 0s  --------------------------------------------------------------------------------------------------------------------------------------------------------------]   0.00%

------------------------- Quickstart Complete ----------------------
The quickstart process is complete.

Ensure your chosen challenge conveyance method is configured properly
before attempting to request certificates. You can find more
information about how to configure your system for each method in the
acmetool documentation:

To request a certificate, run:

$ sudo acmetool want example.com www.example.com

If the certificate is successfully obtained, it will be placed in

Once that's done, we can get a new certificate and let Pound know about it.  We can start by requesting a certificate (usually you do this with both www and non www versions so both match).  If your domain was cooldomain.com you would issue the following command:

acmetool want cooldomain.com www.cooldomain.com

The nice thing about acmetool is it will automatically provide us with a combined pem file so we do not need to manually create the combined pem file each time.  The SSL Certificate will be located at /var/lib/acme/live/cooldomain.com/haproxy where "cooldomain.com" in the path will be whatever the FIRST domain name in the want request was.  To make pound aware of this, we want to make a few changes in Pound (including improving the security a bit).  Edit the pound configuration as follows:

vim /etc/pound.cfg

Remember we already setup our HTTPS, but the Certificate was self signed.  We are also using some insecure protocols by default (SSLv2 and SSLv3) which have major security issues.  So to be sure we fix those issues and use the proper secure Ciphers, you'll want to have your file look similar to this:


        HeadRemove "X-Forwarded-Proto"
        AddHeader "X-Forwarded-Proto: https"
        Port    443
        Client 120
        SSLHonorCipherOrder 1
        Disable SSLv3
        Disable SSLv2

        Cert    "/var/lib/acme/live/cooldomain.com/haproxy"
        Ciphers "AES256+EECDH:AES256+EDH:AES128+EECDH:AES128+EDH"

            Port    6081
        Port        8080
        # Go directly to backend if Varnish is down

Notice the addition of the "Emergency" line.  This makes it so if Varnish is down for whatever reason, it will automatically redirect the traffic to your Apache Server directly.  The advantage is that your website doesn't crash completely, but it may slow down quite a bit.  Be sure to save the file and exit, then restart Pound like this:

systemctl restart pound.service

Now Pound is serving the website using a free SSL Certificate from LetsEncrypt, while pushing your secure traffic through the High Performance Varnish Cache Server, then finally getting to Apache which is waiting eager to serve up requests.

If I missed anything

I try my best to be as accurate and complete as possible.  If I have made any errors, by all means please let me know so I can correct it!  I hope this tutorial helps out, and I am always willing to answer any questions you have - simply ask in the forum and I will respond.

Edit: Moved verification of site to Varnish instead of the web server so it will actually work on Proxy Verification (Sorry about the error!)