Testing HTTPS and Certificates with Apache, OpenSSL and tcpdump in a Virtual Network
This article shows how to setup a secure web server in Ubuntu using Apache, including generating server certificates with OpenSSL. This allows for testing HTTPS, e.g. capturing the packets and observing how SSL/TLS works. It is all performed in a VirtualBox virtual network.
Deploy the Virtual Network
Assuming you have virtnet installed and ready to use, create topology 5 using the following commands:
user@host:~$ cd svn/virtnet/bin/host
user@host:~/svn/virtnet/bin/host$ svn update
user@host:~/svn/virtnet/bin/host$ bash vn-createtopology 5
Other topologies could be used, as long as they have at least three nodes and two subnets, i.e. client, router, server. For example, if you have already created topology 7 for the web security attack demos, then you can re-use those nodes, but in the following instructions replace node2 with node3, and replace node3 with node4.
Now start the nodes, by opening a terminal for each and using vn-ssh, e.g. in the first terminal:
user@host:~/svn/virtnet/bin/host$ bash vn-ssh 1
Then in the other two terminals:
user@host:~/svn/virtnet/bin/host$ bash vn-ssh 2
user@host:~/svn/virtnet/bin/host$ bash vn-ssh 3
Setup the Demo Web Site
The demo university grading website will be used. See the detailed explanation of the website and how to set it up. In brief, on node3 run:
network@node3:~$ sudo bash ~/virtnet/bin/vn-deployrealmyuni
[sudo] password for network: network
vn: You are about to deploy the website for MyUni.
vn: This includes web pages, database and PHP code.
vn: Are you ready to proceed? y/n
y
mysql: [Warning] Using a password on the command line interface can be insecure.
mysql: [Warning] Using a password on the command line interface can be insecure.
vn: Deployment of real MyUni complete.
From now on, commands will be run on the nodes, not the host. The prompt indicates the node the command is to be run on.
Test the (Unsecure) Website
You can now test the network (e.g. using ping) and that the website is accessible via HTTP (HTTPS is not yet setup). On node1 use lynx as the web browser and visit the web site on node3:
network@node1:~$ lynx http://www.myuni.edu/grades/
This should take you to the demo grading system on node3. Read more about the demo website features and using Lynx and Firefox to access on the website.
Optionally, you may use tcpdump to capture traffic on the router (node2) to observe the communications between web browser and server (see if you can capture the login username and password).
Secure the Web Server
An important part of HTTPS is the web server having a digital certificate. This will be used by the browser to confirm that it is indeed communicating with the web server, and no-one is performing a man-in-the-middle attack. So we need to obtain a certificate for our web site www.myuni.edu.
For real web servers, it is best to obtain a certificate from a Certificate Authority (CA) that is widely supported across browsers and operating systems (e.g. I have used StartSSL to obtain free certificates). A second, less-secure but quicker option, is to create a self-signed certificate, however this will usually result in warnings/errors being seen by the web browser. You can easily find instructions for doing this with OpenSSL, e.g. via DigitalOcean, Linode, Ubuntu.
For our web site in the virtual network we will use a certificate signed by a Certificate Authority (i.e. our web site certificate will NOT be self-signed), but that CA will not be a widely supported one. In fact, we will use a CA that we can create ourselves. To overcome the issue of warnings presented at the web browser, we will manually load the root CA certificate into the certificate store of node1. In summary, the steps are:
- CA: Create our own Certificate Authority.
- Server: create a certificate signing request, send to the CA, which will return the web server certificate.
- Server: configure Apache to use HTTPS and the web server certificate.
- Browser: load the CA certificate to avoid any warnings.
Create Certificate Authority
In practice, a Certificate Authority (CA) would be an external node. However for this simple demo we will use node3 as both the CA and the actual web server. So now lets setup the CA on node3.
First our root CA needs its own, self-signed certificate. Generate a RSA public/private key pair. Here we generate a 1024-bit RSA private key using a public exponent (e) of 65537. The key is NOT encrypted with DES (or other ciphers).
network@node3:~$ openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:1024 -pkeyopt rsa_keygen_pubexp:65537 -out cakey.pem
Next create a self-signed certificate. Enter the details for your CA.
network@node3:~$ openssl req -new -x509 -key cakey.pem -out cacert.pem -days 1095
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:AU
State or Province Name (full name) [Some-State]:Qld
Locality Name (eg, city) []:Cairns
Organization Name (eg, company) [Internet Widgits Pty Ltd]:CQUniversity
Organizational Unit Name (eg, section) []:Certificate Authority
Common Name (e.g. server FQDN or YOUR name) []:www.cquni.edu
Email Address []:ca@cquni.edu
Now we setup the CA to handle certificate signing requests from other entities (i.e. our web server). OpenSSL uses some default files and directories, which are specified in /usr/lib/ssl/openssl.cnf. Lets create them:
network@node3:~$ mkdir ./demoCA
network@node3:~$ mkdir ./demoCA/certs
network@node3:~$ mkdir ./demoCA/crl
network@node3:~$ mkdir ./demoCA/newcerts
network@node3:~$ mkdir ./demoCA/private
network@node3:~$ touch ./demoCA/index.txt
network@node3:~$ echo 02 > demoCA/serial
network@node3:~$ mv cacert.pem ./demoCA
network@node3:~$ mv cakey.pem ./demoCA/private
network@node3:~$ ls
cert-myuni-ID.csr demoCA lynx.cfg privkey-myuni-ID.pem virtnet
network@node3:~$ ls demoCA/
cacert.pem certs crl index.txt newcerts private serial
Lastly for the CA setup, OpenSSL has strict policies on the details of the CA matching that of the requesting server. For example, it requires the state of the CA and server to be identical. We can change the policy by editing /usr/lib/ssl/openssl.cnf, in particular the "For the CA policy" section. Edit the configuration file:
network@node3:~$ sudo nano /usr/lib/ssl/openssl.cnf
Find the section "For the CA policy". BEFORE it looks like this:
# For the CA policy
[ policy_match ]
countryName = match
stateOrProvinceName = match
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
Now make changes to relax the policy so AFTER it looks like this:
# For the CA policy
[ policy_match ]
countryName = match
stateOrProvinceName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
Now the CA is setup and ready to process certificate signing requests.
Create Certificate for your Website
To create a certificate for the www.myuni.edu website, first generate a RSA public/private key pair. Here we generate a 1024-bit RSA private key using a public exponent (e) of 65537. The key is NOT encrypted with DES (or other ciphers).
network@node3:~$ openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:1024 -pkeyopt rsa_keygen_pubexp:65537 -out privkey-myuni-ID.pem
The output file (privkey-myuni-ID.pem) is plaintext. It contains the private key, encoded as Base64, in between two lines indicating the begin and end of the key.
Next create a certificate request that will be sent to the Certificate Authority. This takes a private key as input (i.e. the file generated above) and produces a .csr certificate request file as output. This is a new certificate request.
network@node3:~$ openssl req -new -key privkey-myuni-ID.pem -out cert-myuni-ID.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:AU
State or Province Name (full name) [Some-State]:NSW
Locality Name (eg, city) []:Sydney
Organization Name (eg, company) [Internet Widgits Pty Ltd]:MyUni
Organizational Unit Name (eg, section) []:ENTER
Common Name (e.g. server FQDN or YOUR name) []:www.myuni.edu
Email Address []:ID@myuni.edu
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:ENTER
An optional company name []:ENTER
The value of Common Name MUST be the domain of the website, e.g. www.myuni.edu. The other values may be different, depending on the policy of the OpenSSL CA.
Send your certificate request file to the CA. Since in this demo both the server and CA are on node3, there is no actual sending (the file already is available to the CA).
Now the CA processes the certificate signing request using the following command. Make sure all the file names are correct and the certificate is successfully committed to the database of the CA.
network@node3:~$ openssl ca -in cert-myuni-ID.csr -out cert-myuni-ID.pem
Using configuration from /usr/lib/ssl/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 2 (0x2)
Validity
Not Before: May 15 23:06:06 2017 GMT
Not After : May 15 23:06:06 2018 GMT
Subject:
countryName = AU
stateOrProvinceName = NSW
organizationName = MyUni
commonName = www.myuni.edu
emailAddress = ID@myuni.edu
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
81:41:91:CE:33:E7:98:4B:A8:44:DB:70:CB:A2:B3:01:60:82:93:AB
X509v3 Authority Key Identifier:
keyid:4B:2A:C5:8F:04:63:98:0F:D7:99:12:9F:A7:EC:2C:B2:67:14:6E:DC
Certificate is to be certified until May 15 23:06:06 2018 GMT (365 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
The server needs the CAs certificate, so lets get a copy. Note that we will change the extension from .pem to .crt to help in a later step. Also we change the name.
network@node3:~$ cp demoCA/cacert.pem cert-steveca.crt
Now check that the server certificate is ok.
network@node3:~$ openssl verify -CAfile cert-steveca.crt cert-myuni-ID.pem
cert-myuni-ID.pem: OK
The result is that we have the following files on node3.
network@node3:~$ ls
cert-myuni-ID.csr cert-myuni-ID.pem cert-steveca.crt demoCA lynx.cfg privkey-myuni-ID.pem virtnet
Enable HTTPS in Apache
Now you need to enable HTTPS in Apache, including making both certificates available. First lets copy the files into appropriate directories for Apache to read:
network@node3:~$ sudo cp cert-myuni-ID.pem /etc/ssl/certs/
network@node3:~$ sudo cp cert-steveca.crt /etc/ssl/certs/
network@node3:~$ sudo cp privkey-myuni-ID.pem /etc/ssl/private/
For a real web server you should set the permissions on the private key so no-one else can access it. In our virtual network, that is not a problem so I skip that step.
Now edit the configuration file for the SSL enable website:
network@node3:~$ sudo nano /etc/apache2/sites-available/default-ssl.conf
The start of the file looks like this:
ServerAdmin webmaster@localhost
In the file add a line after the ServerAdmin line such that it becomes:
ServerAdmin webmaster@localhost
ServerName www.myuni.edu:443
Change the values of the SSLCertificateFile, SSLCertificateKeyFile and SSLCACertificateFile parameters to refer to your server certificate, server private key and CA certificate, respectively.
BEFORE the file looks like this:
SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
AFTER the file looks like this:
# SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
# SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
SSLCertificateFile /etc/ssl/certs/cert-myuni-ID.pem
SSLCertificateKeyFile /etc/ssl/private/privkey-myuni-ID.pem
SSLCACertificateFile /etc/ssl/certs/cert-steveca.crt
Finally, enable the SSL module, the SSL-based website and restart the server:
network@node3:~$ sudo a2enmod ssl
Considering dependency setenvif for ssl:
Module setenvif already enabled
Considering dependency mime for ssl:
Module mime already enabled
Considering dependency socache_shmcb for ssl:
Enabling module socache_shmcb.
Enabling module ssl.
See /usr/share/doc/apache2/README.Debian.gz on how to configure SSL and create self-signed certificates.
To activate the new configuration, you need to run:
service apache2 restart
network@node3:~$ sudo a2ensite default-ssl
nabling site default-ssl.
To activate the new configuration, you need to run:
service apache2 reload
network@node3:~$ sudo service apache2 restart
From node1 you can now test to see if you can access the website:
network@node1:~$ lynx https://www.myuni.edu/grades/
You should see a warning about a self-signed certificate such as:
SSL error: The certificate is NOT trusted. The certificate is...-Continue? (n)
If you press 'y' to continue you can access the website. The next step will remove this warning.
Load CA Certificate in Client
Although the web server has its own certificate, signed by a CA, we still get a warning message when accessing the web site from node1. This is because node1 does not trust the CA that signed the servers certificate. We will now add the CA's certificate to the list of CA certificates trusted by node1.
Copy the CA's certificate from node3 to node1:
network@node1:~$ scp 192.168.2.21:/home/network/cert-steveca.crt .
The authenticity of host '192.168.2.21 (192.168.2.21)' can't be established.
ECDSA key fingerprint is SHA256:WUrZsAcTnfPtCDbPKKsfKYTU9QMy7a8m0FFLffwWJ3s.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.2.21' (ECDSA) to the list of known hosts.
cert-steveca.crt 100% 1103 1.1KB/s 00:00
Ubuntu keeps are store of trusted CA's certificates, which is used by lynx when it accesses websites. We need to create a directory for extra CA's, add our CA's certificate to it, and then re-configure the store to include the new certificate:
network@node1:~$ sudo mkdir /usr/share/ca-certificates/extra
network@node1:~$ sudo cp cert-steveca.crt /usr/share/ca-certificates/extra/
network@node1:~$ sudo dpkg-reconfigure ca-certificates
Processing triggers for ca-certificates (20160104ubuntu1) ...
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
After running the dpkg-reconfigure command you will be given several options about trust - choose the default - and then presented with a list of CA's. Scroll down to the bottom until you find cert-steveca.crt and then mark it by pressing space. Then ok.
That's it. Now test again with lynx and you should find no errors/warnings when connecting to the secure web server.
Testing HTTPS
Of course you can use your web browser on node1 (e.g. lynx) to access the website. You can also test using openssl directly:
network@node1:~$ openssl s_client -connect www.myuni.edu:443
CONNECTED(00000003)
depth=1 C = AU, ST = Qld, L = Cairns, O = CQUniversity, OU = Certificate Authority, CN = www.cquni.edu, emailAddress = ca@cquni.edu
verify return:1
depth=0 C = AU, ST = NSW, O = MyUni, CN = www.myuni.edu, emailAddress = ID@myuni.edu
verify return:1
---
Certificate chain
0 s:/C=AU/ST=NSW/O=MyUni/CN=www.myuni.edu/emailAddress=ID@myuni.edu
i:/C=AU/ST=Qld/L=Cairns/O=CQUniversity/OU=Certificate Authority/CN=www.cquni.edu/emailAddress=ca@cquni.edu
1 s:/C=AU/ST=Qld/L=Cairns/O=CQUniversity/OU=Certificate Authority/CN=www.cquni.edu/emailAddress=ca@cquni.edu
i:/C=AU/ST=Qld/L=Cairns/O=CQUniversity/OU=Certificate Authority/CN=www.cquni.edu/emailAddress=ca@cquni.edu
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIC6jCCAlOgAwIBAgIBAjANBgkqhkiG9w0BAQsFADCBmDELMAkGA1UEBhMCQVUx
DDAKBgNVBAgMA1FsZDEPMA0GA1UEBwwGQ2Fpcm5zMRUwEwYDVQQKDAxDUVVuaXZl
cnNpdHkxHjAcBgNVBAsMFUNlcnRpZmljYXRlIEF1dGhvcml0eTEWMBQGA1UEAwwN
d3d3LmNxdW5pLmVkdTEbMBkGCSqGSIb3DQEJARYMY2FAY3F1bmkuZWR1MB4XDTE3
MDUxNTIzMDYwNloXDTE4MDUxNTIzMDYwNlowYDELMAkGA1UEBhMCQVUxDDAKBgNV
BAgMA05TVzEOMAwGA1UECgwFTXlVbmkxFjAUBgNVBAMMDXd3dy5teXVuaS5lZHUx
GzAZBgkqhkiG9w0BCQEWDElEQG15dW5pLmVkdTCBnzANBgkqhkiG9w0BAQEFAAOB
jQAwgYkCgYEA5bY+Y40DFX08EjOeE1nwo+JTuM48exEuy/yvRm/CrK3QkUpTMBcI
7HRYrJHILoS2Gbhbh8aCRkmALrb2cfMNNGWdW9kUCOf1sRmz5iGY34cah7cQ1IEu
lRNSRWMBcT722BOxAPFA0RDDAD1doPqjXcKRM/ZSDXvOHmlY1Rvur48CAwEAAaN7
MHkwCQYDVR0TBAIwADAsBglghkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQg
Q2VydGlmaWNhdGUwHQYDVR0OBBYEFIFBkc4z55hLqETbcMuiswFggpOrMB8GA1Ud
IwQYMBaAFEsqxY8EY5gP15kSn6fsLLJnFG7cMA0GCSqGSIb3DQEBCwUAA4GBAKXt
x4YTiKH+vVY3ELadP3acJy7vHh1hETem0v/kxV3RAERxhXs3NZ+3Z201vAuEgR1z
MsMOZ0Gq8YcmSLnVbqqw7paeyXjGUppn8gH4bakXivaGNp5D4TlmXJG8t8XqPJV3
WEj2rDb8ZvpDkNWtd6E5ZDYhPZE1qcey1Kizpqkq
-----END CERTIFICATE-----
subject=/C=AU/ST=NSW/O=MyUni/CN=www.myuni.edu/emailAddress=ID@myuni.edu
issuer=/C=AU/ST=Qld/L=Cairns/O=CQUniversity/OU=Certificate Authority/CN=www.cquni.edu/emailAddress=ca@cquni.edu
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2088 bytes and written 431 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 1024 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: C7B3A7F664196AF9CE78085F5E957793B6F3511FE15DB7360AEAFBF6881C4C78
Session-ID-ctx:
Master-Key: C077085E1E2FDA5D90D25E6F1242C4CEE5C0488FFC2E42BCF845AA383248785BBFD622EF3821255C7791609DD27A1C3B
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 2b 12 1b ff 0f 1f 30 67-4a 31 f7 81 e4 f7 f3 95 +.....0gJ1......
0010 - 7a dc 86 e6 cd 77 4a 6f-02 ac 1f 8d 85 02 3b a6 z....wJo......;.
0020 - 02 dc 0f 85 02 82 75 80-a2 03 57 d9 8b 8b bf 20 ......u...W....
0030 - 0f c4 01 98 11 6d b8 56-39 4f aa 6d 1d e9 ad 47 .....m.V9O.m...G
0040 - d6 7c c5 00 0d d1 d7 e3-34 b1 ad ba 51 ad df 7c .|......4...Q..|
0050 - 70 dc 6a 7e a4 13 61 4b-11 76 7b b8 79 15 d0 60 p.j~..aK.v{.y..`
0060 - 32 72 62 c2 96 b3 1f 84-5a 12 69 5e be 2c b7 38 2rb.....Z.i^.,.8
0070 - 12 c0 05 38 b0 35 ca 4c-66 a7 ee 7c 41 5f 15 9b ...8.5.Lf..|A_..
0080 - aa 5b f3 b0 5b 62 5b 03-fe e9 80 08 99 6b 21 96 .[..[b[......k!.
0090 - b3 77 a9 0d f0 98 3c a7-a3 b0 f6 4e d0 9b 86 25 .w....<....N...%
00a0 - 11 70 0f 58 26 25 99 74-73 fa ff b7 12 7b 79 66 .p.X&%.ts....{yf
00b0 - 21 33 09 d8 77 31 c0 04-9e 34 53 3a 24 8d 35 83 !3..w1...4S:$.5.
Start Time: 1494890943
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
^C
You should see the certificate of the web server, no errors, and eventually the connection is closed.
To see the SSL exchange between browser and server, use tcpdump on the router (node2) to capture the packets while using lynx on node1 to visit the website. Study how SSL works.
Created on Tue, 28 Apr 2015, 2:07pm
Last changed on Thu, 12 Apr 2018, 7:11am