In preparation of my new OpenVPN Server, I needed a PKI (Private Key Infrastructure). A PKI is basically just a way of managing digital certificates. My software of choice for this is OpenSSL, it lets you create certificates for pretty much every usage scenario and SSL is the standard for many encryption scenarios. I actually build a new PKI for my web Servers, but there some issues with it, that convinced me to create an entirely new PKI instead of just a sub CA for OpenVPN. This time i am documenting my approach, mostly to actually have some documentation on the subject, but also to help others avoid the mistakes I made with my old PKI.My first mistake was in the concept: I just created a CA and after that I created the Server Certificates I needed. Usually there is no problem if you use this approach, but if you want to use Certificates for different purposes (Web Servers, E-Mail encryption, VPN …), it is not the worst idea to have a subordinate Certificate Authority (also referred to as intermediate Certificate Authority) for every purpose. Major benefits of using Subordinate Certificate Authorities are added security, easier management and greater flexibility. This also allows you to revoke the entire SubCA, in case it gets compromised or if you decide to stop running the associated service. The downside is, you have to ensure, that the entire chain of trust can be followed.

As already mentioned above I structured my new PKI after the Services the Certificates are used for. For now that means I need to create one CA and two subordinate CAs:
– Root_CA: this one will be only be used to create new SubCAs
– Server_CA: this is a SubCA of the RootCA and will be used to sign Server Certificates
– OpenVPN_CA: this is also a SubCA of the RootCA and will be used for the OpenVPN Clients

After I figured out which structure would suit my needs best it was time to start dealing with the implementation. My first step was creating a new OpenVZ Container for my PKI. This step is not mandatory but at the very least you do not want your root CA on a computer offering services to the Internet. After the machine was up and running, I copied in my prepared openssl.cnf into “/etc/ssl/”.

#
# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
#

# This definition stops the following lines choking if HOME isn't
# defined.
HOME            = .
RANDFILE        = $ENV::HOME/.rnd

# Extra OBJECT IDENTIFIER info:
#oid_file        = $ENV::HOME/.oid
oid_section        = new_oids

# To use this configuration file with the "-extfile" option of the
# "openssl x509" utility, name here the section containing the
# X.509v3 extensions to use:
# extensions        =
# (Alternatively, use a configuration file that has only
# X.509v3 extensions in its main [= default] section.)

[ new_oids ]
# We can add new OIDs in here for use by 'ca' and 'req'.
# Add a simple OID like this:
# testoid1=1.2.3.4
# Or use config file substitution like this:
# testoid2=${testoid1}.5.6

####################################################################
[ ca ]
default_ca    = CA_skelleton.net        # The default ca section

####################################################################
[ CA_skelleton.net ]

dir                = /etc/ssl/CA_skelleton.net        # Where everything is kept
certs            = $dir/certs                # Where the issued certs are kept
crl_dir            = $dir/crl                    # Where the issued crl are kept
database        = $dir/index.txt            # database index file.
new_certs_dir    = $dir/newcerts                # default place for new certs.
certificate        = $dir/cacert_root.pem         # The CA certificate
serial            = $dir/serial                 # The current serial number
crlnumber        = $dir/crlnumber            # the current crl number
crl                = $dir/arl.pem                 # The current CRL
private_key        = $dir/private/cakey_root.pem    # The private key
RANDFILE        = $dir/private/.rand        # private random number file
x509_extensions    = ca_cert                    # The extensions to add to the cert
name_opt         = ca_default                # Subject Name options
cert_opt         = ca_default                # Certificate field options
crl_extensions    = crl_ext
default_days    = 3650                        # how long to certify for
default_crl_hours    = 48                    # how long before next CRL
default_md        = sha1                        # which md to use.
preserve        = no                        # keep passed DN ordering
# A few difference way of specifying how similar the request should look
# For type CA, the listed attributes must be the same, and the optional
# and supplied fields are just that :-)
policy            = policy_match

[ CA_servers_skelleton.net ]
dir                = /etc/ssl/CA_servers_skelleton.net
certs            = $dir/certs
crl_dir            = $dir/crl
database        = $dir/index.txt
new_certs_dir    = $dir/newcerts
certificate        = $dir/cacert_servers.pem
serial            = $dir/serial
crlnumber        = $dir/crlnumber
crl                = $dir/crl_servers.pem
private_key        = $dir/private/cakey_servers.pem
RANDFILE        = $dir/private/.rand
x509_extensions    = server_cert
name_opt        = ca_default
cert_opt        = ca_default
crl_extensions    = crl_ext
default_days    = 1460
default_crl_hours    = 48
default_md        = sha1
preserve        = no
policy            = policy_match

[ CA_OpenVPN_skelleton.net ]
dir                = /etc/ssl/CA_OpenVPN_skelleton.net
certs            = $dir/certs
crl_dir            = $dir/crl
database        = $dir/index.txt
new_certs_dir    = $dir/newcerts
certificate     = $dir/cacert_OpenVPN.pem
serial            = $dir/serial
crlnumber        = $dir/crlnumber
crl                = $dir/crl_OpenVPN.pem
private_key        = $dir/private/cakey_OpenVPN.pem
RANDFILE        = $dir/private/.rand
x509_extensions    = usr_cert
name_opt        = ca_default
cert_opt        = ca_default
crl_extensions    = crl_ext
default_days    = 1095
default_crl_hours    = 48
default_md        = sha1
preserve        = no
policy            = policy_vpnusr

# For the CA policy
[ policy_match ]
countryName        = match
stateOrProvinceName    = match
organizationName    = match
organizationalUnitName    = optional
commonName        = supplied
emailAddress        = optional

[ policy_vpnusr ]
countryName        = supplied
stateOrProvinceName    = supplied
organizationName    = match
organizationalUnitName    = optional
commonName        = supplied
emailAddress        = supplied

# For the 'anything' policy
# At this point in time, you must list all acceptable 'object'
# types.
[ policy_anything ]
countryName        = optional
stateOrProvinceName    = optional
localityName        = optional
organizationName    = optional
organizationalUnitName    = optional
commonName        = supplied
emailAddress        = optional

####################################################################
[ req ]
default_bits        = 4096
default_keyfile     = privkey.pem
distinguished_name    = req_distinguished_name
attributes        = req_attributes
x509_extensions    = v3_ca    # The extentions to add to the self signed cert
# Passwords for private keys if not present they will be prompted for
# input_password = secret
# output_password = secret
# This sets a mask for permitted string types. There are several options.
# default: PrintableString, T61String, BMPString.
# pkix     : PrintableString, BMPString.
# utf8only: only UTF8Strings.
# nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings).
# MASK:XXXX a literal mask value.
# WARNING: current versions of Netscape crash on BMPStrings or UTF8Strings
# so use this option with caution!
string_mask = nombstr
# req_extensions = v3_req # The extensions to add to a certificate request

[ req_distinguished_name ]
countryName                    = Country Name (2 letter code)
countryName_default            = DE
countryName_min                = 2
countryName_max                = 2
stateOrProvinceName            = State or Province Name (full name)
stateOrProvinceName_default    = Skelletonia
localityName                = Locality Name (eg, city)
0.organizationName            = Organization Name (eg, company)
0.organizationName_default    = skelleton.net
# we can do this but it is not needed normally :-)
#1.organizationName        = Second Organization Name (eg, company)
#1.organizationName_default    = World Wide Web Pty Ltd
organizationalUnitName        = Organizational Unit Name (eg, section)
#organizationalUnitName_default    =
commonName                    = Common Name (eg, YOUR name)
commonName_max                = 64
emailAddress                = Email Address
emailAddress_max            = 64
# SET-ex3            = SET extension number 3

[ req_attributes ]
challengePassword            = A challenge password
challengePassword_min        = 4
challengePassword_max        = 20
unstructuredName            = An optional company name

[ usr_cert ]

# These extensions are added when 'ca' signs a request.
# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.
basicConstraints=CA:FALSE
# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.
# This is OK for an SSL server.
# nsCertType            = server
# For an object signing certificate this would be used.
# nsCertType = objsign
# For normal client use this is typical
# nsCertType = client, email
# and for everything including object signing:
# nsCertType = client, email, objsign
# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment
# This will be displayed in Netscape's comment listbox.
nsComment            = "skelleton.net signed Certificate"
# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move
# Copy subject details
# issuerAltName=issuer:copy
#nsCaRevocationUrl        = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName

[ server_cert ]
# extensions to add for Servers implemented after Book Linux-Server P 777
basicConstraints        = critical, CA:FALSE
nsCertType                = server
nsComment                = "skelleton.net signed Certificate"
subjectKeyIdentifier    = hash
authorityKeyIdentifier    = keyid,issuer
keyUsage                = critical, digitalSignature, keyEncipherment
extendedKeyUsage        = serverAuth
crlDistributionPoints    = URI:https://www.skelleton.net/CA_Dist/skelleton.net-server.crl

[ email_cert ]
# certs for mailusers implemented after Book Linux-Server P 777
basicConstraints        = critical, CA:FALSE
nsComment                = "skelleton.net signed Certificate"
subjectKeyIdentifier    = hash
authorityKeyIdentifier    = keyid,issuer
keyUsage                = critical, digitalSignature, keyEncipherment
extendedKeyUsage        = emailProtection

[ vpnusr_cert ]
# certs for mailusers implemented after Book Linux-Server P 777
basicConstraints        = critical, CA:FALSE
nsComment                = "skelleton.net signed Certificate"
subjectKeyIdentifier    = hash
authorityKeyIdentifier    = keyid,issuer
keyUsage                = critical, digitalSignature, keyEncipherment
extendedKeyUsage        = clientAuth
nsCertType                = client

[ ca_cert ]
#certs for subCAs
subjectKeyIdentifier    = hash
authorityKeyIdentifier    = keyid:always,issuer:always
basicConstraints        = critical, CA:true
nsCertType                = sslCA
keyUsage                = critical, digitalSignature, cRLSign, keyCertSign
nsComment                = "skelleton.net signed Certificate Authority"
crlDistributionPoints    = URI:https://www.skelleton.net/CA_Dist/skelleton.net-root.crl

[ v3_req ]

# Extensions to add to a certificate request

basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
crlDistributionPoints    = URI:https://www.skelleton.net/CA_Dist/skelleton.net-root.crl

[ v3_ca ]

# Extensions for a typical CA
# PKIX recommendation.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid:always,issuer:always
# This is what PKIX recommends but some broken software chokes on critical
# extensions.
#basicConstraints = critical,CA:true
# So we do this instead.
basicConstraints = CA:true
keyUsage                = critical, digitalSignature, cRLSign, keyCertSign
nsComment                = "skelleton.net signed Certificate Authority"
crlDistributionPoints    = URI:https://www.skelleton.net/CA_Dist/skelleton.net-root.crl
# Key usage: this is typical for a CA certificate. However since it will
# prevent it being used as a test self-signed certificate it is best
# left out by default.
# keyUsage = cRLSign, keyCertSign
# Some might want this also
# nsCertType = sslCA, emailCA
# Include email address in subject alt name: another PKIX recommendation
# subjectAltName=email:copy
# Copy issuer details
# issuerAltName=issuer:copy
# DER hex encoding of an extension: beware experts only!
# obj=DER:02:03
# Where 'obj' is a standard or added object
# You can even override a supported extension:
# basicConstraints= critical, DER:30:03:01:01:FF

[ crl_ext ]

# CRL extensions.
# Only issuerAltName and authorityKeyIdentifier make any sense in a CRL.

# issuerAltName=issuer:copy
authorityKeyIdentifier=keyid:always,issuer:always

[ proxy_cert_ext ]
# These extensions should be added when creating a proxy certificate

# This goes against PKIX guidelines but some CAs do it and some software
# requires this to avoid interpreting an end user certificate as a CA.

basicConstraints=CA:FALSE

# Here are some examples of the usage of nsCertType. If it is omitted
# the certificate can be used for anything *except* object signing.

# This is OK for an SSL server.
# nsCertType            = server

# For an object signing certificate this would be used.
# nsCertType = objsign

# For normal client use this is typical
# nsCertType = client, email

# and for everything including object signing:
# nsCertType = client, email, objsign

# This is typical in keyUsage for a client certificate.
# keyUsage = nonRepudiation, digitalSignature, keyEncipherment

# This will be displayed in Netscape's comment listbox.
nsComment            = "skelleton.net signed Certificate"

# PKIX recommendations harmless if included in all certificates.
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer:always

# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
# An alternative to produce certificates that aren't
# deprecated according to PKIX.
# subjectAltName=email:move

# Copy subject details
# issuerAltName=issuer:copy

#nsCaRevocationUrl        = http://www.domain.dom/ca-crl.pem
#nsBaseUrl
#nsRevocationUrl
#nsRenewalUrl
#nsCaPolicyUrl
#nsSslServerName

# This really needs to be in place for it to be a proxy certificate.
proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo

After the openssl config was on the Server, i created the necessary files and folders:

cd /etc/openssl
mkdir CA_skelleton.net
mkdir CA_skelleton.net/{certs,crl,newcerts,private}
mkdir CA_servers_skelleton.net
mkdir CA_servers_skelleton.net/{certs,crl,newcerts,private}
mkdir CA_skelleton.net
mkdir CA_OpenVPN_skelleton.net/{certs,crl,newcerts,private}
chmod 700 CA_skelleton.net/private
chmod 700 CA_skelleton.net/private
chmod 700 CA_skelleton.net/private
touch CA_skelleton.net/index.txt
touch CA_servers_skelleton.net/index.txt
touch CA_OpenVPN_skelleton.net/index.txt

The next step was generating the .rand files for each CA. I am still not why or if they are necessary. My best guess is, they are used to salt the private keys generated by OpenSSL. In any case a lot of tutorials had this step so I just created them.

openssl rand - out ./CA_skelleton.net/private/.rand
openssl rand - out ./CA_servers_skelleton.net/private/.rand
openssl rand - out ./CA_OpenVPN_skelleton.net/private/.rand

After these preparations were done, I started generating the certificates. First of course the Root CA Certificate. The certificate creating consists of three steps:
1. Generate the private key.
2. Generate the Certificate Signing Request (csr). This step will ask you to fill in the Certificate Information. The most importent one is common name, it is the name of the Certificate and with my openssl.cnf it must be unique.
3. Sign the Certificate ( since this is the root CA certificate i had to selfsign)
Here are the commands I used (from the folder /etc/ssl):

openssl genrsa -out ./CA_skelleton.net/private/cakey_root.pem -rand ./CA_skelleton.net/private.rand 4096
openssl req -new -key ./CA_skelleton.net/private/cakey_root.pem -out ./CA_skelleton.net/cacert_root_req.pem
openssl ca -create_serial -out ./CA_skelleton.net/cacert_root.pem -days 3650 -keyfile ./CA_Skelleton.net/private/cakey_root.pem -selfsign -infiles cacert_root_req.pem

I generated a key without password protection, it is absolutely not recommended to do this with a Certificate Authority Certificate, especially the root CA Certificate, but not having a password on the key makes scripting easier. And since the password would likely end up being plain text in the script anyway, I left it out. If you want to add a password to your private key, add -aes256 to the genrsa command. If you add a password to your key, you will have to enter your password each time you sign something with that CA. Administrators in Home Environments can probably get away with the more convenient solution of not using a password to protect their Certificate Authorities, as long as these CAs will not go into a real productive use. In a business Environment however, especially where your CA certificate could end up in the trust list of some browsers, you should not even consider using no password on the CA keys. My new root CA was ready for use, so I took it for a test drive and created my sub CAs. The basic steps for creating the certificate are the same. The one exception was the signing, the subordinate CAs get signed by the root CA instead of self signing:

cd ./CA_servers_skelleton.net
openssl genrsa -out private/cakey_servers.pem rand private/.rand 4096
openssl req -new -key private/cakey_servers.pem -out cacert_servers_req.pem
openssl ca -name CA_skelleton.net -in cacert_servers_req.pem -out cacert_servers.pem

I used the -name parameter to specify which CA in the openssl.cnf should sign the certificate. In this case I could have left it out, because my Root CA is also defined as default CA. But from my point of view it is best to always explicitly state which CA I am using in order to avoid signing with the wrong one. I created the other CA in exactly the same manner. After creating those 3 certificates they were in the places in which they were created, but there was also a copy of every certificate in the “newcerts” folder of the root CA. The certificates in this folder are named with the serial number of the certificate. I have read, that it is good practice to archive and index these certificates. So I created a script to do that for me, since the archiving was obviously going to be a repetitive task. If you do not want to use a script for this, following steps have to be taken to archive a certificate:
1. Move the certificate from the newcerts folder to the certs folder. Certificates in the newcerts folder are named after their Serial Number instead of their common name.
2. Generate the hash of the certificate and create following symlink hashvalue.index
The index is an integer number for the “version” of the certificate, starting with 0. If you revoke a certificate and reissue a certificate with the same name, it will have the same hash as the old one. In this case you would leave the old hasvalue.0 symlink alone and create a new hashvalue.1 symlink and point it to the new certificate.

#!/bin/bash

#this script archives issued certificates by moving them from the newcerts to certs and and link the hash value as index
#script must be called scriptname "/path/to/newcerts" "/path/to/certs"
cert_archiver_log="/etc/ssl/scriptlog.log"
CA_newcerts_dir=$1
CA_certs_dir=$2

echo $(date +"%d.%m.%Y %T") START Starting Archiving of certificates in the directory $CA_newcerts_dir >> $cert_archiver_log
if [ ! "$(ls -A ${CA_newcerts_dir}/*.pem)" ]; then
				#check if there are actually any certs in the newcerts folder
				echo no new certificates
				echo $(date +"%d.%m.%Y %T") FINISH Ending Archiving there are no new certificates in the directory $CA_newcerts_dir >> $cert_archiver_log
else
	for CertSN in ${CA_newcerts_dir}/*.pem; do
		echo certsn: $CertSN
		echo moving $certSN to ${CA_certs_dir}/${CertSN##*/}
		mv $CertSN ${CA_certs_dir}/${CertSN##*/}
		#The Number at the end should increase, if the cert exists already -> cert has been reissued
		indexfile=$(openssl x509 -hash -noout -in ${CA_certs_dir}/${CertSN##*/})
		if [ ! -e ${CA_certs_dir}/${indexfile}.0 ]; then
			echo the certificate has not been issued before creating link ${indexfile}.0
			ln -s ${CA_certs_dir}/${CertSN##*/} ${CA_certs_dir}/${indexfile}.0
			echo $(date +"%d.%m.%Y %T") Archived Certificate ${CertSN##*/} >> $cert_archiver_log
		else
			echo certificate has been issued before incrementing index
			echo $(date +"%d.%m.%Y %T") Certificate ${CertSN##*/} has been reissued, incermenting index >> $cert_archiver_log
			i=0
			success=0
			while [ ! $success == 1 ]
			do
				i=$((i + 1))
				echo check if ${indexfile}.$i exists
				if [ ! -e ${CA_certs_dir}/${indexfile}.$i ]; then
					ln -s ${CA_certs_dir}/${CertSN##*/} ${CA_certs_dir}/${indexfile}.$i
					echo $(date +"%d.%m.%Y %T") Archived Certificate ${CertSN##*/} >> $cert_archiver_log
					success=1
				fi
			done
		fi
	done;
fi
echo $(date +"%d.%m.%Y %T") FINISH Ending Archiving >> $cert_archiver_log

With the script prepared and working, it was finally time to create a bunch of certificates for my servers. Like always I needed to create a key, a request and sign the request with the CA:

cd /etc/ssl/CA_servers_skelleton.net
openssl genrsa -out www.skelleton.net.key.pem -rand ./private/.rand 4096
openssl req -new  -key www.skelleton.net.key.pem -out ./requests/www.skelleton.net.cert.req.pem
openssl ca -create_serial -name CA_servers_skelleton.net -in ./requests/www.skelleton.net.req.pem -out www.skelleton.net.cert.pem

I used -create_serial option, because this was the first certificate from this CA, all following certificates do not need this option. Before deploying the certificate it is always a good idea to check, if it is a correct certificate for the intended purpose. Luckily OpenSSL also has the means to verify a certificate.

 openssl verify -purpose sslserver -CAfile /etc/ssl/CA_skelleton.net/cacert_root.pem -untrusted /etc/ssl/CA_servers_skelleton.net/cacert_servers.pem /etc/ssl/CA_servers_skelleton.net/www.skelleton.net.cert.pem
#if your certifcate works the output will look like this
/etc/ssl/CA_servers_skelleton.net/www.skelleton.net.cert.pem: OK

In order to verify the certificate, you need to supply the entire certificate chain. Your root CA certificate will be used as “CAfile” and any intermediary CA certificates have to be supplied by “-untrusted”. After the verification was successful, I moved the new key and certificate to my webserver and implemented them. I am going to document this in another article, since this one is getting way to long already.
With the certificates ready for deployment, there was only one thing left to do: the certificate revoke lists. All created certificates list a web address for the certificate revoke list. But since I did not create a certificate revoke list yet, I had nothing to put online. Fortunately OpenSSL does not require revoked certificates to exist, in order to create a revoke list. Since these would be the first crl lists of my Certificate Authorities I needed to initialize the crlnumber files before creating the crl.

#initializing the crlnumber file only do this before you create your first crl for each CA
echo 01 > /etc/ssl/CA_skelleton.net/crlnumber
echo 01 > /etc/ssl/CA_servers_skelleton.net/crlnumber
#now  generate the crl
openssl ca -name CA_skelleton.net -gencrl -out /etc/ssl/CA_skelleton.net/arl.pem
openssl ca -name CA_servers_skelleton.net -gencrl -out /etc/ssl/CA_servers_skelleton.net/crl_servers.pem

These were the final steps in creating my own PKI. There were still some things that needed to be automated, but those would go beyond the scope of this article.

I googled a lot for this, here are the most helpful resources I found:
The OpenSSL documentation
Article: on phildev.net
PDF File How to Set Up an OpenSSL TEST CA for Interoperability Testing with CertiPath on www.carillon.ca
Article: Create a OpenVPN Certificate Authority on macfreek.nl
Article: Create your oen CA or root CA, subordinate CA on itsecworks.wordpress.com
And the Book Linux-Server from Galileo Computing was also quite helpful