HighSecLab Wiki

OpenSSL: Generating Certificates and a Custom Certificate Authority (CA)

Properties2
tagsplatforms, security, openssl
aliasesOpenssl, OpenSSL: Generating Certificates and a Custom Certificate Authority (CA)

8 min read

OpenSSL: Generating Certificates and a Custom Certificate Authority (CA)

OpenSSL is a robust, commercial-grade, and full-featured toolkit for the Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocols. It is also a general-purpose cryptography library. This document guides you through using OpenSSL to create your own Certificate Authority (CA) and then issuing a server certificate signed by that CA. This setup is useful for internal services, development environments, or secure private networks where trusted public CAs are not practical or necessary.

1. Create a Certificate Authority (CA)

A CA is an entity that issues digital certificates. In this setup, you will create a self-signed CA that you control.

Step 1.1: Generate the CA Private Key

This command generates a strong RSA private key for your CA. This key must be kept highly secure.

openssl genrsa -out ca.key 4096
  • genrsa: Generates an RSA private key.
  • -out ca.key: Specifies the output file for the private key.
  • 4096: Specifies the key length in bits (4096 is recommended for strong security).

Step 1.2: Create the CA Certificate (Self-Signed)

This command creates the self-signed CA certificate using the private key generated in the previous step.

openssl req -x509 -new -nodes -key ca.key -sha256 -days 3650 -out ca.crt
  • req: Command for Certificate Request and Certificate Generation.
  • -x509: Outputs a self-signed certificate instead of a certificate request.
  • -new: Generates a new certificate request.
  • -nodes: No DES encryption for the private key (key will not be passphrase protected in this file, use with caution or remove for security).
  • -key ca.key: Specifies the CA’s private key.
  • -sha256: Uses SHA256 for the digest algorithm.
  • -days 3650: Sets the validity period of the certificate to 10 years (3650 days).
  • -out ca.crt: Specifies the output file for the CA certificate.

You will be prompted to enter information for the CA certificate’s Subject (e.g., Country Name, Organization Name, Common Name). For Common Name, use something descriptive like “My Custom CA”.

2. Create the Server Certificate (Signed by Your CA)

Now, you will generate a private key and a Certificate Signing Request (CSR) for your server, and then have your custom CA sign it.

Step 2.1: Generate the Server Private Key

Generate a private key for your server. This key will be used by your server to secure its communications.

openssl genrsa -out server.key 2048
  • 2048: Common key length for server certificates.

Step 2.2: Create a Certificate Signing Request (CSR) for the Server

The CSR contains information about your server and its public key. You send this to a CA (in this case, your own) to get it signed.

openssl req -new -key server.key -out server.csr

You will be prompted for server details. For “Common Name,” use the fully qualified domain name (FQDN) of your server (e.g., my-server.example.com).

Step 2.3: Create an Extension File for Subject Alternative Names (SANs)

Modern certificates often require Subject Alternative Names (SANs) to secure multiple hostnames or IP addresses with a single certificate. Create a file named server.ext with content similar to this:

authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = @alt_names
 
[alt_names]
DNS.1 = my-server.example.com
DNS.2 = another.example.com
IP.1 = 192.168.1.10
  • Adjust DNS.1, DNS.2, IP.1 to match your server’s hostnames and IP addresses.

Step 2.4: Sign the Server CSR with Your CA Key

This is the final step where your custom CA uses its private key to sign the server’s CSR, generating the actual server certificate.

openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
-out server.crt -days 825 -sha256 -extfile server.ext
  • x509: Command for X.509 certificate display and signing.
  • -req -in server.csr: Specifies that the input is a CSR.
  • -CA ca.crt: Specifies the CA certificate to use for signing.
  • -CAkey ca.key: Specifies the CA’s private key to use for signing.
  • -CAcreateserial: Generates a serial number file (ca.srl) to keep track of issued certificates.
  • -out server.crt: Specifies the output file for the signed server certificate.
  • -days 825: Sets the validity period for the server certificate.
  • -extfile server.ext: Includes the SANs defined in the server.ext file.

3. Export Server Certificate to PKCS12 Format (for Windows Devices)

PKCS12 (.pfx or .p12) is a common archive file format for storing many cryptography objects in a single file, often used for importing certificates and their private keys into Windows environments or web servers like IIS.

openssl pkcs12 -export \
  -out server.pfx \
  -inkey server.key \
  -in server.crt \
  -certfile ca.crt \
  -name "My Server Certificate"
  • -export: Exports to PKCS12 format.
  • -out server.pfx: The output PKCS12 file.
  • -inkey server.key: The server’s private key.
  • -in server.crt: The server’s certificate.
  • -certfile ca.crt: Includes the CA certificate in the bundle, creating a complete trust chain.
  • -name "My Server Certificate": A friendly name for the certificate within the PKCS12 file.

You will be prompted to create an export password for the .pfx file.

4. CA with a Configuration File (CNF)

Using a configuration file (openssl.cnf) provides more control over certificate generation and allows for automating prompts. You can define default values, extensions, and policies directly in the CNF file. This is particularly useful for scripting CA and certificate management.

A comprehensive openssl.cnf for a CA and certificate generation would typically include sections for:

  • Default values ([ default ])
  • Policy rules ([ policy_match ])
  • CA-specific settings ([ v3_ca ])
  • Client/Server certificate extensions ([ v3_req ], [ server_ext ], [ client_ext ])

Refer to the OpenSSL documentation for detailed examples of openssl.cnf for certificate generation.

5. Common Ad-Hoc OpenSSL Commands

This section covers frequently used OpenSSL commands for various ad-hoc tasks.

5.1. View Certificate Details (Human-Readable)

To display the contents of a certificate in a human-readable format.

openssl x509 -in certificate.crt -text -noout
  • -in certificate.crt: Specifies the input certificate file.
  • -text: Displays the certificate in human-readable text format.
  • -noout: Prevents the output of the encoded version of the certificate.

5.2. View CSR Details

To inspect the contents of a Certificate Signing Request (CSR).

openssl req -in server.csr -text -noout
  • -in server.csr: Specifies the input CSR file.
  • -text: Displays the CSR in human-readable text format.
  • -noout: Prevents the output of the encoded version of the CSR.

5.3. Verify a Private Key Matches a Certificate

To check if a private key corresponds to a given certificate. The output (modulus) should be identical if they match.

openssl x509 -noout -modulus -in certificate.crt | openssl md5
openssl rsa -noout -modulus -in private.key | openssl md5

If the MD5 hashes of the moduli are the same, the key and certificate match.

5.4. Verify a Private Key Matches a CSR

To check if a private key corresponds to a given Certificate Signing Request (CSR).

openssl req -noout -modulus -in server.csr | openssl md5
openssl rsa -noout -modulus -in private.key | openssl md5

If the MD5 hashes of the moduli are the same, the key and CSR match.

5.5. Convert PFX/P12 to PEM

To extract the certificate, private key, and CA certificates from a PKCS12 (.pfx or .p12) file into PEM format.

openssl pkcs12 -in your_cert.pfx -out your_cert.pem -nodes
  • -in your_cert.pfx: Specifies the input PKCS12 file.
  • -out your_cert.pem: Specifies the output PEM file.
  • -nodes: Skips encrypting the private key in the output PEM file (use with caution if the PEM file is stored insecurely).

5.6. Convert PEM to DER

To convert a certificate from PEM (Base64 encoded ASCII) to DER (binary) format.

openssl x509 -in certificate.pem -outform DER -out certificate.der
  • -outform DER: Specifies the output format as DER.

5.7. Generate a Strong Private Key

To generate a new RSA private key (without immediately creating a CSR or certificate).

openssl genrsa -out new_private.key 4096
  • 4096: Recommended key length.

5.8. Generate a CSR from an existing Private Key

If you have a private key and need to generate a new CSR for it.

openssl req -new -key private.key -out new_csr.csr

5.9. Test a certificate locally with s_server

s_server is useful when you want to present a certificate locally and inspect how clients behave against it. The -www flag also gives you a simple status page for quick checks in a browser or with curl.

openssl s_server \
  -accept 10443 \
  -cert vpn.example.com-certificate.pem \
  -key vpn.example.com.key \
  -cert_chain vpn.example.com-intermediate.pem \
  -www
  • -accept 10443: Listens locally on port 10443.
  • -cert: The leaf/server certificate to present.
  • -key: The private key for the certificate.
  • -cert_chain: Sends the intermediate certificate to clients as part of the chain.
  • -www: Returns a simple built-in test page over HTTPS.

This is a practical way to validate that the leaf certificate, private key, and intermediate chain work together before importing them into another service.

5.10. Build a PKCS#12 bundle from leaf and intermediate certificates

If you need a .pfx/.p12 for import into another system, first build a combined PEM containing the leaf certificate followed by the intermediate certificate.

  1. Create a combined PEM (leaf + intermediate):
cat vpn.example.com-certificate.pem \
  vpn.example.com-intermediate.pem \
  > vpn-combined.pem
  1. Create the PKCS#12 bundle:
openssl pkcs12 -export \
  -out vpn-fullchain.pfx \
  -inkey vpn.example.com.key \
  -in vpn-combined.pem \
  -name "vpn-cert"
  • -inkey: The private key that matches the leaf certificate.
  • -in vpn-combined.pem: The combined PEM containing the leaf certificate and intermediate certificate.
  • -name "vpn-cert": The friendly name stored in the PKCS#12 bundle.

Note: do not include the GoDaddy root certificate or cross certificate in the combined PEM. The working combination here was leaf + intermediate only.

Graph View

Backlinks