Posts Tagged

PostgreSQL Password Config

Are you sure that your passwords are secure?

Progressive Secrets Management with HashiCorp and Spring Vault

Access data or credentials are environment-specific configuration settings whose management requires strict methods for safeguarding by their users. A storage of these credentials in various environments that is both gapless and secure as well as a confidential handling represents a challenge for access authorization and secure recording.
Modern methods simplify these processes and allow a secure management and storage of passwords in different environments for different apps. One of those methods is the use of HashiCorp Vault, which centrally saves and manages passwords by use of various mechanisms, such as key/value or dynamic processes.
Also, Spring offers a high-level abstraction of HashiCorp Vault – which is called “Spring Vault” –, which features a client-side support for existing Spring applications and thus simplifies the transition to HashiCorp Vault. Spring Vault offers REST interfaces for the access to the passwords that are saved in HashiCorp. A short explanation of the use of these two new technologies illustrates their benefits.

by Walid El Sayed Aly

All highly sensitive data that you want to protect are so-called secrets. A secret can be any element used for authentication or authorization, for instance, username, password, API token or TLS certificate. However, secrets are also sensitive or confidential data like credit card numbers, e-mails or tax identification numbers. We must know that secrets need different ways of saving, storing and managing than other kinds of data to fulfill their special safety requirements. There are new methods on the market that maximize the protection of sensitive data and their use.
Vault by HashiCorp saves, stores and manages passwords, certificates, API keys and other secrets in consideration of strict security criteria. With HashiCorp Vault, secrets can have certainly configurable lifecycles with an individually determined period of validity for passwords [1]. HashiCorp Vault was written in the programming language “Go” and works with the intelligence threading „Goroutine“. Goroutines are lightweight, concurring functions, which interact performantly, intelligently and faster than normal Java threading mechanisms. If, for example, a Goroutine is blocked, a new Goroutine will be generated automatically within the Go runtime.

HashiCorp Vault is an executable application that can be started with the command vault server. The Dev environment can be started with the parameter -dev: vault server -dev. Configuration files for the production environment are written in HCL, HashiCorp Configuration Language. HCL resembles the JSON format but has an additional function for adding commentaries.

Vault has a consistent interface for each secret and a secure access control. In addition, it records a detailed monitoring protocol. Vault can save any random key/value passwords and encrypts these data before saving them. It works with dynamic secrets. That is a safety mechanism that creates passwords only when required. Thus, the client can use a password just once. Also, leasing, revocation and renewal of passwords are possible. Vault renews passwords automatically and can also restore them.

Illustration. 1: A high-level Representation of the Vault Architecture [2]

The client can request a password for his application via an HTTP API (Ill. 1). Then Vault generates a dynamic password and sends it back to the client. Additionally, it logs that process. There is no clear separation between the internal and external components. The memory backend used by Vault does not have a trustworthy design. The safety barrier automatically encrypts all data leaving Vault by using the AES encryption (Advanced Encryption Standard) in the Galois counter modus (GCM). If the safety barrier reads data, the GCM authentication label will be verified in the decryption process to spot manipulations.

Vault supports Cloud backends such as Amazon Web Services (AWS) or database management systems like MySQL, PostgreSQL, Oracle Database or NoSQL databases like MongoDB. Each of theses components can be used as a Mount system. You can mount, unmount or remount single backends. With Mount, a backend will be mounted with every secret. Multiple backends of the same type can be made available by stating configuration different Mount points. At unmounting, the secret backend is not available anymore, and all its password data that had been physically saved will be deleted. At remounting, the backend is shifted to another backend without deleting the data. The different backends can be moved with vault mount/unmount/remount. All backend systems that have been mounted before can be viewed with the command vault mounts 1.2). Each single component has a detailed documentation on how to save passwords or sensible data.


Illustration. 2: Example of Vault Backend Systems

Vault Compared to Other Systems
There are more systems that aim at managing and saving secrets according to current requirements. Examples are, for instance:

  • Amazon Key Management Service (KMS) is a service by AWS that saves secrets in the hardware safety module HSM. In contrast to Vault, KMS focuses on how passwords are saved.
  • Keywhiz is an open-source project for the management of secrets. It offers a RESTfull API for password management. Clients can authenticate via certificates or cookies. The passwords are filed in memory with Keywhiz. In doing so, the data can be saved in a cache server. Keywhiz also offers a web UI for data management, although only for premium customers.
  • You can also manage passwords in a single way with Puppet. Single ways are symmetrical key algorithms that use just one key to achieve safe communication, privacy, integrity, and authentication. Puppet expects that a user’s passwords are encrypted in a format that also other systems accept. Passwords for Unix systems, for example, must be generated in the SHA1 format. Puppet is a simple method for saving secrets. Its safety level is not very high compared to Vault.

Spring Vault

Spring Vault offers an abstraction of Vault for the Spring framework [3]. At the time this article was written, Spring Vault was available in version 2.0. Spring Vault has been supporting HashiCorp Vault since version 1.0. Version 2.0 contains many new features, such as important changes like Vault repositories and a reactive Vault client. The Spring team uses the concept of Spring data repositories for the Vault repositories. Thereby, developers can use all CRUD functionalities with Vault passwords. The Vault repositories are activated with the annotation EnableVaultRepositories to activate Spring data as well. Repositories are meant to reduce boilerplate code and implement on various persistence memories. For reactive programming, Spring Vault offers the ReactiveVaultOperations, which is a central interface that specifies a fundamental set of Vault operations. Spring Vault’s reactive client support is based on modular authentication steps and Spring’s functional WebClient via ReactorNetty, which has a fully non-blocking, event-driven HTTP client.

ReactiveVaultTemplate encapsulates Vault’s main interaction, registers at the initialization in Vault and uses the token during the entire lifetime. Besides, in Spring Vault 2.0 it is possible to save the authentication steps, such as the token authentication or the sending of credentials to Vault. Of course, Spring Vault uses RestTemplate of Spring Web as the primary interface. Additionally, Spring Vault supports Apache HTTP Components, Java’s java.net.URLConnection (HttpURLConnection), Netty and OkHttp 3 by square.

Example

Two examples will show how to manage microservices in the Cloud with Spring Vault and HashiCorp Vault. A static example demonstrates how to save secrets as key/value on a physical hardware. The dynamic management is illustrated by a database as secret backend. We will use PostgreSQL DBMS as database. The project demo and the sources are available under GitHub [4].

You can start Vault in the console and should export the surrounding variables in the bash_profile:

First, we define the Vault address and give Vault a particular token. Then we define the path to a certificate that we have created locally earlier. Now we can start Vault:

We define our backend in the Vault configuration file. In this case Vault starts in-memory. In addition, we configure our listeners. These are the address and ports to which Vault needs to react. Vault supports exclusively the TCP protocol. Listing 1 shows an example of a Vault configuration file. tls_cert_file and tls_key_file are mandatory fields. Disable_mlock is an interesting configuration for the development environment. If disable_mlock is true, mlock will not get active. Mlock locks the virtual port range of the calling process in RAM.

  Listing 1: Vault Configuration File:

After starting the Vault, you can execute various configuration files that are necessary for the environment, like the roles and set-up of the databases, the AppId authentication or the Cloud Foundry. For our example we need a configuration of PostgreSQL. For this purpose, we first have to mount PostgreSQL. After that the connection information regarding the database is committed to Vault and finally the roles are defined.

In this manner you can also connect other database systems, such as Oracle Database or MySQL. Illustration 3 shows the first dynamic PostgreSQL secrets with Vault. Vault generates with that command dynamic access data to PostgreSQL for us, which can only be used with the defined token: vault read postgresql/creds/readonly. Now Vault is ready for use. At start.spring.io, you can get a Spring Boot initialization project, and a Maven or Gradle project will be produced with the necessary dependencies. For this example, the dependencies for Vault configuration, Spring WEB, JPA, Lombok and PostgreSQL were used.

Ill. 3: Dynamic Passwords with Vault and PostgreSQL 

 

Key/Value as Static Example

The application’s context configuration is defined in Java and derived from the abstract Spring class AbstractVaultConfiguration. For this you need to implement the two methods clientAuthentication and vaultEndpoint. You can also define sslConfiguration and create the paths for the trustStore certificates. Listing 2 shows a Vault configuration with Spring Vault.

Listing 2: Spring Vault Java Configuration:

Finally, you can initialize the Vault template from the context and already write, read and delete passwords as key/value in Vault (listing 3). Here, the passwords are saved unencrypted in Vault.

PostgreSQL Database as Dynamic Example

Right after the Vault server has been started and the configuration to PostgreSQL has been defined in Vault, you can dynamically commit the PostgreSQL access data to the application with the support of Vault and Spring. The application itself is not supposed to take care of the secrets any longer. In other words, the access data is not saved anymore in some property file on a machine, but Vault takes care of them in its backend. The application is meant to just commit the URL to the database. And of course, it needs to show Vault a correct token. It does not matter if the application is in the Cloud or not. There is only one place for the management of all secrets. In the dynamic VaultConfig file (listing 1), the DataSource is also defined additionally (listing 4).

Data Source Configuration:

The application also commits the user name and password as variables. To illustrate this, a customer entity has been created in PostgreSQL (listing 5).

Listing 5: Customer Entity:

Now you can access the database and modify the customer entity with the aid of Spring Vault.

The application first accesses the Vault backend to get the dynamic access data (listing 6). The application needs to give Vault the correct token. If the request was successful, you will get back a VaultResponseSupport with postgresql/creds/readonly. The application gets the access data with the response. These will be committed to the dynamic Vault configuration. Finally, the application gets access to PostgreSQL. The secrets are valid as long as the application is online. Whenever the application starts afresh, new secrets will be generated. No other application gets access to the database with the same secrets.

Listing 6: Dynamic PostgreSQL with Vault:

These two examples demonstrate that it is possible to manage passwords safely in the Cloud. However, it is of course a disadvantage that the Cloud services depend on Vault. Therefore, the Vault server must be available to one hundred percent. Vault offers a replication of the data with Vault Enterprise. There is a library of clusters with various Vault ties (besseres Wort für Knoten?). The entire communication between the primary and secondary clusters is encrypted by TLS sessions. There are additional features in the enterprise modus, like support of the hardware safety module HSM and a web interface for data management.

Conclusion

We all agree that we must put special care on the handling and securing of sensitive data like passwords or credit card data. However, the old methods of encrypting passwords or not even encrypting them but saving them locally, are completely outdated. These data should be hashed or managed with modern technologies. Applications as well as operations must not know the secrets. The use of Vault is easy and efficient and therefore all the more advisable for the management of highly sensitive data. The discussed examples show that you can manage passwords statically as well as dynamically with HashiCorp Vault. Spring Vault offers a reasonable abstraction of HashiCorp Vault and considerably simplifies the transition from existing Spring applications to HashiCorp Vault.

Links & Literature

[1] https://www.vaultproject.io/

[2] http://bit.ly/2im2I5W

[3] http://projects.spring.io/spring-vault/

[4] http://bit.ly/2zFWyBR