Every IT asset has at least one local, privileged login account.
This includes workstations, servers, network devices, databases,
applications and more. Some assets also have privileged accounts used
to run services or authenticate one application to another.
Passwords for privileged accounts are used to install software, manage the device and perform technical support functions. They are often "all powerful," having unlimited access to system functions and data. Consequently, compromise of privileged passwords is effectively compromise of the device.
Secure management of access to privileged accounts is essential to IT security. This document identifies technical challenges and offers solutions for effectively managing large numbers of sensitive passwords.
In a typical enterprise-scale organization there are thousands of servers, workstations and network devices. Normally, there is a single, shared administrator password for every type of device. For example, one password may be used for each workstation of a given type or for every server with a given configuration. This is convenient for data center and desktop support staff: if they need to perform maintenance or an upgrade on a workstation or server, they know how to log in.
Such static and well-known privileged passwords create both operational challenges and security problems:
These problems create security vulnerabilities. For example, if administrator passwords don't change, then former IT workers retain them beyond their term of employment. This clearly violates internal controls: former employees should not have administrative access to corporate systems.
In most organizations, strong internal controls are mandatory. Privacy protection legislation such as HIPAA and GLB, as well as legislation regarding corporate governance such as SOX, requires that systems containing sensitive data be secured against unauthorized access. Effective management of access to privileged accounts is therefore not an option, but a requirement.
The obvious way to eliminate static and shared privileged passwords is to change them regularly. If every sensitive password were randomized daily, control problems would be alleviated.
Since IT users often need to sign into privileged accounts, randomizing passwords is only half of the solution. Additional functions are required to control access by IT users to these accounts:
The combined solution, capable of both randomizing large numbers of passwords and controlling access to password values or to the underlying accounts, can be complex. The following section describes some of the technical challenges that must be overcome in order to successfully deploy such a solution.
Describing a basic process for periodically randomizing and archiving administrator credentials is easy, while implementing such a process in a manner that scales well to thousands of devices, that is secure and fail-safe can be challenging.
The following sections describe some of the technical challenges such a system must address.
Every type of IT asset has a local administrator password. This is true even if network credentials are used in the normal course of business to manage the device, since a local administrator password must be used to attach each device to the network in the first place.
To be effective, a system for managing administrator passwords should support a broad array of platforms. This includes workstations, Windows servers, Unix servers, network routers, database servers, ERP applications, midrange servers (iSeries, VMS, etc.), mainframe computers, directories and more. In short, every device that contains sensitive data or whose operation is critical to the business should be supported.
A password management system can easily make connections to servers, which have fixed network addresses, are always on and are continuously connected to the network. It is much harder for a central password management server to connect to mobile laptops, for several reasons:
In short, while it is easy for laptops to contact a central server, it is nearly impossible for the reverse to happen reliably.
To reliably secure local administrator passwords on workstations, a password management system should include technology to overcome location, connectivity, address and firewall challenges.
A large organization may have thousands of workstations, servers and applications. If each of these IT assets gets a new administrator password daily, the total number of passwords that must be securely managed, including historical data, quickly grows into the millions of passwords.
Note that historical passwords need to be stored along with current ones, since in the event that a managed device crashes and is restored from backup media, its old password will be needed.
A scalable solution for managing administrator passwords must be able to randomize tens of thousands of passwords daily and to keep permanent records of millions of historical passwords.
A robust system for managing administrator passwords must ensure that the password kept in its database for a given administrator account always matches the password on the system in question. This should be true even if an attempt to change passwords failed in the middle of an update.
For instance, if a password management system sets a new password on an IT asset and experiences a connection failure, it is not clear whether the new or old password is actually in effect -- should the value stored in the database be updated?
A robust system for managing administrator passwords must ensure that the password it stores in its database is always the right one -- even if a fault occurred in the middle of a password update.
A password management system must be fault tolerant. If it becomes unavailable, IT workers would not be able to do their jobs -- making failure of the system catastrophic.
Hardware servers, including "appliances" (note) sometimes fail, due to disk crashes, power supplies burning up, etc. Network connections, especially over wide area links, also sometimes fail. Whole data centers can fail as well, due to power outages, earthquakes, hurricanes, tornados, fires or floods.
If one component of a privileged access management system fails, the accounts it secures must still be available. This is typically accomplished by running at least two servers, ideally at different sites. This means that if one server or one data center goes offline, IT staff elsewhere will be able to keep retrieving passwords and doing their jobs.
Fault tolerance between servers and sites requires data replication between servers. Such data replication must take place in real-time. The alternative -- scheduled, batch replication -- is inadequate. Consider, for example, a backup system that runs nightly. If a password management server were to fail just before a backup cycle begins, then the day's new passwords would be lost. If passwords are changed daily, the current administrator password for almost every system would be lost: a catastrophic event.
Compromise of even a single privileged password represents business risk. Compromise of many privileged passwords may represent catastrophic business risk. Consequently, a system for securing access to privileged accounts must protect these passwords cryptographically. It should protect passwords both when they are stored (at rest) and in transit: between users and itself, between replicated servers and between itself and target devices.
Networks are increasingly being segmented, to create a layered defense against intruders. This creates situations where the privileged access management system is attached to one network segment while an IT asset to which it controls access is attached to another segment.
To manage passwords on a system on the far side of a firewall, a password management system must be able to send password updates over the firewall. This may not be simple: many network protocols are insecure by design (e.g., SMB for Windows, SQL*Net for Oracle, plaintext LDAP, plaintext HTTP, etc.) and are blocked by firewall administrators for good reason.
To overcome this problem, an effective password management system must be able to replace network protocols that are native to a given target system with its own protocol. The password management system's network protocol must be appropriate to pass over a firewall.
Sensitive passwords are not limited to those used by human IT workers. There are also service accounts, used to run attended software such as web servers and application passwords. There are also application passwords, used by one service on one computer to authenticate itself to another service, possibly on another computer.
On many systems, service passwords are static and application passwords are embedded in scripts, programs or text files. These passwords unlock login IDs that are often just as powerful as administrator accounts.
An effective solution for managing sensitive passwords should include mechanisms for managing service and application passwords, in addition to managing the administrator passwords used by IT workers. This calls for two specific capabilities:
Not every IT worker should be able to access every privileged account. Likewise, applications invoking an API to retrieve a password should only be able to get passwords for services to which they legitimately need to be able to connect.
To enforce such security policies, a password management system must include a flexible access control infrastructure, capable of determining whether a given user of the system -- human or software agent -- should be granted access to a given privileged account.
Every action in the password management system, including looking up assets and their passwords and changing access control policies should be auditable. This creates a chain of accountability between users and their actions.
It also makes sense to link auditable events to alerts. For example, if a legitimate user retrieves a given server's administrator password, the owner of that server might wish to receive an e-mail about the event.
To create accountability, to meet audit requirements and to enable system owners to promptly respond to anomalous administrator activity, a privileged access management system must include detailed logs of user sessions, must retain its audit data indefinitely and must be able to act on, rather than just record, security events.
Each of the requirements set forth in the previous section can be addressed with a suitable architectural element in the password management solution. These architectural components are described in the following sections:
A rich set of connectors should be provided, to integrate with a broad range of target system types.
Client software should be available, to be installed on user workstations, which periodically contacts a central cluster of password management servers and requests new passwords for locally managed accounts.
This "pull mode" approach eliminates the problems with a central server "pushing" out passwords to devices with intermittent connectivity and dynamic IP addresses.
Multiple, concurrently-active password management servers should be supported, each of which can push new passwords to servers and each of which can provide new passwords to workstations on demand.
As the need for scalability grows, the number of servers can be increased. Servers should be placed behind a load balancer to hide this complexity from users and workstations.
It is not feasible to manually configure thousands of devices for periodic password changes. Instead, a privileged access management system requires an auto-discovery infrastructure to:
A reliable protocol is required, especially for workstations, to confirm password updates before updating stored passwords.
Historical passwords should be retained indefinitely. In the event that an IT asset was damaged and had to be recovered from backup media, passwords from the date the backup was made will be available.
As mentioned in (_label_req-ha), multiple servers are required. Not only should the servers each be able to randomize passwords in a multi-master configuration, but each server should house a complete data set and should replicate all local updates to that data to every other server.
Multiple servers should be installed in different data centers. This provides the opportunity for performance tuning, by having a local server manage passwords on local assets. It also provides for fault tolerance in the event of a disaster at one data center. If one data center goes offline, the password management servers at other data centers can keep working and will contain a full data set.
Design of an encryption system for a password management system revolves around key management: How are keys generated? How are keys associated with data, with servers, with end users and with managed devices? Key management is an advanced topic and deserves separate treatment, beyond what this white paper can cover. That said, some basic observations can be made:
In order to cross firewalls without exposing insecure protocols, the password management system must have components on both sides of the firewall. To avoid the need to fragment password storage into one database per network segment, it makes sense to provide a proxy server -- i.e., a server installed on one network segment whose purpose is to run connectors and update passwords on another network segment.
The communication between a primary password management server and a password management proxy server can be a simple, encrypted protocol over an arbitrarily numbered TCP port. This is robust, secure, bandwidth efficient and easy for firewall administrators to understand and forward.
In order to manage passwords used to start services, the password management system must be able to execute plug-in code, after successfully randomizing a password. The function of this installation-specific code is to notify network components of the new password value.
Some plug-ins are common. For example, the Windows Service Control Manager, Scheduler and IIS web server all store passwords in secondary storage (outside of the security database) in order to execute processes as named users. Since other programs may have the same requirement, the infrastructure for notifying programs of new passwords must be extensible (hence plug-ins).
In order to manage passwords used by one application to authenticate to another, an API must be exposed, to enable applications to acquire current credentials. For example, a web application might use the API to get a database password and use that password to connect to a database and read data which is then displayed in a web page.
This type of API creates a circular problem: how does an application which needs a password authenticate itself to the password management system? The obvious answer is that it must have its own (static) password, but this approach is clearly undesirable, as it reduces security of the application password (now randomized) back to a static password -- but the point of a privileged access management system is precisely eliminate static password.
Some options for authenticating applications to the API include:
A simple access control model maps privileges between individual passwords and individual users. For example, user X is allowed to retrieve the current password for login ID Y on system Z.
As the number of systems, managed user accounts and IT users grows, this model breaks down -- there are simply too many relationships.
A more powerful model is to insert security groups between users and managed systems. Essentially users are collected into groups (each user can belong to multiple groups) and groups are assigned privileges to groups. For example, users A, B and C belong to group G. Members of group G are allowed to retrieve the current password for login ID X on system Y and login ID Z on system W.
This model may also be difficult to manage in large environments -- users must be explicitly attached to groups (an administrative burden where there are many users and their responsibilities change often) and large numbers of managed systems must be manually attached to multiple groups.
The best model is to define both user groups and managed system policies and to define access controls (privileges) between the two. For example, users A, B and C belong to user group UG1. Managed systems R, S and T belong to policy P1. Members of user group UG1 are allowed to connect to privileged accounts on systems in policy P1.
This model provides for maximum flexibility and minimum administrative burden. It can be optimized further by automating association of users with user groups and managed systems with policies.
Logging is straightforward -- record every event as it takes place and provide reports that are either user-centric or system-centric to show event history.