APIs are essential for building applications that are open and can integrate with other applications and services, yet with the openness of APIs comes a challenge: APIs can create additional security risks, as they increase the number of ways in which malicious actors could get into applications. Therefore, when developing APIs, careful design and security are important. As part of my internship at phpList, I had the opportunity to work on performing a risk assessment for the REST API.

On a personal note, it has been a challenging and exciting experience. I would like to share the research with the community as a learning opportunity for readers and myself. Please note that all high severity risks have been addressed and resolved already in the various related code repositories.

Recap on the phpList REST API

The phpList REST API provides functions for superusers to manage lists and subscribers. It uses functionality from the phpList core module. This new REST API can also be used to provide REST access to an existing phpList 3 installation. For this, phpList 3 installation and the phpList 4 installation with the REST API need to share the same database.

Local demo with Postman

You can try out the REST API by using pre-prepared requests and the Postman GUI tool. Install Postman as a browser extension or stand-alone app, open the phpList 4 REST API Demo collection and click “Run in Postman”.
For more information on how to install and try phpList REST API, please read the documentation.

Tools used for the Security Review

Reading “The Web Application Hacker’s Handbook“, helped me in the process of reviewing the security of phpList REST API. Another tool that I used was BurpSuite Community Edition, which was mentioned in the book. Burp is widely used by security professionals to find security vulnerabilities in web applications and I had the opportunity to play around with it for this risk assessment. Burp can test any HTTP endpoint. The process is to proxy the client’s traffic through Burp.

Burp interface

 

I also used cURL to consume the API calls. It is an open source software command line tool that is used to transfer data with URLs.

The REST API topic has been also covered in several sites such as OWASP (The Open Web Application Security Project) REST Security, which is a good starter reference to learn about the main security challenges of REST.

Implementation of the API during testing

Here is a technical overview of how the REST API was implemented during my testing.

Session creation

For authentication a user has to call the /api/v2/sessions route and provide username and password within the POST body:

POST /app.php/api/v2/sessions HTTP/1.1
Host: 10.211.55.4:82
User-Agent: curl/7.54.0
Accept: */*
Content-Type: application/json
Content-Length: 58
Connection: close

{
    "login_name": "admin",
    "password": "admin1234"
}

The controller then queries the existing phpList database for an administrator and user with this password:

public function postAction(Request $request): View
{
   $this->validateCreateRequest($request);
   $administrator = $this->administratorRepository->findOneByLoginCredentials(
       $request->get('login_name'),
       $request->get('password')
   );
   if ($administrator === null) {
       throw new UnauthorizedHttpException('', 'Not authorized', null, 1500567098);
   }

   $token = $this->createAndPersistToken($administrator);

   return View::create()->setStatusCode(Response::HTTP_CREATED)->setData($token);
}

The controller is creating a new session key of 256 bytes and returning the MD5 representation of it. The user can then use those credentials for authentication.

Session verification

For authentication purposes a trait AuthenticationTrait is implemented, this trait implements a function requireAuthentication:

private function requireAuthentication(Request $request): Administrator
{
   $administrator = $this->authentication->authenticateByApiKey($request);
   if ($administrator === null) {
       throw new AccessDeniedHttpException(
           'No valid session key was provided as basic auth password.',
           null,
           1512749701
       );
   }

   return $administrator;
}

When this function is invoked, it calls \PhpList\Core\Security\Authentication::authenticateByApiKey, this function is implemented as following.

As one can see, it searches the AdministratorTokenRepository for an unexpired key and then verifies whether the token belongs to an administrator:

public function authenticateByApiKey(Request $request)
{
   $apiKey = $request->headers->get('php-auth-pw');
   if (empty($apiKey)) {
       return null;
   }

   $token = $this->tokenRepository->findOneUnexpiredByKey($apiKey);
   if ($token === null) {
       return null;
   }

   /** @var Administrator|null $administrator */
   $administrator = $token->getAdministrator();
   if ($administrator === null) {
       return null;
   }

   try {
       // This checks for cases where a super user created a session key and then got their super user
       // privileges removed during the lifetime of the session key.
     // In addition, this will load the lazy-loaded model from the database,
       // which will check that the model really exists in the database (i.e., it has not been deleted).
       if (!$administrator->isSuperUser()) {
           $administrator = null;
       }
   } catch (EntityNotFoundException $exception) {
       $administrator = null;
   }

   return $administrator;
}

This is done by querying the table phplist_admintoken for the current session key.

In the controllers, authentication is enforced by calling \PhpList\RestBundle\Controller\SessionController::validateCreateRequest in every function such as the following:

public function cgetAction(Request $request): View
{
   $this->requireAuthentication($request);

   return View::create()->setData($this->subscriberListRepository->findAll());
}

Lists Route

\PhpList\RestBundle\Controller\ListController implements cgetAction, getAction and deleteAction. Those functions allow an administrator to:

1. Get a collection of lists
2. Get information about a single list
3. Delete a list

The information is returned to the user as a JSON string with a Content-Type of application/json:

HTTP/1.1 200 OK
Date: Thu, 21 Jun 2018 14:45:33 GMT
Server: Apache/2.4.18 (Ubuntu)
Cache-Control: no-cache, private
Content-Length: 210
Connection: close
Content-Type: application/json

{"name":"newsletter","description":"Sign up to our newsletter<h1><script>alert(1)<\/script>","creation_date":"2018-06-20T13:29:07+02:00","list_position":0,"subject_prefix":"",
"public":true,"category":"","id":2}

Subscribers route

\PhpList\RestBundle\Controller\SubscriberController
implements postAction. This function allows an administrator to:

Create a subscriber

The information is returned to the user as a JSON string with a Content-Type of application/json:

HTTP/1.1 201 Created
Date: Thu, 21 Jun 2018 15:05:17 GMT
Server: Apache/2.4.18 (Ubuntu)
Cache-Control: no-cache, private
Content-Length: 219
Connection: close
Content-Type: application/json

{"creation_date":"2018-06-21T17:05:17+02:00","email":"xhenitest@xheni.me",
"confirmed":false,"blacklisted":false,"bounce_count":0,
"unique_id":"498f3de0e0b033665f58bb4b3c3f3c21","html_email":false,"disabled":false,"id":7}

The controller performs the required authentication checks and doesn’t expose any functionalities not yet available to superusers. The security level is thus adequate.

Security concerns and mitigation

Note: High severity risks have already been mitigated; the other risks identified have been addressed by hardening the pertinent code.

Disabled admins can be authenticated

Administrators that are disabled in phpList are still able to login using the REST API.

Mitigation: Adjust the authenticateByApiKey method to check if the user is disabled.
Note: This issue has already been fixed.

Invalid login attempts are not logged

Invalid login attempts using the REST API are not logged. The regular web interface does log invalid attempts, however.
From these logs, you can monitor activity and potentially discover any patterns or excessive usage activity.

Mitigation: Add logging of login events to REST API.

Insecure storage of sensitive information

As the session tokens are stored unencrypted, a leak of the phplist_admintoken table will allow a malicious actor to authenticate as phpList administrator.

Mitigation: Adjust the login logic to use a strong and expensive hashing algorithm such as bcrypt or scrypt.

Insecure querying of a secret (“Timing attack”)

As the session token is stored in a VARCHAR(255) field in the database this may be vulnerable to timing attacks.
An attacker may be able to infer a correct session token based on the responses. This risk is, however, quite a bit mitigated, as timing attacks require a close proximity to the server as well as a lot of session tokens. (due to detecting timing differences in milliseconds).

Mitigation: This would be addressed as well by using a hashing algorithm with a salt (e.g. bcrypt).

Stored Cross-Site Scripting

Some browsers such as Internet Explorer require the nosniff header to be set and potentially dangerous characters to be encoded. Otherwise, other websites embedding this resource could trigger an XSS vulnerability.
According to Wikipedia:

MIME sniffing was, and still is, used by some web browsers, including notably Microsoft’s Internet Explorer, in an attempt to help web sites which do not correctly signal the MIME type of web content display correctly.[1] However, doing this opens up a serious security vulnerability,[2] in which, by confusing the MIME sniffing algorithm, the browser can be manipulated into interpreting data in a way that allows an attacker to carry out operations that are not expected by either the site operator or user, such as cross-site scripting.

So when Internet Explorer sees HTML in a page it can be tricked to execute it. That’s why the nosniff header is required, which tells the browser “do not try to magically interpret what this could be”.

Mitigation:

1. Add an X-Content-Type-Options header with nosniff option.
2. Add a Content-Security-Policy header disabling all script execution.
3. Add X-Frame-Options header and set it to DENY.
4. Use JSON_HEX_TAG when using json_encode. This also encoded characters such as < and >.

The security headers have been added, and the fourth one is additional hardening that I’m still working on.

Security headers added to the response

Future Development

Some additional fixes remain a work in progress and I will submit them soon. Meanwhile, everyone should feel free to open their Pull Requests and report any issues identified. Check out the contributor guidelines on GitHub to get started!