Forensic Aspects of Microsoft Remote Access VPN

Rédigé par Théo Letailleur - 28/08/2023 - dans CSIRT - Téléchargement

As remote work surges, VPNs gain significance. With employees using their devices in uncontrolled networks, VPNs are certainly now a serious option for attackers to gain an initial foothold on the corporate network. Microsoft offers a VPN solution called Remote Access Service. This article sheds light on Microsoft VPN service's inner workings, and provides forensic aspects to improve incident response and the monitoring of this service.

Introduction

The rise of remote work forced companies to take a closer look at remote access solutions and in particular virtual private networks (VPN). With employees using their devices in uncontrolled networks, VPNs are certainly now a serious option for attackers to gain an initial foothold on the corporate network.

Microsoft has its own solution called Remote Access. In a classic Active Directory environment, the Remote Access Server Role can be configured with one or more of the following role services1:

  1. Direct Access and VPN (RAS): to connect clients to the organization;
  2. Routing Service: to route network traffic between subnets;
  3. Web Application Proxy service: acts as a reverse-proxy to allow clients outside the corporate network to access web applications inside the corporate network.

NB: Using Remote Access in Microsoft Azure is not supported.

This article focuses on the first role, Direct Access and VPN, and more precisely on the classical VPN since this is the configuration I encountered. Moreover, Microsoft indicates that Direct Access should only be used for clients earlier than Windows 10, so let's shift our focus to the future, shall we! The motivation to write this article comes from the fact that there is little to no literature regarding forensic (or even offensive) aspects of this service.

Microsoft Remote Access VPN

Microsoft provided thorough tutorials2 to install a remote access server as a VPN, build the necessary infrastructure and deploy VPN clients to workstations. I followed it to build my lab, it's a bit long but I recommend it to you if you want to try or better know how it works.

Roles

Several Microsoft Windows server roles are needed to build up the infrastructure in an Active Directory domain. Besides a domain controller, the following roles are needed:

  • The Certificate Authority must be installed and configured on the domain controller, so the AD CS role is required.
  • A Network Policy and Access Service (NPS) role must be installed and configured on a Windows server. It will handle the RADIUS authentication for the VPN server.
  • Finally, the Remote Access role must be installed on a Windows server. It can be configured either as a Direct Access server or a VPN server. Only the latter must be configured.

On-premises deployment

In the end, my lab was composed of:

  • Three Microsoft Windows servers 2022: a domain controller, a NPS server, and a RAS VPN server,
  • One Windows 10 Pro workstation: the VPN client.

Below is an architecture diagram that describes the Microsoft VPN infrastructure and authentication flow:

Microsoft Remote Access VPN deployment
Microsoft Remote Access VPN deployment

Please excuse any approximations regarding the architecture. However, this is what you can anticipate when encountering a Microsoft VPN infrastructure during your engagements.

Several authentication methods are available. The following figure describes the different options to configure the VPN security properties and what I configured during my testing.

VPN Security properties
VPN Security properties

As you can see, there are several options: from machine certificate authentication and strong encrypted authentication algorithms, to no encryption and no authentication, in order to obtain remote access. I have not evaluated the potential weaknesses in the different authentication options, but there is likely interesting research to be done here.

VPN Client profile deployment

Based on the configuration I set in my lab, users that can connect to the VPN are part of a dedicated AD group called "VPN Users". A user certificate is pushed by GPO to those specific users that allows them to authenticate to the VPN.

VPN client certificate
VPN client certificate

The certificate is clearly used to authenticate the user but not the machine.

Once the certificate is pushed to the user profile, the VPN connection profile must be configured. It can be done either manually, or with a Powershell script3 (then deployed by Microsoft Intune for instance).

While configuring the VPN connection, we set the VPN URL and the authentication mode. Some Protected Extensible Authentication Protocol (PEAP) settings must be provided, such as the NPS server's FQDN and the authentication method.

VPN connection settings
VPN connection settings

Finally, the VPN connection can be selected. In addition to the certificate, AD credentials of the user must be entered (can be automatically saved though).

VPN client authentication
VPN client authentication

Once the VPN authentication is successful, the VPN network interface is provided a new IP address from the pool specified in the VPN server settings.

VPN PPP interface
VPN network interface

More details regarding the artifacts of the Microsoft VPN connections are discussed in the next part.

 

Powershell administration

VPN connections can be managed with Powershell. For instance, the following Powershell command allows to disconnect the user NEBULAD\alice:

Disconnect-VpnUser -Username NEBULAD\alice

Disconnect-VpnUser is part of the Powershell module VpnClient. This module has the following cmdlets that can be used to interact with VPN clients and their configuration.

The following table describes each cmdlet as specified in the Microsoft documentation.4

Powershell VpnClient cmdlet descriptions
Cmdlet Description
Add-VpnConnection Adds a VPN connection to the Connection Manager phone book.
Set-VpnConnection Changes the configuration settings of an existing VPN connection profile.
Remove-VpnConnection Removes specified VPN connection profiles.
Get-VpnConnection Retrieves the specified VPN connection profile information.
New-EapConfiguration Generates an XML file with the specified EAP configuration.
Set-VpnConnectionProxy Configures web proxy information for the specified VPN connection.
New-VpnServerAddress Creates a VPN server address object.
Add-VpnConnectionRoute Adds a route to a VPN connection.
Remove-VpnConnectionRoute Removes a route from a VPN connection.
Set-VpnConnectionIPsecConfiguration Sets the IPsec parameters of a VPN connection.
Add-VpnConnectionTriggerApplication Adds applications that automatically trigger a VPN connection when launched.
Remove-VpnConnectionTriggerApplication Removes a trigger application from a VPN connection object.
Add-VpnConnectionTriggerDnsConfiguration Adds a DNS suffix or name to VPN connection trigger properties.
Remove-VpnConnectionTriggerDnsConfiguration Removes DNS suffixes from VPN connection DNS trigger properties.
Set-VpnConnectionTriggerDnsConfiguration Changes DNS servers for an existing DNS suffix.
Get-VpnConnectionTrigger Gets trigger properties of a VPN connection.
Add-VpnConnectionTriggerTrustedNetwork Adds DNS suffixes as trusted networks to the VPN profile.
Remove-VpnConnectionTriggerTrustedNetwork Removes DNS suffixes previously configured as trusted networks from the VPN profile.
Set-VpnConnectionTriggerTrustedNetwork Sets the trusted network list.

How to investigate a compromise in a RAS VPN infrastructure

Scenarios

At least two scenarios can be considered when facing a compromise in a RAS VPN infrastructure:

First, the attackers compromise a user's laptop from spearphishing (for instance) and take advantage of its VPN connection to pivot to the corporate network.

Scenario 1: the attacker takes advantage of the legitimate VPN connection
Scenario 1: the attacker takes advantage of the legitimate VPN connection

Second, the attackers compromise a user's laptop from spearphishing, steal its AD credentials and user certificate. They can configure the VPN connection from their own Windows virtual machine to connect to the corporate network by stealing the user's identity.

Scenario 2: the attacker steals user's identity to initiate a VPN connection from a rogue workstation
Scenario 2: the attacker steals user's identity to initiate a VPN connection from a rogue workstation

For this to work, the attackers must:

  1. Steal the domain user's password
  2. Export the user's certificate:
    1. The certutil command can be executed to verify its presence: certutil -store -user My
    2. Then to dump it: certutil -store -user My 0 alice.cer
  3. Export the (public) certificate of the root Certificate Authority (CA) that issued the user certificate:
    1. To verify its presence: certutil -ca
    2. In my setup, the certutil command to extract the root CA is: certutil -ca.cert -config DC1.nebulad.lab\nebulad-DC1-CA dc1_ca.cer
  4. Install both certificates in a user's context on the rogue workstation
  5. Properly configure any VPN and PEAP properties:
    1. To easily get the configuration, the Get-VpnConnection Powershell cmdlet5 can be used.
      Get VPN and EAP configuration
      Get VPN and EAP configuration

       

I could successfully reproduce this scenario on a workstation that is not part of the NEBULAD domain, so there is by default no conditional access. However, there is a way to add Microsoft Entra ID (Azure AD) conditional access to RADIUS attributes (NPS server) of the VPN connection properties6, but I did not try myself.

By the way, it is possible to connect with another user from the domain that is member of the group allowed to connect to the VPN. To illustrate this situation:

  • Alice and Bob are part of the group "VPN Users" allowed to connect to the VPN;
  • Charlie steals Alice's certificate and Bob's Windows user account credentials;
  • Charlie installs Alice's certificate on its rogue workstation and can authenticate to the VPN with Bob's credentials.

So whenever attackers get a user VPN certificate, they may authenticate with any other allowed users (if they have their password).

Let's see what pieces of evidence we can find in a Microsoft Remote Access VPN infrastructure.

VPN Server

Logging capabilities

The VPN service has logging capabilities but by default, only errors and warnings are logged. In order to generate the VPN connection logs, I recommend enabling the logging of all events in the Routing and Remote Access page of the VPN server.

VPN server logging
VPN server logging

There is also a quite interesting admin panel called Remote Access Management Console. This console has several dashboards that indicate the operational status of the server and statistics on the currently connected clients.

In the following dashboard, we can see the health status of the server, how many clients are connected, and the amount of data transferred:

VPN general dashboard
VPN general dashboard

Then the Remote Client Status page shows details about every connected client. For each client, we can get:

  • Its username;
  • Its host name (though it was empty in my example);
  • Its ISP Address. In my example, we can see that it's a private IP address, but it's due to my lab setup. The ISP (Internet Service Provider) address is actually the IP address of the client from the Internet, as the name suggests.
  • The IP Address: provided by the VPN;
  • The protocol used to connect;
  • The duration of the connection, and when it started;
  • Bytes in/out during the session;
  • And we have access details (protocol, port, and IP address). In the picture below, I just opened Microsoft Edge after connecting to the VPN. We can see the IP addresses from Microsoft Edge and Akamai.

By default, the remote client activity is refreshed every minute. It can be modified by clicking on Configure Refresh Interval.

VPN Remote Client Status
VPN Remote Client Status

As highlighted in the figure above, there is also a button that allows to disconnect all clients. It can be quite handy during a compromise! I tried and it effectively terminates the session. It probably executes the Powershell command I provided earlier (Disconnect-VpnUser).

Those details are great but they are volatile, not written to the disk. However, the reporting feature can be enabled to keep the connection and access details in a Windows Internal Database7 (WID). Microsoft WID databases - Microsoft SQL format - are used by some Windows server components to store their data. To enable the reporting, go to Reporting page and click on Configure Accounting task.

Configure connection details reporting
Configure connection details reporting

On this window, tick the Use inbox accounting box, select the appropriate retention settings related to your specifications and Apply.

Once enabled, we obtain a new dashboard similar to the Remote Client Status page, but the data and notably the access details, are now persistent. 🎉 There is also the possibility to filter time periods included in the report.

Connection details reporting
Connection details reporting

This data can be quite useful to detect potential exfiltration or C2 communication. Only the accessed IP addresses are collected though, no domain name. As your trained eyes probably noticed, there is no "button" to export the data, which is quite painful for a forensic investigator. The WID location on the disk is %SystemRoot%\Windows\WID\Data, in Master Database File format (used by MSSQL). It is possible to directly access the WID with SQL Server Management Studio and browse or export the data. To connect, specify the following named pipe for the server name: np:\\.\pipe\MICROSOFT##WID\tsql\query. The data related to the RAS VPN reports is stored in the RaAcctDb database.

RaAcctDb Windows Internal Database
RaAcctDb Windows Internal Database

The database is composed of four tables:

  1. ConnectionTable contains information about the VPN client IP addresses (external and internal);
  2. SessionTable contains the state of the VPN session, the connection id (from ConnectionTable), the user name, the start time, the duration, and other pieces of information such as the number of bytes sent and received during the session;
  3. ServerEndpointTable, as shown in the figure above, contains IP addresses, protocols and ports accessed by the VPN clients;
  4. EndpointsAccessedTable maps the server endpoint ids to the session ids.

As you see in the figure above, IP addresses are hexadecimal encoded. Data can be exported in several ways: Excel, CSV, or other DB.

Once the reporting is enabled, we can also use Powershell to extract this data.

  • Get-RemoteAccessConnectionStatistics8 provides statistics of currently active and historical VPN connections;
  • Get-RemoteAccessConnectionStatisticsSummary9 provides summary statistics of currently active and historical VPN connections;
  • and Get-RemoteAccessUserActivity10 displays resources accessed over active and historical VPN connections.
VPN Clients Powershell reporting
VPN Clients Powershell reporting

The end of the article presents a Python script to conveniently dump these data into CSV files.

Basically, these tables provide precious information about VPN connections. We can determine who was connected, when, and from where. Additionally, we can ascertain which internal and external resources they accessed.

Evtx Logs

VPN logs also have dedicated event IDs. Events are logged into the System channel (C:\Windows\System32\winevt\logs\System.evtx) under RemoteAccess provider11. The following table lists only logs of interest to look for on the server:

Remote Access VPN server EVTX logs
Channel Provider Event ID Category Message sample
System RemoteAccess 20250 Audit The user NEBULAD\alice has connected and has been successfully authenticated on port VPN2-4.
System RemoteAccess 20253 Connection error The user NEBULAD\alice connected to port VPN2-4 has been disconnected because no network protocols were successfully negotiated.
System RemoteAccess 20255 Connection error

The following error occurred in the Point to Point Protocol module on port: VPN2-4, UserName: <Unauthenticated user>. Connection time expired.

System RemoteAccess 20271 Authentication and Accounting The user NEBULAD\alice connected from 10.0.102.242 but failed an authentication attempt due to the following reason: The connection was forbidden by a strategy configured on your RAS/VPN server
System RemoteAccess 20272 Audit The user NEBULAD\alice connected on port VPN2-4 on 21/08/2023 at 16:16 and disconnected on 21/08/2023 at 17:57.  The user was active for 101 minutes 27 seconds.  15464434 bytes were sent and 751775 bytes were received. The reason for disconnecting was demande utilisateur. The tunnel used was WAN Miniport (IKEv2). The quarantine state was .
System RemoteAccess 20274 Audit The user NEBULAD\alice connected on port VPN2-4 has been assigned address 10.0.102.222
System RemoteAccess 20275 Audit The user with ip address 10.0.102.222 has disconnected

Based on my experience, the following order of event IDs can be encountered:

  • Connection: 20250 => 20274
    • Interesting pieces of evidence: start date of the connection, username, assigned IP address
  • Connection error (bad username, password or certificate): 20271 => 20255
    • Interesting pieces of evidence: date of attempt, source IP address, username
  • Connection error: 20253
    • Interesting pieces of evidence: date of attempt, username
  • Disconnection: 20272 => 20275
    • Interesting pieces of evidence: start date of the connection, end date of the connection, session time, username, assigned IP address, bytes IN/OUT transferred, reason.

As with RDP investigation, it is important to keep track of the VPN session timeline in order to focus on precise periods of potential compromise.

Text Logs

Text formatted logs can be found under the C:\Windows\System32\Logfiles directory with the name INXXXX.log, XXXX for the date. For instance, the filename I found was IN2308.log: 23 for year 2023, 08 for August. Actually the logs have a CSV format but the columns' header was not included. The columns' header specified in the Microsoft documentation related to the NPS server12 (I did not find such documentation for the VPN server though) kinda matches but not perfectly. The following sample has been extracted, with the additional header:

ComputerName,ServiceName,Record-Date,Record-Time,Packet-Type,User-Name,Fully-Qualified-Distinguished-Name,Called-Station-ID,Calling-Station-ID,Callback-Number,Framed-IP-Address,NAS-Identifier,NAS-IP-Address,NAS-Port,Client-Vendor,Client-IP-Address,Client-Friendly-Name,Event-Timestamp,Port-Limit,NAS-Port-Type,Connect-Info,Framed-Protocol,Service-Type,Authentication-Type,Policy-Name,Reason-Code,Class,Session-Timeout,Idle-Timeout,Termination-Action,EAP-Friendly-Name,Acct-Status-Type,Acct-Delay-Time,Acct-Input-Octets,Acct-Output-Octets,Acct-Session-Id,Acct-Authentic,Acct-Session-Time,Acct-Input-Packets,Acct-Output-Packets,Acct-Terminate-Cause,Acct-Multi-Ssn-ID,Acct-Link-Count,Acct-Interim-Interval,Tunnel-Type,Tunnel-Medium-Type,Tunnel-Client-Endpt,Tunnel-Server-Endpt,Acct-Tunnel-Conn,Tunnel-Pvt-Group-ID,Tunnel-Assignment-ID,Tunnel-Preference,MS-Acct-Auth-Type,MS-Acct-EAP-Type,MS-RAS-Version,MS-RAS-Vendor,MS-CHAP-Error,MS-CHAP-Domain,MS-MPPE-Encryption-Types,MS-MPPE-Encryption-Policy,Proxy-Policy-Name,Provider-Type,Provider-Name,Remote-Server-Address,MS-RAS-Client-Name,MS-RAS-Client-Version
"VPN","RAS",08/22/2023,16:00:39,1,"NEBULAD\alice",,"10.0.102.226","10.0.102.242",,,"VPN",,7,,,"VPN",,,5,,1,2,,,0,"311 1 10.0.102.227 08/15/2023 14:19:27 223",,,,,,,,,"206",,,,,,,,,9,1,"10.0.102.242","10.0.102.226",,,,,,,"MSRASV5.20",311,,,,,"Stratégie du service Routage et accès à distance Microsoft",2,"Serveurs d’authentification du service Routage et accès à distance Microsoft","10.0.102.227",,
"VPN","RAS",08/22/2023,16:00:39,2,,,,,,,,,,,,,,,,,1,2,,,0,"311 1 10.0.102.227 08/15/2023 14:19:27 223",,,,,,,,,"206",,,,,,,,,,,,,,,,,,,,,,"0x014E4542554C4144",14,2,"Stratégie du service Routage et accès à distance Microsoft",2,"Serveurs d’authentification du service Routage et accès à distance Microsoft","10.0.102.227",,
"VPN","RAS",08/22/2023,16:00:39,4,"NEBULAD\alice",,"10.0.102.226","10.0.102.242",,"10.0.102.221","VPN",,7,,,"VPN",1692712839,1,5,,1,2,,,0,"311 1 10.0.102.227 08/15/2023 14:19:27 223",,,,,1,,,,"206",3,,,,,"25",1,,9,1,"10.0.102.242","10.0.102.226",,,,,,,"MSRASV5.20",311,,"0x014E4542554C4144",,,"Stratégie du service Routage et accès à distance Microsoft",1,,,,
"VPN","RAS",08/22/2023,16:02:57,4,"NEBULAD\alice",,"10.0.102.226","10.0.102.242",,"10.0.102.221","VPN",,7,,,"VPN",1692712839,1,5,,1,2,,,0,"311 1 10.0.102.227 08/15/2023 14:19:27 223",,,,,2,,25263,32269,"206",3,138,180,97,1,"25",1,,9,1,"10.0.102.242","10.0.102.226",,,,,,,"MSRASV5.20",311,,"0x014E4542554C4144",,,"Stratégie du service Routage et accès à distance Microsoft",1,,,,
"VPN","RAS",08/22/2023,16:05:42,1,"NEBULAD\charlie",,"10.0.102.226","10.0.102.242",,,"VPN",,7,,,"VPN",,,5,,1,2,,,0,,,,,,,,,,"207",,,,,,,,,9,1,"10.0.102.242","10.0.102.226",,,,,,,"MSRASV5.20",311,,,,,"Stratégie du service Routage et accès à distance Microsoft",2,"Serveurs d’authentification du service Routage et accès à distance Microsoft","10.0.102.227",,
"VPN","RAS",08/22/2023,16:05:42,3,,,,,,,,,,,,,,,,,,,,,112,,,,,,,,,,"207",,,,,,,,,,,,,,,,,,,,,,,,,"Stratégie du service Routage et accès à distance Microsoft",2,"Serveurs d’authentification du service Routage et accès à distance Microsoft","10.0.102.227",,
"VPN","RAS",08/22/2023,16:06:20,1,"NEBULAD\bob",,"10.0.102.226","10.0.102.242",,,"VPN",,7,,,"VPN",,,5,,1,2,,,0,,,,,,,,,,"208",,,,,,,,,9,1,"10.0.102.242","10.0.102.226",,,,,,,"MSRASV5.20",311,,,,,"Stratégie du service Routage et accès à distance Microsoft",2,"Serveurs d’authentification du service Routage et accès à distance Microsoft","10.0.102.227",,
"VPN","RAS",08/22/2023,16:06:20,11,,,,,,,,,,,,,,,,,,,,,0,,30,,,,,,,,"208",,,,,,,,,,,,,,,,,,,,,,,,,"Stratégie du service Routage et accès à distance Microsoft",2,"Serveurs d’authentification du service Routage et accès à distance Microsoft","10.0.102.227",,
"VPN","RAS",08/22/2023,16:06:20,1,"NEBULAD\bob",,"10.0.102.226","10.0.102.242",,,"VPN",,7,,,"VPN",,,5,,1,2,,,0,,,,,,,,,,"209",,,,,,,,,9,1,"10.0.102.242","10.0.102.226",,,,,,,"MSRASV5.20",311,,,,,"Stratégie du service Routage et accès à distance Microsoft",2,"Serveurs d’authentification du service Routage et accès à distance Microsoft","10.0.102.227",,

In my opinion, important fields in these logs are:

  • The time event specified by Record-Date and Record-Time;
  • The User-Name;
  • The IP address in Called-Station-ID is the VPN server;
  • The IP address in Calling-Station-ID is the external IP address of the connecting user;

If enabled, there are also debugging logs in the C:\Windows\tracing folder. They are in a text format, very verbose and related to the inner workings of the Remote Access service. I did not find a particular interest for an investigation, but it's good to keep that in mind in case you spot a weird crash on that service. Those logs would probably help.

VPN Client

VPN logs generated client side also have dedicated event IDs. They are logged into the Application channel (C:\Windows\System32\winevt\Logs\Application.evtx) under RasClient provider13. The following table lists only logs of interest to look for on the client:

Remote Access VPN client EVTX logs
Channel Provider Event ID Category Message sample
Application RasClient 20220 Connection Establishment Error

The connection to <Server Name> made using device <Device Name> was terminated. The error code returned on failure is <err_code>

=> I did not get this one during my research but it seems related to an error on the server certificate checking, could be interesting

Application RasClient 20221 Connection Establishment

The user NEBULAD\alice has started dialing a VPN connection using a per-user connection profile named VPN Nebulad. The connection settings are:
Dial-in User =
VpnStrategy = IKEv2
DataEncryption = Requested

[... other settings ...]

Application RasClient 20222 Connection Establishment The user NEBULAD\alice is trying to establish a link to the Remote Access Server for the connection named VPN Nebulad using the following device:
Server address/Phone Number = 10.0.102.226 (VPN server IP address)
Device = WAN Miniport (IKEv2)
Port = VPN2-1
MediaType = VPN.
Application RasClient 20223 Connection Establishment The user NEBULAD\alice has successfully established a link to the Remote Access Server using the following device:
Server address/Phone Number = 10.0.102.226
Device = WAN Miniport (IKEv2)
Port = VPN2-1
MediaType = VPN.
Application RasClient 20224 Connection Establishment The link to the Remote Access Server has been established by user NEBULAD\alice.
Application RasClient 20225 Connection Establishment The user NEBULAD\alice has dialed a connection named VPN Nebulad to the Remote Access Server which has successfully connected. The connection parameters are:
TunnelIpAddress = 10.0.102.223
TunnelIpv6Address = None
Dial-in User = NEBULAD\bob.
Application RasClient 20226 Connection Termination The user NEBULAD\alice dialed a connection named VPN Nebulad which has terminated. The reason code returned on termination is 63114. (ERROR_USER_DISCONNECTION)
Application RasClient 20227 Connection Establishment Error The user NEBULAD\alice dialed a connection named VPN Nebulad which has failed. The error code returned on failure is 812. (ERROR_SERVER_POLICY)

Based on my experience, the following order of event ids can be encountered:

  • Connection: 20221 => 20222 => 20223 => 20224 => 20225
    • Interesting pieces of evidence: start date of the initiation of the connection, source username, "dial-in" username (the username used in the Windows authentication), VPN connection name, VPN server IP address and assigned ip address.
  • Connection error: 20227
    • Interesting pieces of evidence: source username, VPN connection name, and the error code.
  • Disconnection: 20226
    • Interesting pieces of evidence: end date of the connection, the username, and the VPN connection name.

It's important to correlate both client and server logs to have the best comprehension of the incident.

In addition, if the user selected "Remember my sign-in info" on the VPN connection, the cached credentials are stored in HKEY_CURRENT_USERS\SOFTWARE\Microsoft\RAS EAP\UserEapInfo. In this key there is a value called EapInfo that contains the cached credentials.

$ xxd EapInfo_export.bin
00000000: 4541 5032 1900 0000 51a5 8c7f 9649 3c4f  EAP2....Q....I<O
00000010: 8797 9053 5693 3b77 e104 0000 0200 0000  ...SV.;w........
00000020: e104 0000 0000 0000 0000 0000 0000 0000  ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0100 0000 0100 0000 b904 0000 1a00 0000  ................
00000050: 0000 0000 0300 0000 0000 0000 0000 0000  ................
00000060: 616c 6963 6500 0000 0000 0000 0000 0000  alice...........
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
[... lots of null bytes ...]
00000250: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000260: 0000 4e45 4255 4c41 4400 0000 0000 0000  ..NEBULAD.......
00000270: 0000 0000 0000 0000 0000 0000 0000 0000  ................
[... lots of null bytes ...]
00000360: 0000 0000 9201 0000 0100 0000 d08c 9ddf  ................
00000370: 0115 d111 8c7a 00c0 4fc2 97eb 0100 0000  .....z..O.......
00000380: 0b94 34b9 7170 6d47 8fc8 0eaa e68a 2beb  ..4.qpmG......+.
00000390: 0000 0000 0200 0000 0000 0366 0000 c000  ...........f....
000003a0: 0000 1000 0000 e919 4777 ceb9 45b7 594c  ........Gw..E.YL
000003b0: 2ead d951 8ad8 0000 0000 0480 0000 a000  ...Q............
000003c0: 0000 1000 0000 f78f 8caf a0ae 16cf 2976  ..............)v
000003d0: 79ae 8229 6066 0801 0000 4c18 9668 f23f  y..)`f....L..h.?
000003e0: 8e11 b2d2 14e5 a780 a624 b0fa e077 5a0f  .........$...wZ.
000003f0: 51d5 f044 f11b d691 3778 5b12 07a8 cd2b  Q..D....7x[....+
00000400: 6453 710f a1e0 446b bb55 551d b3dc 0af8  dSq...Dk.UU.....
00000410: 1ccb 4d1b bb47 2eaf b73b b1df c223 49f2  ..M..G...;...#I.
00000420: df55 4c7c cfaa 0182 9982 410c 651f 1cc9  .UL|......A.e...
00000430: 14b1 8f40 8b32 1fbb 2743 ff66 86f9 c459  ...@.2..'C.f...Y
00000440: 9e34 fc52 2112 3321 713a bd54 548d 56b0  .4.R!.3!q:.TT.V.
00000450: 3adb 50a1 5687 fa02 3f9a e5ac cc1a ee1c  :.P.V...?.......
00000460: 9985 14a0 20c0 30df 0bff 0728 1f12 5bc9  .... .0....(..[.
00000470: 9fd8 e00b 197d 250a 06c6 117c 72a3 984e  .....}%....|r..N
00000480: d1ab a167 20bc 5f02 20ed 5f11 48a5 f075  ...g ._. ._.H..u
00000490: e85b b4cd 7ae7 1944 dd22 a052 4cb5 61d6  .[..z..D.".RL.a.
000004a0: 1ced 66ef 3251 6317 3f24 5e64 86d8 e39f  ..f.2Qc.?$^d....
000004b0: 2582 a5b2 c122 501a 3a17 a90a 11bf 5f70  %...."P.:....._p
000004c0: 6a65 3b67 bd52 f51f 396c 18ab 28f1 5212  je;g.R..9l..(.R.
000004d0: ec1c 1061 0709 bc60 bfe6 0579 1eac a537  ...a...`...y...7
000004e0: 8ef3 1400 0000 1d56 6c88 4003 ae0b 31e4  .......Vl.@...1.
000004f0: 0f7f 85eb fd18 b8d2 6547 0000 0000 0000  ........eG......
00000500: 0000 0000 0000 0000                      ........

Regarding the EAP2 structure itself, I did not find much about it. If you have the information I would be interested!

We can however guess most of its content:

  • At 0x60 there is the username;
  • at 0x262 we find the domain;
  • and last but not least, at 0x368, we can determine the signature of a DPAPI protected password: 01000000d08c9ddf01...

There are several ways to recover the password. I extracted the protected password from EapInfo value in a new file then I unprotected it with mimikatz's dpapi::blob command in the context of my user alice. I am not sure whether there is a dpapi::something that can directly recover the protected password from the path of the registry value for this EAP2 structure. If there is, please let me know! The following figure shows the result of this procedure:

Protected Eap password retrieval
Protected Eap password retrieval

Results show the clear-text password in hexadecimal encoding: Adminadmin2023!.

NPS Server

The NPS server also has valuable text logs in addition to the VPN server. They are also under the C:\Windows\System32\Logfiles directory with the name INXXXX.log. The following sample has been extracted from this log file, with the additional header:

ComputerName,ServiceName,Record-Date,Record-Time,Packet-Type,User-Name,Fully-Qualified-Distinguished-Name,Called-Station-ID,Calling-Station-ID,Callback-Number,Framed-IP-Address,NAS-Identifier,NAS-IP-Address,NAS-Port,Client-Vendor,Client-IP-Address,Client-Friendly-Name,Event-Timestamp,Port-Limit,NAS-Port-Type,Connect-Info,Framed-Protocol,Service-Type,Authentication-Type,Policy-Name,Reason-Code,Class,Session-Timeout,Idle-Timeout,Termination-Action,EAP-Friendly-Name,Acct-Status-Type,Acct-Delay-Time,Acct-Input-Octets,Acct-Output-Octets,Acct-Session-Id,Acct-Authentic,Acct-Session-Time,Acct-Input-Packets,Acct-Output-Packets,Acct-Terminate-Cause,Acct-Multi-Ssn-ID,Acct-Link-Count,Acct-Interim-Interval,Tunnel-Type,Tunnel-Medium-Type,Tunnel-Client-Endpt,Tunnel-Server-Endpt,Acct-Tunnel-Conn,Tunnel-Pvt-Group-ID,Tunnel-Assignment-ID,Tunnel-Preference,MS-Acct-Auth-Type,MS-Acct-EAP-Type,MS-RAS-Version,MS-RAS-Vendor,MS-CHAP-Error,MS-CHAP-Domain,MS-MPPE-Encryption-Types,MS-MPPE-Encryption-Policy,Proxy-Policy-Name,Provider-Type,Provider-Name,Remote-Server-Address,MS-RAS-Client-Name,MS-RAS-Client-Version
"NPS","IAS",08/21/2023,14:50:01,1,"NEBULAD\alice","NEBULAD\alice","10.0.102.226","10.0.102.242",,,"VPN",,7,0,"10.0.102.226","vpn",,,5,,1,2,5,"Connexions de réseau privé virtuel (VPN)",0,"311 1 10.0.102.227 08/15/2023 14:19:27 157",,,,,,,,,"140",,,,,,,,,9,1,"10.0.102.242","10.0.102.226",,,,,,,"MSRASV5.20",311,,,,,"Connexions de réseau privé virtuel (VPN)",1,,,,
[...]
"NPS","IAS",08/21/2023,14:50:01,1,"NEBULAD\alice","NEBULAD\alice","10.0.102.226","10.0.102.242",,,"VPN",,7,0,"10.0.102.226","vpn",,,5,,1,2,11,"Connexions de réseau privé virtuel (VPN)",0,"311 1 10.0.102.227 08/15/2023 14:19:27 160",,,,,,,,,"143",,,,,,,,,9,1,"10.0.102.242","10.0.102.226",,,,,,,"MSRASV5.20",311,,"0x014E4542554C4144",,,"Connexions de réseau privé virtuel (VPN)",1,,,,
"NPS","IAS",08/21/2023,14:50:01,2,,"NEBULAD\alice",,,,,,,,0,"10.0.102.226","vpn",,,,,1,2,11,"Connexions de réseau privé virtuel (VPN)",0,"311 1 10.0.102.227 08/15/2023 14:19:27 161",,,,"Microsoft: Mot de passe sécurisé (EAP-MSCHAP version 2)",,,,,"144",,,,,,,,,,,,,,,,,,,,,,"0x014E4542554C4144",14,2,"Connexions de réseau privé virtuel (VPN)",1,,,,

"NPS","IAS",08/21/2023,16:16:21,1,"NEBULAD\alice","NEBULAD\alice","10.0.102.226","10.0.102.146",,,"VPN",,7,0,"10.0.102.226","vpn",,,5,,1,2,5,"Connexions de réseau privé virtuel (VPN)",0,"311 1 10.0.102.227 08/15/2023 14:19:27 162",,,,,,,,,"145",,,,,,,,,9,1,"10.0.102.146","10.0.102.226",,,,,,,"MSRASV5.20",311,,,,,"Connexions de réseau privé virtuel (VPN)",1,,,,
[...]
"NPS","IAS",08/21/2023,16:16:21,1,"NEBULAD\alice","NEBULAD\alice","10.0.102.226","10.0.102.146",,,"VPN",,7,0,"10.0.102.226","vpn",,,5,,1,2,11,"Connexions de réseau privé virtuel (VPN)",0,"311 1 10.0.102.227 08/15/2023 14:19:27 171",,,,"Microsoft: Mot de passe sécurisé (EAP-MSCHAP version 2)",,,,,"154",,,,,,,,,9,1,"10.0.102.146","10.0.102.226",,,,,,,"MSRASV5.20",311,,,,,"Connexions de réseau privé virtuel (VPN)",1,,,,
"NPS","IAS",08/21/2023,16:16:21,2,,"NEBULAD\alice",,,,,,,,0,"10.0.102.226","vpn",,,,,1,2,11,"Connexions de réseau privé virtuel (VPN)",0,"311 1 10.0.102.227 08/15/2023 14:19:27 171",,,,"Microsoft: Mot de passe sécurisé (EAP-MSCHAP version 2)",,,,,"154",,,,,,,,,,,,,,,,,,,,,,"0x014E4542554C4144",14,2,"Connexions de réseau privé virtuel (VPN)",1,,,,

 

Domain controller

Finally, on the domain controller, some security authentication logs are generated:

  • Event ID 4776: The domain controller attempted to validate the credentials for an account (alice)
    • The domain controller attempted to validate the credentials for an account
      
      Authentication package:	MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
      Logon account:	alice
      Source Workstation:	
      Error Code :	0x0 (Success)
  • Event ID 4624: An account was successfully logged on (WKS1$, computer's account of alice's workstation). 10.0.102.223 is the IP address assigned by the VPN to the workstation.
    • An account was successfully logged on.
      
      Subject:
      	Security ID: NULL SID
      	Account Name: -
      	Account Domain: -
      	Logon ID: 0x0
      
      Logon Information:
      	Logon Type: 3
      	Restricted Admin Mode: -
      	Virtual Account: No
      	Elevated Token: Yes
      
      Impersonation Level: Impersonation
      
      New Logon:
      	Security ID: NEBULAD\WKS1$
      	Account Name: WKS1$
      	Account Domain: NEBULAD.LAB
      	Logon ID: 0x61E80DC
      
      Network Information:
      	Workstation Name: -
      	Source Network Address: 10.0.102.223
      	Source Port: 63390

And that's about it for the domain controller. It's not surprising since most of the authentication process is handled by the NPS and the VPN server.

 

The goodies

Velociraptor Artifact

There is no Velociraptor15 client artifact (VQL) so far. Therefore, the finality of this research was to write a new Velociraptor artifact, called Windows.EventLogs.RemoteAccessVPN.

Velociraptor Artifact RemoteAccessVPN
Velociraptor Artifact RemoteAccessVPN

This client artifact hunts the related events IDs from both client and server sides. It also hunts text logs from NPS and VPN server located in %SystemRoot%\System32\Logfiles. The following figure shows the results of a hunt on the workstation WKS1 of my lab.

Velociraptor artifact RemoteAccessVPN Client results
Velociraptor artifact RemoteAccessVPN Client results

The following figure shows the results of a hunt on the VPN server of my lab.

Velociraptor artifact RemoteAccessVPN VPN server results
Velociraptor artifact RemoteAccessVPN VPN server results

The following figure shows the results of a hunt on the NPS server of my lab.

Velociraptor artifact RemoteAccessVPN VPN server results
Velociraptor artifact RemoteAccessVPN NPS server results

This artifact is being merged to the Artifact Exchange16 (community contributed artifacts): https://github.com/Velocidex/velociraptor-docs/pull/662.

You can also find it on Synacktiv GitHub repository: https://github.com/synacktiv/forensic-msvpn.

Chainsaw rules

In order to detect the Microsoft VPN connections and errors with the great EVTX parser project chainsaw17 from WithSecure Labs, I implemented 7 new rules. They allow to automatically find all related event IDs and determine when a VPN session starts and ends, from both client and server sides.

Chainsaw rules to detect Microsoft VPN activity
Chainsaw rules to detect Microsoft VPN activity

The rules are:

  • Microsoft Remote Access VPN - Client - Connection establishment
  • Microsoft Remote Access VPN - Client - Connection termination
  • Microsoft Remote Access VPN - Client - Connection error
  • Microsoft Remote Access VPN - Server - User logon
  • Microsoft Remote Access VPN - Server - User logoff
  • Microsoft Remote Access VPN - Server - Authentication error
  • Microsoft Remote Access VPN - Server - Connection error

You can find my Pull Request on chainsaw's GitHub project which contains these new rules (subject to changes): https://github.com/WithSecureLabs/chainsaw/pull/142.

You can also find it on Synacktiv GitHub repository: https://github.com/synacktiv/forensic-msvpn.

Python script to export RaAcctDb database

In order to conveniently dump the whole content of the RaAcctDb Windows Internal Database without installing Microsoft SQL Server Management Studio, a Python script has been developed. The script csv_export_raacctdb.py allows dumping the RaAcctDb WID database in CSV files. IP addresses and timestamp are converted in human-readable formats. Instructions to run the script - and the script itself - can be found on Synacktiv GitHub repository: https://github.com/synacktiv/forensic-msvpn.

PS C:\Users\Administrateur.NEBULAD> python .\csv_export_raactdb.py .\out\
PS C:\Users\Administrateur.NEBULAD> dir .\out

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----        27/08/2023     17:27            270 ConnectionTable.csv
-a----        27/08/2023     17:27           1943 EndpointsAccessedTable.csv
-a----        27/08/2023     17:27           6385 ServerEndpointTable.csv
-a----        27/08/2023     17:27            417 SessionTable.csv

Some samples of the export:

$ head -n 5 ConnectionTable.csv |column -s, -t
ConnectionId  Hostname  ClientIPv4Address  ClientIPv6Address  ClientISPAddressType  ClientISPAddress  ConnectionType  TransitionTechnology  TunnelType
1             -         10.0.102.223       ::                 2                     10.0.102.242      2               0                     9
2             -         10.0.102.221       ::                 2                     10.0.102.242      2               0                     9
3             -         10.0.102.222       ::                 2                     10.0.102.242      2               0                     9

$ head -n 5 EndpointsAccessedTable.csv |column -s, -t
ServerEndpointId  SessionId
1                 1
2                 1
3                 1
4                 1

$ head -n 5 ServerEndpointTable.csv |column -s, -t
ServerEndpointId  ServerIpAddressType  ServerIpAddress  Protocol  Port
79                2                    2.16.245.27      6         443
130               2                    2.17.196.96      6         443
111               2                    2.17.196.113     6         443
27                2                    2.17.196.120     6         443

$ head -n 5 SessionTable.csv |column -s, -t
SessionId  ConnectionId  SessionHandle  SessionState  Username       SessionStartTime            SessionDuration  BytesIn   BytesOut   AuthMethod  HealthStatus  NumConcurrentConnections
1          1             4278190115     2             nebulad\alice  2023-08-25 07:43:05.522762  272              94257     1100500    8           0             1
2          2             4278190116     2             nebulad\alice  2023-08-25 09:07:21.589603  17455            926561    3082409    8           0             1
3          3             4278190117     1             nebulad\bob    2023-08-25 13:58:25.471056  178017           11519553  193327184  8           0             1

Conclusion

VPN solutions are more and more targeted by attackers to gain an initial foothold on the corporate network. Microsoft Remote Access VPN is not the most common choice among other more popular solutions integrated to our networks, but as an incident responder it is important to know how to handle forensic analysis in a wide range of products.

Components involved in a Microsoft remote access VPN infrastructure are the:

  • VPN server itself,
  • the domain controller handling the Active Directory and the certificates used to authenticate the different parts,
  • the NPS server handling the RADIUS authentication and access control,
  • and the Windows workstations, acting as the VPN clients.

There are two main scenarios that we considered: VPN connection hijacking and identity theft. For each component, we discovered many pieces of evidence that can help during the investigation:

  • Dashboards in the Remote Access Management Console showing details about every connected client;
  • The reporting feature from Remote Access Management Console that provides persistent connection statistics and user activity;
  • RemoteAccess event IDs, tracing the logon activity on the server side;
  • RasClient event IDs, tracing the logon activity on the client side;
  • And text logs found on both NPS and VPN servers, giving additional traces on VPN activity.

Finally, a Velociraptor artifact, Chainsaw rules and a Python script are provided to help you include this use case in your detection and investigation workflows.

I unfortunately cannot provide raw samples of the events generated in the lab but in case you need more information, feel free to contact me.

Taking into account the insights gained from this forensic analysis, it is strongly recommended to enable the logging of all VPN events as well as the VPN reporting feature. This proactive measure will not only strengthen our ability to investigate incidents but also enhance overall security and facilitate detection of abnormal or suspicious activities.

 

Kudos to Catarina De Faria Cristas for her great presentation about Microsoft RDS forensic analysis18 which inspired me to do this article.