Back

VPC Flow Logs – Analyzing Network Traffic

  • Posted by admin
  • Date July 12, 2022

Configure a custom network with VPC Flow Logs

Create the custom network

  1. In the Console, navigate to Navigation menu > VPC network > VPC networks.
  2. Click Create VPC Network.
  3. Set the following values, leave all others at their defaults:
    Name: vpc-net
    Description: Enter an optional description
  4. For Subnet creation mode, click Custom.
  5. Set the following values, leave all others at their defaults:
    Name: vpc-subnet
    Region: us-west1
    IP address range: 10.1.3.0/24
    Flow Logs: On
  6. Click Done, and then click Create.

Create the firewall rule

  1. In the left menu, click Firewall.
  2. Click Create Firewall Rule.
  3. Set the following values, leave all others at their defaults:
    Name: allow-http-ssh
    Network: vpc-net
    Targets: Specified target tags
    Target tags: http-server
    Source filter: IPv4 ranges
    Source IP ranges: 0.0.0.0/0
    Protocols and ports: Specified protocols and ports, and then check tcp, type: 80, 22
  4. Click Create.

Create an Apache web server

Create the web server

  1. In the Console, navigate to Navigation menu > Compute Engine > VM instances.
  2. Click CREATE INSTANCE.
  3. Set the following values, leave all others at their defaults:
    Name: web-server
    Region: us-west1
    Zone: us-west1-b
    Machine type: f1-micro (1 vCPU, 614 MB memory)
    Firewall: Allow HTTP traffic
  4. Click Management, security, disks, networking, sole tenancy.
  5. Click Networking.
  6. For Network interfaces, click on default to edit.
  7. Set the following values, leave all others at their defaults:
    Network: vpc-net
    Subnetwork: vpc-subnet
  8. Click Done, and then click Create.

Install Apache

  1. Return to the Console, still on the VM instances page (Navigation menu > Compute Engine > VM instances). For web-server, click SSH to launch a terminal and connect.
  2. In the web-server SSH terminal, update the package index: 
    sudo apt-get update
  3. Install the Apache2 package: 
    sudo apt-get install apache2 -y
  4. Create a new default web page by overwriting the default: 
    echo '<!doctype html><html><body><h1>Hello World!</h1></body></html>' | sudo tee /var/www/html/index.html
  5. Exit the SSH terminal: 
    exit

Verify that network traffic is logged

Generate network traffic

  1. Return to the Console, still on the VM instances page (Navigation menu > Compute Engine > VM instances).
  2. To view web-server, click the External IP to access the server.

Find your IP address

  1. Click this whatismyip.host to find your IP v4 address.
  2. Copy your IP address. It will be referred to as YOUR_IP_ADDRESS.

Access the VPC Flow Logs

  1. In the Console, navigate to Navigation menu > Logging > Logs Explorer.
  2. In the Log fields panel, under Resource, click Subnetwork. In the Query results pane, entries from the subnetwork logs appear.
  3. In the Log fields panel, under Log name, click compute.googleapis.com/vpc_flows.
  4. Enter "YOUR_IP_ADDRESS" in the Query search box at the top. Then Click Run Query.
  5. Click on one of the log entries to expand it.
  6. Within the entry, click the arrows to expand the jsonPayload and then the connection. You may have to click Expand all to see the connection.

Export the network traffic to BigQuery to further analyze the logs

Create an export sink

  1. In the Console, in the left pane, click Logs Explorer.
  2. From Resources dropdown, select Subnetwork. Then click Add.
  3. From Log name dropdown, check vpc_flows and click Add. Then, click Run query.
  4. Click More Actions > Create Sink.
  5. For “Sink Name”, type vpc-flows and click NEXT.
  6. For “Select sink service”, select the BigQuery dataset.
  7. For “Sink Destination”, select Create new BigQuery dataset.
  8. For “Dataset ID”, type bq_vpcflows, and then click CREATE DATASET.
  9. Click CREATE SINK. The Logs Router Sinks page appears. You should be able to see the sink you created (vpc-flows). If you are unable to see the sink click on Logs Router.

Generate log traffic for BigQuery

  1. In the Console, navigate to Navigation menu > Compute Engine > VM instances.
  2. Note the External IP address for the web-server instance to use in the next step. It will be referred to as EXTERNAL_IP.
  3. In the Cloud Shell command line, run the following command to store the EXTERNAL_IP in an environment variable. Replace the <EXTERNAL_IP> with the address you just noted. 
    export MY_SERVER=<EXTERNAL_IP>
  4. Access the web-server 50 times from Cloud Shell: 
    for ((i=1;i<=50;i++)); do curl $MY_SERVER; done

Visualize the VPC Flow Logs in BigQuery

  1. In the Console, navigate to Navigation menu (mainmenu.png) > BigQuery.
  2. If prompted, re-enter the Qwiklabs-provided student password and click Sign in.
  3. On the left-hand side, expand the bq_vpcflows dataset to reveal the table. You might have to first expand the Project ID to reveal the dataset.
  4. Click on the name of the table. It should start with compute_googleapis.
  5. Click on Details tab.
  6. Copy the portion of the Table ID that is after the colon(:). It will be referred to as TABLE_ID.
  7. Add the following to the Query Editor and replace your_table_id with TABLE_ID while leaving the accents (`) on both sides: 
    #standardSQL
SELECT
jsonPayload.src_vpc.vpc_name,
SUM(CAST(jsonPayload.bytes_sent AS INT64)) AS bytes,
jsonPayload.src_vpc.subnetwork_name,
jsonPayload.connection.src_ip,
jsonPayload.connection.src_port,
jsonPayload.connection.dest_ip,
jsonPayload.connection.dest_port,
jsonPayload.connection.protocol
FROM
`your_table_id`
GROUP BY
jsonPayload.src_vpc.vpc_name,
jsonPayload.src_vpc.subnetwork_name,
jsonPayload.connection.src_ip,
jsonPayload.connection.src_port,
jsonPayload.connection.dest_ip,
jsonPayload.connection.dest_port,
jsonPayload.connection.protocol
ORDER BY
bytes DESC
LIMIT
15
  8. Click Run.

Analyze the VPC Flow Logs in BigQuery

  1. Create a new query in the Query Editor with the following and replace your_table_id with TABLE_ID while leaving the accents (`) on both sides: 
    #standardSQL
SELECT
jsonPayload.connection.src_ip,
jsonPayload.connection.dest_ip,
SUM(CAST(jsonPayload.bytes_sent AS INT64)) AS bytes,
jsonPayload.connection.dest_port,
jsonPayload.connection.protocol
FROM
`your_table_id`
WHERE jsonPayload.reporter = 'DEST'
GROUP BY
jsonPayload.connection.src_ip,
jsonPayload.connection.dest_ip,
jsonPayload.connection.dest_port,
jsonPayload.connection.protocol
ORDER BY
bytes DESC
LIMIT
15
  2. Click Run.

Add VPC Flow Log aggregation

Setting up aggregation

  1. In the Console, navigate to Navigation menu > VPC network > VPC networks.
  2. Click vpc-net, and then click Edit.
  3. In the Subnets tab, click vpc-subnet:
  4. Click Edit > Configure logs to expose the following fields:
  5. Set the Aggregation Interval to 30 seconds.
  6. Set the Sample rate to 25%.
  7. Click Save. You should see the following message:

Tag:

admin

You may also like

How to install git credential manager core on Ubuntu 22.04 / 20.04
September 28, 2024

Download the GCM wget “https://github.com/git-ecosystem/git-credential-manager/releases/download/v2.5.0/gcm-linux_amd64.2.5.0.deb” -O /tmp/gcmcore.deb Execute the installable file sudo dpkg -i /tmp/gcmcore.deb Configure GCM git-credential-manager configure Configure the GCM store git config –global credential.credentialStore plaintext or export GCM_CREDENTIAL_STORE=plaintext

Configure Azure P2S VPN Gateway with Microsoft Entra Authentication
June 25, 2024

Prerequisites Azure Subscription: Ensure you have an active Azure subscription. Azure AD Directory: You must have an Azure AD directory. User Permissions: Ensure you have permissions to create and manage resources in the Azure portal. Step-by-Step Guide Step 1: Create and Configure the Virtual Network Gateway Create a Virtual Network: …

Hosting Static Website In S3 Bucket
August 24, 2023

Courses

About

© 2024 WebMagic Informatica. All rights reserved.

Facebook Youtube Whatsapp Github Linkedin Instagram