Responding to Cloud Logging Messages with Cloud Functions

gcloud pubsub topics create vm-audit-logs

export ZONE=us-central1-c

gcloud compute instances create --zone $ZONE instance-1

gcloud compute instances stop --zone $ZONE instance-1

1. In the first drop-down, select Compute Engine VM Instance > All instance_id
2. In the second drop-down, select cloudaudit.googleapis.com/activity. Click OK.

1. Click on the Create Export button.
2. Sink Name: instance-insert-sink.
3. Sink Service: Cloud Pub/Sub.
4. Sink Destination: vm-audit-logs (the Cloud Pub/Sub topic you created earlier as the sink destination).
5. Click Create sink
6. Close the acknowledgement dialog

1. In Cloud Shell make a new directory to hold your Cloud Function code:

   mkdir ~/gcf-vm-metadata

2. Execute the below command to create a new Python file called main.py with the below code in the new directory. Read through the code to understand its operation:

   cat > ~/gcf-vm-metadata/main.py <<'EOF'
   import base64
   import json
   import googleapiclient.discovery
   def tag_with_creator(event, context):
       """Adds a custom metadata entry for a new virtual machine.
       Triggered by a Cloud Pub/Sub message containing a Compute Engine
       audit activity Stackdriver log message
       """
       pubsub_message = base64.b64decode(event['data']).decode('utf-8')
       msg_json = json.loads(pubsub_message)
       proto_payload = msg_json['protoPayload']
       resource_name = proto_payload['resourceName']
       email = proto_payload['authenticationInfo']['principalEmail']
       # compute engine API
       compute = googleapiclient.discovery.build(
           'compute', 'v1', cache_discovery=False)
       # full name is of the form
       # projects/$PROJ_NAME/zones/$ZONE/instances/$INST_NAME
       name_tokens = resource_name.split('/')
       project = name_tokens[1]
       zone = name_tokens[3]
       instance_name = name_tokens[5]
       # need to get current vm metadata before we can update it
       vm_details = compute.instances().get(
           project=project, zone=zone, instance=instance_name).execute()
       vm_metadata = vm_details['metadata']
       # add/replace metadata item
       _update_metadata(vm_metadata, 'creator', email)
       response = compute.instances().setMetadata(
           project=project, zone=zone, instance=instance_name,
           body=vm_metadata).execute()
       print('Updated metadata for resource %s' % resource_name)
   def _update_metadata(vm_meta, key, value):
       """Update existing vm metadata with the supplied key/value pair.
       If the key already exists, value is overwritten.
       """
       if 'items' not in vm_meta:
           vm_meta['items'] = []
       for item in vm_meta['items']:
           if item['key'] == key:
               item['value'] = value
               return
       vm_meta['items'].append({
           'key': key,
           'value': value
       })
   EOF
   

3. Execute the following to create a new requirements.txt file with the required Python dependency:

   cat > ~/gcf-vm-metadata/requirements.txt <<'EOF'
   google-api-python-client==1.7.4
   EOF

4. Execute the below command to update the Cloud Function you created earlier. Note that the --source parameter references the directory that contains the main.py source file and requirements.txt dependency file:

   gcloud functions deploy addVmCreatorMetadata --source ~/gcf-vm-metadata --entry-point tag_with_creator

5. Refresh the Cloud Functions page in the Console and verify that the Executed function field has been updated to the new tag_with_creator value.

1. In Cloud Shell, restart the virtual machine you created earlier:

   gcloud compute instances start --zone $ZONE instance-1

2. Once the VM has completed starting, navigate to Logging > Logs.
3. In the first filter drop-down, select Cloud Function. You should see some log messages relating to the creation and update of the function performed in the previous step.
4. Create a new VM in Cloud Shell, which should trigger the Cloud Function:

   gcloud compute instances create --zone $ZONE instance-2

5. Once the VM creation has completed, go back to refresh the Logging view pane with the existing filter intact. You should now see some log messages, indicating that the Cloud Function executed.
6. Examine the log messages and cross-reference with the function code to gain an understanding of the different steps.
7. Navigate to Compute Engine -> VM Instances and click into the instance-2 machine. Verify that the metadata for the newly created VM has been updated to include the email of the account that created it.
8. Scroll down through the details and verify that the Custom Metadata section has the creator key and expected email address added by the Cloud Function