Cloud Functions secured behind Cloud Endpoints

5 min readAug 23, 2020

Google’s Cloud Endpoints has ESPv2 in beta so I pieced together a short journey into on how to set up and deploy a Google Cloud Function that is invoked only via a Cloud Endpoint (and throw and API Key into the mix).

Worth noting, of course, is that this was authored for ESPv2 Beta and is in response to struggling mightily around:

Also worth noting is that Google Cloud’s API Gateway is in Beta which should ultimately replace Cloud Endpoints in this “API proxy” style usage.

The struggle was quite simply on how to set up the Service Account, when to use it and the IAM permissions required for glueing everything together, bearing in mind that what we ultimately want is:

A Cloud Function handling requests
Only from a Cloud Endpoint
With clients secured in some fashion (yeah yeah, API Keys are not really securing things but it’s good enough for illustration)

Given that Google’s own documentation is a little simplistic and Guillaume’s article covers ESP v1, I needed to dive deeper…

The Basics

We need to define some basic variables that we can reuse to make life easier as we go.

PROJECT_ID=awesome-api-99
REGION=europe-west1
ENDPOINT_SERVICE_NAME=api-gw

Make sure this project exists. I just use a non-default Region for illustration. The ENDPOINT_SERVICE_NAME is the name for the Cloud Endpoints Service. It gets deployed in a Cloud Run instance.

I like to set some Cloud SDK defaults as I go but the main next bit is to ensure we’ve got the APIs and Services enabled that we’re going to need.

gcloud config set project $PROJECT_ID
gcloud config set run/region $REGION
gcloud services enable cloudfunctions.googleapis.com
gcloud services enable cloudbuild.googleapis.com
gcloud services enable servicemanagement.googleapis.com
gcloud services enable servicecontrol.googleapis.com
gcloud services enable endpoints.googleapis.com
gcloud services enable run.googleapis.com

Create the Service Account

We’re also going to use a Service Account so that the Cloud Run instance holding the Cloud Endpoint ESP can invoke the Cloud Function.

gcloud iam service-accounts create esp-svc-acc
SVC_ACC=esp-svc-acc@$PROJECT_ID.iam.gserviceaccount.com

Deploy the Cloud Function

I’m not going to write the function for you here, I’m just going to assume that the function is called “my-cloud-function”.

gcloud functions deploy my-cloud-function \
    --runtime python37 --trigger-http \
    --entry-point my-cloud-function

But really, just customise the thing above to suit your function. What’s more interesting here is we are deploying without using the allow-unauthenticated flag so that you can’t call this thing over the interwebs.

Beyond the Basics: the ESPv2 Beta thingy

Now we turn our attention to the Cloud Run ESP instance. From the Google docs we see that we deploy the instance twice… once purely so that we have an instance to reference and another time once we’ve deployed the endpoint configuration and can build it into a customised docker image.

First ESP Deployment

WE deploy the thing so that we can work out the new hostname and config ID.

gcloud run deploy $ENDPOINT_SERVICE_NAME \
 --image="gcr.io/endpoints-release/endpoints-runtime-serverless:2" \
 --allow-unauthenticated --platform managed \
 --service-account=$SVC_ACC

Which will give us an output that ends in something like:

Service [NAME] revision [NAME-REVISION] has been deployed and is serving traffic at URL

What we want to grab is the URL and the hostname of the deployed service. We can do this using some Cloud SDK and awk magic.

ENDPOINT_URL=`gcloud run services describe $ENDPOINT_SERVICE_NAME --platform managed | grep Traffic | head -1 | awk '{ print $2 }'`ENDPOINT_HOSTNAME=`echo $ENDPOINT_URL | awk -Fhttps:// '{print $2}'`

Configure the Cloud Endpoint

The OpenAPI YAML file needs to have the host: field updated with our newly grabbed ENDPOINT_HOSTNAME (this is the hostname of our deployed Cloud Run instance without the https:// bit).

swagger: '2.0'
info:
  title: Some Title
  description: Some Description
host: $ENDPOINT_HOSTNAME

While we’re at it, make sure the thing is configured to require an API Key globally (or whatever your unique requirements are). Also, make sure the x-google-backend fields are set to use your Cloud Function for the specific path in the OpenAPI YAML file.

Deploy the Cloud Endpoint

Deploy the YAML file like:

gcloud endpoints services deploy openapi.yaml

which will output something like

Service Configuration [CONFIG_ID] uploaded for service [HOSTNAME]

We want to get our paws on that CONFIG_ID so either copy it or use some Cloud SDK and awk magic:

CONFIG_ID=`gcloud endpoints configs list --service=$ENDPOINT_HOSTNAME | head -n 2 | tail -n 1 | awk '{print $1}'`

Now we need to rebuild our ESPv2 image by using Googles supplied script.

./gcloud_build_image -s $ENDPOINT_HOSTNAME \
  -c $CONFIG_ID -p $PROJECT_IDESP_VERSION=2.16.0
ESP_IMAGE=gcr.io/$PROJECT_ID/endpoints-runtime-serverless:$ESP_VERSION-$ENDPOINT_HOSTNAME-$CONFIG_IDgcloud run deploy $CLOUD_RUN_SERVICE_NAME \
  --image=$ESP_IMAGE \
  --allow-unauthenticated --platform managed \
  --service-account=$SVC_ACC

Juuuust make sure you get the ESP_VERSION right. I couldn’t figure out yet how to automate that.

Now we have the ESP instance running and the Function running. Al we got to do is link them up with some IAM permissions.

Link Things Up with IAM

Here it got a little funky and, to be honest, I’m not sure what the best way to do things is. However, I do know what worked so just read on and see.

Cloud Run ESP must be able to Invoke the Function

I used:

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member=serviceAccount:$SVC_ACC \
  --role=roles/cloudfunctions.invoker

so that the Service Account that the ESP instances is deployed with can call the function.

The docs tell us to try:

gcloud functions add-iam-policy-binding my-cloud-function \
   --member "serviceAccount:$PROJECT_NUMBER-compute@developer.gserviceaccount.com" \
   --role "roles/cloudfunctions.invoker"

which is not using the Service Account we created but rather the default compute service account. It didn’t work for me but following through with the Service Account did.

Cloud Run ESP must be able to fiddle with service management

gcloud projects add-iam-policy-binding $PROJECT_ID \
   --member "serviceAccount:$SVC_ACC" \
   --role roles/servicemanagement.serviceController

Various docs ask us to grant roles/servicemanagement.configEditor to the Service Account but I can’t understand why.

Wrapping it up

Run some curl or httpie or some postman and voila! It works.

For an API Key go to API & Servicesand select Credentials. Click on Create credentialsand select API Key. That said, I found this doc on API Keys super useful.

I’d like to understand better the IAM configuration for this deployment and why the permissions granted the way I did in my examples resulted in success but not when following Google’s tutorials.