Supporting REST and HTML with a gRPC Microservice

Including Dependencies

Envoy is organized into seperate packages to minimize requirements and improve organization.

The code that makes HTTP annotations work is in a separate annotations.proto file. Standard practice with .proto files differs from Swift Package Manager (SPM).

Although SPM can download dependencies when it runs, for .proto files, you’ll want to download dependencies first to prevent unexpected changes from breaking your application.

Make a new sub-directory (inside the project root) to hold the dependency:

mkdir -p google/api

Download the current version of the annotations.proto from Google using cURL:

curl https://raw.githubusercontent.com/googleapis/googleapis/master/google/api/annotations.proto > google/api/annotations.proto

Download http.proto — it’s a dependency for annotations.proto:

curl https://raw.githubusercontent.com/googleapis/googleapis/master/google/api/http.proto > google/api/http.proto

Add an import statement to your todo.proto file.

Open the todo.proto in a text editor and replace the //TODO: Add AnnotationsImport line with the following:

import "google/api/annotations.proto";

Great, now you have your house in order with all the dependencies installed.

Next up, you’ll cover the basic options you have for using HTTP GET verbs.

Transcoding HTTP GET Verbs

In the original todo.proto, the FetchTodos procedure call takes an empty input then returns a list of todos, similar to an HTTP GET request.

Modify the FetchTodos definition, inserting an annotation between the curly braces. Your finished code should look like this:

rpc FetchTodos (Empty) returns (TodoList) {
  option (google.api.http) = {
    get: "/v1/todos"
  };
}

The code tells the Envoy proxy to convert an inbound HTTP GET of http://yourserver/v1/todos to a gRPC call of FetchTodos. It then converts the response of TodoList to a JSON array.

Another available service that is similar to an HTTP GET is CompleteTodo. Unlike FetchTodos, this service has an input parameter. When using HTTP GET, input parameters are usually coded in the URL. The annotations support this pattern.

Find the CompleteTodo service and insert this annotation between the curly braces:

option (google.api.http) = {
  get: "/v1/todos/{todoID}/complete"
};

With this, you tell Envoy to extract a value from the URL and assign it to todoID — capitalization matters here. The todo.proto definition for CompleteTodo expects a message of type TodoID.

Look at the definition for the TodoID message:

message TodoID {
  string todoID = 1;
}

One of the fields is a string type, called todoID. At runtime, Envoy uses the string extracted from the URL to create a TodoID then passes a gRPC call to the server.

Transcoding an HTTP POST Verb

For an HTTP POST request, you must specify what the body of the POST contains.

First, find the entry for CreateTodo, and add this annotation:

option (google.api.http) = {
  post: "/v1/todos"
  body: "*"
};

The body line indicates that the payload will contain keys and values at the root level.

Envoy will attempt to decode them into the needed message. Any fields missing from the request payload will be assigned default values in gRPC.

Still in todo.proto, observe how it defines a Todo. It should look like this:

message Todo {
  optional string todoID = 1;
  string title = 2;
  bool completed = 3;
}

The todoID is optional, and completed is a bool which has a default value of false.

When the client creates the body of the POST, it uses JSON:

{
"title": "Buy Spinach and Olive Oil"
}

Using an asterisk for the body is just one pattern. For this tutorial, you’ll stick to the asterisk.

An example of this form is below — it would change the service and create a new message:

In the example above, the body expects a JSON object that maps to a Todo message. That would require changing the server and client code, which is beyond the scope of this tutorial.

rpc CreateTodo(CreateTodoRequest) returns (Todo) {
  option (google.api.http) = {
    post: "/v1/todos"
    body: "todo"
  };
}

message CreateTodoRequest {
  Todo todo = 1;
}

rpc CreateTodo(CreateTodoRequest) returns (Todo) {
  option (google.api.http) = {
    post: "/v1/todos"
    body: "todo"
  };
}

message CreateTodoRequest {
  Todo todo = 1;
}

By now, you can see a pattern for annotating gRPC procedure calls with HTTP. There’s still more to learn, so keep reading.

Transcoding Other Verbs

In todo.proto, there is one call left to explore: DeleteTodo. It uses the TodoID, similarly to how CompleteTodo uses it, but there is a different HTTP verb.

Try it out for yourself. Annotate DeleteTodo like this:

option (google.api.http) = {
  delete: "/v1/todos/{todoID}"
};

Similar to CompleteTodo above, you tell Envoy to extract a value from the URL and assign it to todoID.

Additionally, gRPC supports PUT and UPDATE, as well as others. Google’s gRPC Service Configuration Reference for gRPC Transcoding explains the implementation. It also covers how to use URL query values and a few other tricks.

Generating an Annotated Protobuf File

At this point, you’ve annotated todo.proto and put the imports in place, and you’re ready to generate a todo.pb file for Envoy to use.

Save your changes to todo.proto. Make sure your working directory is the root for your project. Execute this command to tell protoc to generate todo.pb:

protoc -I. --include_imports --include_source_info --descriptor_set_out=todo.pb todo.proto

Here’s what you’re doing with that command:

• -I. tells protoc to look for imports starting in the current directory.
• --include_source_info and --include_imports work together with --descriptor_set_out to create todo.pb as a self-contained, meaning it needs no dependency references at runtime.

Copy the new todo.pb to the envoy folder so it’s adjacent to the Envoy configuration files.

Before you configure Envoy to do the transcoding, open docker-compose.yml in a text editor and overwrite volumes within the Envoy section with the following:

- ./envoy/grpc-envoy.yml:/etc/envoy/envoy.yaml
- ./envoy/todo.pb:/data/todo.pb:ro

The first line will now copy grpc-envoy.yml into the server, and the second line will copy todo.pb into the server’s container.

Ok, you’re almost to the good part. Keep going! The last step is to configure Envoy to actually do the transcoding.

Configuring Envoy for Transcoding

Open the envoy directory then open grpc-envoy.yml in a text editor. This file is a sample taken from the Envoy documentation and is a basic, bare configuration to support transcoding.

The first entry for admin assigns the administration website to port 9901. In the section for static_resouces there are listeners and clusters.

Envoy uses one listener for one port. Skim through the configuration to take note of a few more attributes:

• There’s a single listener watching port 8082.
• There’s an entry for stat_prefix, which is just the prefix that any log entries will have.
• In the routes section, note that the server is going to match using the “/” prefix, meaning it’ll match everything.
• You can also see that it’ll send traffic to a cluster named grpc, which is defined further down.
• And before Envoy routes any traffic, it’ll apply the http_filters.