gRPC – gRPC Blog

Blog: gRPConf 2025 Announcement

Mon, 30 Jun 2025 00:00:00 +0000

Attention gRPC community!

Mark your calendars for August 26th, 2025, as gRPConf returns to the Google Cloud Campus in Sunnyvale, California. The schedule is now live!

This is your chance to dive deep into the world of gRPC, connect with fellow developers, and stay ahead of the curve in all things gRPC.

Why Attend?

Learn: Explore the latest gRPC advancements, best practices, and real-world use cases through a series of informative talks and workshops.
Connect: Network with a vibrant community of gRPC experts, users, and open-source contributors.
Share: Discuss your own gRPC experiences, challenges, and successes with peers who share your passion.
Innovate: Gain inspiration and discover new ways to leverage gRPC in your own projects.

Register Now!

Early bird registration is available until July 30th for $50. Keep an eye on the official gRPC website and social media channels for updates.

We can’t wait to see you at gRPConf 2025!

Blog: How robots talk: building distributed robots with gRPC and WebRTC

Wed, 18 Jun 2025 00:00:00 +0000

If you’re building any kind of distributed machine, like robots, drones, or IoT devices, two questions come up fast:

How do you send structured commands and data efficiently?
How do you keep communication resilient when you have an unreliable network?

In this article, I’ll share the approach we use at Viam, an open-source robotics platform that combines gRPC for structured RPCs and WebRTC for peer-to-peer streaming. Even if you’re not building robots, the same architectural ideas can help you design more scalable and adaptable systems.

Bridging the gap between lab-grade robotics and real-world systems

ROS (Robot Operating System) has long been the go-to framework for building and prototyping robotics systems, especially in research and academia. It works well on trusted, local networks because that’s what it was designed for.

But today’s robots don’t stay on tightly-regulated lab benches. They’re deployed on factory floors, out at sea, or in remote fields, where network conditions are unpredictable and reliability matters.

ROS doesn’t handle network communication out of the box. If you need to talk to a robot over the internet, you’re on your own to architect the solution. Modern robotics requires more than local LAN protocols. It needs cloud-ready, peer-to-peer, resilient communication stacks.

Rather than relying on a distributed messaging stack like MQTT or relaying commands through the cloud, Viam uses gRPC and WebRTC to power its robotics platform. Used together, these protocols offer a modern alternative that complements ROS’s strengths. gRPC provides structured, language-agnostic APIs with fast, efficient Protobuf serialization. And WebRTC adds low-latency, peer-to-peer connections, which is ideal for streaming sensor data or real-time video between machines.

Some robot builders are also layering gRPC interfaces on top of ROS, blending ROS’s rich ecosystem with the modern transport and scalability of gRPC.

Why gRPC makes sense for robots

Modern robots are no longer single-purpose machines in fixed environments. They’re mobile, multi-part systems that need to collaborate with sensors, perception services, and control loops, often in real time.

gRPC supports these needs in several key ways:

Low-latency, real-time control: gRPC enables continuous, bidirectional streams, allowing systems to send real-time pose updates to a robotic arm, while simultaneously receiving telemetry, without additional round-trip delays.
Cross-platform, multi-language consistency: gRPC automatically generates client libraries (stubs) for languages like Python, Go, C++, and others, making it easier to bridge different devices, SDKs, and environments with a consistent interface.
Lightweight and efficient serialization: gRPC also helps in bandwidth-constrained environments or on embedded devices, enabled by Protobuf’s lightweight binary encoding.
Security benefits: gRPC is designed with end-to-end encryption (TLS by default) and authentication capabilities baked in, ensuring secure communication between machine parts, whether across local networks or the public internet, without requiring developers to bolt on separate security layers.
Service abstraction for machine parts: With gRPC and Protobuf, every part of a robot, from motors to cameras to sensors, can be modeled as a standardized, language-agnostic service.

In Viam’s public API, each robot component is defined as a gRPC service using Protobuf. Components like arms, cameras, and sensors expose typed methods that can be called remotely. Here’s an excerpt from the arm.proto file that defines basic movement and pose retrieval methods for a robotic arm:

// An ArmService services all arms associated with a robot
service ArmService {
 rpc MoveToPosition (MoveToPositionRequest) returns (MoveToPositionResponse);
 rpc GetEndPosition (GetEndPositionRequest) returns (GetEndPositionResponse);
 rpc GetJointPositions(GetJointPositionsRequest) returns (GetJointPositionsResponse);
}

These proto services can then generate SDKs to help you work with your machines. This example uses the Python SDK to move an arm. Under the hood, gRPC handles encoding the command into a Protobuf message and sending it over HTTP/2 (or WebRTC, depending on the environment).

from viam.components.arm import ArmClient

arm = ArmClient.from_robot(robot=robot, name="my-arm")

# Move the arm to a target 3D position
arm.move_to_position(x=0.1, y=0.2, z=0.3, orientation=None, world_state=None)

Why WebRTC is also part of the picture

At first glance, gRPC and WebRTC might seem redundant. Both can stream data and send structured messages. But they solve very different challenges in robotics.

WebRTC excels at direct, peer-to-peer connections, which is essential when:

Devices are on local networks or constrained by NAT/firewalls
You want to bypass a central relay or cloud server
You’re streaming high-bandwidth sensor data (like real-time video or LIDAR) between machines

Viam uses gRPC for orchestration and WebRTC as the underlying transport, combining them for fast, structured messaging with minimal routing overhead.

Flexible transport layers with gRPC

While WebRTC enables peer-to-peer communication in distributed machines, gRPC also offers flexibility to swap underlying transport layers.

Here is an example gRPC server that runs over WebRTC and other transport protocols. Viam also uses Unix domain sockets (UDS) for local messaging between the viam-server and internal modules, Bluetooth for provisioning machines or proxying network traffic, and serial ports for embedded peripherals.

Because gRPC defines a consistent interface at the API layer, we can change the transport depending on the environment or device without rewriting client logic, a huge strength when building systems that span cloud, local, and constrained networks.

A real-world example: coordinating a claw game

Picture a remote-controlled claw machine, like the ones you’d find in an arcade. It has two main components: a camera and a robotic arm. A user interface, such as a web app or mobile app, sends control commands and receives a live video stream to help guide the claw to its prize.

Here’s how gRPC and WebRTC work together behind the scenes:

Initialization: The system establishes control channels using gRPC over HTTP/2. Each component registers its services (e.g. ArmService, CameraService).
Connection: gRPC coordinates the exchange of peer metadata and signaling to set up a WebRTC session between the SDK and machine parts.
Real-time operation: WebRTC now handles the media and data streams directly between peers. Commands are sent using gRPC method calls, routed over the WebRTC transport. Video and sensor streams flow the other way.

This dual-protocol approach allows real-time interaction, smooth streaming, and resilient fallback behavior, even in unpredictable network conditions.

Why this pattern matters for the broader community

The combination of gRPC’s structure and WebRTC’s flexibility opens the door to a new class of distributed systems, not just in robotics, but anywhere you need:

Real-time, bi-directional communication between loosely-coupled systems
Typed, versioned APIs that span languages and devices
Transport-agnostic flexibility, with the option to go peer-to-peer when needed

gRPC structures the conversation. WebRTC carries it across the network.

Viam for real-time robot control

gRPC has evolved far beyond simple request/response APIs. Across many industries beyond robotics, the way machines communicate is more critical than ever. gRPC and WebRTC aren’t just faster protocols, they’re influential building blocks for the next generation of distributed, resilient, and intelligent machines.

If you’re building systems that need to talk to each other across devices, environments, and networks, it might be time to explore a more scalable and adaptable communication layer.

While you could engineer a similar stack on your own, Viam offers these capabilities out of the box. The on-machine functionality is open-source and free to operate, with optional cloud services available on a usage-based model when you scale to fleets.

Technical review by Nick Hehr (Viam)

Blog: gRPC and AI: A Powerful Partnership

Tue, 20 May 2025 00:00:00 +0000

We recently showcased the exciting potential of Large Language Models (LLMs) to transform gRPC development at gRPConf 2024. If you’re curious to see how AI can streamline your gRPC workflows, read on! We would love to emphasize that these capabilities are accessible to you today using Gemini Code Assist.

In our gRPConf demonstration, we explored how LLMs, particularly Google’s Gemini, can streamline your gRPC workflows in three key ways. Now let’s see how can you utilize LLM’s gRPC knowledge using Gemini Code Assist as an example:

Swagger to Protobuf: Explore Your API as a gRPC Service: Curious to see what your existing REST APIs would look like as high-performance gRPC services? Open your OpenAPI YAML file, then write prompts like

“Convert OpenAPI schema to Protobuf”

to explore the potential of gRPC for your current services, accelerating adoption, and discovering new possibilities for your API architecture.
Protobuf to JUnit Test Cases: Supercharge Your Testing Workflow: Want to ensure your gRPC services are robust and reliable? You can use prompts like

“Create JUnit tests for a service implementing PetStoreService in petstore.proto”

to automatically generate comprehensive JUnit test cases from your Protobuf definitions. Experience how you can quickly create tests in multiple languages (Java, Go, Python, etc.), enabling you to focus on building innovative features with confidence.
Natural Language to Protobuf: Unleash Your Service Ideas: Have a vision for a new gRPC service? You can directly describe your service concepts in plain English, using prompts like

“Help me create a pet store gRPC service, and it should list all the pets, create a new pet, and get pet information by ID.”

to generate Protobuf schemas. Discover how you can quickly prototype new services, foster collaboration between teams, and bring your ideas to life with ease.

LLMs like Gemini possess a significant understanding of gRPC concepts and code structures, enabling you to leverage them in your development workflow. We’re excited to continue exploring the potential of LLMs to further streamline workflows, automate tasks, and empower developers to build and maintain high-quality gRPC services.

At the same time, feel free to start trying Gemini Code Assist with your own prompts today!

Catch the recording of our demo in the below YouTube video to see these prompt-based transformations in action. And don’t miss the opportunity to connect with the gRPC community and explore more innovations at the upcoming gRPC Conference on August 26th, Sunnyvale, CA. You can

Sign up for the upcoming gRPC Conference in 2025 at https://events.linuxfoundation.org/grpconf/
Submit a talk proposal before May 28th, 11:59 PM PDT at: https://events.linuxfoundation.org/grpconf/program/cfp/

Stay tuned for more exciting advancements in the gRPC and AI space at the upcoming gRPC Conference in 2025!

Blog: Highlights in gRPConf 2024:Customer Showcase, Developer Engagement, Birds of Feathers Discussions and more.

Mon, 25 Nov 2024 00:00:00 +0000

We loved hosting gRPConf 2024 this summer in Sunnyvale, California. Thank you to hundreds of developers from Netflix, Apple, Turso, Microsoft, Cisco, Coinbase, LinkedIn, Datadog, Mercari, Aspect Build, Signeen, Authzed, Reboot.dev, to name a few, who joined our special event. We were overwhelmed by the various topics and advanced use cases around the gRPC ecosystem. If you missed the opportunity to join in person, here are the video recordings that highlight the presentations during the day:

We received a wealth of feedbacks from everyone throughout the day which has helped us shape our future roadmap. Stay tuned to gRPC on many platforms for new releases, announcements and events:

Blog: Announcing Our New Mailing List grpc-io-announce

Wed, 06 Nov 2024 00:00:00 +0000

Want to stay ahead of the curve on critical gRPC announcements? We’ve got you covered!

Introducing grpc-io-announce, a new mailing list dedicated to delivering essential updates directly to your inbox. This high-signal channel cuts through the noise and focuses on what matters most, including:

Security vulnerabilities and reliability issues: Be the first to know about security concerns and get timely updates on fixes and mitigations.
Major platform support changes: Stay informed about changes in support for different platforms.
Project and governance news: Get the latest updates on major project milestones and governance decisions.

Note: Release announcements and feature launch announcements will stay in the existing grpc-io group.

No more scouring through grpc-io groups or worrying about missing critical information. Subscribe to grpc-io-announce and get peace of mind knowing you won’t miss critical information.

Join the group today: grpc-io-announce group

For more information, please see P6 grpc-io-announce.

We’re committed to keeping you informed and ensuring a smooth gRPC experience.

Blog: Configuring proxyless gRPC with Kubernetes Gateway API and OpenTelemetry metrics for Cloud Service Mesh now available in preview

Thu, 12 Sep 2024 00:00:00 +0000

We are excited to announce that configuring proxyless gRPC with the Kubernetes Gateway API and OpenTelemetry metrics for Cloud Service Mesh are now available as preview features. This integration allows you to leverage the powerful capabilities of the Kubernetes Gateway API to manage traffic to your gRPC services within the Cloud Service Mesh environment, along with the added benefit of gRPC OpenTelemetry metrics for enhanced observability.

What’s new:

Kubernetes Gateway API Integration: Now you can use the Kubernetes Gateway API to define and manage traffic routing for services running in your Cloud Service Mesh. This provides a more standardized and flexible approach to managing ingress and traffic management within your Kubernetes clusters.
gRPC OpenTelemetry Metrics for Cloud Service Mesh: Gain deeper insights into your gRPC services with integrated OpenTelemetry metrics, providing valuable observability data for monitoring and troubleshooting.

Benefits:

Simplified traffic management: The Kubernetes Gateway API provides a declarative way to manage traffic routing, making it easier to configure and manage ingress for your services.
Enhanced observability: gRPC OpenTelemetry metrics for Cloud Service Mesh provide valuable insights into the performance and health of your gRPC services, aiding in monitoring and troubleshooting.

Get started:

To learn more about how to configure Cloud Service Mesh with the Kubernetes Gateway API and try out this new feature, check out our documentation and tutorials. More information on OpenTelemetry Metrics for Cloud Service Mesh can be found in our documentation.

We are eager to hear your feedback on these new features. Please share your thoughts and experiences through our community channels.

Blog: gRPConf 2024 Schedule

Wed, 03 Jul 2024 00:00:00 +0000

Attention gRPC community!

Mark your calendars for August 27th, 2024, as gRPConf returns to the Google Cloud Campus in Sunnyvale, California. The schedule is now live!

This is your chance to dive deep into the world of gRPC, connect with fellow developers, and stay ahead of the curve in all things gRPC.

Why Attend?

Learn: Explore the latest gRPC advancements, best practices, and real-world use cases through a series of informative talks and workshops.
Connect: Network with a vibrant community of gRPC experts, users, and open-source contributors.
Share: Discuss your own gRPC experiences, challenges, and successes with peers who share your passion.
Innovate: Gain inspiration and discover new ways to leverage gRPC in your own projects.

Register Now!

Early bird registration is available until July 30th for $50. Keep an eye on the official gRPC website and social media channels for updates.

We can’t wait to see you at gRPConf 2024!

Blog: Celebrate gRPC Day

Tue, 12 Mar 2024 00:00:00 +0000

The gRPC core engineering team is excited to share a new video playlist that accelerates your getting started with gRPC and helps you better understand recently introduced innovations such as gRPC Microservices Observability in GCP (for the full announcement, see blog and codelab.

The materials are for any audience, including individuals starting their gRPC journey as well as established gRPC coders, and place the core technologies of gRPC at your fingertips.

The video series includes the following clips:

Individuals new to gRPC should start with sessions #1-3, while seasoned gRPC developers can jump into the advanced topics covered by videos #4-7.

Checkout the YouTube playlist gRPC Day to start watching!

Blog: Can gRPC replace REST and WebSockets for Web Application Communication?

Mon, 04 Dec 2023 00:00:00 +0000

Welcome to this guest blog entry from our friends at Postman!

In the rapidly evolving landscape of web development, efficiency and performance often stand at the forefront of adoption of new technologies. The work being done with the gRPC-Web library marks a pivotal shift in how developers can utilize the speed and power of gRPC into client-server communication in web applications by replacing REST and some aspects of WebSockets. Let’s look at a comparative analysis with traditional RESTful calls and WebSocket connections, and offer practical code samples with gRPC-Web to draw comparisons in each approach.

Understanding gRPC-Web and Its Place in Modern Web Development

The genesis of gRPC-Web lies in the quest for more responsive, low-latency web applications. It extends the capabilities of gRPC – a high-performance, open-source universal RPC framework – to the browser, enabling direct communication with gRPC services normally designated to server-to-server communication. gRPC is structured around a serialization format called Protocol Buffers (Protobuf), which facilitates smaller payloads and well-defined interface descriptions that streamline the development process.

Before diving into the technicalities, let’s examine the potential shift that gRPC-Web represents. Unlike the native gRPC protocol, which requires HTTP/2, gRPC-Web relaxes this requirement, allowing it to support any HTTP/* protocols available in a browser environment. In a WebSocket scenario, a persistent connection is maintained for full-duplex communication, but WebSockets can introduce complexity in managing various connection states. gRPC-Web offers a compelling alternative with its server streaming capabilities, leading to a more efficient real-time data flow. At present, gRPC-Web does not support client-side streaming due to browser constraints.

The Mechanics of gRPC-Web: How It Works

To integrate gRPC-Web into your web application, a specific architecture must be adopted. At the heart of this architecture lies the Envoy proxy, which serves as a bridge between the web application and the gRPC server (this is necessary to allow abstraction of network protocols). Envoy translates the gRPC-Web calls into gRPC calls, handling the HTTP/1.1 to HTTP/2 conversion, allowing browsers to enjoy the benefits of gRPC.

Let’s break down the steps involved in this communication model:

The browser initiates a gRPC-Web client call.
The Envoy proxy receives the call, which includes the Protobuf-defined request.
Envoy then translates this into an HTTP/2 gRPC call and forwards it to the gRPC server.
The gRPC server processes the request and returns the response to Envoy.
Envoy converts the gRPC response back into gRPC-Web format and sends it to the client.

Using this proxy system, gRPC-Web facilitates a robust client-server interaction that is performant, and offers data clarity and precision thanks to the strong data typing of Protobuf.

The Theory of Transitioning from REST to gRPC-Web

For developers accustomed to REST, the leap to gRPC-Web may seem challenging. The transition can be smooth with a proper understanding of the involved components and a step-by-step approach.

Consider a typical RESTful fetch call:

fetch('https://api.example.com/data', {
 method: 'GET',
 headers: {
 'Accept': 'application/json',
 },
})
.then(response => response.json())
.then(data => console.log(data))
.catch(error => console.error('Error:', error));

The above code retrieves JSON data from a RESTful API service. Notice the use of the fetch API, HTTP method, and content type headers. The response is processed as a JSON object, and error handling is baked into the promise chain. What is not indicated in this code is the data validation that is needed to ensure the data you received in the payload matches an expected schema, and that the data received in the schema is properly set in the data type you require, and the user experience of handling erroneous data.

Let’s reimagine this with gRPC-Web:

const { ExampleRequest, ExampleResponse } = require('./generated/example_pb.js');
const { ExampleServiceClient } = require('./generated/example_grpc_web_pb.js');

const client = new ExampleServiceClient('https://api.example.com');

const request = new ExampleRequest();

client.getExampleData(request, {}, (err, response) => {
 if (err) {
 console.error('Error:', err);
 } else {
 console.log(response.toObject());
 }
});

In this gRPC-Web example, we begin by importing the necessary Protobuf definitions and client stub. A client instance is created, specifying the service URL. We construct a request object, and the getExampleData method is called on the client, passing the request and a callback function for handling the response or error.

Notice the stark difference in approach: gRPC-Web calls are strongly-typed, and serialization/deserialization is handled by the library, not manually by the developer. This type safety and automation can drastically reduce the potential for human error and streamline the development process. If you receive an object, it has already been fully validated.

Advantages of gRPC-Web Over REST

While REST has been the cornerstone of web APIs for years, its simplicity can sometimes be a limitation when it comes to complex web applications. While gRPC-Web can work with any HTTP/* protocols supported in the browser, gRPC-Web leverages many HTTP/2 features, bringing a host of improvements. Here are some advantages of HTTP/2 and gRPC-Web:

It works with existing services: There’s nothing new to build other than the Envoy proxy, so implementing gRPC-Web will allow you to access any existing gRPC service. This can be an advantage for applications, including mobile apps, utilizing JavaScript libraries.
Type Safety: With gRPC-Web, both requests and responses are strongly typed based on the Protobuf definitions. This contract between the client and server is explicit, reducing the likelihood of miscommunication and bugs.
Efficient Serialization: Protobuf, the serialization format used by gRPC, is more efficient than JSON or XML, leading to quicker serialization and smaller message sizes. This can be particularly beneficial for performance and can lead to cost savings in terms of bandwidth. HTTP/1.1 allows data to be sent in text mode or binary mode, but not both. HTTP/2 is binary-only and encoding/decoding binary into text is less error-prone than encoding/decoding a binary file into text to send a mixed payload over REST.
Clear API Contracts: Using Protobuf for service definition creates a clear, language-agnostic API contract. This can be used to generate client and server code in multiple languages, providing a seamless experience for developers.

Setting Up a gRPC-Web Environment

Getting started with gRPC-Web will require defining services and message payloads using Protobuf, setting up the gRPC backend service (or a mock server for the time being), and configuring an Envoy proxy to translate between gRPC-Web and gRPC.

First, you define your service in a .proto file:

syntax = "proto3";

package example;

service ExampleService {
 rpc GetExampleData(ExampleRequest) returns (ExampleResponse);
}

message ExampleRequest {
 string query = 1;
}

message ExampleResponse {
 repeated string data = 1;
}

This .proto file defines a simple service with a single RPC method, GetExampleData, along with the request and response message formats. Since the operation sends a single ExampleRequest message in the request, and expects to receive a single ExampleResponse message in the response, this unary RPC call mimics a RESTful request.

Next, generate the client stub code for your service using the protoc command-line tool with the appropriate gRPC-Web plugin. (Here is an example from the gRPC-Web quickstart documentation). This process will create the JavaScript client files you’ll need to make gRPC-Web calls from the browser.

After your gRPC server is implemented in the language of your choice, you’ll configure an Envoy proxy. Here is another example from the gRPC-Web quickstart documentation.

Here is some YAML syntax of an Envoy configuration which enables gRPC-Web as part of the larger configuration linked above.

http_filters:
- name: envoy.filters.http.grpc_web
- name: envoy.filters.http.router

With these elements in place, you can start making gRPC-Web calls from your web application.

Defining Service Methods with Protobuf

When defining your service methods, Protobuf acts as the single source of truth by defining the message structure for requests and responses. This strict schema allows for automatic client and server code generation in multiple languages. For JavaScript in particular, this code generation streamlines the call process for the browser client.

Using the example .proto file above, the generated JavaScript client code would use these definitions to ensure that only the correct data types are sent and received. This process handles much of the manual data validation and parsing that can be error-prone with RESTful services.

Replacing a Typical WebSocket Connection with gRPC-Web

WebSockets provide a full-duplex communication channel over a single long-lived connection. In scenarios where gRPC-Web cannot fully replace WebSockets due to its lack of client streaming capabilities, it can still be used for efficient server-to-client streaming.

Here’s a typical WebSocket implementation example:

const socket = new WebSocket('ws://example.com/data');

socket.onmessage = function(event) {
 const receivedData = JSON.parse(event.data);
 console.log(receivedData);
};

socket.onerror = function(error) {
 console.error('WebSocket Error:', error);
};

The WebSocket API is straightforward, but managing the state and lifecycle of the connection can become complex.

Now, let’s explore how server-side streaming would look with gRPC-Web:

const { Empty } = require('./generated/common_pb.js');
const { DataServiceClient } = require('./generated/data_grpc_web_pb.js');

const client = new DataServiceClient('https://api.example.com');

const request = new Empty();

const stream = client.dataStream(request, {});

stream.on('data', (response) => {
 console.log(response.toObject());
});

stream.on('error', (err) => {
 console.error('Stream Error:', err);
});

stream.on('end', () => {
 console.log('Stream ended.');
});

While there is more code involved to replace WebSockets with gRPC-Web, you can set up a server-side streaming call where the server can continuously send messages to the client. The client uses event listeners to handle incoming messages, errors, and the end of the stream. It’s a different paradigm than WebSockets but one that can be more efficient and easier to manage in the context of supported use cases.

Many chat applications over WebSockets utilize single client sends and server streamed events, which gRPC-Web can replace. Even in scenarios where a multiplayer game was developed, an RPC call of “MoveCharacters” could take a single message from the browser from where you move your character, and stream back all of the movements of other players or computer-controlled characters.

Is It Time to Replace REST and WebSockets?

This article has begun to scratch the surface of replacing REST and WebSockets with gRPC-Web, focusing on the reasons for doing so and how to get started with practical code samples. More work would be needed to fully incorporate error handling, and to show performance benchmarking, which are beyond the scope of this document.

Many technical aspects of gRPC and gRPC-Web, using Envoy, can replace REST and WebSockets in modern web application development. While there are few public-facing gRPC APIs, it’s a matter of time before we see more companies adopting the performant nature of HTTP/2 and HTTP/3 based APIs, and consider alternative emerging technologies for web applications.

gRPC Support in Postman

If you work with APIs, you probably use Postman. Did you know that Postman supports gRPC? Our VS Code extension supports gRPC requests, too, while you’re building your application. We had a great time attending gRPC Conf this year and meeting with the community. Keep an eye on our blog for upcoming updates and articles about gRPC. If you’d like a little history on gRPC, Protobuf, and how they’re used within Postman, you can also check out our Postman Academy course.

Technical reviews by Kevin Swiber (Postman), Eryu Xia (Google)

Blog: Highlights in gRPConf 2023:Customer Showcase, Developer Engagement, Birds of Feathers Discussions and more.

Fri, 27 Oct 2023 00:00:00 +0000

We loved hosting gRPConf 2023 last month in Sunnyvale, California. Thank you to hundreds of developers from Apple, Cisco, Intel, LinkedIn, Netflix, Salesforce, to name a few, who joined our special event. We were overwhelmed by the various topics and advanced use cases around the gRPC ecosystem. If you missed the opportunity to join in person, here are the video recordings that highlight the presentations during the day:

We received a wealth of feedbacks from everyone throughout the day which has helped us shape our future roadmap. Stay tuned to gRPC on many platforms for new releases, announcements and events:

Blog: Announcing gRPConf 2023!

Wed, 05 Jul 2023 00:00:00 +0000

gRPConf 2023 is happening!

Join us at the Google Cloud Campus on September 20, 2023 for a full day of talks, demos, case studies, and code labs. Experts will discuss real-world implementations of gRPC, best practices for developers, and deep dives into specific topics. This is a must-attend event for anyone using gRPC in their applications today, and those considering gRPC for their enterprise microservices.

There will be ample time to meet project leads and key members of the gRPC ecosystem, network with peers, and ask questions. This will be a great opportunity for us to hear from members of the larger gRPC community and gain valuable feedback on the entire gRPC ecosystem.

If you’re interested in speaking at gRPConf, please submit a proposal. The deadline is July 9, 2023. Spots are still available and topics might include:

gRPC in-production
User Stories + Case Studies
Implementation
Ecosystem + Tooling
Codelabs

Speakers who are chosen will be notified by August 1, 2023, and a full schedule announcement will follow shortly after.

Early bird registration is now open and is only $50. To register or find more information about the event, check out the event page here. For any other questions please contact us at grpconf@google.com

We hope to see you all there!

Blog: gRPConf 2023 Schedule

Wed, 05 Jul 2023 00:00:00 +0000

Hello gRPC Community!

We are excited to announce that the gRPConf 2023 schedule is now live. We have a schedule with 19 total sessions (5 keynote, 14 breakout) led by a strong group of experts that are using gRPC across a variety of applications and companies. Their experiences and expertise will provide huge value for all attendees whether you are already using gRPC or are still considering it. There will be something for everyone as the talks will span topics such as tooling, implementations, production use, and user stories.

In addition to our lineup of speakers, this year’s conference is designed to provide plenty of time to hear from the community as well. We’ll be providing breakfast, lunch, and an evening reception to give everyone the chance to network, meet project leads, and ask questions. The feedback gathered from the larger gRPC community is incredibly valuable, so we’ve also built in time for “Birds of a Feather” discussions where we can have more targeted discussions moderated by gRPC team leaders.

You can view the entire schedule here. Early Bird Registration is also still open for $50 until September 6th. If you’re interested in joining us please register now.

We’ll see you on September 20th!

Blog: gRPC performance benchmarks on GKE

Tue, 01 Mar 2022 00:00:00 +0000

gRPC performance benchmarks have now been transitioned to run on GKE, with similar results but much increased flexibility.

Background

gRPC performance testing requires a test driver and workers (one or more clients and a server), as described in gRPC performance benchmarks. Each test may have a different configuration, or scenario, that is passed to the driver and specified as a JSON file. Previously, the driver was run by the continuous integration process, and the workers were run on long-lived GCE VMs. This gave rise to several limitations:

Tests ran sequentially and were difficult to parallelize, since they ran on (the same) fixed VMs.
The state of the VMs was not guaranteed to be the same at the start of each test.
Running manual experiments required configuring new VMs, which was a manual process, or reusing existing VMs, with the risks of collision with other users and of having VMs in an unknown state.

Benchmarking on Kubernetes

The core of the current framework is a custom controller for managing Kubernetes resources of kind LoadTest. This controller must be deployed to a Kubernetes cluster before load tests can be run on it. The controller is implemented with kubebuilder. The code for the controller is stored in the Test Infra repository. For further documentation on individual LoadTest fields, see the LoadTest implementation.

LoadTest configurations specify driver, client and server pods to be created for the test. Once the configuration is applied to the cluster (for instance, with kubectl apply -f), the controller will create the pods and the test will run. If multiple configurations are applied to the cluster, the controller will create pods as long as there are resources available, allowing tests to run in parallel.

Examples include basic configurations that can be applied directly, and templates that require additional steps and parameter substitution.

Basic configurations rely on clone, build and runtime worker images that are bundled with each release of the controller. The clone and build images are used to build gRPC binaries that are passed to the runtime container. These configurations are suitable as examples and for one-off testing.
Template configurations rely on worker images that are built before starting the tests. These prebuilt images include the gRPC binary, eliminating the need to clone and build before each test. Template substitution is used to point to the location of the worker images. These configurations are suitable for running a batch of tests on the same gRPC version, or for running the same test repeatedly.

In addition to the controller, the Test Infra repository contains a set of tools, including a test runner and tools to build and delete prebuilt worker images, as well as a dashboard implementation.

The tools related to prebuilt workers use gcloud internally and are dependent on GKE. Other than that, all components of the framework are built on Kubernetes itself and independent of GKE. That is, it should be possible to deploy the controller and run tests on a custom Kubernetes cluster or on another cloud provider’s Kubernetes offering.

Cluster setup

The cluster running benchmark jobs must be configured with node pools dimensioned for the number of simultaneous tests that it should support. The controller uses pool as a node selector for the various pod types. Worker pods have mutual anti-affinity, so one node is required per pod.

For example, the node pools that are used in our continuous integration setup are configured as follows:

Pool name	Node count	Machine type	Kubernetes labels
system	2	e2-standard-8	default-system-pool:true, pool:system
drivers-ci	8	e2-standard-2	pool:drivers-ci
workers-c2-8core-ci	8	c2-standard-8	pool:workers-c2-8core-ci
workers-c2-30core-ci	8	c2-standard-30	pool:workers-c2-30core-ci

Since each scenario in our tests requires one driver and two workers, this configuration supports four simultaneous tests on 8-core machines and four on 30-core machines. Drivers require few resources, and do not have mutual anti-affinity. We find it convenient to schedule them on two-core machines with a node count set to the required number of drivers, rather than on a larger shared machine, because that allows the driver pool to be resized together with the worker pools. The controller itself is scheduled in the system pool.

In addition to the pools used in continuous integration, our cluster contains additional node pools that can be used for ad hoc testing:

Pool name	Node count	Machine type	Kubernetes labels
drivers	8	e2-standard-8	default-driver-pool:true, pool:drivers
workers-8core	8	e2-standard-8	default-worker-pool:true, pool:workers-8core
workers-32core	8	e2-standard-32	pool:workers-32core

Some pools are labeled with default-*-pool labels. These labels specify which pool to use if it is not specified in the LoadTest configuration. With the configuration above, these tests (for instance, the tests specified in the examples) will use the drivers and workers-8core pools, and not interfere with continuous integration jobs. The default labels are defined as part of the controller build: if they are not set, the controller will only run tests where the pool labels are specified explicitly.

Controller deployment

The steps for building and deploying a controller are described in the deployment documentation.

Continuous integration

Our continuous integration setup is described in the gRPC OSS benchmarks README in the gRPC Core repository. The main continuous integration job uses the script grpc_e2e_performance_gke.sh to generate the data presented on the dashboard linked to the gRPC performance benchmarks page.

Each continuous integration run has three phases:

Generate test configurations.
Build and push worker images.
Run the tests.

Each continuous integration run executes 122 tests using the 8-core worker pool, and 98 test using the 30-core worker pool. Each test runs one test scenario. Tests using C++, C#, Java and Python workers run on both pools. Tests using Node.js, PHP and Ruby workers run only on the 8-core pool. Config generation for all these combinations takes negligible time (~1s).

The configurations used in continuous integration require worker images that include the gRPC binaries to be tested. These images depend only on the worker’s language, so these prebuilt images are built in advance and pushed to an image repository. This process takes approximately 20 minutes.

A test runner manages the rate at which tests are applied to the cluster, collects test results and logs, and deletes tests once they complete successfully. Two tests at a time are allowed to run on each pool. This phase takes approximately 50 minutes to complete.

Each test scenario is configured to take 30s to run, plus a 5s warm-up period (15s for Java). This places a lower bound on the time needed to run each test. The observed run time of the 122 tests in the 8-core pool (16 of them in Java), running two tests at a time, implies that the overhead introduced by pod creation and deletion is modest, at ~12.8s per test.

Config generation

Since we are running hundreds of tests that mostly share the same components (driver and workers for the various languages), it becomes necessary to generate configurations that includes repetitive driver and worker configurations and differ only in the scenario being tested. In addition, each configuration must have a unique name, since that is a requirement for resources applied to a Kubernetes cluster.

We handle these issues by generating load test configurations with a tool. The tool is stored in the gRPC Core repository, where test scenarios are also defined.

Prebuilt images

The configurations generated for continuous integration use a set of prebuilt images. These images are built and pushed to an image repository before running tests. The images are deleted at the end of each test run.

For details on the tools used to prepare and delete images, see Using prebuilt images with gRPC OSS benchmarks.

Test runner

The test runner takes the test configurations generated previously, applies each configuration to the cluster, polls each LoadTest resource for completion, collects artifacts such as results and pod logs, and (optionally) deletes resources once each test completes successfully.

The test runner maintains separate queues for tests that require the same resources on the cluster (e.g. 8-core or 30-core worker nodes). Tests configurations belonging to the same queue are not applied ot the cluster at once, but according to a concurrency level set for each queue. Our continuous integration tests run in two queues (corresponding to 8-core and 30-core worker nodes). The concurrency level of each queue is set to two.

Once a configuration is applied to the cluster, the controller creates client, driver and server pods to run the test, monitors test execution, and updates the status of the LoadTest resource.

The design of the test runner can be explained as follows:

The use of the test runner allows the continuous integration job to wait for all tests to complete, collect tests artifacts, and prepare a report with results.
The use of separate queues (indicated by annotations in each test configuration) allows tests that do not require the same cluster resources to be managed independently of each other.
The use of limited concurrency levels reduces the number of tests applied to the cluster at a time. This has several benefits:
1. The load on the test runner is reduced, since there are fewer LoadTest resources on the cluster at a time, and the runner polls these resources periodically for completion. The polling interval in our continuous integration is set to 5s.
2. The load on the controller is reduced, since there are fewer LoadTest resources at a time for it to control.
3. Each test can have a shorter timeout period, since the time takes for the controller to start each test is more predictable. Timeouts are necessary to account for error cases where client or server pods hang and prevent the test from completing. These cases are rare but can accumulate and consume cluster resources, preventing other tests from running. Tests in our continuous integration have a timeout of 15 minutes.
4. The concurrency level can be set lower than the capacity of the cluster, allowing a user to run a batch of tests without preventing other users from running tests simultaneously.
The option of deleting each test once it completes successfully (and after results and logs are collected) provides better control of the lifecycle of each test.
1. The default behavior of the controller is to keep a LoadTest resource and associated pods on the cluster until it reaches a set TTL, and then delete it. Our continuous integration specifies a TTL of 24h for each test.
2. Pods belonging to a completed LoadTest are in terminated state, and so consume no resources on the cluster. However, terminated pods can be garbage-collected at any time.
3. If we let pods belonging to all completed tests stay in our continuous integration cluster, we find that they are garbage-collected within one hour.
4. If we delete LoadTest resources for tests that complete successfully, the associated pods are also deleted. In this case, the pods belonging to unsuccessful tests, which are few in number and may be useful for debugging, stay on the cluster until the 24h TTL is reached.

Dashboard

Test results from continuous integration are saved to BigQuery. The data stored in BigQuery is then replicated to a Postgres database for visualization on a dashboard.

The code for the dashboard, as well as the configuration of the main continuous integration dashboard, are stored in the Test Infra repository. This brings the following benefits:

The main dashboard is maintained by updating the stored configuration, instead of updating it directly in the UI.
Users can deploy their own dashboard, using their own configuration.

This contrasts with the dashboard for the previous benchmarks, built using Perfkit Explorer, which was maintained by updating it directly in the UI, and could not be easily replicated by users.

For details, see dashboard implementation.

Results

The following observations can be made on the results and user experience of the gRPC benchmarking on GKE:

Performance metrics (latency, QPS, etc.) produce the same or better results as the old benchmarks on GCE.
The overhead of pod creation and deletion for each test in GKE is small (less than 15s) in our benchmarking cluster.
Test images are dockerized and started again for each test, resulting in several benefits:
1. Results are more consistent.
2. Runtime errors are rare.
3. System is split into well-defined components, resulting in simpler upgrades.
4. Tests can be easily parallelized, resulting in faster execution times.
5. Experiments are easier to perform.

Examples of best practices and insights derived from experimentation:

Use c2 instances for clients and servers (instance types matter a lot for the observed latency and its variance and also for the measured throughput).
GKE pod-to-pod networking only has a very small overhead over raw GCE networking. You can get raw GCE networking performance by setting hostnetworking:true for the benchmark pods.
For Java under Docker, the JVM may not be able to detect the number of available processors automatically. This can lead to very pessimistic results, since gRPC uses the detected number of processors for sizing thread pools for processing events. A workaround is to set the number of processors explicitly. This workaround is implemented here.

Running your own

The code in the Test Infra repository allows any user to create a cluster, deploy a controller, run gRPC benchmarks, and display results on their own dashboards. If you are interested in performance, and run your own benchmarks, let us know!

Blog: Running gRPC and Protobuf on ARM64 (on Linux)

Wed, 23 Jun 2021 00:00:00 +0000

ARM processors have recently been gaining importance in many areas of compute, including those that were traditionally considered to be an x86_64-only domain. Thanks to the momentum around the ARM ecosystem, we can expect adoption of ARM platforms to grow significantly. The same is true for providing software that supports ARM-based platforms.

Since the main purpose of gRPC and protocol buffers is to interconnect distributed systems, their role in supporting ARM compute is especially important. With the emergence of ARM Cloud computing, we expect many systems will actually be hybrid compositions of x86 and ARM servers, mixing and matching the qualities of each ecosystem as needed. gRPC and protocol buffers are the ideal building blocks that can enable users to build systems that span multiple architectures seamlessly.

To meet the very high demand from gRPC users to support ARM, the gRPC team has decided to support selected ARM-based platforms officially and fully. Some time ago, we started an effort to test everything and fix any problems we encounter. Today, we are happy to announce that gRPC and protocol buffers implementations in C++, C#, Go, Java, Node, PHP, Python and Ruby are ready for production workloads for ARM64 Linux (see more details below).

The current state of things is best described by a list of general areas we completed:

Bugfixes/improvements: We put a number of fixes in place to make sure gRPC and protobuf work reliably on ARM64 Linux.
Package & Distribution: For the languages that provide binary architecture-specific packages, we added ARM64 Linux packages, and we started publishing them on every release as part of our standard release process. Having ready-to-use binary packages greatly improves the developer experience. For languages that don’t ship binary packages, we tested that the build works as expected and that one can install gRPC and protobuf on ARM64 Linux without problems.
Continuous Testing: We invested significant resources setting up continuous testing ensuring that gRPC and protobuf are fully tested and prevent any future regressions. Testing large projects with many tests and support for multiple languages was definitely a challenge, especially since the open source ecosystem for testing on ARM architecture is still in its infancy, but we successfully tested gRPC and protobuf with a combination of cross-compilation, emulated tests and running tests on real hardware.

Overview of gRPC and Protobuf support on ARM64 Linux

In the following table you can see the detailed status broken down by language. Each entry summarizes the support level for both gRPC and Protobuf on ARM64 Linux.

Language	continuously tested	distribution/packages	additional info
C++	✔️	Build from source using cmake or bazel (same approach as on x86_64)
C#	✔️	Grpc.Core nuget packages now have aarch64 Linux support (starting from `v2.38.1`)	Grpc.Tools nuget package has now support for codegen on aarch64 Linux
Go	✔️ ¹	Use the standard way of installing libraries in golang (same experience as on x86_64)
Java	✔️	Maven artifacts published with each release work well on aarch64 Linux	aarch64 protoc and grpc-java protoc plugin are published with each release
Node/Javascript	✔️	Use existing npm packages (they are platform-independent)
PHP	✔️	Existing PECL and composer packages work well on aarch64 Linux
Python	✔️	Pre-built wheels for aarch64 Linux are published with each release (starting from `v1.38.1`)	grpcio-tools package has now support for codegen on aarch64 Linux
Ruby	✔️	Pre-built native gems for aarch64 Linux not available yet. In order to use grpc-ruby and protobuf-ruby, users need to build the gems from source	Continuous tests are in place and they are passing consistently, but we don't yet provide pre-built packages. gRPC and protobuf in ruby are safe to use, but the installation experience is suboptimal

¹ continuous testing is in place for grpc-go and in-progress for protobuf-go. protobuf-go has been tested on aarch64 manually and has been found to work reliably.

ARM64 / aarch64 / ARMv8 terminology

While the terms ARM64, aarch64 and ARMv8 denote slightly different things, in practice they are often used interchangeably. For purposes of this article, they all mean essentially the same thing and they can be treated as synonyms.

Official ARM64 support is currently Linux-only

At this point we only officially support gRPC and Protobuf on ARM64 on Linux. We do realize that there’s demand for ARM64 support for other platforms as well (e.g. MacOS X with the new Apple M1 Silicon), but some of these platforms come with significant challenges (provisioning the hardware, lack of emulation, etc). Rather than delaying the ARM64 support release due to these complications, it made more sense to stay focused on delivering official ARM64 support to Linux first - so there we go.

Appendix: list of changes/fixes/improvements

grpc/grpc repository

grpc/grpc-java repository

grpc/grpc-go repostitory

https://github.com/grpc/grpc-go/pull/4344

protocolbuffers/protobuf repository

Blog: The future of gRPC in C# belongs to grpc-dotnet

Sat, 01 May 2021 00:00:00 +0000

Update 2023-10-02: The maintenance period for Grpc.Core has been extended again, until at least October 2024. See announcement for up to date info on Grpc.Core.

Update 2022-05-03: The maintenance period for Grpc.Core has been extended until May 2023. See announcement for more info on the future of Grpc.Core.

TL;DR grpc-dotnet (the Grpc.Net.Client and Grpc.AspNetCore.Server nuget packages) is now the recommended gRPC implementation for .NET/C#. The original gRPC C# implementation (the Grpc.Core nuget package) will enter maintenance mode and won’t be getting any new features and will only receive important bug fixes and security fixes going forward. The ultimate plan is to phase out Grpc.Core completely at some point in the future. This announcement describes the reasons why we have decided to do so and lays out the plan in more detail.

In September 2019 we announced general availability of a new gRPC C# implementation that is no longer based on the gRPC C core native library and that’s using the HTTP/2 protocol implementation that was added in .NET Core 3 and ASP.NET Core 3. We refer to this implementation as “grpc-dotnet”.

When we were introducing the grpc-dotnet implementation, we announced that both gRPC C# implementations (the new pure C# grpc-dotnet implementation and the original gRPC C# implementation based on C core native library) would co-exist side by side, letting the users choose which implementation works best for them. That made a lot of sense since grpc-dotnet was brand new back then and required a just-released .NET Core framework, while the original gRPC C# implementation had been stable for a long time, had lots of users and worked on even very old .NET Framework versions. Things needed some time to settle.

Since then, the new grpc-dotnet implementation has come a long way: it has been adopted by many users and became very popular, it has been used by a lot of applications in production environments, and has also added a lot of interesting new features. In addition, its main prerequisite, the .NET Core 3 framework, has been around for a while now and its adoption numbers are growing.

At the same time, while the original gRPC C# implementation (often referred to as “Grpc.Core”, the name of its nuget package) definitely has its place and it is hugely popular, we are now nearing a point where some of the design decisions that made perfect sense back in 2016 (when gRPC C# was released as GA) no longer have the weight they used to. For example, we decided to base the gRPC C# implementation on a native library because in 2016, there was no usable C# HTTP/2 library that we could depend on. By depending on the C core native library instead, we were able to deliver a stable, high performance gRPC library much faster than if we had to implement everything in C# from scratch. But from today’s perspective, taking a native dependency doesn’t make that much sense anymore since HTTP/2 support is now built into the .NET Core framework. The benefits of having a native dependency are diminishing, while the maintenance burden of having one is staying the same.

Out of the two stable C# implementations, the grpc-dotnet implementation is definitely the one that has more future potential. It is a more modern implementation that is well-integrated with the modern versions of .NET and it’s likely going to be more aligned with where the C# community will be a few years from now. It is also a pure C# implementation (no native components), which makes it much more contribution friendly, leads to better debuggability and it is simply also something that C# enthusiasts like to see.

Because the maintenance costs of having two official implementations of gRPC for C# are nontrivial and because grpc-dotnet seems to be the best choice for all users in the long run, we would like to announce the intent to phase out the original gRPC C# implementation (nuget package Grpc.Core) in favor of the more modern and more forward-looking grpc-dotnet implementation.

The details of the plan are described in the following sections, along with further explanation why it makes sense. To help understand the consequences of the decision to phase out Grpc.Core, we have also come up with a list of frequently asked questions and provided answers to them.

What makes grpc-dotnet the preferred implementation

Simply said, grpc-dotnet seems to be a better bet for the future. Some of the most important points were already mentioned. Here is a more detailed list of reasons why we believe grpc-dotnet will serve the users’ needs better:

It’s a more modern implementation, based on features of the recent version of the .NET framework. As such, it will probably be the more viable one of the two implementations in the future.
It’s more aligned with where the C# / .NET community is now and in the future. Staying aligned with where the community is heading seems to be the best bet for the future of gRPC in C#.
The implementation is much more agile and contribution friendly - because it’s internally based on well known primitives / APIs (ASP.NET core serving APIs and HTTP2 client) and it’s implemented in pure C#, the code is much more accessible to C# developers (both to users that just want to understand how things work and to potential contributors who would author PRs). The grpc-dotnet codebase is relatively small, it takes seconds to build, running the tests is easy and quick. In the long run, easier development and contribution friendliness should make up for some of the features that are missing today and make it a superior choice for the users – that is, lowering the barrier to contribute and fix/improve stuff translates into more stuff being fixed and better user experience after some time.
Having a library that is implemented in pure C# is something that’s generally favored by the .NET community, compared to an implementation that depends on a native component. While C# has good support for interoperating with native libraries, it is a technique that most C# developers are not familiar with and it looks like a black box to them. Native interop is tricky to get it right and has many downsides (e.g. more complicated development and build process, complex debugging, hard to maintain, hard to get community contributions, hard to provide support for multiple platforms). With Grpc.Core we were able to overcome most of these challenges (so things work these days), but it has been a lot of effort, the solutions are sometimes complex and fragile and maintaining it is costly and requires a lot of expertise.

NOTE: the Google.Protobuf library for C# is already written purely in C# (no native components), so having a pure C# implementation of gRPC gets rid of native components from developers’ microservice stack completely

Why not keep Grpc.Core forever?

Developing two implementations of gRPC in C# isn’t free. It costs valuable resources and we believe that the engineering time would be better spent on making gRPC in C# easier to use and on adding new features (and fixing bugs of course), rather than needing to work on two different codebases that both serve the same purpose. Also, having two separate implementations necessarily fragments the user base to some extent and splits the contributors’ efforts into two. Also just the simple act of users needing to choose which of the two implementations they want to bet on comes with uncertainty and inherent risk (none of which we want for our users).

By making grpc-dotnet the recommended implementation and by making the Grpc.Core implementation “maintenance only” (and eventually phasing it out), we are aiming to achieve the following goals:

Free up engineering resources to work on better features and better usability.
Unify the gRPC C# user base. This will lead to directing all community work and contributions toward a single implementation. It also removes the inherent friction caused by the user needing to choose which of the two official implementations to use.
Address some of the well known pain points of Grpc.Core that would be too difficult to address by other means.
Futureproof C#/.NET implementation of gRPC by staying aligned with the .NET community.

The plan

Phase 1: Grpc.Core becomes “Maintenance Only”

When: Effective immediately (May 2021)

From now on, we are no longer going to provide new features or enhancements for Grpc.Core. Important bugs and security issues will continue to be addressed in a normal way.

We will publish Grpc.Core releases normally, with the usual 6 weekly cadence.

The releases will be based on the newest grpc C core native library build, so all new features that don’t require C# specific work will also be included.

Phase 2: Grpc.Core becomes “Deprecated”

When: 1 year from now (May 2022)

Once this milestone is reached, Grpc.Core will no longer be officially supported and all the users will be strongly advised to only use grpc-dotnet from this point onwards.

The Grpc.Core nuget packages will remain available in the nuget.org repository, but no more fixes will be provided (= not even security fixes).

Future of Grpc.Tools and Grpc.Core.Api nuget packages

Both packages will continue to be fully supported, since they are strictly speaking not part of Grpc.Core and they are also used by grpc-dotnet.

the Grpc.Tools nuget package which provides the codegen build integration for C# projects will continue to be supported (and will potentially get improvements) – as it’s used by both Grpc.Core and grpc-dotnet. This package is independent of C core.
Grpc.Core.Api package is a prerequisite for grpc-dotnet so it will potentially evolve over time as well (but it’s a pure C# API only package and since it only contains the public API surface, changes are very infrequent)

Q & A

I’m a current Grpc.Core user, what does this mean for me?

While we are going to continue supporting Grpc.Core for a while (see the deprecation schedule for details), you’ll have to migrate your project to grpc-dotnet if you wish to continue getting updates and bug fixes in the future.

How do I migrate my existing project to grpc-dotnet?

Since Grpc.Core and grpc-dotnet are two distinct libraries, there are going to be some code changes necessary in your project. Since both implementations share the same API for invoking and handling RPCs (we’ve intentionally designed them to be that way), we believe that the code changes necessary should be fairly minimal. For many applications you’ll simply need to change the way you configure your gRPC channels and servers; that is usually only a small portion of your app’s implementation and tends to be separate from the business logic.

For more tips on how to migrate from Grpc.Core to grpc-dotnet, see Migrating gRPC services from C-core to ASP.NET Core.

We plan to publish a more detailed migration guide in the future to facilitate the migration from Grpc.Core to grpc-dotnet.

I’d like to use gRPC in C# for a new project. Which implementation should I choose?

We strongly recommend only using grpc-dotnet for new projects. We are going to stop supporting Grpc.Core in the future.

Does this mean I need to stop using Grpc.Core right now?

No, Grpc.Core will continue to be supported for a while (see the deprecation schedule). There should be enough time for you to assess the situation and plan your migration.

I’m not using gRPC directly in my code, but I’m using the Google Cloud Client Libraries (which do use Grpc.Core under the hood). How does this impact me?

This deprecation does not currently impact existing users of Google Cloud Client Libraries.

Since Grpc.Core is an integral part of these client libraries, security and bug fixes for Grpc.Core will continue to be provided for Google Cloud Client Libraries.

The client libraries for which the extended support will be provided:

Note that the extended support of Grpc.Core will only be provided for Grpc.Core when used as part of these client libraries. For other use cases than the Google Cloud Client Libraries, Grpc.Core won’t be officially supported past the deprecation date and users must migrate existing workloads to grpc-dotnet before the deprecation happens.

Where can I find the list of supported features?

Our documentation on github has a comparison of supported features.

I have an important Grpc.Core use case that is not covered by this document.

We welcome your feedback! Write to us through the grpc-io Google Group, or any other of the main gRPC community channels.

Blog: Analyzing gRPC messages using Wireshark

Wed, 03 Feb 2021 00:00:00 +0000

Wireshark is an open source network protocol analyzer that can be used for protocol development, network troubleshooting, and education. Wireshark lets you analyze gRPC messages that are transferred over the network, and learn about the binary format of these messages.

In this post, you’ll learn how to configure and use the Wireshark gRPC dissector and the Protocol Buffers (Protobuf) dissector, which are protocol-specific components that allow you to analyze gRPC messages with Wireshark.

Features

The main features of the gRPC and Protobuf dissectors are the following:

Support dissecting (decoding) gRPC messages serialized in the protocol buffer wire format or as JSON
Support dissecting gRPC messages of unary, server streaming, client streaming, and bidirectional streaming RPC calls
Enhanced dissection of serialized protocol buffers data by allowing you to do the following:
- Load relevant .proto files
- Register your own subdissectors for protocol buffer fields of type byte or string

Capturing gRPC traffic

This post focuses on the analysis of captured gRPC messages. To learn how to store network traffic in capture files, see Capturing Live Network Data from the Wireshark User’s Guide.

Note

Currently, Wireshark can only parse plain text gRPC messages. While Wireshark supports TLS dissection, it requires per-session secret keys. As of the time of writing, the only Go gRPC supports the exporting such keys. To learn how to export keys using Go gRPC – and other languages as support becomes available – see How to Export TLS Master keys of gRPC.

Example

Let’s walk through the setup necessary to analyze previously-captured messages that were generated by a slightly extended version of the address book app used in the Protocol Buffers tutorials.

Address book `.proto` files

The app’s main protocol file is addressbook.proto:

syntax = "proto3";
package tutorial;
import "google/protobuf/timestamp.proto";

message Person {
 string name = 1;
 int32 id = 2; // Unique ID number for this person.
 string email = 3;

 enum PhoneType {
 MOBILE = 0;
 HOME = 1;
 WORK = 2;
 }

 message PhoneNumber {
 string number = 1;
 PhoneType type = 2;
 }

 repeated PhoneNumber phone = 4;
 google.protobuf.Timestamp last_updated = 5;
 bytes portrait_image = 6;
}

message AddressBook {
 repeated Person people = 1;
}

This file is identical to the Protocol Buffers tutorial version, except for the additional portrait_image field.

Note the import statement at the top of the file, it is used to import Timestamp, which is one of many Protocol Buffers Well-Known Types.

Our variant of the app also defines a person-search service that can be used to search for address book entries based on selected Person attributes. The service is defined in person_search_service.proto:

syntax = "proto3";
package tutorial;
import "addressbook.proto";

message PersonSearchRequest {
 repeated string name = 1;
 repeated int32 id = 2;
 repeated string phoneNumber = 3;
}

service PersonSearchService {
 rpc Search (PersonSearchRequest) returns (stream Person) {}
}

Because the service uses the Person type defined in addressbook.proto, the address book .proto is imported at the start of the file.

Setting protobuf search paths

Wireshark gives the most meaningful decodings when it knows about the .proto files used by the apps whose messages you are analyzing.

You can tell Wireshark where to find .proto files by setting the Protobuf Search Paths in the preferences accessible from the Edit menu under Preferences > Protocols > Protobuf.

If our example app’s .proto files are in the d:/protos/my_proto_files directory, and the official Protobuf library directory is d:/protos/protobuf-3.4.1/include, then add these two paths as source directories like this:

By selecting the Load all files option for the app’s protocol directory you enable preloading of message definitions from the addressbook.proto and person_search_service.proto files.

Loading a capture file

From the Wireshark SampleCaptures page, download the following sample gRPC capture file created by running the app and issuing a search request: grpc_person_search_protobuf_with_image.pcapng.

Select Open from the File menu to load the capture file in Wireshark. Wireshark displays, in order, all of the network traffic from the capture file in the Packet-list pane at the top of the window.

Select an entry from the packet-list pane and Wireshark will decode it and show its details in the lower pane like this:

Select an entry from the details pane to see the byte sequence corresponding to that entry:

Setting port traffic type

The app’s server-side port is 50051. The client-side port, which is different for each RPC call, is 51035 in the sample capture file.

You need to tell Wireshark that these ports are carrying HTTP2 traffic. Do this through the Decode As dialog, which you access from the Analyze menu (or right-click on an entry from the packet-list pane). You only need to register the server-side port:

Look at the packet-list pane and you’ll see that Wireshark is now decoding HTTP2 and gRPC messages:

Decoding the search request message

Select the first gRPC message sent to port 50051, it corresponds to the sample’s service request message. This is how Wireshark dissects the gRPC request:

By examining the HTTP2 message header path field, you’ll see the URL to the app’s service (/tutorial.PersonSearchService), followed by the name of the invoked RPC (Search).

The content-type, which is set by the gRPC library, informs Wireshark that the HTTP2 message content is a gRPC message. By examining the decoded Protocol Buffers message of the sample gRPC request, you can see that the search request is for the names “Jason” and “Lily”.

Decoding the server-streamed response

Since the Search RPC response is server-streaming, Person objects can be returned to the client one after another.

Select the second Person message returned in the response stream to see its details:

By registering subdissectors, you can have Wireshark further decode fields of type byte or string. For example, to learn how to register a PNG decoder for the portrait_image field, see Protobuf field subdissectors.

History of gRPC and Protocol Buffers support

Here is a brief annotated list of Wireshark versions as they relate to the support of gRPC and Protocol Buffers:

v2.6.0: first release of gRPC and Protobuf dissectors, without support for .proto files or streaming RPCs.
v3.2.0: improved dissection of serialized protocol buffers data based on .proto files, and support of streaming RPCs.
v3.3.0: improved and enhanced .proto file support, such as capture-file search on protocol buffer field values.
v3.4.0: Protocol Buffers Timestamp time is displayed as locale date-time string.

Learn more

Interested in learning more? Start with the Wireshark User’s Guide. For more details concerning the example used in this post, as well as other sample capture files containing gRPC messages, see the gRPC dissector and Protocol Buffers dissector wiki pages.

Blog: Interceptors in gRPC-Web

Thu, 18 Jun 2020 00:00:00 +0000

We’re pleased to announce support for interceptors in gRPC-web as of release 1.1.0. While the current design is based on gRPC client interceptors available from other gRPC languages, it also includes gRPC-web specific features that should make interceptors easy to adopt and use alongside modern web frameworks.

Introduction

Similar to other gRPC languages, gRPC-web supports unary and server-streaming interceptors. For each kind of interceptor, we’ve defined an interface containing a single intercept() method:

UnaryInterceptor
StreamInterceptor

This is how the UnaryInterceptor interface is declared:

/*
* @interface
*/
const UnaryInterceptor = function() {};

/**
 * @template REQUEST, RESPONSE
 * @param {!Request<REQUEST, RESPONSE>} request
 * @param {function(!Request<REQUEST,RESPONSE>):!Promise<!UnaryResponse<RESPONSE>>}
 * invoker
 * @return {!Promise<!UnaryResponse<RESPONSE>>}
 */
UnaryInterceptor.prototype.intercept = function(request, invoker) {};

The intercept() method takes two parameters:

A request of type grpc.web.Request
An invoker, which performs the actual RPC when invoked

The StreamInterceptor interface declaration is similar, except that the invoker return type is ClientReadableStream instead of Promise. For implementation details, see interceptor.js.

Note

A StreamInterceptor can be applied to any RPC with a ClientReadableStream return type, whether it’s a unary or a server-streaming RPC.

What can I do with an interceptor?

An interceptor allows you to do the following:

Update the original gRPC request before passing it along — for example, you might inject extra information such as auth headers
Manipulate the behavior of the original invoker function, such as bypassing the call so that you can use a cached result instead
Update the response before it’s returned to the client

You’ll see some examples next.

Unary interceptor example

The code given below illustrates a unary interceptor that does the following:

It prepends a string to the gRPC request message before the RPC.
It prepends a string to the gRPC response message after it’s received.

This simple unary interceptor is defined as a class that implements the UnaryInterceptor interface:

/**
 * @constructor
 * @implements {UnaryInterceptor}
 */
const SimpleUnaryInterceptor = function() {};

/** @override */
SimpleUnaryInterceptor.prototype.intercept = function(request, invoker) {
 // Update the request message before the RPC.
 const reqMsg = request.getRequestMessage();
 reqMsg.setMessage('[Intercept request]' + reqMsg.getMessage());

 // After the RPC returns successfully, update the response.
 return invoker(request).then((response) => {
 // You can also do something with response metadata here.
 console.log(response.getMetadata());

 // Update the response message.
 const responseMsg = response.getResponseMessage();
 responseMsg.setMessage('[Intercept response]' + responseMsg.getMessage());

 return response;
 });
};

Stream interceptor example

More care is needed to intercept server-streamed responses from a ClientReadableStream using a StreamInterceptor. These are the main steps to follow:

Create a ClientReadableStream-wrapper class, and use it to intercept stream events such as the reception of server responses.
Create a class that implements StreamInterceptor and that uses the stream wrapper.

The following sample stream-wrapper class intercepts responses and prepends a string to response messages:

/**
 * A ClientReadableStream wrapper.
 *
 * @template RESPONSE
 * @implements {ClientReadableStream}
 * @constructor
 * @param {!ClientReadableStream<RESPONSE>} stream
 */
const InterceptedStream = function(stream) {
 this.stream = stream;
};

/** @override */
InterceptedStream.prototype.on = function(eventType, callback) {
 if (eventType == 'data') {
 const newCallback = (response) => {
 // Update the response message.
 const msg = response.getMessage();
 response.setMessage('[Intercept response]' + msg);
 // Pass along the updated response.
 callback(response);
 };
 // Register the new callback.
 this.stream.on(eventType, newCallback);
 } else {
 // You can also override 'status', 'end', and 'error' eventTypes.
 this.stream.on(eventType, callback);
 }
 return this;
};

/** @override */
InterceptedStream.prototype.cancel = function() {
 this.stream.cancel();
 return this;
};

The intercept() method of the sample interceptor returns a wrapped stream:

/**
 * @constructor
 * @implements {StreamInterceptor}
 */
const TestStreamInterceptor = function() {};

/** @override */
TestStreamInterceptor.prototype.intercept = function(request, invoker) {
 return new InterceptedStream(invoker(request));
};

Binding interceptors

By passing an array of interceptor instances using an appropriate option key, you can bind interceptors to a client when the client is instantiated:

const promiseClient = new MyServicePromiseClient(
 host, creds, {'unaryInterceptors': [interceptor1, interceptor2, interceptor3]});

const client = new MyServiceClient(
 host, creds, {'streamInterceptors': [interceptor1, interceptor2, interceptor3]});

Note

Interceptors are executed in reverse order for request processing, and in order for response processing, as illustrated here:

Feedback

Found a problem with grpc-web or need a feature? File an issue over the grpc-web repository. If you have general questions or comments, then consider posting to the gRPC mailing list or sending us an email at grpc-web-team@google.com.

Blog: Announcing gRPC-JS 1.0

Mon, 20 Apr 2020 00:00:00 +0000

We are excited to announce the release of version 1.0 of gRPC-JS (@grpc/grpc-js), a pure-TypeScript reimplementation of the original Node gRPC library, grpc.

Features

gRPC-JS supports the following features, which should cover most use cases:

Clients
Automatic reconnection
Servers
Streaming
Metadata
Partial compression support: clients can decompress response messages
Pick first and round robin load balancing policies
Client Interceptors
Connection Keepalives
HTTP Connect support (proxies)

Should I use @grpc/grpc-js or grpc?

The original Node gRPC library (grpc) will no longer receive feature updates and we plan to deprecate it in a year, so we recommend that you use gRPC-JS, @grpc/grpc-js.

However, some advanced features haven’t been ported to gRPC-JS yet, such as full compression support or support for other load balancing policies. If you need one of these features, you should use the grpc library, but open a feature request over gRPC-JS to let us know which features you are missing the most.

Blog: Kotlin, meet gRPC

Thu, 16 Apr 2020 00:00:00 +0000

As developers work to modernize applications, they need foundational tools that are simple and scalable. gRPC is a high-performance, open-source, universal RPC framework originally developed at Google that developers are adopting in tremendous numbers, helping them connect services more easily and reliably. Paired with Kotlin, the second most popular JVM-based programming language in the world, developers can build everything from mobile apps to cloud microservices. To help, we’ve open-sourced gRPC Kotlin for the JVM, allowing you to use gRPC with your Kotlin projects.

For the full announcement, including how to deploy gRPC Kotlin services on Cloud Run, see the Google Cloud blog, Kotlin, meet gRPC: a new open-source project for modern apps.

For more information, see the following Kotlin pages:

Blog: gRPC comes to Cloud Run

Tue, 17 Mar 2020 00:00:00 +0000

Cloud Run is a serverless platform offered by Google Cloud that lets you run stateless server containers in a fully managed environment. Most Cloud Run apps use HTTP JSON REST to serve requests, but since September 2019, apps can also use unary gRPC services!

Get all the details from Not just for HTTP anymore: gRPC comes to Cloud Run.

Blog: Improvements to gRPC's CMake Build System

Mon, 16 Mar 2020 00:00:00 +0000

For the past few months, Kitware Inc. has been working with the gRPC team to improve gRPC’s CMake support. The goal of the effort was to modernize gRPC’s CMake build with the most current features and techniques CMake has to offer. This has improved the user experience for gRPC developers choosing to use gRPC’s CMake as a build system. During the effort the CMake build was looked at as a whole, and CMake related issues in GitHub were explored and resolved. A number of improvements were made that will give developers and end users a better experience when building gRPC with CMake.

One of the more exciting changes is the ability to seamlessly add gRPC to any CMake project and have all of its dependent libraries built using a simple CMake file. Prior to our recent changes this was a multi-step process. The user had to build and install each of gRPC’s dependencies separately, then build and install gRPC before finally building their own project. Now, this can all be done in one step. The following CMake code clones and builds the latest stable release of gRPC as documented here:

cmake_minimum_required(VERSION 3.15)
project(my_exe)

include(FetchContent)

FetchContent_Declare(
 gRPC
 GIT_REPOSITORY https://github.com/grpc/grpc
 GIT_TAG v1.28.0
 )
set(FETCHCONTENT_QUIET OFF)
FetchContent_MakeAvailable(gRPC)

add_executable(my_exe my_exe.cc)
target_link_libraries(my_exe grpc++)

At configure time CMake uses git to clone the gRPC repository using the specified tag. Then gRPC will be added to the current CMake project via add_subdirectory and built as part of the project.

What has changed?

We have addressed many of the CMake-related issues on GitHub, with bug fixes, documentation updates, and new features. All the fixes and features are available starting from gRPC 1.28.0 release.

We’ve improved the documentation for building gRPC from source and adding gRPC as a dependency to your CMake project giving developers several options for using gRPC from CMake from simply linking to a pre-built gRPC to downloading and building gRPC as part of the project.
The CMake build now generates pkgconfig (*.pc) files in the installation directory, just like the Makefile build. This allows for pkgconfig to correctly find and report a CMake built version of gRPC.
If you are using CMake v3.13 or newer, you can now build & install gRPC and its dependencies in a single step, rather than building and installing each component separately.
The CMake build now has configuration options to enable or disable building of every protoc plugin. For example, running CMake with -DgRPC_BUILD_GRPC_PYTHON_PLUGIN=OFF will disable building the Python plugin. You can view and edit these options in cmake-gui (or ccmake) as you are configuring your build of gRPC.
When building and installing gRPC as shared libraries, CMake now sets the .so version so the libraries are correctly versioned. (for example, libgrpc.so.9.0.0, libgrpc++.so.1.27.0-dev, etc).
We’ve added examples showing how to build gRPC using the CMake FetchContent module, and how to cross-compile gRPC for the Raspberry Pi.
CMake can now find libc-ares even if c-ares has been built with Autotools instead of CMake. This allows gRPC to be built against the distribution-provided version of c-ares if it has been built with Autotools.
If gRPC is built without testing enabled, the dependent testing frameworks will automatically be disabled, in order to avoid unnecessary compilation.
Some issues with parallel builds have been addressed.

As a bonus, there is one extra change that wasn’t technically part of this effort, but also contributes to simpler and easier cmake build:

To build the boringssl dependency, a much more lightweight cmake build is now used, which eliminates some odd build-time dependencies (e.g. golang).

Blog: .NET Core ❤ gRPC

Mon, 23 Sep 2019 00:00:00 +0000

The .NET team at Microsoft has been working in close collaboration with the gRPC team since November 2018 on a new fully managed implementation of gRPC for .NET Core.

We’re pleased to announce that grpc-dotnet is now available with .NET Core 3.0 today!

How to get it?

grpc-dotnet packages have just been released to NuGet.org and are already available for use in your projects. These packages also require the latest .NET Core 3.0 shared framework. You can download the .NET Core 3.0 SDK for your dev machine and build servers from the .NET Core 3.0 download page to acquire the shared framework.

Getting Started

As gRPC is now a first-class citizen in .NET ecosystem, gRPC templates are included as part of the .NET SDK. To get started, navigate to a console window after installing the SDK and run the following commands.

dotnet new grpc -o GrpcGreeter
cd GrpcGreeter
dotnet run

To create a gRPC client and test with the newly created gRPC Greeter service, you can follow the rest of this tutorial here.

Doesn’t gRPC already work with .NET Core?

There are currently two official implementations of gRPC for .NET:

Grpc.Core: The original gRPC C# implementation based on the native gRPC Core library.
grpc-dotnet: The new implementation written entirely in C# with no native dependencies and based on the newly released .NET Core 3.0.

The implementations coexist side-by-side and each has its own advantages in terms of available features, integrations, supported platforms, maturity level and performance. Both implementations share the same API for invoking and handling RPCs, thus limiting the lock-in and enabling users to choose the implementation that satisfies their needs the best.

What’s new?

Unlike the existing C-Core based implementation (Grpc.Core), the new libraries (grpc-dotnet) make use of the existing networking primitives in the .NET Core Base Class Libraries (BCL). The diagram below highlights the difference between the existing Grpc.Core library and the new grpc-dotnet libraries.

On the server side, the Grpc.AspNetCore.Server package integrates into ASP.NET Core, allowing developers to benefit from ecosystem of common cross-cutting concerns of logging, configuration, dependency injection, authentication, authorization etc. which have already been solved by ASP.NET Core. Popular libraries in the ASP.NET ecosystem such as Entity Framework Core (ORM), Serilog (Logging library), and Identity Server among others now work seamlessly with gRPC.

On the client side, the Grpc.Net.Client package builds upon the familiar HttpClient API that ships as part of .NET Core. As with the server, gRPC clients greatly benefit from the ecosystem of packages that build upon HttpClient. It is now possible to use existing packages such as Polly(Resilience and fault-handling library) and HttpClientFactory(manage HTTPClient lifetimes) with gRPC clients.

The diagram below captures the exhaustive list of all new .NET packages for gRPC and their relationship with the existing packages.

In addition to the newly published packages that ship as part of grpc-dotnet, we’ve also made improvements that benefit both stacks. Visual Studio 2019 ships with language grammar support for protobuf files and automatic generation of gRPC server/client code upon saving a protobuf file without requiring full project rebuilds due to design-time builds.

Feedback

We’re excited about improving the gRPC experience for .NET developers. Give it a try and let us know about feature ideas or bugs you may encounter using the grpc-dotnet issue tracker.

Blog: Dear gRPC

Fri, 08 Mar 2019 00:00:00 +0000

Dear gRPC,

We messed up. We are so sorry that we missed your birthday this year. Last year we celebrated with cake and fanfare, but this year we dropped the ball. Please don’t think that we love you any less.

You’re 4 now and that’s a big milestone! You’re part of so much amazing technology at companies like Salesforce, Netflix, Spotify, Fanatics, and of course, Google. In fact just this week the biggest API Google has went production-ready with gRPC.

We’re proud of you, gRPC, and we’re going to make this up to you. For starters - we got you a puppy! He’s an adorable Golden Retriever and his name is PanCakes. He loves to run back and forth with toys, packets, or messages. He’s super active and no matter how much we train him, we just can’t get him to REST. PanCakes is going to be your best friend, and ambassador.

Even though it’s a bit late, we still want to throw you a party, gRPC. Our friends at CNCF have planned a big event for you on March 21, and there’s going to be lots of people there! They’ll be sharing stories about the cool things they’ve built, and meeting new people. It’s an entire day all about you, and everyone there is going to learn so much. There will be other puppies who can play with PanCakes! Some of the amazing dogs from Canine Companions for Independence will be there to greet conference attendees and share how they help their humans live a more independent life.

We are so excited to see what this year holds for you, gRPC!

~ gRPC Maintainers

Blog: The state of gRPC in the browser

Tue, 08 Jan 2019 00:00:00 +0000

gRPC 1.0 was released in August 2016 and has since grown to become one of the premier technical solutions for application communications. It has been adopted by startups, enterprise companies, and open source projects worldwide. Its support for polyglot environments, focus on performance, type safety, and developer productivity has transformed the way developers design their architectures.

So far the benefits have largely only been available to mobile app and backend developers, whilst frontend developers have had to continue to rely on JSON REST interfaces as their primary means of information exchange. However, with the release of gRPC-Web, gRPC is poised to become a valuable addition in the toolbox of frontend developers.

In this post, I’ll describe some of the history of gRPC in the browser, explore the state of the world today, and share some thoughts on the future.

Beginnings

In the summer of 2016, both a team at Google and Improbable¹ independently started working on implementing something that could be called “gRPC for the browser”. They soon discovered each other’s existence and got together to define a spec² for the new protocol.

The gRPC-Web Spec

It is currently impossible to implement the HTTP/2 gRPC spec³ in the browser, as there is simply no browser API with enough fine-grained control over the requests. For example: there is no way to force the use of HTTP/2, and even if there was, raw HTTP/2 frames are inaccessible in browsers. The gRPC-Web spec starts from the point of view of the HTTP/2 spec, and then defines the differences. These notably include:

Supporting both HTTP/1.1 and HTTP/2.
Sending of gRPC trailers at the very end of request/response bodies as indicated by a new bit in the gRPC message header⁴.
A mandatory proxy for translating between gRPC-Web requests and gRPC HTTP/2 responses.

The Tech

The basic idea is to have the browser send normal HTTP requests (with Fetch or XHR) and have a small proxy in front of the gRPC server to translate the requests and responses to something the browser can use.

The Two Implementations

The teams at Google and Improbable both went on to implement the spec in two different repositories^5,⁶, and with slightly different implementations, such that neither entirely conformed to the spec, and for a long time neither was compatible with the other’s proxy^7,⁸.

The Improbable gRPC-Web client⁹ is implemented in TypeScript and available on npm as @improbable-eng/grpc-web¹⁰. There is also a Go proxy available, both as a package that can be imported into existing Go gRPC servers¹¹, and as a standalone proxy that can be used to expose an arbitrary gRPC server to a gRPC-Web frontend¹².

The Google gRPC-Web client¹³ is implemented in JavaScript using the Google Closure library¹⁴ base. It is available on npm as grpc-web¹⁵. It originally shipped with a proxy implemented as an NGINX extension¹⁶, but has since doubled down on an Envoy proxy HTTP filter¹⁷, which is available in all versions since v1.4.0.

Feature Sets

The gRPC HTTP/2 implementations all support the four method types: unary, server-side, client-side, and bi-directional streaming. However, the gRPC-Web spec does not mandate any client-side or bi-directional streaming support specifically, only that it will be implemented once WHATWG Streams¹⁸ are implemented in browsers.

The Google client supports unary and server-side streaming, but only when used with the grpcwebtext mode. Only unary requests are fully supported in the grpcweb mode. These two modes specify different ways to encode the protobuf payload in the requests and responses.

The Improbable client supports both unary and server-side streaming, and has an implementation that automatically chooses between XHR and Fetch based on the browser capabilities.

Here’s a table that summarizes the different features supported:

Client / Feature	Transport	Unary	Server-side streams	Client-side & bi-directional streaming
Improbable	Fetch/XHR ️	✔️	✔️	❌¹⁹
Google (`grpcwebtext`)	XHR ️	✔️	✔️	❌
Google (`grpcweb`)	XHR ️	✔️	❌²⁰	❌

For more information on this table, please see my compatibility test repo on github.

The compatibility tests may evolve into some automated test framework to enforce and document the various compatibilities in the future.

Compatibility Issues

Of course, with two different proxies also come compatibility issues. Fortunately, these have recently been ironed out, so you can expect to use either client with either proxy.

The Future

The Google implementation announced version 1.0 and general availability in October 2018²¹ and has published a road map of future goals²², including:

An efficient JSON-like message encoding
In-process proxies for Node, Python, Java and more
Integration with popular frameworks (React, Angular, Vue)
Fetch API transport for memory efficient streaming
Bi-directional streaming support

Google is looking for feedback on what features are important to the community, so if you think any of these are particularly valuable to you, then please fill in their survey²³.

Recent talks between the two projects have agreed on promoting the Google client and Envoy proxy as preferred solutions for new users. The Improbable client and proxy will remain as alternative implementations of the spec without the Google Closure dependency, but should be considered experimental. A migration guide will be produced for existing users to move to the Google client, and the teams are working together to converge the generated APIs.

Conclusion

The Google client will continue to have new features and fixes implemented at a steady pace, with a team dedicated to its success, and it being the official gRPC client. It doesn’t have Fetch API support like the Improbable client, but if this is an important feature for the community, it will be added. The Google team and the greater community are collaborating on the official client to the benefit of the gRPC community at large. Since the GA announcement the community contributions to the Google gRPC-Web repo has increased dramatically.

When choosing between the two proxies, there’s no difference in capability, so it becomes a matter of your deployment model. Envoy will suit some scenarios, while an in-process Go proxy has its own advantages.

If you’re getting started with gRPC-Web today, first try the Google client. It has strict API compatibility guarantees and is built on the rock-solid Google Closure library base used by Gmail and Google Maps. If you need Fetch API memory efficiency or experimental websocket client-side and bi-directional streaming, the Improbable client is a good choice, and it will continue to be used and maintained by Improbable for the foreseeable future.

Either way, gRPC-Web is an excellent choice for web developers. It brings the portability, performance, and engineering of a sophisticated protocol into the browser, and marks an exciting time for frontend developers!

References

improbable.io/games/blog/grpc-web-moving-past-restjson-towards-type-safe-web-apis ↩
github.com/grpc/grpc/blob/master/doc/PROTOCOL-WEB.md ↩
github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md ↩
github.com/grpc/grpc/blob/master/doc/PROTOCOL-WEB.md#protocol-differences-vs-grpc-over-http2 ↩
github.com/improbable-eng/grpc-web ↩
github.com/grpc/grpc-web ↩
github.com/improbable-eng/grpc-web/issues/162 ↩
github.com/grpc/grpc-web/issues/91 ↩
github.com/improbable-eng/grpc-web/tree/master/client/grpc-web ↩
npmjs.com/package/@improbable-eng/grpc-web ↩
github.com/improbable-eng/grpc-web/tree/master/go/grpcweb ↩
github.com/improbable-eng/grpc-web/tree/master/go/grpcwebproxy ↩
github.com/grpc/grpc-web/tree/master/javascript/net/grpc/web ↩
developers.google.com/closure ↩
npmjs.com/package/grpc-web ↩
github.com/grpc/grpc-web/tree/master/net/grpc/gateway ↩
envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/grpc_web_filter ↩
streams.spec.whatwg.org ↩
The Improbable client supports client-side and bi-directional streaming with an experimental websocket transport. This is not part of the gRPC-Web spec, and is not recommended for production use. ↩
grpcweb allows server streaming methods to be called, but it doesn’t return data until the stream has closed. ↩
gRPC-Web is Generally Available ↩
github.com/grpc/grpc-web/blob/master/doc/roadmap.md ↩
docs.google.com/forms/d/1NjWpyRviohn5jaPntosBHXRXZYkh_Ffi4GxJZFibylM ↩

Blog: Visualizing gRPC Language Stacks

Tue, 11 Dec 2018 00:00:00 +0000

Here is a high level overview of the gRPC Stacks. Each of the 10 default languages supported by gRPC has multiple layers, allowing you to customize what pieces you want in your application.

There are three main stacks in gRPC: C-core, Go, and Java. Most of the languages are thin wrappers on top of the C-based gRPC core library:

Wrapped Languages:

For example, a Python application calls into the generated Python stubs. These calls pass through interceptors, and into the wrapping library where the calls are translated into C calls. The gRPC C-core will encode the RPC as HTTP/2, optionally encrypt the data with TLS, and then write it to the network.

One of the cool things about gRPC is that you can swap these pieces out. For example, you could use C++ instead, and use an In-Process transport. This would save you from having to go all the way down to the OS network layer. Another example is trying out the QUIC protocol, which allows you to open new connections quickly. Being able to run over a variety of transports based on the environment makes gRPC really flexible.

For each of the wrapped languages, the default HTTP/2 implementation is built into the C-core library, so there is no need to include an outside one. However, as you can see, it is possible to bring your own (such as with Cronet, the Chrome networking library).

Go

In gRPC-Go, the stack is much simpler, due to not having to support so many configurations. Here is a high level overview of the Go stack:

The structure is a little different here. Since there is only one language, the flow from the top of the stack to the bottom is more linear. Unlike wrapped languages, gRPC Go can use either its own HTTP/2 implementation, or the Go net/http package.

Java

Here is a high level overview of the gRPC-Java stack:

Again, the structure is a little different. Java supports HTTP/2, QUIC, and In Process like the C-core. Unlike the C-Core though, applications commonly can bypass the generated stubs and interceptors, and speak directly to the Java Core library. Each structure is slightly different based on the needs of each language implementation of gRPC. Also unlike wrapped languages, gRPC Java separates the HTTP/2 implementation into pluggable libraries (such as Netty, OkHttp, or Cronet).

Blog: gRPC-Web is Generally Available

Tue, 23 Oct 2018 00:00:00 +0000

We are excited to announce the GA release of gRPC-Web, a JavaScript client library that enables web apps to communicate directly with gRPC backend services, without requiring an HTTP server to act as an intermediary. “GA” means that gRPC-Web is now Generally Available and stable and qualified for production use.

With gRPC-Web, you can now easily build truly end-to-end gRPC application architectures by defining your client and server-side data types and service interfaces with Protocol Buffers. This has been a hotly requested feature for a while, and we are finally happy to say that it is now ready for production use. In addition, being able to access gRPC services opens up new an exciting possibilities for web based tooling around gRPC.

The Basics

gRPC-Web, just like gRPC, lets you define the service “contract” between client (web) and backend gRPC services using Protocol Buffers. The client can then be auto generated. To do this, you have a choice between the Closure compiler or the more widely used CommonJS. This development process removes the need to manage concerns such as creating custom JSON serialization and deserialization logic, wrangling HTTP status codes (which can vary across REST APIs), managing content type negotiation etc.

From a broader architectural perspective, gRPC-Web enables end-to-end gRPC. The diagram below illustrates this:

Figure 1. gRPC with gRPC-Web (left) and gRPC with REST (right)

In the gRPC-Web universe on the left, a client application speaks Protocol Buffers to a gRPC backend server that speaks Protocol Buffers to other gRPC backend services. In the REST universe on the right, the web app speaks HTTP to a backend REST API server that then speaks Protocol Buffers to backend services.

Advantages of using gRPC-Web

gRPC-Web will offer an ever-broader feature set over time, but here’s what’s in 1.0 today:

End-to-end gRPC — Enables you to craft your entire RPC pipeline using Protocol Buffers. Imagine a scenario in which a client request goes to an HTTP server, which then interacts with 5 backend gRPC services. There’s a good chance that you’ll spend as much time building the HTTP interaction layer as you will building the entire rest of the pipeline.
Tighter coordination between frontend and backend teams — With the entire RPC pipeline defined using Protocol Buffers, you no longer need to have your “microservices teams” alongside your “client team.” The client-backend interaction is just one more gRPC layer amongst others.
Easily generate client libraries — With gRPC-Web, the server that interacts with the “outside” world, i.e. the membrane connecting your backend stack to the internet, is now a gRPC server instead of an HTTP server, that means that all of your service’s client libraries can be gRPC libraries. Need client libraries for Ruby, Python, Java, and 4 other languages? You no longer need to write HTTP clients for all of them.

A gRPC-Web example

The previous section illustrated some of the high-level advantages of gRPC-Web for large-scale applications. Now let’s get closer to the metal with an example: a simple TODO app. In gRPC-Web you can start with a simple todos.proto definition like this:

syntax = "proto3";

package todos;

message Todo {
 string content = 1;
 bool finished = 2;
}

message GetTodoRequest {
 int32 id = 1;
}

service TodoService {
 rpc GetTodoById (GetTodoRequest) returns (Todo);
}

CommonJS client-side code can be generated from this .proto definition with the following command:

protoc echo.proto \
 --js_out=import_style=commonjs:./output \
 --grpc-web_out=import_style=commonjs:./output

Now, fetching a list of TODOs from a backend gRPC service is as simple as:

const {GetTodoRequest} = require('./todos_pb.js');
const {TodoServiceClient} = require('./todos_grpc_web_pb.js');

const todoService = new proto.todos.TodoServiceClient('http://localhost:8080');
const todoId = 1234;

var getTodoRequest = new proto.todos.GetTodoRequest();
getTodoRequest.setId(todoId);

var metadata = {};
var getTodo = todoService.getTodoById(getTodoRequest, metadata, (err, response) => {
 if (err) {
 console.log(err);
 } else {
 const todo = response.todo();
 if (todo == null) {
 console.log(`A TODO with the ID ${todoId} wasn't found`);
 } else {
 console.log(`Fetched TODO with ID ${todoId}: ${todo.content()}`);
 }
 }
});

Once you declare the data types and a service interface, gRPC-Web abstracts away all the boilerplate, leaving you with a clean and human-friendly API (essentially the same API as the current Node.js for gRPC API, just transferred to the client).

On the backend, the gRPC server can be written in any language that supports gRPC, such as Go, Java, C++, Ruby, Node.js, and many others. The last piece of the puzzle is the service proxy. From the get-go, gRPC-Web will support Envoy as the default service proxy, which has a built-in envoy.grpc_web filter that you can apply with just a few lines of configuration.

Next Steps

Going GA means that the core building blocks are firmly in place and ready for usage in production web applications. But there’s still much more to come for gRPC-Web. Check out the official road map to see what the core team envisions for the near future.

If you’re interested in contributing to gRPC-Web, there are a few things we would love community help with:

Front-end framework integration — Commonly used front-end frameworks like React, Vue and Angular don’t yet offer official support for gRPC-Web. But we would love to see these frameworks support it since the integration between these frontend frameworks and gRPC-Web can be a vehicle to deliver user-perceivable performance benefits to applications. If you are interested in building out support for these frontend frameworks, let us know on the gRPC.io mailing list, filing a feature request on github or via the feature survey form below.
Language-specific proxy support — As of the GA release, Envoy is the default proxy for gRPC-Web, offering support via a special module. NGINX is also supported. But we’d also love to see development of in-process proxies for specific languages since they obviate the need for special proxies—such as Envoy and nginx—and would make using gRPC-Web even easier.

We’d also love to get feature requests from the community. Currently the best way to make feature requests is to fill out the gRPC-Web road map features survey. When filling up the form, list features you’d like to see and also let us know if you’d like to contribute to the development of those features in the I’d like to contribute to section. The gRPC-Web engineers will be sure to take that information to heart over the course of the project’s development.

Most importantly, we want to thank all the Alpha and Beta users who have given us feedback, bug reports and pull requests contributions over the course of the past year. We would certainly hope to maintain this momentum and make sure this project brings tangible benefits to the developer community.

Blog: A short introduction to Channelz

Wed, 05 Sep 2018 00:00:00 +0000

Channelz is a tool that provides comprehensive runtime info about connections at different levels in gRPC. It is designed to help debug live programs, which may be suffering from network, performance, configuration issues, etc. The gRFC provides a detailed explanation of channelz design and is the canonical reference for all channelz implementations across languages. The purpose of this blog is to familiarize readers with channelz service and how to use it for debugging issues. The context of this post is set in gRPC-Go, but the overall idea should be applicable across languages. At the time of writing, channelz is available for gRPC-Go and gRPC-Java. Support for C++ and wrapped languages is coming soon.

Let’s learn channelz through a simple example which uses channelz to help debug an issue. The helloworld example from our repo is slightly modified to set up a buggy scenario. You can find the full source code here: client, server.

Client setup: The client will make 100 SayHello RPCs to a specified target and load balance the workload with the round robin policy. Each call has a 150ms timeout. RPC responses and errors are logged for debugging purposes.

Running the program, we notice in the log that there are intermittent errors with error code DeadlineExceeded, as shown in Figure 1.

However, there’s no clue about what is causing the deadline exceeded error and there are many possibilities:

Network issue, for example: connection lost
Proxy issue, for example: dropped requests/responses in the middle
Server issue, for example: lost requests or just slow to respond

Figure 1. Program log screenshot

Let’s turn on grpc INFO logging for more debug info and see if we can find something helpful.

Figure 2. gRPC INFO log

As shown in Figure 2, the info log indicates that all three connections to the server are connected and ready for transmitting RPCs. No suspicious event shows up in the log. One thing that can be inferred from the info log is that all connections are up all the time, therefore the lost connection hypothesis can be ruled out.

To further narrow down the root cause of the issue, we will ask channelz for help.

Channelz provides gRPC internal networking machinery stats through a gRPC service. To enable channelz, users just need to register the channelz service to a gRPC server in their program and start the server. The code snippet below shows the API for registering channelz service to a grpc.Server. Note that this has already been done for our example client.

import "google.golang.org/grpc/channelz/service"

// s is a *grpc.Server
service.RegisterChannelzServiceToServer(s)

// call s.Serve() to serve channelz service

A web tool called grpc-zpages has been developed to conveniently serve channelz data through a web page. First, configure the web app to connect to the gRPC port that’s serving the channelz service (see instructions from the previous link). Then, open the channelz web page in the browser. You should see a web page like Figure 3. Now we can start querying channelz!

Figure 3. Channelz main page

As the error is on the client side, let’s first click on TopChannels. TopChannels is a collection of root channels which don’t have parents. In gRPC-Go, a top channel is a ClientConn created by the user through NewClient, and used for making RPC calls. Top channels are of Channel type in channelz, which is an abstraction of a connection that an RPC can be issued to.

Figure 4. TopChannels result

So we click on the TopChannels, and a page like Figure 4 appears, which lists all the live top channel(s) with related info.

As shown in Figure 5, there is only one top channel with id = 2 (Note that text in square brackets is the reference name of the in memory channel object, which may vary across languages).

Looking at the Data section, we can see there are 15 calls failed out of 100 on this channel.

Figure 5. Top Channel (id = 2)

On the right hand side, it shows the channel has no child Channels, 3 Subchannels (as highlighted in Figure 6), and 0 Sockets.

Figure 6. Subchannels owned by the Channel (id = 2)

A Subchannel is an abstraction over a connection and used for load balancing. For example, you want to send requests to “google.com”. The resolver resolves “google.com” to multiple backend addresses that serve “google.com”. In this example, the client is set up with the round robin load balancer, so all live backends are sent equal traffic. Then the (logical) connection to each backend is represented as a Subchannel. In gRPC-Go, a SubConn can be seen as a Subchannel.

The three Subchannels owned by the parent Channel means there are three connections to three different backends for sending the RPCs to. Let’s look inside each of them for more info.

So we click on the first Subchannel ID (i.e. “4[]”) listed, and a page like Figure 7 renders. We can see that all calls on this Subchannel have succeeded. Thus it’s unlikely this Subchannel is related to the issue we are having.

Figure 7. Subchannel (id = 4)

So we go back, and click on Subchannel 5 (i.e. “5[]”). Again, the web page indicates that Subchannel 5 also never had any failed calls.

Figure 8. Subchannel (id = 6)

And finally, we click on Subchannel 6. This time, there’s something different. As we can see in Figure 8, there are 15 out of 34 RPC calls failed on this Subchannel. And remember that the parent Channel also has exactly 15 failed calls. Therefore, Subchannel 6 is where the issue comes from. The state of the Subchannel is READY, which means it is connected and is ready to transmit RPCs. That rules out network connection problems. To dig up more info, let’s look at the Socket owned by this Subchannel.

A Socket is roughly equivalent to a file descriptor, and can be generally regarded as the TCP connection between two endpoints. In grpc-go, http2Client and http2Server correspond to Socket. Note that a network listener is also considered a Socket, and will show up in the channelz Server info.

Figure 9. Subchannel (id = 6) owns Socket (id = 8)

We click on Socket 8, which is at the bottom of the page (see Figure 9). And we now see a page like Figure 10.

The page provides comprehensive info about the socket like the security mechanism in use, stream count, message count, keepalives, flow control numbers, etc. The socket options info is not shown in the screenshot, as there are a lot of them and not related to the issue we are investigating.

The Remote Address field suggests that the backend we are having a problem with is “127.0.0.1:10003”. The stream counts here correspond perfectly to the call counts of the parent Subchannel. From this, we can know that the server is not actively sending DeadlineExceeded errors. This is because if the DeadlineExceeded error is returned by the server, then the streams would all be successful. A client side stream’s success is independent of whether the call succeeds. It is determined by whether a HTTP2 frame with EOS bit set has been received (refer to the gRFC for more info). Also, we can see that the number of messages sent is 34, which equals the number of calls, and it rules out the possibility of the client being stuck somehow and results in deadline exceeded. In summary, we can narrow down the issue to the server which serves on 127.0.0.1:10003. It may be that the server is slow to respond, or some proxy in front of it is dropping requests.

Figure 10. Socket (id = 8)

As you see, channelz has helped us pinpoint the potential root cause of the issue with just a few clicks. You can now concentrate on what’s happening with the pinpointed server. And again, channelz may help expedite the debugging at the server side too.

We will stop here and let readers explore the server side channelz, which is simpler than the client side. In channelz, a Server is also an RPC entry point like a Channel, where incoming RPCs arrive and get processed. In grpc-go, a grpc.Server corresponds to a channelz Server. Unlike Channel, Server only has Sockets (both listen socket(s) and normal connected socket(s)) as its children.

Here are some hints for the readers:

Look for the server with the address (127.0.0.1:10003).
Look at the call counts.
Go to the Socket(s) owned by the server.
Look at the Socket stream counts and message counts.

You should notice that the number of messages received by the server socket is the same as sent by the client socket (Socket 8), which rules out the case of having a misbehaving proxy (dropping request) in the middle. And the number of messages sent by the server socket is equal to the messages received at client side, which means the server was not able to send back the response before deadline. You may now look at the server code to verify whether it is indeed the cause.

Server setup: The server side program starts up three GreeterServers, with two of them using an implementation (server) that imposes no delay when responding to the client, and one using an implementation (slowServer) which injects a variable delay of 100ms - 200ms before sending the response.

As you can see through this demo, channelz helped us quickly narrow down the possible causes of an issue and is easy to use. For more resources, see the detailed channelz gRFC. Find us on GitHub at github.com/grpc/grpc-go.

Blog: gRPC on HTTP/2 Engineering a Robust, High-performance Protocol

Mon, 20 Aug 2018 00:00:00 +0000

In a previous article, we explored how HTTP/2 dramatically increases network efficiency and enables real-time communication by providing a framework for long-lived connections. In this article, we’ll look at how gRPC builds on HTTP/2’s long-lived connections to create a performant, robust platform for inter-service communication. We will explore the relationship between gRPC and HTTP/2, how gRPC manages HTTP/2 connections, and how gRPC uses HTTP/2 to keep connections alive, healthy, and utilized.

gRPC Semantics

To begin, let’s dive into how gRPC concepts relate to HTTP/2 concepts. gRPC introduces three new concepts: channels¹, remote procedure calls (RPCs), and messages. The relationship between the three is simple: each channel may have many RPCs while each RPC may have many messages.

Let’s take a look at how gRPC semantics relate to HTTP/2:

Channels are a key concept in gRPC. Streams in HTTP/2 enable multiple concurrent conversations on a single connection; channels extend this concept by enabling multiple streams over multiple concurrent connections. On the surface, channels provide an easy interface for users to send messages into; underneath the hood, though, an incredible amount of engineering goes into keeping these connections alive, healthy, and utilized.

Channels represent virtual connections to an endpoint, which in reality may be backed by many HTTP/2 connections. RPCs are associated with a connection (this association is described further on). RPCs are in practice plain HTTP/2 streams. Messages are associated with RPCs and get sent as HTTP/2 data frames. To be more specific, messages are layered on top of data frames. A data frame may have many gRPC messages, or if a gRPC message is quite large² it might span multiple data frames.

Resolvers and Load Balancers

In order to keep connections alive, healthy, and utilized, gRPC utilizes a number of components, foremost among them name resolvers and load balancers. The resolver turns names into addresses and then hands these addresses to the load balancer. The load balancer is in charge of creating connections from these addresses and load balancing RPCs between connections.

A DNS resolver, for example, might resolve some host name to 13 IP addresses, and then a RoundRobin balancer might create 13 connections - one to each address - and round robin RPCs across each connection. A simpler balancer might simply create a connection to the first address. Alternatively, a user who wants multiple connections but knows that the host name will only resolve to one address might have their balancer create connections against each address 10 times to ensure that multiple connections are used.

Resolvers and load balancers solve small but crucial problems in a gRPC system. This design is intentional: reducing the problem space to a few small, discrete problems helps users build custom components. These components can be used to fine-tune gRPC to fit each system’s individual needs.

Connection Management

Once configured, gRPC will keep the pool of connections - as defined by the resolver and balancer - healthy, alive, and utilized.

When a connection fails, the load balancer will begin to reconnect using the last known list of addresses³. Meanwhile, the resolver will begin attempting to re-resolve the list of host names. This is useful in a number of scenarios. If the proxy is no longer reachable, for example, we’d want the resolver to update the list of addresses to not include that proxy’s address. To take another example: DNS entries might change over time, and so the list of addresses might need to be periodically updated. In this manner and others, gRPC is designed for long-term resiliency.

Once resolution is finished, the load balancer is informed of the new addresses. If addresses have changed, the load balancer may spin down connections to addresses not present in the new list or create connections to addresses that weren’t previously there.

Identifying Failed Connections

The effectiveness of gRPC’s connection management hinges upon its ability to identify failed connections. There are generally two types of connection failures: clean failures, in which the failure is communicated, and the less-clean failure, in which the failure is not communicated.

Let’s consider a clean, easy-to-observe failure. Clean failures can occur when an endpoint intentionally kills the connection. For example, the endpoint may have gracefully shut down, or a timer may have been exceeded, prompting the endpoint to close the connection. When connections close cleanly, TCP semantics suffice: closing a connection causes the FIN handshake to occur. This ends the HTTP/2 connection, which ends the gRPC connection. gRPC will immediately begin reconnecting (as described above). This is quite clean and requires no additional HTTP/2 or gRPC semantics.

The less clean version is where the endpoint dies or hangs without informing the client. In this case, TCP might undergo retry for as long as 10 minutes before the connection is considered failed. Of course, failing to recognize that the connection is dead for 10 minutes is unacceptable. gRPC solves this problem using HTTP/2 semantics: when configured using KeepAlive, gRPC will periodically send HTTP/2 PING frames. These frames bypass flow control and are used to establish whether the connection is alive. If a PING response does not return within a timely fashion, gRPC will consider the connection failed, close the connection, and begin reconnecting (as described above).

In this way, gRPC keeps a pool of connections healthy and uses HTTP/2 to ascertain the health of connections periodically. All of this behavior is opaque to the user, and message redirecting happens automatically and on the fly. Users simply send messages on a seemingly always-healthy pool of connections.

Keeping Connections Alive

As mentioned above, KeepAlive provides a valuable benefit: periodically checking the health of the connection by sending an HTTP/2 PING to determine whether the connection is still alive. However, it has another equally useful benefit: signaling liveness to proxies.

Consider a client sending data to a server through a proxy. The client and server may be happy to keep a connection alive indefinitely, sending data as necessary. Proxies, on the other hand, are often quite resource constrained and may kill idle connections to save resources. Google Cloud Platform (GCP) load balancers disconnect apparently-idle connections after 10 minutes, and Amazon Web Services Elastic Load Balancers (AWS ELBs) disconnect them after 60 seconds.

With gRPC periodically sending HTTP/2 PING frames on connections, the perception of a non-idle connection is created. Endpoints using the aforementioned idle kill rule would pass over killing these connections.

A Robust, High Performance Protocol

HTTP/2 provides a foundation for long-lived, real-time communication streams. gRPC builds on top of this foundation with connection pooling, health semantics, efficient use of data frames and multiplexing, and KeepAlive.

Developers choosing protocols must choose those that meet today’s demands as well as tomorrow’s. They are well served by choosing gRPC, whether it be for resiliency, performance, long-lived or short-lived communication, customizability, or simply knowing that their protocol will scale to extraordinarily massive traffic while remaining efficient all the way. To get going with gRPC and HTTP/2 right away, check out gRPC’s Getting Started guides.

Footnotes

In Go, a gRPC channel is called ClientConn because the word “channel” has a language-specific meaning. ↩
gRPC uses the HTTP/2 default max size for a data frame of 16kb. A message over 16kb may span multiple data frames, whereas a message below that size may share a data frame with some number of other messages. ↩
This is the behavior of the RoundRobin balancer, but not every load balancer does or must behave this way. ↩

Blog: gRPC + JSON

Wed, 15 Aug 2018 00:00:00 +0000

So you’ve bought into this whole RPC thing and want to try it out, but aren’t quite sure about Protocol Buffers. Your existing code encodes your own objects, or perhaps you have code that needs a particular encoding. What to do?

Fortunately, gRPC is encoding agnostic! You can still get a lot of the benefits of gRPC without using Protobuf. In this post we’ll go through how to make gRPC work with other encodings and types. Let’s try using JSON.

gRPC is actually a collection of technologies that have high cohesion, rather than a singular, monolithic framework. This means its possible to swap out parts of gRPC and still take advantage of gRPC’s benefits. Gson is a popular library for Java for doing JSON encoding. Let’s remove all the protobuf related things and replace them with Gson:

- Protobuf wire encoding
- Protobuf generated message types
- gRPC generated stub types
+ JSON wire encoding
+ Gson message types

Previously, Protobuf and gRPC were generating code for us, but we would like to use our own types. Additionally, we are going to be using our own encoding too. Gson allows us to bring our own types in our code, but provides a way of serializing those types into bytes.

Let’s continue with the Key-Value store service. We will be modifying the code used my previous So You Want to Optimize gRPC post.

What is a Service Anyways?

From the point of view of gRPC, a Service is a collection of Methods. In Java, a method is represented as a MethodDescriptor. Each MethodDescriptor includes the name of the method, a Marshaller for encoding requests, and a Marshaller for encoding responses. They also include additional detail, such as if the call is streaming or not. For simplicity, we’ll stick with unary RPCs which have a single request and single response.

Since we won’t be generating any code, we’ll need to write the message classes ourselves. There are four methods, each which have a request and a response type. This means we need to make eight messages:

 static final class CreateRequest {
 byte[] key;
 byte[] value;
 }

 static final class CreateResponse {
 }

 static final class RetrieveRequest {
 byte[] key;
 }

 static final class RetrieveResponse {
 byte[] value;
 }

 static final class UpdateRequest {
 byte[] key;
 byte[] value;
 }

 static final class UpdateResponse {
 }

 static final class DeleteRequest {
 byte[] key;
 }

 static final class DeleteResponse {
 }

Because GSON uses reflection to determine how the fields in our classes map to the serialized JSON, we don’t need to annotate the messages.

Our client and server logic will use the request and response types, but gRPC needs to know how to produce and consume these messages. To do this, we need to implement a Marshaller. A marshaller knows how to convert from an arbitrary type to an InputStream, which is then passed down into the gRPC core library. It is also capable of doing the reverse transformation when decoding data from the network. For GSON, here is what the marshaller looks like:

 static <T> Marshaller<T> marshallerFor(Class<T> clz) {
 Gson gson = new Gson();
 return new Marshaller<T>() {
 @Override
 public InputStream stream(T value) {
 return new ByteArrayInputStream(gson.toJson(value, clz).getBytes(StandardCharsets.UTF_8));
 }

 @Override
 public T parse(InputStream stream) {
 return gson.fromJson(new InputStreamReader(stream, StandardCharsets.UTF_8), clz);
 }
 };
 }

Given a Class object for a some request or response, this function will produce a marshaller. Using the marshallers, we can compose a full MethodDescriptor for each of the four CRUD methods. Here is an example of the Method descriptor for Create:

 static final MethodDescriptor<CreateRequest, CreateResponse> CREATE_METHOD =
 MethodDescriptor.newBuilder(
 marshallerFor(CreateRequest.class),
 marshallerFor(CreateResponse.class))
 .setFullMethodName(
 MethodDescriptor.generateFullMethodName(SERVICE_NAME, "Create"))
 .setType(MethodType.UNARY)
 .build();

Note that if we were using Protobuf, we would use the existing Protobuf marshaller, and the method descriptors would be generated automatically.

Sending RPCs

Now that we can marshal JSON requests and responses, we need to update our KvClient, the gRPC client used in the previous post, to use our MethodDescriptors. Additionally, since we won’t be using any Protobuf types, the code needs to use ByteBuffer rather than ByteString. That said, we can still use the grpc-stub package on Maven to issue the RPC. Using the Create method again as an example, here’s how to make an RPC:

 ByteBuffer key = createRandomKey();
 ClientCall<CreateRequest, CreateResponse> call =
 chan.newCall(KvGson.CREATE_METHOD, CallOptions.DEFAULT);
 KvGson.CreateRequest req = new KvGson.CreateRequest();
 req.key = key.array();
 req.value = randomBytes(MEAN_VALUE_SIZE).array();

 ListenableFuture<CreateResponse> res = ClientCalls.futureUnaryCall(call, req);
 // ...

As you can see, we create a new ClientCall object from the MethodDescriptor, create the request, and then send it using ClientCalls.futureUnaryCall in the stub library. gRPC takes care of the rest for us. You can also make blocking stubs or async stubs instead of future stubs.

Receiving RPCs

To update the server, we need to create a key-value service and implementation. Recall that in gRPC, a Server can handle one or more Services. Again, what Protobuf would normally have generated for us we need to write ourselves. Here is what the base service looks like:

 static abstract class KeyValueServiceImplBase implements BindableService {
 public abstract void create(
 KvGson.CreateRequest request, StreamObserver<CreateResponse> responseObserver);

 public abstract void retrieve(/*...*/);

 public abstract void update(/*...*/);

 public abstract void delete(/*...*/);

 /* Called by the Server to wire up methods to the handlers */
 @Override
 public final ServerServiceDefinition bindService() {
 ServerServiceDefinition.Builder ssd = ServerServiceDefinition.builder(SERVICE_NAME);
 ssd.addMethod(CREATE_METHOD, ServerCalls.asyncUnaryCall(
 (request, responseObserver) -> create(request, responseObserver)));

 ssd.addMethod(RETRIEVE_METHOD, /*...*/);
 ssd.addMethod(UPDATE_METHOD, /*...*/);
 ssd.addMethod(DELETE_METHOD, /*...*/);
 return ssd.build();
 }
 }

KeyValueServiceImplBase will serve as both the service definition (which describes which methods the server can handle) and as the implementation (which describes what to do for each method). It serves as the glue between gRPC and our application logic. Practically no changes are needed to swap from Proto to GSON in the server code:

final class KvService extends KvGson.KeyValueServiceImplBase {

 @Override
 public void create(
 KvGson.CreateRequest request, StreamObserver<KvGson.CreateResponse> responseObserver) {
 ByteBuffer key = ByteBuffer.wrap(request.key);
 ByteBuffer value = ByteBuffer.wrap(request.value);
 // ...
 }

After implementing all the methods on the server, we now have a fully functioning gRPC Java, JSON encoding RPC system. And to show you there is nothing up my sleeve:

./gradlew :dependencies | grep -i proto
# no proto deps!

Optimizing the Code

While Gson is not as fast as Protobuf, there’s no sense in not picking the low hanging fruit. Running the code we see the performance is pretty slow:

./gradlew installDist
time ./build/install/kvstore/bin/kvstore

INFO: Did 215.883 RPCs/s

What happened? In the previous optimization post, we saw the Protobuf version do nearly 2,500 RPCs/s. JSON is slow, but not that slow. We can see what the problem is by printing out the JSON data as it goes through the marshaller:

{"key":[4,-100,-48,22,-128,85,115,5,56,34,-48,-1,-119,60,17,-13,-118]}

That’s not right! Looking at a RetrieveRequest, we see that the key bytes are being encoded as an array, rather than as a byte string. The wire size is much larger than it needs to be, and may not be compatible with other JSON code. To fix this, let’s tell GSON to encode and decode this data as base64 encoded bytes:

 private static final Gson gson =
 new GsonBuilder().registerTypeAdapter(byte[].class, new TypeAdapter<byte[]>() {
 @Override
 public void write(JsonWriter out, byte[] value) throws IOException {
 out.value(Base64.getEncoder().encodeToString(value));
 }

 @Override
 public byte[] read(JsonReader in) throws IOException {
 return Base64.getDecoder().decode(in.nextString());
 }
 }).create();

Using this in our marshallers, we can see a dramatic performance difference:

./gradlew installDist
time ./build/install/kvstore/bin/kvstore

INFO: Did 2,202.2 RPCs/s

Almost 10x faster than before! We can still take advantage of gRPC’s efficiency while bringing our own encoders and messages.

Conclusion

gRPC lets you use encoders other than Protobuf. It has no dependency on Protobuf and was specially made to work with a wide variety of environments. We can see that with a little extra boilerplate, we can use any encoder we want. While this post only covered JSON, gRPC is compatible with Thrift, Avro, Flatbuffers, Cap’n Proto, and even raw bytes! gRPC lets you be in control of how your data is handled. (We still recommend Protobuf though due to strong backwards compatibility, type checking, and performance it gives you.)

All the code is available on GitHub if you would like to see a fully working implementation.

Blog: Take the gRPC Survey!

Tue, 14 Aug 2018 00:00:00 +0000

The gRPC Project wants your feedback!

The gRPC project is looking for feedback to improve the gRPC experience. To do this, we are running a gRPC user survey. We invite you to participate and provide input that will help us better plan and prioritize.

gRPC User Survey

Who : If you currently use gRPC, have used gRPC in the past, or have any interest in it, we would love to hear from you.

Where: Please take this 15 minute survey by Friday, 24th August.

Why: gRPC is a broadly applicable project with a variety of use cases. We want to use this survey to help us understand what works well, and what needs to be fixed.

Spread the word!

Please help us spread the word on this survey by posting it on your social networks and sharing with your friends. Every single feedback is precious, and we would like as much of it as possible!

Survey Short link: http://bit.ly/gRPC18survey

Blog: Gracefully clean up in gRPC JUnit tests

Tue, 26 Jun 2018 00:00:00 +0000

It is best practice to always clean up gRPC resources such as client channels, servers, and previously attached Contexts whenever they are no longer needed.

This is even true for JUnit tests, because otherwise leaked resources may not only linger in your machine forever, but also interfere with subsequent tests. A not-so-bad case is that subsequent tests can’t pass because of a leaked resource from the previous test. The worst case is that some subsequent tests pass that wouldn’t have passed at all if the previously passed test had not leaked a resource.

So cleanup, cleanup, cleanup… and fail the test if any cleanup is not successful.

A typical example is

public class MyTest {
 private Server server;
 private ManagedChannel channel;
 ...
 @After
 public void tearDown() throws InterruptedException {
 // assume channel and server are not null
 channel.shutdownNow();
 server.shutdownNow();
 // fail the test if cleanup is not successful
 assert channel.awaitTermination(5, TimeUnit.SECONDS) : "channel failed to shutdown";
 assert server.awaitTermination(5, TimeUnit.SECONDS) : "server failed to shutdown";
 }
 ...
}

or to be more graceful

public class MyTest {
 private Server server;
 private ManagedChannel channel;
 ...
 @After
 public void tearDown() throws InterruptedException {
 // assume channel and server are not null
 channel.shutdown();
 server.shutdown();
 // fail the test if cannot gracefully shutdown
 try {
 assert channel.awaitTermination(5, TimeUnit.SECONDS) : "channel cannot be gracefully shutdown";
 assert server.awaitTermination(5, TimeUnit.SECONDS) : "server cannot be gracefully shutdown";
 } finally {
 channel.shutdownNow();
 server.shutdownNow();
 }
 }
 ...
}

However, having to add all this to every test so it shuts down gracefully gives you more work to do, as you need to write the shutdown boilerplate by yourself. Because of this, the gRPC testing library has helper rules to make this job less tedious.

Initially, a JUnit rule GrpcServerRule was introduced to eliminate the shutdown boilerplate. This rule creates an In-Process server and channel at the beginning of the test, and shuts them down at the end of test automatically. However, users found this rule too restrictive in that it does not support transports other than In-Process transports, multiple channels to the server, custom channel or server builder options, and configuration inside individual test methods.

A more flexible JUnit rule GrpcCleanupRule was introduced in gRPC release v1.13, which also eliminates the shutdown boilerplate. However unlike GrpcServerRule, GrpcCleanupRule does not create any server or channel automatically at all. Users create and start the server by themselves, and create channels by themselves, just as in plain tests. With this rule, users just need to register every resource (channel or server) that needs to be shut down at the end of test, and the rule will then shut them down gracefully automatically.

You can register resources either before running test methods

public class MyTest {
 @Rule
 public GrpcCleanupRule grpcCleanup = new GrpcCleanupRule();
 ...
 private String serverName = InProcessServerBuilder.generateName();
 private Server server = grpcCleanup.register(InProcessServerBuilder
 .forName(serverName).directExecutor().addService(myServiceImpl).build().start());
 private ManagedChannel channel = grpcCleanup.register(InProcessChannelBuilder
 .forName(serverName).directExecutor().build());
 ...
}

or inside each individual test method

public class MyTest {
 @Rule
 public GrpcCleanupRule grpcCleanup = new GrpcCleanupRule();
 ...
 private String serverName = InProcessServerBuilder.generateName();
 private InProcessServerBuilder serverBuilder = InProcessServerBuilder
 .forName(serverName).directExecutor();
 private InProcessChannelBuilder channelBuilder = InProcessChannelBuilder
 .forName(serverName).directExecutor();
 ...

 @Test
 public void testFooBar() {
 ...
 grpcCleanup.register(
 serverBuilder.addService(myServiceImpl).build().start());
 ManagedChannel channel = grpcCleanup.register(
 channelBuilder.maxInboundMessageSize(1024).build());
 ...
 }
}

Now with GrpcCleanupRule you don’t need to worry about graceful shutdown of gRPC servers and channels in JUnit test. So try it out and clean up in your tests!

Blog: gRPC Meets .NET SDK And Visual Studio: Automatic Codegen On Build

Tue, 26 Jun 2018 00:00:00 +0000

As part of Microsoft’s move towards its cross-platform .NET offering, they have greatly simplified the project file format, and allowed a tight integration of third-party code generators with .NET projects. We are listening, and now proud to introduce integrated compilation of Protocol Buffer and gRPC service .proto files in .NET C# projects starting with the version 1.17 of the Grpc.Tools NuGet package, now available from Nuget.org.

You no longer need to use hand-written scripts to generate code from .proto files: The .NET build magic handles this for you. The integrated tools locate the proto compiler and gRPC plugin, standard Protocol Buffer imports, and track dependencies before invoking the code generators, so that the generated C# source files are never out of date, at the same time keeping regeneration to the minimum required. In essence, .proto files are treated as first-class sources in a .NET C# project.

A Walkthrough

In this blog post, we’ll walk through the simplest and probably the most common scenario of creating a library from .proto files using the cross-platform dotnet command. We will implement essentially a clone of the Greeter library, shared by client and server projects in the C# Helloworld example directory .

Create a new project

Let’s start by creating a new library project.

~/work$ dotnet new classlib -o MyGreeter
The template "Class library" was created successfully.

~/work$ cd MyGreeter
~/work/MyGreeter$ ls -lF
total 12
-rw-rw-r-- 1 kkm kkm 86 Nov 9 16:10 Class1.cs
-rw-rw-r-- 1 kkm kkm 145 Nov 9 16:10 MyGreeter.csproj
drwxrwxr-x 2 kkm kkm 4096 Nov 9 16:10 obj/

Observe that the dotnet new command has created the file Class1.cs that we won’t need, so remove it. Also, we need some .proto files to compile. For this exercise, we’ll copy an example file examples/protos/helloworld.proto from the gRPC distribution.

~/work/MyGreeter$ rm Class1.cs
~/work/MyGreeter$ wget -q https://raw.githubusercontent.com/grpc/grpc/master/examples/protos/helloworld.proto

(on Windows, use del Class1.cs, and, if you do not have the wget command, just open the above URL and use a Save As… command from your Web browser).

Next, add required NuGet packages to the project:

~/work/MyGreeter$ dotnet add package Grpc
info : PackageReference for package 'Grpc' version '1.17.0' added to file '/home/kkm/work/MyGreeter/MyGreeter.csproj'.
~/work/MyGreeter$ dotnet add package Grpc.Tools
info : PackageReference for package 'Grpc.Tools' version '1.17.0' added to file '/home/kkm/work/MyGreeter/MyGreeter.csproj'.
~/work/MyGreeter$ dotnet add package Google.Protobuf
info : PackageReference for package 'Google.Protobuf' version '3.6.1' added to file '/home/kkm/work/MyGreeter/MyGreeter.csproj'.

Add `.proto` files to the project

Next comes an important part. First of all, by default, a .csproj project file automatically finds all .cs files in its directory, although Microsoft now recommends suppressing this globbing behavior, so we too decided against globbing .proto files. Thus the .proto files must be added to the project explicitly.

Second of all, it is important to add a property PrivateAssets="All" to the Grpc.Tools package reference, so that it will not be needlessly fetched by the consumers of your new library. This makes sense, as the package only contains compilers, code generators and import files, which are not needed outside of the project where the .proto files have been compiled. While not strictly required in this simple walkthrough, it must be your standard practice to do that always.

So edit the file MyGreeter.csproj to add the helloworld.proto so that it will be compiled, and the PrivateAssets property to the Grpc.Tools package reference. Your resulting project file should now look like this:

<Project Sdk="Microsoft.NET.Sdk">

 <PropertyGroup>
 <TargetFramework>netstandard2.0</TargetFramework>
 </PropertyGroup>

 <ItemGroup>
 <PackageReference Include="Google.Protobuf" Version="3.6.1" />
 <PackageReference Include="Grpc" Version="1.17.0" />

 <!-- The Grpc.Tools package generates C# sources from .proto files during
 project build, but is not needed by projects using the built library.
 It's IMPORTANT to add the 'PrivateAssets="All"' to this reference: -->
 <PackageReference Include="Grpc.Tools" Version="1.17.0" PrivateAssets="All" />

 <!-- Explicitly include our helloworld.proto file by adding this line: -->
 <Protobuf Include="helloworld.proto" />
 </ItemGroup>

</Project>

Build it!

At this point you can build the project with the dotnet build command to compile the .proto file and the library assembly. For this walkthrough, we’ll add a logging switch -v:n to the command, so we can see that the command to compile the helloworld.proto file was in fact run. You may find it a good idea to always do that the very first time you compile a project!

Note that many output lines are omitted below, as the build output is quite verbose.

~/work/MyGreeter$ dotnet build -v:n

Build started 11/9/18 5:33:44 PM.
 1:7>Project "/home/kkm/work/MyGreeter/MyGreeter.csproj" on node 1 (Build target(s)).
 1>_Protobuf_CoreCompile:
 /home/kkm/.nuget/packages/grpc.tools/1.17.0/tools/linux_x64/protoc
 --csharp_out=obj/Debug/netstandard2.0
 --plugin=protoc-gen-grpc=/home/kkm/.nuget/packages/grpc.tools/1.17.0/tools/linux_x64/grpc_csharp_plugin
 --grpc_out=obj/Debug/netstandard2.0 --proto_path=/home/kkm/.nuget/packages/grpc.tools/1.17.0/build/native/include
 --proto_path=. --dependency_out=obj/Debug/netstandard2.0/da39a3ee5e6b4b0d_helloworld.protodep helloworld.proto
 CoreCompile:

 [ ... skipping long output ... ]

 MyGreeter -> /home/kkm/work/MyGreeter/bin/Debug/netstandard2.0/MyGreeter.dll

Build succeeded.

If at this point you invoke the dotnet build -v:n command again, protoc would not be invoked, and no C# sources would be compiled. But if you change the helloworld.proto source, then its outputs will be regenerated and then recompiled by the C# compiler during the build. This is a regular dependency tracking behavior that you expect from modifying any source file.

Of course, you can also add .cs files to the same project: It is a regular C# project building a .NET library, after all. This is done in our RouteGuide example.

Where are the generated files?

You may wonder where the proto compiler and gRPC plugin output C# files are. By default, they are placed in the same directory as other generated files, such as objects (termed the “intermediate output” directory in the .NET build parlance), under the obj/ directory. This is a regular practice of .NET builds, so that autogenerated files do not clutter the working directory or accidentally placed under source control. Otherwise, they are accessible to the tools like the debugger. You can see other autogenerated sources in that directory, too:

~/work/MyGreeter$ find obj -name '*.cs'
obj/Debug/netstandard2.0/MyGreeter.AssemblyInfo.cs
obj/Debug/netstandard2.0/Helloworld.cs
obj/Debug/netstandard2.0/HelloworldGrpc.cs

(use dir /s obj\*.cs if you are following this walkthrough from a Windows command prompt).

There Is More To It

While the simplest default behavior is adequate in many cases, there are many ways to fine-tune your .proto compilation process in a large project. We encourage you to read the documentation file BUILD-INTEGRATION.md for available options if you find that the default arrangement does not suit your workflow. The package also extends the Visual Studio’s Properties window, so you may set some options per file in the Visual Studio interface.

“Classic” .csproj projects and Mono are also supported.

As with any initial release of a complex feature, we are thrilled to receive your feedback. Did something not work as expected? Do you have a scenario that is not easy to cover with the new tools? Do you have an idea how to improve the workflow in general? Please read the documentation carefully, and then open an issue in the gRPC code repository on GitHub. Your feedback is important to determine the future direction for our build integration work!

Blog: gRPC ❤ Kotlin

Tue, 19 Jun 2018 00:00:00 +0000

Did you know that gRPC Java now has out of box support for Kotlin projects built with Gradle? Kotlin is a modern, statically typed language developed by JetBrains that targets the JVM and Android. It is generally easy for Kotlin programs to interoperate with existing Java libraries. To improve this experience further, we have added support to the protobuf-gradle-plugin so that the generated Java libraries are automatically picked up by Kotlin. You can now add the protobuf-gradle-plugin to your Kotlin project, and use gRPC just like you would with a typical Java project.

Native Kotlin support

Looking for native Kotlin support of gRPC? See Kotlin, meet gRPC.

The following examples show you how to configure a project for a JVM application and an Android application using Kotlin.

Kotlin gRPC client and server

The full example can be found here.

Configuring gRPC for a Kotlin project is the same as configuring it for a Java project.

Below is a snippet of the example project’s build.gradle highlighting some Kotlin related sections:

apply plugin: 'kotlin'
apply plugin: 'com.google.protobuf'

// Generate IntelliJ IDEA's .idea & .iml project files.
// protobuf-gradle-plugin automatically registers *.proto and the gen output files
// to IntelliJ as sources.
// For best results, install the Protobuf and Kotlin plugins for IntelliJ.
apply plugin: 'idea'

buildscript {
 ext.kotlin_version = '1.2.21'

 repositories {
 mavenCentral()
 }
 dependencies {
 classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.5'
 classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
 }
}

dependencies {
 compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"
 // The rest of the projects dep are added below, refer to example URL
}

// The standard protobuf block, same as normal gRPC Java projects
protobuf {
 protoc { artifact = 'com.google.protobuf:protoc:3.5.1-1' }
 plugins {
 grpc { artifact = "io.grpc:protoc-gen-grpc-java:${grpcVersion}" }
 }
 generateProtoTasks {
 all()*.plugins { grpc {} }
 }
}

Now Kotlin source files can use the proto generated messages and gRPC stubs. By default, Kotlin sources should be placed in src/main/kotlin and src/test/kotlin. If needed, run ./gradlew generateProto generateTestProto and refresh IntelliJ for the generated sources to appear in the IDE. Finally, run ./gradlew installDist to build the project, and use ./build/install/examples/bin/hello-world-client or ./build/install/examples/bin/hello-world-server to run the example.

You can read more about configuring Kotlin here.

Kotlin Android gRPC application

The full example can be found here.

Configuring gRPC for a Kotlin Android project is the same as configuring it for a normal Android project.

In the top level build.gradle file:

buildscript {
 ext.kotlin_version = '1.2.21'

 repositories {
 google()
 jcenter()
 }
 dependencies {
 classpath 'com.android.tools.build:gradle:3.0.1'
 classpath "com.google.protobuf:protobuf-gradle-plugin:0.8.5"
 classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
 }
}

allprojects {
 repositories {
 google()
 jcenter()
 }
}

And in the app module’s build.gradle file:

apply plugin: 'com.android.application'
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'
apply plugin: 'com.google.protobuf'

repositories {
 mavenCentral()
}

dependencies {
 compile "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
 // refer to full example for remaining deps
}

protobuf {
 // The normal gRPC configuration for Android goes here
}

android {
 // Android Studio 3.1 does not automatically pick up 'src/main/kotlin' as source files
 sourceSets {
 main.java.srcDirs += 'src/main/kotlin'
 }
}

Just like the non-Android project, run ./gradlew generateProto generateProto to run the proto code generator and ./gradlew build to build the project.

Finally, test out the Android app by opening the project in Android Studio and selecting Run > Run 'app'.

We are excited about improving the gRPC experience for Kotlin developers. Please add enhancement ideas or bugs to the protobuf-gradle-plugin issue tracker or the grpc-java issue tracker.

Blog: So You Want to Optimize gRPC - Part 2

Mon, 16 Apr 2018 00:00:00 +0000

How fast is gRPC? Pretty fast if you understand how modern clients and servers are built. In part 1, I showed how to get an easy 60% improvement. In this post I show how to get a 10000% improvement.

Setup

As in part 1, we will start with an existing, Java based, key-value service. The service will offer concurrent access for creating, reading, updating, and deleting keys and values. All the code can be seen here if you want to try it out.

Server Concurrency

Let’s look at the KvService class. This service handles the RPCs sent by the client, making sure that none of them accidentally corrupt the state of storage. To ensure this, the service uses the synchronized keyword to ensure only one RPC is active at a time:

private final Map<ByteBuffer, ByteBuffer> store = new HashMap<>();

@Override
public synchronized void create(
 CreateRequest request, StreamObserver<CreateResponse> responseObserver) {
 ByteBuffer key = request.getKey().asReadOnlyByteBuffer();
 ByteBuffer value = request.getValue().asReadOnlyByteBuffer();
 simulateWork(WRITE_DELAY_MILLIS);
 if (store.putIfAbsent(key, value) == null) {
 responseObserver.onNext(CreateResponse.getDefaultInstance());
 responseObserver.onCompleted();
 return;
 }
 responseObserver.onError(Status.ALREADY_EXISTS.asRuntimeException());
}

While this code is thread safe, it comes at a high price: only one RPC can ever be active! We need some way of allowing multiple operations to happen safely at the same time. Otherwise, the program can’t take advantage of all the available processors.

Breaking the Lock

To solve this, we need to know a little more about the semantics of our RPCs. The more we know about how the RPCs are supposed to work, the more optimizations we can make. For a key-value service, we notice that operations to different keys don’t interfere with each other. When we update key ‘foo’, it has no bearing on the value stored for key ‘bar’. But, our server is written such that operations to any key must be synchronized with respect to each other. If we could make operations to different keys happen concurrently, our server could handle a lot more load.

With the idea in place, we need to figure out how to modify the server. The synchronized keyword causes Java to acquire a lock on this, which is the instance of KvService. The lock is acquired when the create method is entered, and released on return. The reason we need synchronization is to protect the store Map. Since it is implemented as a HashMap, modifications to it change the internal arrays. Because the internal state of the HashMap will be corrupted if not properly synchronized, we can’t just remove the synchronization on the method.

However, Java offers a solution here: ConcurrentHashMap. This class offers the ability to safely access the contents of the map concurrently. For example, in our usage we want to check if a key is present. If not present, we want to add it, else we want to return an error. The putIfAbsent method atomically checks if a value is present, adds it if not, and tells us if it succeeded.

Concurrent maps provide stronger guarantees about the safety of putIfAbsent, so we can swap the HashMap to a ConcurrentHashMap and remove synchronized:

private final ConcurrentMap<ByteBuffer, ByteBuffer> store = new ConcurrentHashMap<>();

@Override
public void create(
 CreateRequest request, StreamObserver<CreateResponse> responseObserver) {
 ByteBuffer key = request.getKey().asReadOnlyByteBuffer();
 ByteBuffer value = request.getValue().asReadOnlyByteBuffer();
 simulateWork(WRITE_DELAY_MILLIS);
 if (store.putIfAbsent(key, value) == null) {
 responseObserver.onNext(CreateResponse.getDefaultInstance());
 responseObserver.onCompleted();
 return;
 }
 responseObserver.onError(Status.ALREADY_EXISTS.asRuntimeException());
}

If at First You Don’t Succeed

Updating create was pretty easy. Doing the same for retrieve and delete is easy too. However, the update method is a little trickier. Let’s take a look at what it’s doing:

@Override
public synchronized void update(
 UpdateRequest request, StreamObserver<UpdateResponse> responseObserver) {
 ByteBuffer key = request.getKey().asReadOnlyByteBuffer();
 ByteBuffer newValue = request.getValue().asReadOnlyByteBuffer();
 simulateWork(WRITE_DELAY_MILLIS);
 ByteBuffer oldValue = store.get(key);
 if (oldValue == null) {
 responseObserver.onError(Status.NOT_FOUND.asRuntimeException());
 return;
 }
 store.replace(key, oldValue, newValue);
 responseObserver.onNext(UpdateResponse.getDefaultInstance());
 responseObserver.onCompleted();
}

Updating a key to a new value needs two interactions with the store:

Check to see if the key exists at all.
Update the previous value to the new value.

Unfortunately ConcurrentMap doesn’t have a straightforward method to do this. Since we may not be the only ones modifying the map, we need to handle the possibility that our assumptions have changed. We read the old value out, but by the time we replace it, it may have been deleted.

To reconcile this, let’s retry if replace fails. It returns true if the replace was successful. (ConcurrentMap asserts that the operations will not corrupt the internal structure, but doesn’t say that they will succeed!) We will use a do-while loop:

@Override
public void update(
 UpdateRequest request, StreamObserver<UpdateResponse> responseObserver) {
 // ...
 ByteBuffer oldValue;
 do {
 oldValue = store.get(key);
 if (oldValue == null) {
 responseObserver.onError(Status.NOT_FOUND.asRuntimeException());
 return;
 }
 } while (!store.replace(key, oldValue, newValue));
 responseObserver.onNext(UpdateResponse.getDefaultInstance());
 responseObserver.onCompleted();
}

The code wants to fail if it ever sees null, but never if there is a non-null previous value. One thing to note is that if another RPC modifies the value between the store.get() call and the store.replace() call, it will fail. This is a non-fatal error for us, so we will just try again. Once it has successfully put the new value in, the service can respond back to the user.

There is one other possibility that could happen: two RPCs could update the same value and overwrite each other’s work. While this may be okay for some applications, it would not be suitable for APIs that provide transactionality. It is out of scope for this post to show how to fix this, but be aware it can happen.

Measuring the Performance

In the last post, we modified the client to be asynchronous and use the gRPC ListenableFuture API. To avoid running out of memory, the client was modified to have at most 100 active RPCs at a time. As we now see from the server code, performance was bottlenecked on acquiring locks. Since we have removed those, we expect to see a 100x improvement. The same amount of work is done per RPC, but a lot more are happening at the same time. Let’s see if our hypothesis holds:

Before:

./gradlew installDist
time ./build/install/kvstore/bin/kvstore
Apr 16, 2018 10:38:42 AM io.grpc.examples.KvRunner runClient
INFO: Did 24.067 RPCs/s

real 1m0.886s
user 0m9.340s
sys 0m1.660s

After:

Apr 16, 2018 10:36:48 AM io.grpc.examples.KvRunner runClient
INFO: Did 2,449.8 RPCs/s
real 1m0.968s
user 0m52.184s
sys 0m20.692s

Wow! From 24 RPCs per second to 2,400 RPCs per second. And we didn’t have to change our API or our client. This is why understanding your code and API semantics is important. By exploiting the properties of the key-value API, namely the independence of operations on different keys, the code is now much faster.

One noteworthy artifact of this code is the user timing in the results. Previously the user time was only 9 seconds, meaning that the CPU was active only 9 of the 60 seconds the code was running. Afterwards, the usage went up by more than 5x to 52 seconds. The reason is that more CPU cores are active. The KvServer is simulating work by sleeping for a few milliseconds. In a real application, it would be doing useful work and not have such a dramatic change. Rather than scaling per the number of RPCs, it would scale per the number of cores. Thus, if your machine had 12 cores, you would expect to see a 12x improvement. Still not bad though!

More Errors

If you run this code yourself, you will see a lot more log spam in the form:

Apr 16, 2018 10:38:40 AM io.grpc.examples.KvClient$3 onFailure
INFO: Key not found
io.grpc.StatusRuntimeException: NOT_FOUND

The reason is that the new version of the code makes API level race conditions more apparent. With 100 times as many RPCs happening, the chance of updates and deletes colliding with each other is more likely. To solve this we will need to modify the API definition. Stay tuned for the next post showing how to fix this.

Conclusion

There are a lot of opportunities to optimize your gRPC code. To take advantage of these, you need to understand what your code is doing. This post shows how to convert a lock-based service into a low-contention, lock-free service. Always make sure to measure before and after your changes.

Blog: So You Want to Optimize gRPC - Part 1

Tue, 06 Mar 2018 00:00:00 +0000

A common question with gRPC is how to make it fast. The gRPC library offers users access to high performance RPCs, but it isn’t always clear how to achieve this. Because this question is common enough I thought I would try to show my thought process when tuning programs.

Setup

Consider a basic key-value service that is used by multiple other programs. The service needs to be safe for concurrent access in case multiple updates happen at the same time. It needs to be able to scale up to use the available hardware. Lastly, it needs to be fast. gRPC is a perfect fit for this type of service; let’s look at the best way to implement it.

For this blog post, I have written an example client and server using gRPC Java. The program is split into three main classes, and a protobuf file describing the API:

KvClient is a simulated user of the key value system. It randomly creates, retrieves, updates, and deletes keys and values. The size of keys and values it uses is also randomly decided using an exponential distribution.
KvService is an implementation of the key value service. It is installed by the gRPC Server to handle the requests issued by the client. To simulate storing the keys and values on disk, it adds short sleeps while handling the request. Reads and writes will experience a 10 and 50 millisecond delay to make the example act more like a persistent database.
KvRunner orchestrates the interaction between the client and the server. It is the main entry point, starting both the client and server in process, and waiting for the the client to execute its work. The runner does work for 60 seconds and then records how many RPCs were completed.
kvstore.proto is the protocol buffer definition of our service. It describes exactly what clients can expect from the service. For the sake of simplicity, we will use Create, Retrieve, Update, and Delete as the operations (commonly known as CRUD). These operations work with keys and values made up of arbitrary bytes. While they are somewhat REST like, we reserve the right to diverge and add more complex operations in the future.

Protocol buffers (protos) aren’t required to use gRPC, they are a very convenient way to define service interfaces and generate client and server code. The generated code acts as glue code between the application logic and the core gRPC library. We refer to the code called by a gRPC client the stub.

Starting Point

Client

Now that we know what the program should do, we can start looking at how the program performs. As mentioned above, the client makes random RPCs. For example, here is the code that makes the creation request:

private void doCreate(KeyValueServiceBlockingStub stub) {
 ByteString key = createRandomKey();
 try {
 CreateResponse res = stub.create(
 CreateRequest.newBuilder()
 .setKey(key)
 .setValue(randomBytes(MEAN_VALUE_SIZE))
 .build());
 if (!res.equals(CreateResponse.getDefaultInstance())) {
 throw new RuntimeException("Invalid response");
 }
 } catch (StatusRuntimeException e) {
 if (e.getStatus().getCode() == Code.ALREADY_EXISTS) {
 knownKeys.remove(key);
 logger.log(Level.INFO, "Key already existed", e);
 } else {
 throw e;
 }
 }
}

A random key is created, along with a random value. The request is sent to the server, and the client waits for the response. When the response is returned, the code checks that it is as expected, and if not, throws an exception. While the keys are chosen randomly, they need to be unique, so we need to make sure that each key isn’t already in use. To address this, the code keeps track of keys it has created, so as not to create the same key twice. However, it’s possible that another client already created a particular key, so we log it and move on. Otherwise, an exception is thrown.

We use the blocking gRPC API here, which issues a requests and waits for a response. This is the simplest gRPC stub, but it blocks the thread while running. This means that at most one RPC can be in progress at a time from the client’s point of view.

Server

On the server side, the request is received by the service handler:

private final Map<ByteBuffer, ByteBuffer> store = new HashMap<>();

@Override
public synchronized void create(
 CreateRequest request, StreamObserver<CreateResponse> responseObserver) {
 ByteBuffer key = request.getKey().asReadOnlyByteBuffer();
 ByteBuffer value = request.getValue().asReadOnlyByteBuffer();
 simulateWork(WRITE_DELAY_MILLIS);
 if (store.putIfAbsent(key, value) == null) {
 responseObserver.onNext(CreateResponse.getDefaultInstance());
 responseObserver.onCompleted();
 return;
 }
 responseObserver.onError(Status.ALREADY_EXISTS.asRuntimeException());
}

The service extracts the key and value as ByteBuffers from the request. It acquires the lock on the service itself to make sure concurrent requests don’t corrupt the storage. After simulating the disk access of a write, it stores it in the Map of keys to values.

Unlike the client code, the service handler is non-blocking, meaning it doesn’t return a value like a function call would. Instead, it invokes onNext() on the responseObserver to send the response back to the client. Note that this call is also non-blocking, meaning that the message may not yet have been sent. To indicate we are done with the message, onCompleted() is called.

Performance

Since the code is safe and correct, let’s see how it performs. For my measurement I’m using my Ubuntu system with a 12 core processor and 32 GB of memory. Let’s build and run the code:

./gradlew installDist
time ./build/install/kvstore/bin/kvstore
Feb 26, 2018 1:10:07 PM io.grpc.examples.KvRunner runClient
INFO: Starting
Feb 26, 2018 1:11:07 PM io.grpc.examples.KvRunner runClient
INFO: Did 16.55 RPCs/s

real 1m0.927s
user 0m10.688s
sys 0m1.456s

Yikes! For such a powerful machine, it can only do about 16 RPCs per second. It hardly used any of our CPU, and we don’t know how much memory it was using. We need to figure out why it’s so slow.

Optimization

Analysis

Let’s understand what the program is doing before we make any changes. When optimizing, we need to know where the code is spending its time in order to know what we can optimize. At this early stage, we don’t need profiling tools yet, we can just reason about the program.

The client is started and serially issues RPCs for about a minute. Each iteration, it randomly decides what operation to do:

void doClientWork(AtomicBoolean done) {
 Random random = new Random();
 KeyValueServiceBlockingStub stub = KeyValueServiceGrpc.newBlockingStub(channel);

 while (!done.get()) {
 // Pick a random CRUD action to take.
 int command = random.nextInt(4);
 if (command == 0) {
 doCreate(stub);
 continue;
 }
 /* ... */
 rpcCount++;
 }
}

This means that at most one RPC can be active at any time. Each RPC has to wait for the previous one to complete. And how long does each RPC take to complete? From reading the server code, most of the operations are doing a write which takes about 50 milliseconds. At top efficiency, the most operations this code can do per second is about 20:

20 queries = 1000ms / (50 ms / query)

Our code can do about 16 queries in a second, so that seems about right. We can spot check this assumption by looking at the output of the time command used to run the code. The server goes to sleep when running queries in the simulateWork method. This implies that the program should be mostly idle while waiting for the RPCs to complete.

We can confirm this is the case by looking at the real and user times of the command above. They say that the amount of wall clock time was 1 minute, while the amount of cpu time was 10 seconds. My powerful, multicore CPU was only busy 16% of the time. Thus, if we could get the program to do more work during that time, it seems like we could get more RPCs complete.

Hypothesis

Now we can state clearly what we think is the problem, and propose a solution. One way to speed up programs is to make sure the CPU is not idling. To do this, we issue work concurrently.

In gRPC Java, there are three types of stubs: blocking, non-blocking, and listenable future. We have already seen the blocking stub in the client, and the non-blocking stub in the server. The listenable future API is a compromise between the two, offering both blocking and non-blocking like behavior. As long as we don’t block a thread waiting for work to complete, we can start new RPCs without waiting for the old ones to complete.

Experiment

To test our hypothesis, let’s modify the client code to use the listenable future API. This means that we need to think more about concurrency in our code. For example, when keeping track of known keys client-side, we need to safely read, modify, and write the keys. We also need to make sure that in case of an error, we stop making new RPCs (proper error handling will be covered in a future post). Lastly, we need to update the number of RPCs made concurrently, since the update could happen in another thread.

Making all these changes increases the complexity of the code. This is a trade off you will need to consider when optimizing your code. In general, code simplicity is at odds with optimization. Java is not known for being terse. That said, the code below is still readable, and program flow is still roughly from top to bottom in the function. Here is the doCreate() method revised:

private void doCreate(KeyValueServiceFutureStub stub, AtomicReference<Throwable> error) {
 ByteString key = createRandomKey();
 ListenableFuture<CreateResponse> res = stub.create(
 CreateRequest.newBuilder()
 .setKey(key)
 .setValue(randomBytes(MEAN_VALUE_SIZE))
 .build());
 res.addListener(() -> rpcCount.incrementAndGet(), MoreExecutors.directExecutor());
 Futures.addCallback(res, new FutureCallback<CreateResponse>() {
 @Override
 public void onSuccess(CreateResponse result) {
 if (!result.equals(CreateResponse.getDefaultInstance())) {
 error.compareAndSet(null, new RuntimeException("Invalid response"));
 }
 synchronized (knownKeys) {
 knownKeys.add(key);
 }
 }

 @Override
 public void onFailure(Throwable t) {
 Status status = Status.fromThrowable(t);
 if (status.getCode() == Code.ALREADY_EXISTS) {
 synchronized (knownKeys) {
 knownKeys.remove(key);
 }
 logger.log(Level.INFO, "Key already existed", t);
 } else {
 error.compareAndSet(null, t);
 }
 }
 });
}

The stub has been modified to be a KeyValueServiceFutureStub, which produces a Future when called instead of the response itself. gRPC Java uses an extension of this called ListenableFuture, which allows adding a callback when the future completes. For the sake of this program, we are not as concerned with getting the response. Instead we care more if the RPC succeeded or not. With that in mind, the code mainly checks for errors rather than processing the response.

The first change made is how the number of RPCs is recorded. Instead of incrementing the counter outside of the main loop, we increment it when the RPC completes.

Next, we create a new object for each RPC which handles both the success and failure cases. Because doCreate() will already be completed by the time RPC callback is invoked, we need a way to propagate errors other than by throwing. Instead, we try to update an reference atomically. The main loop will occasionally check if an error has occurred and stop if there is a problem.

Lastly, the code is careful to only add a key to knownKeys when the RPC is actually complete, and only remove it when known to have failed. We synchronize on the variable to make sure two threads don’t conflict. Note: although the access to knownKeys is threadsafe, there are still race conditions. It is possible that one thread could read from knownKeys, a second thread delete from knownKeys, and then the first thread issue an RPC using the first key. Synchronizing on the keys only ensures that it is consistent, not that it is correct. Fixing this properly is outside of the scope of this post, so instead we just log the event and move on. You will see a few such log statements if you run this program.

Running the Code

If you start up this program and run it, you’ll notice that it doesn’t work:

WARNING: An exception was thrown by io.grpc.netty.NettyClientStream$Sink$1.operationComplete()
java.lang.OutOfMemoryError: unable to create new native thread
 at java.lang.Thread.start0(Native Method)
 at java.lang.Thread.start(Thread.java:714)
 ...

What?! Why would I show you code that fails? The reason is that in real life making a change often doesn’t work on the first try. In this case, the program ran out of memory. Odd things begin to happen when a program runs out of memory. Often, the root cause is hard to find, and red herrings abound. A confusing error message says “unable to create new native thread” even though we didn’t create any new threads in our code. Experience is very helpful in fixing these problems rather than debugging. Since I have debugged many OOMs, I happen to know Java tells us about the straw that broke the camel’s back. Our program started using way more memory, but the final allocation that failed happened, by chance, to be in thread creation.

So what happened? There was no pushback to starting new RPCs. In the blocking version, a new RPC couldn’t start until the last one completed. While slow, it also prevented us from creating tons of RPCs that we didn’t have memory for. We need to account for this in the listenable future version.

To solve this, we can apply a self-imposed limit on the number of active RPCs. Before starting a new RPC, we will try to acquire a permit. If we get one, the RPC can start. If not, we will wait until one is available. When an RPC completes (either in success or failure), we return the permit. To accomplish this, we will using a Semaphore:

private final Semaphore limiter = new Semaphore(100);

private void doCreate(KeyValueServiceFutureStub stub, AtomicReference<Throwable> error)
 throws InterruptedException {
 limiter.acquire();
 ByteString key = createRandomKey();
 ListenableFuture<CreateResponse> res = stub.create(
 CreateRequest.newBuilder()
 .setKey(key)
 .setValue(randomBytes(MEAN_VALUE_SIZE))
 .build());
 res.addListener(() -> {
 rpcCount.incrementAndGet();
 limiter.release();
 }, MoreExecutors.directExecutor());
 /* ... */
}

Now the code runs successfully, and doesn’t run out of memory.

Results

Building and running the code again looks a lot better:

./gradlew installDist
time ./build/install/kvstore/bin/kvstore
Feb 26, 2018 2:40:47 PM io.grpc.examples.KvRunner runClient
INFO: Starting
Feb 26, 2018 2:41:47 PM io.grpc.examples.KvRunner runClient
INFO: Did 24.283 RPCs/s

real 1m0.923s
user 0m12.772s
sys 0m1.572s

Our code does 46% more RPCs per second than previously. We can also see that we used about 20% more CPU than previously. As we can see our hypothesis turned out to be correct and the fix worked. All this happened without making any changes to the server. Also, we were able to measure without using any special profilers or tracers.

Do the numbers make sense? We expect to issue mutation (create, update, and delete) RPCs each about with 1/4 probability. Reads are also issue 1/4 of the time, but don’t take as long. The mean RPC time should be about the weighted average RPC time:

 .25 * 50ms (create)
.25 * 10ms (retrieve)
.25 * 50ms (update)
+.25 * 50ms (delete)
------------
40ms

At 40ms on average per RPC, we would expect the number of RPCs per second to be:

25 queries = 1000ms / (40 ms / query)

That’s approximately what we see with the new code. The server is still serially handling requests, so it seems like we have more work to do in the future. But for now, our optimizations seem to have worked.

Conclusion

There are a lot of opportunities to optimize your gRPC code. To take advantage of these, you need to understand what your code is doing, and what your code is supposed to do. This post shows the very basics of how to approach and think about optimization. Always make sure to measure before and after your changes, and use these measurements to guide your optimizations.

In Part 2, we will continue optimizing the server part of the code.

Blog: gRPC and Deadlines

Mon, 26 Feb 2018 00:00:00 +0000

TL;DR: Always set a deadline. This post explains why we recommend being deliberate about setting deadlines, with useful code snippets to show you how.

When you use gRPC, the gRPC library takes care of communication, marshalling, unmarshalling, and deadline enforcement. Deadlines allow gRPC clients to specify how long they are willing to wait for an RPC to complete before the RPC is terminated with the error DEADLINE_EXCEEDED. By default this deadline is a very large number, dependent on the language implementation. How deadlines are specified is also language-dependent. Some language APIs work in terms of a deadline, a fixed point in time by which the RPC should complete. Others use a timeout, a duration of time after which the RPC times out.

In general, when you don’t set a deadline, resources will be held for all in-flight requests, and all requests can potentially reach the maximum timeout. This puts the service at risk of running out of resources, like memory, which would increase the latency of the service, or could crash the entire process in the worst case.

To avoid this, services should specify the longest default deadline they technically support, and clients should wait until the response is no longer useful to them. For the service this can be as simple as providing a comment in the .proto file. For the client this involves setting useful deadlines.

There is no single answer to “What is a good deadline/timeout value?”. Your service might be as simple as the Greeter in our quick start guides, in which case 100 ms would be fine. Your service might be as complex as a globally-distributed and strongly consistent database. The deadline for a client query will be different from how long they should wait for you to drop their table.

So what do you need to consider to make an informed choice of deadline? Factors to take into account include the end to end latency of the whole system, which RPCs are serial, and which can be made in parallel. You should to be able to put numbers on it, even if it’s a rough calculation. Engineers need to understand the service and then set a deliberate deadline for the RPCs between clients and servers.

In gRPC, both the client and server make their own independent and local determination about whether the remote procedure call (RPC) was successful. This means their conclusions may not match! An RPC that finished successfully on the server side can fail on the client side. For example, the server can send the response, but the reply can arrive at the client after their deadline has expired. The client will already have terminated with the status error DEADLINE_EXCEEDED. This should be checked for and managed at the application level.

Setting a deadline

As a client you should always set a deadline for how long you are willing to wait for a reply from the server. Here are examples using the Greeting service from the Quick start pages:

C++

ClientContext context;
time_point deadline = std::chrono::system_clock::now() +
 std::chrono::milliseconds(100);
context.set_deadline(deadline);

Go

clientDeadline := time.Now().Add(time.Duration(*deadlineMs) * time.Millisecond)
ctx, cancel := context.WithDeadline(ctx, clientDeadline)

Java

response = blockingStub.withDeadlineAfter(deadlineMs, TimeUnit.MILLISECONDS).sayHello(request);

This sets the deadline to 100ms from when the client RPC is set to when the response is picked up by the client.

Checking deadlines

On the server side, the server can query to see if a particular RPC is no longer wanted. Before a server starts work on a response it is very important to check if there is still a client waiting for it. This is especially important to do before starting expensive processing.

C++

if (context->IsCancelled()) {
 return Status(StatusCode::CANCELLED, "Deadline exceeded or Client cancelled, abandoning.");
}

Go

if ctx.Err() == context.Canceled {
 return status.New(codes.Canceled, "Client cancelled, abandoning.")
}

Java

if (Context.current().isCancelled()) {
 responseObserver.onError(Status.CANCELLED.withDescription("Cancelled by client").asRuntimeException());
 return;
}

Is it useful for a server to continue with the request, when you know your client has reached their deadline? It depends. If the response can be cached in the server, it can be worth processing and caching it; particularly if it’s resource heavy, and costs you money for each request. This will make future requests faster as the result will already be available.

Adjusting deadlines

What if you set a deadline but a new release or server version causes a bad regression? The deadline could be too small, resulting in all your requests timing out with DEADLINE_EXCEEDED, or too large and your user tail latency is now massive. You can use a flag to set and adjust the deadline.

C++

#include <gflags/gflags.h>
DEFINE_int32(deadline_ms, 20*1000, "Deadline in milliseconds.");

ClientContext context;
time_point deadline = std::chrono::system_clock::now() +
 std::chrono::milliseconds(FLAGS_deadline_ms);
context.set_deadline(deadline);

Go

var deadlineMs = flag.Int("deadline_ms", 20*1000, "Default deadline in milliseconds.")

ctx, cancel := context.WithTimeout(ctx, time.Duration(*deadlineMs) * time.Millisecond)

Java

@Option(name="--deadline_ms", usage="Deadline in milliseconds.")
private int deadlineMs = 20*1000;

response = blockingStub.withDeadlineAfter(deadlineMs, TimeUnit.MILLISECONDS).sayHello(request);

Now the deadline can be adjusted to wait longer to avoid failing, without the need to cherry-pick a release with a different hard coded deadline. This lets you mitigate the issue for users until the regression can be debugged and resolved.

Blog: gRPC-Go Engineering Practices

Mon, 22 Jan 2018 00:00:00 +0000

It’s the start of the new year, and almost the end of my first full year on the gRPC-Go project, so I’d like to take this opportunity to provide an update on the state of gRPC-Go development and give some visibility into how we manage the project. For me, personally, this is the first open source project to which I’ve meaningfully contributed, so this year has been a learning experience for me. Over this year, the team has made constant improvements to our work habits and communication. I still see room for improvement, but I believe we are in a considerably better place than we were a year ago.

Repo Health

When I first joined the gRPC-Go team, they had been without their previous technical lead for a few months. At that time, we had 45 open PRs, the oldest of which was over a year old at the time. As a new team member and maintainer, the accumulation of stale PRs made it difficult to assess priorities and understand the state of things. For our contributors, neglecting PRs was both disrespectful and an inconvenience when we started asking for rebases due to other commits. To resolve this, we made a concerted effort to either merge or close all of those PRs, and we now hold weekly meetings to review the status of every active PR to prevent the situation from reoccurring.

At the same time, we had 103 open issues, many of which were already fixed or outdated or untriaged. Since then, we fixed or closed 85 of those and put in place a process to ensure we triage and prioritize new issues on a weekly rotation. Similarly to our PRs, we also review our assigned and high-priority issues in a weekly meeting.

Our ongoing SLO for new issues and PRs is 1 week to triage and first response.

We also revamped our labels for issues and PRs to help with organization. We typically apply a priority (P0-P3) and a type (for example, Bug, Feature, or Performance) to every issue. We also have a collection of status labels we apply in various situations. The type labels are also applied to PRs to aid in generating our release notes.

Versioning and Backward Compatibility

We have recently documented our versioning policy. Our goal is to maintain full backward compatibility except in limited circumstances, including experimental APIs and mitigating security risks (most notably #1392). If you notice a behavior regression, please don’t hesitate to open an issue in our repo (please be reasonable).

gRFC

The gRPC proposal repo contains proposals for substantial feature changes for gRPC that need to be designed upfront, called gRFCs. The purpose of this process is to provide visibility and solicit feedback from the community. Each change is discussed on our mailing list and debated before the change is made. We leveraged this before making the backward-compatibility-breaking metadata change (gRFC L7), and also for designing the new resolver/balancer API (gRFC L9).

Regression Testing

Every PR in our repo must pass our unit and end-to-end tests. Our current test coverage is 85%. Anytime a regression is identified, we add a test that covers the failing scenario, both to prove to ourselves that the problem is resolved by the fix, and to prevent it from reoccurring in the future. This helps us improve our overall coverage numbers as well. We also intend to re-enable coverage reporting for all PRs, but in a non-blocking fashion (related issue).

In addition to testing for correctness, any PR that we suspect will impact performance is run though our benchmarks. We have a set of benchmarks both in our open source repo and also within Google. These comprise a variety of workloads that we believe are most important for our users, both streaming and unary, and some are specifically designed to measure our optimal QPS, throughput, or latency.

Releases

The GA release of gRPC-Go was made in conjunction with the other languages in July of 2016. The team performed several patch releases between then and the end of 2016, but none included release notes. Our subsequent releases have improved in regularity (a minor release is performed every six weeks) and in the quality of the release notes. We also are responsive with patch releases, back-porting bug fixes to older releases either on demand or for more serious issues within a week.

When performing a release, in addition to the tests in our repo, we also run a full suite of inter-op tests with other gRPC language implementations. This process has been working well for us, and we will cover more about this in a future blog post.

Non-Open Source Work

We have taken an “open source first” approach to developing gRPC. This means that, wherever possible, gRPC functionality is added directly into the open source project. However, to work within Google’s infrastructure, our team sometimes needs to provide additional functionality on top of gRPC. This is typically done through hooks like the stats API or interceptors or custom resolvers.

To keep Google’s internal version of gRPC up-to-date with the open source version, we do weekly or on-demand imports. Before an import, we run every test within Google that depends upon gRPC. This gives us another way in which we can catch problems before performing releases in Open Source.

Looking Forward

In 2018, we intend to do more of the same, and maintain our SLOs around addressing issues and accepting contributions to the project. We also would like to more aggressively tag issues with the Help Wanted label for anyone looking to contribute to have a bigger selection of issues to choose from.

For gRPC itself, one of our main focuses right now is performance, which we hope will transparently benefit many of our users. In the near-term, we have some exciting changes we’re wrapping up that should provide a 30+% reduction in latency with high concurrency, resulting in a QPS improvement of ~25%. Once that work is done, we have a list of other performance issues that we’ll be tackling next.

On user experience, we want to provide better documentation, and are starting to improve our godoc with better comments and more examples. We want to improve the overall experience of using gRPC, so we will be working closely on projects around distributed tracing, monitoring, and testing to make gRPC services easier to manage in production. We want to do more, and we are hoping that starting with these and listening to feedback will help us ship improvements steadily.

Blog: The gRPC Meetup Kit

Thu, 14 Sep 2017 00:00:00 +0000

If you have ever wanted to run an event around gRPC, but didn’t know where to start, or weren’t sure what content is available - we have released the gRPC Meetup Kit!

The meetup kit includes a 15 minute presentation on the basic concepts of gRPC, with accompanying slides and video for either reference or playback, as well as a 45-minute codelab that takes you through the basics of gRPC in Node.js and Go. At the end of the codelab participants will have a solid understanding of the fundamentals of gRPC.

If you are thinking about running a gRPC event, make sure to contact us to receive gRPC stickers and/or organise office hours over Hangouts with the gRPC team!

Blog: gRPC-Go performance Improvements

Tue, 22 Aug 2017 00:00:00 +0000

For past few months we’ve been working on improving gRPC-Go performance. This includes improving network utilization, optimizing CPU usage and memory allocations. Most of our recent effort has been focused around revamping gRPC-Go flow control. After several optimizations and new features we’ve been able to improve quite significantly, especially on high-latency networks. We expect users that are working with high-latency networks and large messages to see an order of magnitude performance gain. Benchmark results at the end.

This blog summarizes the work we have done so far (in chronological order) to improve performance and lays out our near-future plans.

Recently Implemented Optimizations

Expanding stream window on receiving large messages

Code link

This is an optimization used by gRPC-C to achieve performance benefits for large messages. The idea is that when there’s an active read by the application on the receive side, we can effectively bypass stream-level flow control to request the whole message. This proves to be very helpful with large messages. Since the application is already committed to reading and has allocated enough memory for it, it makes sense that we send a proactive large window update (if necessary) to get the whole message rather than receiving it in chunks and sending window updates when we run low on window.

This optimization alone provided a 10x improvement for large messages on high-latency networks.

Decoupling application reads from connection flow control

Code link

After having several discussions with gRPC-Java and gRPC-C core team we realized that gRPC-Go’s connection-level flow control was overly restrictive in the sense that window updates on the connection depended upon if the application had read data from it or not. It must be noted that it makes perfect sense to have stream-level flow control depended on application read but not so much for connection-level flow control. The rationale is as follows: A connection is shared by several streams (RPCs). If there were at least one stream that read slowly or didn’t read at all, it would hamper the performance or completely stall other streams on that connection. This happens because we won’t send out window updates on the connection until that slow or inactive stream read data. Therefore, it makes sense to decouple the connection’s flow control from application reads.

However, this begs at least two questions:

Won’t a client be able to send as much data as it wants to the server by creating new streams when one runs out?
Why even have connection-level flow control if the stream-level flow control is enough?

The answer to the first question is short and simple: no. A server has an option to limit the number of streams that it intends to serve concurrently. Therefore, although at first it may seem like a problem, it really is not.

The need for connection-level flow control:

It is true that stream-level flow control is sufficient to throttle a sender from sending too much data. But not having connection-level flow control (or using an unlimited connection-level window) makes it so that when things get slower on a stream, opening a new one will appear to make things faster. This will only take one so far since the number of streams are limited. However, having a connection-level flow control window set to the Bandwidth Delay Product (BDP) of the network puts an upper-bound on how much performance can realistically be squeezed out of the network.

Piggyback window updates

Code link

Sending a window update itself has a cost associated to it; a flush operation is necessary, which results in a syscall. Syscalls are blocking and slow. Therefore, when sending out a stream-level window update, it makes sense to also check if a connection-level window update can be sent using the same flush syscall.

BDP estimation and dynamic flow control window

Code link

This feature is the latest and in some ways the most awaited optimization feature that has helped us close the final gap between gRPC and HTTP/1.1 performance on high latency networks.

Bandwidth Delay Product (BDP) is the bandwidth of a network connection times its round-trip latency. This effectively tells us how many bytes can be “on the wire” at a given moment, if full utilization is achieved.

The algorithm to compute BDP and adapt accordingly was first proposed by @ejona and later implemented by both gRPC-C core and gRPC-Java (note that it isn’t enabled in Java yet). The idea is simple and powerful: every time a receiver gets a data frame it sends out a BDP ping (a ping with unique data only used by BDP estimator). After this, the receiver starts counting the number of bytes it receives (including the ones that triggered the BDP ping) until it receives the ack for that ping. This total sum of all bytes received in about 1.5 RTT (Round-Trip Time) is an approximation of the effective BDP * 1.5. If this is close to our current window size (say, more than 2/3rd of it) we must increase the window size. We put our window sizes (both streaming and connection) to be twice the BDP we sampled(total sum of all bytes received).

This algorithm by itself could cause the BDP estimation to increase indefinitely; an increase in window will result in sampling more bytes which in turn will cause the window to be increased further. This phenomenon is called buffer-bloat and was discovered by earlier implementations in gRPC-C core and gRPC-Java. The solution to this is to calculate the bandwidth for every sample and check if it is greater than the maximum bandwidth noted so far. If so, only then increase our window sizes. The bandwidth, as we know, can be calculated by dividing the sample by RTT * 1.5 (remember the sample was for one and a half round trips). If the bandwidth doesn’t increase with an increase in sampled bytes that’s indicative that this change is because of an increased window size and doesn’t really reflect the nature of the network itself.

While running experiments on VMs in different continents we realized that every once in awhile a rogue, unnaturally fast ping-ack at the right time (really the wrong time) would cause our window sizes to go up. This happens because such a ping-ack would cause us to notice a decreased RTT and calculate a high bandwidth value. Now if that sample of bytes was greater than 2/3rd of our window then we would increase the window sizes. However, this ping ack was an aberration and shouldn’t have changed our perception of the network RTT al together. Therefore, we keep a running average of the RTTs we note weighted by a constant rather than the total number of samples to heed more to recent RTTs and less to the ones in past. It is important because networks might change over time.

During implementation, we experimented with several tuning parameters, such as the multiplier to compute the window size from the sample size to select the best settings, that balanced between growth and accuracy.

Given that we’re always bound by the flow control of TCP which for most cases is upper bounded at 4MB, we bound the growth of our window sizes by the same number: 4MB.

BDP estimation and dynamically adjusting window sizes is turned-on by default and can be turned off by setting values manually for connection and/or stream window sizes.

Near-future efforts

We are now looking into improving our throughput by better CPU utilization, the following efforts are in-line with that.

Reducing flush syscalls

We noticed a bug in our transport layer which causes us to make a flush syscall for every data frame we write, even if the same goroutine has more data to send. We can batch a lot of these writes to use only one flush. This in fact will not be a big change to the code itself.

In our efforts to get rid of unnecessary flushes we recently combined the headers and data write for unary and server streaming RPCs to one flush on the client-side. Link to code

Another related idea proposed by one of our users @petermattic in this PR was to combine a server response to a unary RPC into one flush. We are currently looking into that as well.

Reducing memory allocation

For every data frame read from the wire a new memory allocation takes place. The same holds true at the gRPC layer for every new message for decompressing and decoding. These allocations result in excessive garbage collection cycles, which are expensive. Reusing memory buffers can reduce this GC pressure, and we are prototyping approaches to do so. As requests need buffers of differing sizes, one approach would be to maintain individual memory pools of fixed sizes (powers of two). So now when reading x bytes from the wire we can find the nearest power of 2 greater than x and reuse a buffer from our cache if available or allocate a new one if need be. We will be using golang sync Pools so we don’t have to worry about garbage collection. However, we will need to run sufficient tests before committing to this.

Results

Benchmark on a real network:
- Server and client were launched on two VMs in different continents. RTT of ~152ms.
- Client made an RPC with a payload and server responded back with an empty message.
- The time taken for each RPC was measured.
- Code link

Message size	gRPC	HTTP 1.1
1 KB	~152 ms	~152 ms
10 KB	~152 ms	~152 ms
10 KB	~152 ms	~152 ms
1 MB	~152 ms	~152 ms
10 MB	~622 ms	~630 ms
100 MB	~5 sec	~5 sec

Benchmark on simulated network:
- Server and client were launched on the same machine and different network latencies were simulated.
- Client made an RPC with 1MB of payload and server responded back with an empty message.
- The time taken for each RPC was measured.
- Following tables show time taken by first 10 RPCs.
- Code link

No-latency Network

gRPC	HTTP 2.0	HTTP 1.1
5.097809ms	16.107461ms	18.298959ms
4.46083ms	4.301808ms	7.715456ms
5.081421ms	4.076645ms	8.118601ms
4.338013ms	4.232606ms	6.621028ms
5.013544ms	4.693488ms	5.83375ms
3.963463ms	4.558047ms	5.571579ms
3.509808ms	4.855556ms	4.966938ms
4.864618ms	4.324159ms	6.576279ms
3.545933ms	4.61375ms	6.105608ms
3.481094ms	4.621215ms	7.001607ms

Network with RTT of 16ms

gRPC	HTTP 2.0	HTTP 1.1
118.837625ms	84.453913ms	58.858109ms
36.801006ms	22.476308ms	20.877585ms
35.008349ms	21.206222ms	19.793881ms
21.153461ms	20.940937ms	22.18179ms
20.640364ms	21.888247ms	21.4666ms
21.410346ms	21.186008ms	20.925514ms
19.755766ms	21.818027ms	20.553768ms
20.388882ms	21.366796ms	21.460029ms
20.623342ms	20.681414ms	20.586908ms
20.452023ms	20.781208ms	20.278481ms

Network with RTT of 64ms

GRPC	HTTP 2.0	HTTP 1.1
455.072669ms	275.290241ms	208.826314ms
195.43357ms	70.386788ms	70.042513ms
132.215978ms	70.01131ms	71.19429ms
69.239273ms	70.032237ms	69.479335ms
68.669903ms	70.192272ms	70.858937ms
70.458108ms	69.395154ms	71.161921ms
68.488057ms	69.252731ms	71.374758ms
68.816031ms	69.628744ms	70.141381ms
69.170105ms	68.935813ms	70.685521ms
68.831608ms	69.728349ms	69.45605ms

Blog: 2017-08-17 Community Meeting Update

Thu, 17 Aug 2017 00:00:00 +0000

Next Community Meeting: Thursday, August 31, 2017 11am Pacific Time (US and Canada)

General Announcements

Call for Papers: CloudNativeCon

CloudNativeCon gathers all CNCF (Cloud Native Computing Foundation) projects under a single roof. Presenters will be talking about their experiences with Kubernetes, Prometheus, OpenTracing, Fluentd, Linkerd, gRPC, CoreDNS, containerd, rkt and CNI. The call for papers period will be ending on Monday, August 21, 2017. The conference will be taking place on the 6th and 7th of December, 2017. If you submit a talk, please add an entry to the spreadsheet linked in the gRPC Community Meeting Working Doc.

Release Updates

1.6 will be available soon. The required changes have already been merged and published in the protocol buffer libraries. Keep your eyes peeled for the upcoming release.

Platform Updates

No platform updates.

Language Updates

No language specific updates.

Blog: Announcing out-of-the-box support for gRPC in the Flatbuffers serialization library

Thu, 17 Aug 2017 00:00:00 +0000

The recent release of Flatbuffers version 1.7 introduced truly zero-copy support for gRPC out of the box.

Flatbuffers is a serialization library that allows you to access serialized data without first unpacking it or allocating any additional data structures. It was originally designed for games and other resource constrained applications, but is now finding more general use, both by teams within Google and in other companies such as Netflix and Facebook.

Flatbuffers enables maximum throughput by directly using gRPC’s slice buffers with zero-copy for common use cases. An incoming rpc can be processed directly from gRPCs internal buffers, and constructing a new message will write directly to these buffers without intermediate steps.

This is currently, fully supported in the C++ implementation of FlatBuffers, with more languages to come. There is also an implementation in Go, which is not entirely zero copy, but still very low on allocation cost (see below).

Example Usage

Let’s look at an example of how this works.

Use Flatbuffers as an IDL

Start with an .fbs schema (similar to .proto, if you are familiar with protocol buffers) that declares an RPC service:

table HelloReply {
 message:string;
}

table HelloRequest {
 name:string;
}

table ManyHellosRequest {
 name:string;
 num_greetings:int;
}

rpc_service Greeter {
 SayHello(HelloRequest):HelloReply;
 SayManyHellos(ManyHellosRequest):HelloReply (streaming: "server");
}

To generate C++ code from this, run: flatc --cpp --grpc example.fbs, much like in protocol buffers.

Generated Server Implementation

The server implementation is very similar to protocol buffers, except now the request and response messages are of type flatbuffers::grpc::Message<HelloRequest> *. Unlike protocol buffers, where these types represent a tree of C++ objects, here they are merely handles to a flat object in the underlying gRPC slice. You can access the data directly:

auto request = request_msg->GetRoot();
auto name = request->name()->str();

Building a response is equally simple

auto msg_offset = mb_.CreateString("Hello, " + name);
auto hello_offset = CreateHelloReply(mb_, msg_offset);
mb_.Finish(hello_offset);
*response_msg = mb_.ReleaseMessage<HelloReply>();

The client code is the same as that generated by protocol buffers, except for the FlatBuffer access and construction code.

See the full example here. To compile it, you need gRPC. The same repo has a similar example for Go.

Read more about using and building FlatBuffers for your platform on the flatbuffers site.

Blog: gRPC Load Balancing

Thu, 15 Jun 2017 00:00:00 +0000

This post describes various load balancing scenarios seen when deploying gRPC. If you use gRPC with multiple backends, this document is for you.

A large scale gRPC deployment typically has a number of identical back-end instances, and a number of clients. Each server has a certain capacity. Load balancing is used for distributing the load from clients optimally across available servers.

Why gRPC?

gRPC is a modern RPC protocol implemented on top of HTTP/2. HTTP/2 is a Layer 7 (Application layer) protocol, that runs on top of a TCP (Layer 4 - Transport layer) protocol, which runs on top of IP (Layer 3 - Network layer) protocol. gRPC has many advantages over traditional HTTP/REST/JSON mechanism such as

Binary protocol (HTTP/2)
Multiplexing many requests on one connection (HTTP/2)
Header compression (HTTP/2)
Strongly typed service and message definition (Protobuf)
Idiomatic client/server library implementations in many languages

In addition, gRPC integrates seamlessly with ecosystem components like service discovery, name resolver, load balancer, tracing and monitoring, among others.

Load balancing options

Proxy or Client side?

Note: Proxy load balancing is also known as server-side load balancing in some literature.

Deciding between proxy versus client-side load balancing is a primary architectural choice. In Proxy load balancing, the client issues RPCs to the a Load Balancer (LB) proxy. The LB distributes the RPC call to one of the available backend servers that implement the actual logic for serving the call. The LB keeps track of load on each backend and implements algorithms for distributing load fairly. The clients themselves do not know about the backend servers. Clients can be untrusted. This architecture is typically used for user facing services where clients from open internet can connect to servers in a data center, as shown in the picture below. In this scenario, clients make requests to LB (#1). The LB passes on the request to one of the backends (#2), and the backends report load to LB (#3).

In Client side load balancing, the client is aware of multiple backend servers and chooses one to use for each RPC. The client gets load reports from backend servers and the client implements the load balancing algorithms. In simpler configurations server load is not considered and client can just round-robin between available servers. This is shown in the picture below. As you can see, the client makes request to a specific backend (#1). The backends respond with load information (#2), typically on the same connection on which client RPC is executed. The client then updates its internal state.

The following table outlines the pros and cons of each model.

	Proxy	Client Side
Pros	Simple client No client-side awareness of backend Works with untrusted clients	High performance because elimination of extra hop
Cons	LB is in the data path Higher latency LB throughput may limit scalability	Complex client Client keeps track of server load and health Client implements load balancing algorithm Per-language implementation and maintenance burden Client needs to be trusted, or the trust boundary needs to be handled by a lookaside LB.

Proxy Load Balancer options

Proxy load balancing can be L3/L4 (transport level) or L7 (application level). In transport level load balancing, the server terminates the TCP connection and opens another connection to the backend of choice. The application data (HTTP/2 and gRPC frames) are simply copied between the client connection to the backend connection. L3/L4 LB by design does very little processing, adds less latency compared with L7 LB, and is cheaper because it consumes fewer resources.

In L7 (application level) load balancing, the LB terminates and parses the HTTP/2 protocol. The LB can inspect each request and assign a backend based on the request contents. For example, a session cookie sent as part of HTTP header can be used to associate with a specific backend, so all requests for that session are served by the same backend. Once the LB has chosen an appropriate backend, it creates a new HTTP/2 connection to that backend. It then forwards the HTTP/2 streams received from the client to the backend(s) of choice. With HTTP/2, LB can distribute the streams from one client among multiple backends.

L3/L4 (Transport) vs L7 (Application)

Use case	Recommendation
RPC load varies a lot among connections	Use Application level LB
Storage or compute affinity is important	Use Application level LB and use cookies or similar for routing requests to correct backend
Minimizing resource utilization in proxy is more important than features	Use L3/L4 LB
Latency is paramount	Use L3/L4 LB

Client side LB options

Thick client

A thick client approach means the load balancing smarts are implemented in the client. The client is responsible for keeping track of available servers, their workload, and the algorithms used for choosing servers. The client typically integrates libraries that communicate with other infrastructures such as service discovery, name resolution, quota management, etc.

Lookaside Load Balancing

Note: A lookaside load balancer is also known as an external load balancer or one-arm load balancer

With lookaside load balancing, the load balancing smarts are implemented in a special LB server. Clients query the lookaside LB and the LB responds with best server(s) to use. The heavy lifting of keeping server state and implementation of LB algorithm is consolidated in the lookaside LB. Note that client might choose to implement simple algorithms on top of the sophisticated ones implemented in the LB. gRPC defines a protocol for communication between client and LB using this model. See Load Balancing in gRPC doc for details.

The picture below illustrates this approach. The client gets at least one address from lookaside LB (#1). Then the client uses this address to make a RPC (#2), and server sends load report to the LB (#3). The lookaside LB communicates with other infrastructure such as name resolution, service discovery, and so on (#4).

Recommendations and best practices

Depending upon the particular deployment and constraints, we suggest the following.

Setup	Recommendation
Very high traffic between clients and servers Clients can be trusted	Thick client-side load balancing Client side LB with ZooKeeper/Etcd/Consul/Eureka. ZooKeeper example.
Traditional setup - Many clients connecting to services behind a proxy Need trust boundary between servers and clients	Proxy Load Balancing L3/L4 LB with GCLB (if using GCP) L3/L4 LB with haproxy - config file Nginx coming soon If need session stickiness - L7 LB with Envoy as proxy
Microservices - N clients, M servers in the data center Very high performance requirements (low latency, high traffic) Client can be untrusted	Look-aside Load Balancing Client-side LB using gRPC-LB protocol. Roll your own implementation (Q2’17), hosted gRPC-LB in the works.
Existing Service-mesh like setup using Linkerd or Istio	Service Mesh Use built-in LB with Istio, or Envoy.

Blog: gRPC in Helm

Mon, 15 May 2017 00:00:00 +0000

Helm is the package manager for Kubernetes. Helm provides its users with a customizable mechanism for managing distributed applications and controlling their deployment.

I have the good fortune to be a member of the phenomenal open-source Kubernetes Helm community serving as a core contributor. My first day working with the Helm team was spent prototyping the architecture for the next generation of Helm. By the end of that day, we had procured the preliminary RPC protocol data model used to enable communication between Helm and its in-cluster server component, Tiller.

We chose to use protocol buffers - the default framework gRPC uses for serialization and over-the-air transmission - as our data definition language. By the end of that first day hacking with the Helm team, gRPC and protocol buffers proved to be a powerful combination. We had successfully had achieved communication between the Helm client and Tiller server using code generated from the protobuf and gRPC service definitions. As a personal preference, we found that the protobuf files and resulting generated gRPC code provided an aesthetic, nearly self-documenting developer experience compared to something like Swagger.

Within a few days, the Helm team was scoping and implementing features for our users. By choosing gRPC/Proto we had reduced the typical time spent bikeshedding that, in general, inevitably evolves from API modeling and churning out boilerplate server code. If we had not reaped the benefits of gRPC/protobuf from day 1, we would have spent significantly more time pivoting up and down the stack, as opposed to honing our focus on what matters: the users and the features they requested.

In addition to serving as the Helm/Tiller communication protocol, one of our more interesting applications of protocol buffers is that we use it to model what’s referred to in Kubernetes parlance as a “Chart”. Charts are an encapsulation of Kubernetes manifests that enable you to define, install, and upgrade Kubernetes applications. For more complex Kubernetes applications, the set of manifests may be large. By virtue of its inherent compression capabilities, protocol buffers and gRPC allowed us to mitigate the nuisance of transmitting bulky and sprawling Kubernetes manifests.

For a deeper dive into:

The Helm proto, see: https://github.com/kubernetes/helm/tree/master/_proto/hapi
Its generated counterpart, see: https://github.com/kubernetes/helm/tree/master/pkg/proto/hapi
The interface to our Helm client, see: https://github.com/kubernetes/helm/tree/master/pkg/helm

In summary, protobuf and gRPC provided Helm with:

Clearly defined message and protocol semantics for client and server communications.
Increased feature development via a reduction in time spent on boilerplate server code / API modeling.
High performance transmission of data through generated code and compression.
Minimized cognitive cycles spent going from 0 to client/server communications.

Blog: Migration to Google Cloud Platform — gRPC & grpc-gateway

Wed, 12 Apr 2017 00:00:00 +0000

In our previous blog post we gave an overview of our migration to Google Cloud Platform from Amazon Web Services. In this post we will drill down into the role that gRPC and grpc-gateway played in that migration and share some lessons which we picked up along the way.

Most people have REST APIs, don’t you? What’s the problem?

Yes, we actually still have REST APIs that clients use because migrating the client APIs was out of scope. To be fair, you can make REST APIs work and there are a lot of useful REST APIs out there. Having said that, the issues that we had with REST lie in the details.

No Canonical REST Specification

There is no single REST specification that is canonical. There are best practices, but no true canon. For that reason, there isn’t unanimous agreement on when to use specific HTTP methods and response codes. Beyond that, not all of the possible HTTP methods and response codes are supported across all platforms… This forces REST API implementers to compensate for these deficiencies using techniques that work for them but create more variance in REST APIs across the board. At best, REST APIs are really REST-ish dialects.

Harder on Developers

REST APIs aren’t exactly great from a developer’s standpoint either. First, because REST is tied to HTTP there is no simple mapping to an API in my language of choice. If I’m using Go or Java there is no “interface” that I can use in my code to stub it out. I can create one, but it is extra-linguistic to the REST API definition.

Second, REST APIs spread information that is necessary to interpreting the intent of the request across various components of the request. You have the HTTP method, the request URI, the request payload, and it can get even more complicated if request headers are involved in the semantics.

Third, it is great that I can use curl from the command line to hit an API, but it comes at the cost of having to shoehorn the API into that ecosystem. Normally that use case only matters for letting people quickly try out an API — and if that is high on your list of requirements then by all means feel free to use REST… Just keep it simple.

No Declarative REST API Description

The fourth problem with a REST APIs is that, at least until Swagger arrived on the scene, there was no declarative way to define a REST API and include type information. It may sound pedantic, but there are legitimate reasons to want a proper definition that includes type information in general. To reinforce the point, look at the lines of PHP server code below, which were extracted from various files, that set the “hidePin” field on “yak” which was then returned to the client. The actual line of code that executed on the server was a function of multiple parameters, so imagine that the one which was run was basically chosen at random:

// Code omitted…
$yak->hidePin=false;

// Code omitted…
$yak->hidePin=true;

// Code omitted…
$yak->hidePin=0;

// Code omitted…
$yak->hidePin=1;

What is the type of the field hidePin? You cannot say for certain. It could be a boolean or an integer or whatever happens to have been written there by the server, but in any case now your clients have to be able to deal with these possibilities which makes them more complicated.

Problems can also arise when the client’s definition of a type varies from that which the server expects. Have a look at the server code below which processed a JSON payload sent up by a client:

// Code omitted...
switch ($fieldName) {
 // Code omitted...
 case “recipientID”:
 // This is being added because iOS is passing the recipientID
 // incorrectly and we still want to capture these events
 // … expected fall through …

 case “Recipientid”:
 $this->yakkerEvent->recipientID = $value;
 break;
 // Code omitted...
}
// Code omitted...

In this case, the server had to deal with an iOS client that sent a JSON object whose field name used unexpected casing. Again, not insurmountable but all of these little disconnects compound and work together to steal time away from the problems that really move the ball down the field.

gRPC can address the issues with REST

If you’re not familiar with gRPC, it’s a “high performance, open-source universal remote procedure call (RPC) framework” that uses Google Protocol Buffers as the Interface Description Language (IDL) for describing a service interface as well as the structure of the messages exchanged. This IDL can then be compiled to produce language-specific client and server stubs. In case that seemed a little obtuse, I’ll zoom into the aspects that are important.

gRPC is Declarative, Strongly-Typed, and Language Independent

gRPC descriptions are written using an Interface Description Language that is independent of any specific programming language, yet its concepts map onto the supported languages. This means that you can describe your ideal service API, the messages that it supports, and then use “protoc”, the protocol compiler, to generate client and server stubs for your API. Out of the box, you can produce client and server stubs in C/C++, C#, Node.js, PHP, Ruby, Python, Go and Java. You can also get additional protoc plugins which can create stubs for Objective-C and Swift.

Those issues that we had with “hidePin” and “recipientID” vs.”Recipientid” fields above go away because we have a single, canonical declaration that establishes the types used, and the language-specific code generation ensures that we don’t have typos in the client or server code regardless of their implementation language.

gRPC Means No hand-rolling of RPC Code is Required

This is a very powerful aspect of the gRPC ecosystem. Often times developers will hand roll their RPC code because it just seems more straightforward. However, as the number of types of clients that you need to support increases, the carrying costs of this approach also increase non-linearly. Imagine that you start off with a service that is called from a web browser. At some point down the road, the requirements are updated and now you have to support Android and iOS clients. Your server is likely fine, but the clients now need to be able to speak the same RPC dialect and often times there are differences that creep in. Things can get even worse if the server has to compensate for the differences amongst the clients. On the other hand, using gRPC you just add the protocol compiler plugins and they generate the Android and iOS client stubs. This cuts out a whole class of problems. As a bonus, if you don’t modify the generated code — and you should not have to — then any performance improvements in the generated code will be picked up.

gRPC has Compact Serialization

gRPC uses Google protocol buffers to serialize messages. This serialization format is very compact because, among other things, field names are not included in the serialized form. Compare this to a JSON object where each instance of an object carries a full copy of its field names, includes extra curly braces, etc. For a low-volume application this may not be an issue, but it can add up quickly.

gRPC Tooling is Extensible

Another very useful feature of the gRPC framework is that it is extensible. If you need support for a language that is not currently supported, there is a way to create plugins for the protocol compiler that allows you to add what you need.

gRPC Supports Contract Updates

An often overlooked aspect of service APIs is how they may evolve over time. At best, this is often a secondary consideration. If you are using gRPC, and you adhered to a few basic rules, your messages can be forward and backward compatible.

Grpc-gateway — because REST will be with us for a while…

You’re probably thinking: gRPC is great but I have a ton of REST clients to deal with. Well, there is another tool in this ecosystem and it is called grpc-gateway. Grpc-gateway “generates a reverse-proxy server which translates a RESTful JSON API into gRPC”. So if you want to support REST clients you can, and it doesn’t cost you any real extra effort. If your existing REST clients are pretty far from the normal REST APIs, you can use custom marshallers with grpc-gateway to compensate.

Migration and gRPC + grpc-gateway

As mentioned previously, we had a lot of PHP code and REST endpoints which we wanted to rework as part of the migration. By using the combination of gRPC and grpc-gateway, we were able to define gRPC versions of the legacy REST APIs and then use grpc-gateway to expose the exact REST endpoints that clients were used to. With these alternative implementations in place we were able to move traffic between the old and new systems using combinations of DNS updates as well as our Experimentation and Configuration System without causing any disruption to the existing clients. We were even able to leverage the existing test suites to verify functionality and establish parity between the old and new systems. Lets walk through the pieces and how they fit together.

gRPC IDL for “/api/getMessages”

Below is the gRPC IDL that we defined to mimic the legacy Yik Yak API in GCP. We’ve simplified the example to only contain the “/api/getMessages” endpoint which clients use to get the set of messages centered around their current location.

// APIRequest Message — sent by clients
message APIRequest {
 // userID is the ID of the user making the request
 string userID = 1;
 // Other fields omitted for clarity…
}

// APIFeedResponse contains the set of messages that clients should
// display.
message APIFeedResponse {
 repeated APIPost messages = 1;
 // Other fields omitted for clarity…
}

// APIPost defines the set of post fields returned to the clients.
message APIPost {
 string messageID = 1;
 string message = 2;
 // Other fields omitted for clarity…
}

// YYAPI service accessed by Android, iOS and Web clients.
service YYAPI {
 // Other endpoints omitted…

 // APIGetMessages returns the list of messages within a radius of
 // the user’s current location.
 rpc APIGetMessages (APIRequest) returns (APIFeedResponse) {
 option (google.api.http) = {
 get: “/api/getMessages” // Option tells grpc-gateway that an HTTP
 // GET to /api/getMessages should be
 // routed to the APIGetMessages gRPC
 // endpoint.
 };
 }

 // Other endpoints omitted…
}

Protoc Generated Go Interfaces for YYAPI Service

The IDL above is then compiled to Go files by the protoc compiler to produce client proxies and server stubs as below.

// Client API for YYAPI service
type YYAPIClient interface {
 APIGetMessages(ctx context.Context, in *APIRequest, opts ...grpc.CallOption) (*APIFeedResponse, error)
}

// NewYYAPIClient returns an implementation of the YYAPIClient interface which
// clients can use to call the gRPC service.
func NewYYAPIClient(cc *grpc.ClientConn) YYAPIClient {
 // Code omitted for clarity..
}

// Server API for YYAPI service
type YYAPIServer interface {
 APIGetMessages(context.Context, *APIRequest) (*APIFeedResponse, error)
}

// RegisterYYAPIServer registers an implementation of the YYAPIServer with an
// existing gRPC server instance.
func RegisterYYAPIServer(s *grpc.Server, srv YYAPIServer) {
 // Code omitted for clarity..
}

Grpc-gateway Generated Go-code for REST Reverse Proxy of YYAPI Service

By using the google.api.http option in our IDL above, we tell the grpc-gateway system that it should route HTTP GETs for “/api/getMessages” to the APIGetMessages gRPC endpoint. In turn, it creates the HTTP to gRPC reverse proxy and allows you to set it up by calling the generated function below.

// RegisterYYAPIHandler registers the http handlers for service YYAPI to “mux”.
// The handlers forward requests to the grpc endpoint over “conn”.
func RegisterYYAPIHandler(ctx context.Context, mux *runtime.ServeMux, conn *grpc.ClientConn) error {
 // Code omitted for clarity
}

So again, from a single gRPC IDL description you can obtain client and server interfaces and implementation stubs in your language of choice as well as REST reverse proxies for free.

gRPC — I heard there were some rough edges?

We started working with gRPC for Go late in Q1 of 2016 and there were definitely some rough edges at the time.

Early Adopter Issues

We ran into Issue 674, a resource leak inside of the Go gRPC client code which could cause gRPC transports to hang when under heavy load. The gRPC team was very responsive and the fix was merged into the master branch within days.

We ran into a resource leak in the generated code for grpc-gateway. However, by the time we found that issue, it had already been fixed by that team and merged into master.

The last early-adopter type issue that we ran into was around the Go’s gRPC client not supporting the GOAWAY packet that was part of the gRPC protocol spec. Fortunately, this one did not impact us in production. It only manifested during the repo case we had put together for Issue 674.

All in all this was fairly reasonable given how early we were.

Load Balancing

Now, if you are going to use gRPC this is definitely one area that you need to think through carefully. By default, gRPC uses HTTP2 instead of HTTP1. HTTP2 is able to open a connection to a server and reuse it for multiple requests among other things. If you use it in that mode, you won’t distribute requests amongst all of the servers in your load balancing pool. At the time that we executing the migration, existing load balancers didn’t handle HTTP2 traffic very well if at all.

At the time the gRPC team didn’t have a Load Balancing Proposal, so we burned a lot of cycles trying to force our system to do some type of client-side load balancing. In the end, since most of our raw gRPC communications took place within the data center, and everything was deployed using Kubernetes, it was simpler to dial the remote server every time thereby forcing the system to spread the load out amongst the servers in the Kubernetes Service. Given our setup it only added about 1 ms to the overall response time, so it was a simple work around.

So was that the end of the load balancing issues? Not exactly. Once we had our basic gRPC-based system up and running we started running load tests against it, and noticed some interesting behaviors. Below is the per gRPC server CPU load graph over time, do you notice anything curious about it?

The server with the heaviest load was running at around 50% CPU, while the most lightly loaded server was running at around 20% CPU even after several minutes of warmup. It turned out that even though we were dialing every time, we had an nghttp2 ingress as part of our network topology which would tend to send inbound requests to servers to whom it had already connected and thereby causing uneven distribution. After removing the nghttp2 ingress, our CPU graphs showed much less variance in the load distribution.

Conclusion

REST APIs have their issues, but they are not going away anytime soon. If you are up for trying something a little cleaner, then definitely consider using gRPC (along with grpc-gateway if you still need to expose a REST API). Even though we hit some issues early on, gRPC was a net gain for us. It gave us a path forward to more tightly defined APIs. It also allowed us to stand up new implementations of the legacy REST APIs in GCP which teed us up to seamlessly migrate traffic from the AWS implementations to the new GCP ones in a controlled manner.

Having discussed our use of Go, gRPC and Google Cloud Platform, we are ready to discuss how we built a new geo store on top of Google Bigtable and the Google S2 Library — the subject of our next post.

Blog: Building gRPC services with bazel and rules_protobuf

Thu, 13 Oct 2016 00:00:00 +0000

gRPC makes it easier to build high-performance microservices by providing generated service entrypoints in a variety of different languages. Bazel complements these efforts with a capable and fast polyglot build environment.

rules_protobuf extends bazel and makes it easier develop gRPC services.

It does this by:

Building protoc (the protocol buffer compiler) and all the necessary protoc-gen-* plugins.
Building the protobuf and gRPC libraries required for gRPC-related code to compile.
Abstracting away protoc plugin invocation (you don’t have to necessarily learn or remember how to call protoc).
Regenerating and recompiling outputs when protobuf source files change.

In this post I’ll provide background about how bazel works (Part 1) and how to get started building gRPC services with rules_protobuf (Part 2). If you’re already a bazel aficionado, you can skip directly to Part 2.

To best follow along, install bazel and clone the rules_protobuf repository:

git clone https://github.com/pubref/rules_protobuf
cd rules_protobuf
~/rules_protobuf$

Great. Let’s get started!

1: About Bazel

Bazel is Google’s open-source version of their internal build tool called “Blaze”. Blaze originated from the challenges of managing a large monorepo with code written in a variety of languages. Blaze was the inspiration for other capable and fast build tools including Pants and Buck. Bazel is conceptually simple but there are some core concepts & terminology to understand:

Bazel command: a function that does some type of work when called from the command line. Common ones include bazel build (compile a library), bazel run (run a binary executable), bazel test (execute tests), and bazel query (tell me something about the build dependency graph). See all with bazel help.
Build phases: the three stages (loading, analysis, and execution) that bazel goes through when calling a bazel command.
The WORKSPACE file: a required file that defines the project root. It is primarily used to declare external dependencies (external workspaces).
BUILD files: the presence of a BUILD file in a directory defines it as a package. BUILD files contain rules that define targets which can be selected using the target pattern syntax. Rules are written in a python-like language called skylark. Syklark has stronger deterministic guarantees than python but is intentionally minimal, excluding language features such as recursion, classes, and lambdas.

1.1: Package Structure

To illustrate these concepts, let’s look at the package structure of the rules_protobuf examples subdirectory. Let’s look at the file tree, showing only those folder having a BUILD file:

tree -P 'BUILD|WORKSPACE' -I 'third_party|bzl' examples/
.
├── BUILD
├── WORKSPACE
└── examples
    ├── helloworld
    │   ├── cpp
    │   │   └── BUILD
    │   ├── go
    │   │   ├── client
    │   │   │   └── BUILD
    │   │   ├── greeter_test
    │   │   │   └── BUILD
    │   │   └── server
    │   │   └── BUILD
    │   ├── grpc_gateway
    │   │   └── BUILD
    │   ├── java
    │   │   └── org
    │   │   └── pubref
    │   │   └── rules_protobuf
    │   │   └── examples
    │   │   └── helloworld
    │   │   ├── client
    │   │   │   └── BUILD
    │   │   └── server
    │   │   └── BUILD
    │   └── proto
    │      └── BUILD
    └── proto
    └── BUILD

1.2: Targets

To get a list of targets within the examples/ folder, use a query. This says “Ok bazel, show me all the callable targets in all packages within the examples folder, and say what kind of thing it is in addition to its path label”:

~/rules_protobuf$ bazel query //examples/... --output label_kind | sort | column -t

cc_binary rule //examples/helloworld/cpp:client
cc_binary rule //examples/helloworld/cpp:server
cc_library rule //examples/helloworld/cpp:clientlib
cc_library rule //examples/helloworld/proto:cpp
cc_library rule //examples/proto:cpp
cc_proto_compile rule //examples/helloworld/proto:cpp.pb
cc_proto_compile rule //examples/proto:cpp.pb
cc_test rule //examples/helloworld/cpp:test
filegroup rule //examples/helloworld/proto:protos
filegroup rule //examples/proto:protos
go_binary rule //examples/helloworld/go/client:client
go_binary rule //examples/helloworld/go/server:server
go_library rule //examples/helloworld/go/server:greeter
go_library rule //examples/helloworld/grpc_gateway:gateway
go_library rule //examples/helloworld/proto:go
go_library rule //examples/proto:go_default_library
go_proto_compile rule //examples/helloworld/proto:go.pb
go_proto_compile rule //examples/proto:go_default_library.pb
go_test rule //examples/helloworld/go/greeter_test:greeter_test
go_test rule //examples/helloworld/grpc_gateway:greeter_test
grpc_gateway_proto_compile rule //examples/helloworld/grpc_gateway:gateway.pb
java_binary rule //examples/helloworld/java/org/pubref/rules_protobuf/examples/helloworld/client:netty
java_binary rule //examples/helloworld/java/org/pubref/rules_protobuf/examples/helloworld/server:netty
java_library rule //examples/helloworld/java/org/pubref/rules_protobuf/examples/helloworld/client:client
java_library rule //examples/helloworld/java/org/pubref/rules_protobuf/examples/helloworld/server:server
java_library rule //examples/helloworld/proto:java
java_library rule //examples/proto:java
java_proto_compile rule //examples/helloworld/proto:java.pb
java_proto_compile rule //examples/proto:java.pb
js_proto_compile rule //examples/helloworld/proto:js
js_proto_compile rule //examples/proto:js
py_proto_compile rule //examples/helloworld/proto:py.pb
ruby_proto_compile rule //examples/proto:rb.pb

We’re not limited to targets in our own workspace. As it turns out, the Google Protobuf repo is named as an external repository (more on this later) and we can also address targets in that workspace in the same way. Here’s a partial list:

~/rules_protobuf$ bazel query @com_github_google_protobuf//... --output label_kind | sort | column -t

cc_binary rule @com_github_google_protobuf//:protoc
cc_library rule @com_github_google_protobuf//:protobuf
cc_library rule @com_github_google_protobuf//:protobuf_lite
cc_library rule @com_github_google_protobuf//:protoc_lib
cc_library rule @com_github_google_protobuf//util/python:python_headers
filegroup rule @com_github_google_protobuf//:well_known_protos
java_library rule @com_github_google_protobuf//:protobuf_java
objc_library rule @com_github_google_protobuf//:protobuf_objc
py_library rule @com_github_google_protobuf//:protobuf_python
...

This is possible because the protobuf team provides a BUILD file at the root of their repository. Thanks Protobuf team! Later we’ll learn how to “inject” our own BUILD files into repositories that don’t already have one.

Inspecting the list above, we see a cc_binary rule named protoc. If we bazel run that target, bazel will clone the protobuf repo, build all the dependent libraries, build a pristine executable binary from source, and call it (pass command line arguments to binary rules after the double-dash):

~/rules_protobuf$ bazel run @com_github_google_protobuf//:protoc -- --help
Usage: /private/var/tmp/_bazel_pcj/63330772b4917b139280caef8bb81867/execroot/rules_protobuf/bazel-out/local-fastbuild/bin/external/com_github_google_protobuf/protoc [OPTION] PROTO_FILES
Parse PROTO_FILES and generate output based on the options given:
 -IPATH, --proto_path=PATH Specify the directory in which to search for
 imports. May be specified multiple times;
 directories will be searched in order. If not
 given, the current working directory is used.
 --version Show version info and exit.
 -h, --help Show this text and exit.
...

As we’ll see in a moment, we name the protobuf external dependency with a specific commit ID so there’s no ambiguity about which protoc version we’re using. In this way you can vendor in tools with your project with reliable, repeable, secure precision without bloating your repository by checking in binaries, resorting to git submodules, or similar hacks. Very clean!

Note: the gRPC repository also has a BUILD file: $ bazel query @com_github_grpc_grpc//... --output label_kind

1.3: Target Pattern Syntax

With those examples under our belt, let’s examine the target syntax a bit more. When I first started working with bazel I found the target-pattern syntax somewhat intimidating. It’s actually not too bad. Here’s a closer look:

The @ (at-sign) selects an external workspace. These are established by workspace rules that bind a name to something fetched over the network (or your filesystem).
The // (double-slash) selects the workspace root.
The : (colon) selects a target (rule or file) within a package. Recall that a package is established by the presence of a BUILD file in a subfolder of the workspace.
The / (single-slash) selects a folder within a workspace or package.

A common source of confusion is that the mere presence of a BUILD file defines that filesystem subtree as a package and therefore one must always account for that. For example, if there exists a file qux.js in foo/bar/baz/ and there exists a BUILD file in baz/ also, the file is selected with foo/bar/baz:qux.js and not foo/bar/baz/quz.js

Common shortcut: if there exists a rule having the same name as the package, this is the implied target and can be omitted. For example, there is a :jar target in the //jar package in the external workspace com_google_guava_guava, so the following are equivalent:

deps = ["@com_google_guava_guava//jar:jar"]
deps = ["@com_google_guava_guava//jar"]

1.4: External Dependencies: Workspace Rules

Many large organizations check-in in all the required tools, compilers, linkers, etc to guarantee correct, repeatable builds. With external workspaces, one can effectively accomplish the same thing without bloating your repository.

Note: the bazel convention is to use a fully-namespaced identifier for external dependency names (replacing special chars with underscore). For example, the remote repository URL is https://github.com/google/protobuf.git. This is simplified to the workspace identifier com_github_google_protobuf. Similarly, by convention the jar artifact io.grpc:grpc-netty:jar:1.0.0-pre1 becomes io_grpc_grpc_netty.

1.4.1: Workspace Rules that require a pre-existing WORKSPACE

These rules assume that the remote resource or URL contains a WORKSPACE file at the top of the file tree and BUILD files that define rule targets. These are referred to as bazel repositories.

git_repository: external bazel dependency from a git repository. The rule requires commit (or tag).
http_archive: an external zip or tar.gz dependency from a URL. It is highly recommended to name a sha265 for security.

Note: although you don’t interact directly with the bazel execution_root, you can peek at what these external dependencies look like when unpacked at $(bazel info execution_root)/external/WORKSPACE_NAME.

1.4.2: Workspace Rules that autogenerate a WORKSPACE file for you

The implementation of these repository rules contain logic to autogenerate a WORKSPACE file and BUILD file(s) to make resources available. As always, it is recommended to provide a known sha265 for security to prevent a malicious agent from slipping in tainted code via a compromised network.

http_jar: external jar from a URL. The jar file is available as a java_library dependency as @WORKSPACE_NAME//jar.
maven_jar: external jar from a URL. The jar file is available as a java_library dependency as @WORKSPACE_NAME//jar.
http_file: external file from a URL. The resource is available as a filegroup via @WORKSPACE_NAME//file.

For example, we can peek at the generated BUILD file for the maven_jar guava dependency via:

~/rules_protobuf$ cat $(bazel info execution_root)/external/com_google_guava_guava/jar/BUILD

# DO NOT EDIT: automatically generated BUILD file for maven_jar rule com_google_guava_guava
java_import(
 name = 'jar',
 jars = ['guava-19.0.jar'],
 visibility = ['//visibility:public']
)

filegroup(
 name = 'file',
 srcs = ['guava-19.0.jar'],
 visibility = ['//visibility:public']
)

Note: the external workspace directory won’t exist until you actually need it, so you’ll have to have built a target that requires it, such as bazel build examples/helloworld/java/org/pubref/rules_protobuf/examples/helloworld/client

1.4.3: Workspace Rules that accept a BUILD file as an argument

If a repository has no BUILD file(s), you can put one into its filesystem root to adapt the external resource into bazel’s worldview and make those resources available to your project.

For example, consider Mark Adler’s zlib library. To start, let’s learn what depends on this code. This query says “Ok bazel, for all targets in examples, find all dependencies (a transitive closure set), then tell me which ones depend on the zlib target in the root package of the external workspace com_github_madler_zlib.” Bazel reports this reverse dependency set. We request the output in graphviz format and pipe this to dot to generate the figure:

~/rules_protobuf$ bazel query "rdeps(deps(//examples/...), @com_github_madler_zlib//:zlib)" \
 --output graph | dot -Tpng -O

So we can see that all grpc-related C code ultimately depends on this library. But, there is no BUILD file in Mark’s repo… where did it come from?

By using the variant workspace rule new_git_repository, we can provide our own BUILD file (which defines the cc_library target) as follows:

new_git_repository(
 name = "com_github_madler_zlib",
 remote = "https://github.com/madler/zlib",
 tag: "v1.2.8",
 build_file: "//bzl:build_file/com_github_madler_zlib.BUILD",
)

This new_* family of workspace rules keeps your repository lean and allows you to vendor in pretty much any type of network-available resource. Awesome!

You can also write your own repository rules that have custom logic to pull resources from the net and bind it into bazel’s view of the universe.

1.5: Bazel Summary

When presented with a command and a target-pattern, bazel goes through the following three phases:

Loading: Read the WORKSPACE and required BUILD files. Generate a dependency graph.
Analysis: for all nodes in the graph, which nodes are actually required for this build? Do we have all the necessary resources available?
Execution: execute each required node in the dependency graph and generate outputs.

Hopefully you now have enough conceptual knowledge of bazel to be productive.

1.6: rules_protobuf

rules_protobuf is an extension to bazel that takes care of:

Building the protocol buffer compiler protoc,
Downloading and/or building all the necessary protoc-gen plugins.
Downloading and/or building all the necessary gRPC-related support libraries.
Invoking protoc for you (on demand), smoothing out the idiosyncrasies of different protoc plugins.

It works by passing one or more proto_language specifications to the proto_compile rule. A proto_language rule contains the metadata about how to invoke the plugin and the predicted file outputs, while the proto_compile rule interprets a proto_language spec and builds the appropriate command-line arguments to protoc. For example, here’s how we can generate outputs for multiple languages simultaneously:

 proto_compile(
 name = "pluriproto",
 protos = [":protos"],
 langs = [
 "//cpp",
 "//csharp",
 "//closure",
 "//ruby",
 "//java",
 "//java:nano",
 "//python",
 "//objc",
 "//node",
 ],
 verbose = 1,
 with_grpc = True,
 )

bazel build :pluriproto
# ************************************************************
cd $(bazel info execution_root) && bazel-out/host/bin/external/com_github_google_protobuf/protoc \
--plugin=protoc-gen-grpc-java=bazel-out/host/genfiles/third_party/protoc_gen_grpc_java/protoc_gen_grpc_java \
--plugin=protoc-gen-grpc=bazel-out/host/bin/external/com_github_grpc_grpc/grpc_cpp_plugin \
--plugin=protoc-gen-grpc-nano=bazel-out/host/genfiles/third_party/protoc_gen_grpc_java/protoc_gen_grpc_java \
--plugin=protoc-gen-grpc-csharp=bazel-out/host/genfiles/external/nuget_grpc_tools/protoc-gen-grpc-csharp \
--plugin=protoc-gen-go=bazel-out/host/bin/external/com_github_golang_protobuf/protoc_gen_go \
--descriptor_set_out=bazel-genfiles/examples/proto/pluriproto.descriptor_set \
--ruby_out=bazel-genfiles \
--python_out=bazel-genfiles \
--cpp_out=bazel-genfiles \
--grpc_out=bazel-genfiles \
--objc_out=bazel-genfiles \
--csharp_out=bazel-genfiles/examples/proto \
--java_out=bazel-genfiles/examples/proto/pluriproto_java.jar \
--javanano_out=ignore_services=true:bazel-genfiles/examples/proto/pluriproto_nano.jar \
--js_out=import_style=closure,error_on_name_conflict,binary,library=examples/proto/pluriproto:bazel-genfiles \
--js_out=import_style=commonjs,error_on_name_conflict,binary:bazel-genfiles \
--go_out=plugins=grpc,Mexamples/proto/common.proto=github.com/pubref/rules_protobuf/examples/proto/pluriproto:bazel-genfiles \
--grpc-java_out=bazel-genfiles/examples/proto/pluriproto_java.jar \
--grpc-nano_out=ignore_services=true:bazel-genfiles/examples/proto/pluriproto_nano.jar \
--grpc-csharp_out=bazel-genfiles/examples/proto \
--proto_path=. \
examples/proto/common.proto
# ************************************************************
examples/proto/common_pb.rb
examples/proto/pluriproto_java.jar
examples/proto/pluriproto_nano.jar
examples/proto/common_pb2.py
examples/proto/common.pb.h
examples/proto/common.pb.cc
examples/proto/common.grpc.pb.h
examples/proto/common.grpc.pb.cc
examples/proto/Common.pbobjc.h
examples/proto/Common.pbobjc.m
examples/proto/pluriproto.js
examples/proto/Common.cs
examples/proto/CommonGrpc.cs
examples/proto/common.pb.go
examples/proto/common_pb.js
examples/proto/pluriproto.descriptor_set

The various *_proto_library rules (that we’ll be using below) internally invoke this proto_compile rule, then consume the generated outputs and compile them with the requisite libraries into .class, .so, .a (or whatever) objects.

So let’s make something already! We’ll use bazel and rules_protobuf to build a gRPC application.

2: Building a gRPC service with rules_protobuf

The application will involve communication between two different gRPC services:

2.1: Services

The Greeter service: This is the familiar “Hello World” starter example that accepts a request with a user argument and replies back with the string Hello {user}.
The GreeterTimer service: This gRPC service will repeatedly call a Greeter service in batches and report back aggregate batch times (in milliseconds). In this way we can compare some average rpc times for the different Greeter service implementations.

This is an informal benchmark intended only for demonstration of building gRPC applications. For more formal performance testing, consult the gRPC performance dashboard.

2.2: Compiled Programs

For the demo, we’ll use 6 different compiled programs written in 4 languages:

A GreeterTimer client (go). This command-line interface requires the greetertimer.proto service definition defined locally in the //proto:greetertimer.proto file.
A GreeterTimer server (java). This netty-based server requires both the //proto/greetertimer.proto file and the proto definition defined externally in @org_pubref_rules_protobuf//examples/helloworld/proto:helloworld.proto.
Four Greeter server implementations (C++, java, go, and C#). rules_protobuf already provides these example implementations, so we’ll just use them directly.

2.3: Protobuf Definitions

GreeterTimer accepts a unary TimerRequest and streams back a sequence of BatchResponse until all messages have been processed, at which point the remote procedure call is complete.

service GreeterTimer {
 // Unary request followed by multiple streamed responses.
 // Response granularity will be set by the request batch size.
 rpc timeHello(TimerRequest) returns (stream BatchResponse);
}

TimerRequest includes metadata about where to contact the Greeter service, how many total RPC calls to make, and how frequent to stream back a BatchResponse (configured via the batch size).

message TimerRequest {
 // the host where the grpc server is running
 string host = 1;
 // The port of the grpc server
 int32 port = 2;
 // The total number of hellos
 int32 total = 3;
 // The number of hellos before sending a BatchResponse.
 int32 batchSize = 4;
}

BatchResponse reports the number of calls made in the batch, how long the batch run took, and the number of remaining calls.

message BatchResponse {
 // The number of checks that are remaining, calculated relative to
 // totalChecks in the request.
 int32 remaining = 1;
 // The number of checks actually performed in this batch.
 int32 batchCount = 2;
 // The number of checks that failed.
 int32 errCount = 3;
 // The total time spent, expressed as a number of milliseconds per
 // request batch size (total time spent performing batchSize number
 // of health checks).
 int64 batchTimeMillis = 4;
}

The non-streaming Greeter service takes a unary HelloRequest and responds with a single HelloReply:

service Greeter {
 rpc SayHello (HelloRequest) returns (HelloReply) {}
}

message HelloRequest {
 string name = 1;
 common.Config config = 2;
}

message HelloReply {
 string message = 1;
}

The common.Config message type is not particularly functional here but serves to demonstrate the use of imports. rules_protobuf can help with more complex setups having multiple proto → proto dependencies.

2.4: Build the grpc_greetertimer example application.

This demo application can be cloned at https://github.com/pubref/grpc_greetertimer.

2.4.1: Create the Project Layout

Here’s the directory layout and relevant BUILD files we’ll be using:

mkdir grpc_greetertimer && cd grpc_greetertimer
~/grpc_greetertimer$ mkdir -p proto/ go/ java/org/pubref/grpc/greetertimer/
~/grpc_greetertimer$ touch WORKSPACE
~/grpc_greetertimer$ touch proto/BUILD
~/grpc_greetertimer$ touch proto/greetertimer.proto
~/grpc_greetertimer$ touch go/BUILD
~/grpc_greetertimer$ touch go/main.go
~/grpc_greetertimer$ touch java/org/pubref/grpc/greetertimer/BUILD
~/grpc_greetertimer$ touch java/org/pubref/grpc/greetertimer/GreeterTimerServer.java

2.4.2: The WORKSPACE

We’ll begin by creating the WORKSPACE file with a reference to the rules_protobuf repository. We load the main entrypoint skylark file rules.bzl in the //bzl package and call its protobuf_repositories function with the languages to we want to use (in this case java and go). We also load rules_go for go compile support (not shown).

# File //:WORKSPACE
workspace(name = "org_pubref_grpc_greetertimer")

git_repository(
 name = "org_pubref_rules_protobuf",
 remote = "https://github.com/pubref/rules_protobuf.git",
 tag = "v0.6.0",
)

# Load language-specific dependencies
load("@org_pubref_rules_protobuf//java:rules.bzl", "java_proto_repositories")
java_proto_repositories()

load("@org_pubref_rules_protobuf//go:rules.bzl", "go_proto_repositories")
go_proto_repositories()

Refer to the repositories.bzl file, if you are interested in inspecting the dependencies.

Bazel won’t actually fetch something unless we actually need it by some other rule later, so let’s go ahead and write some code. We’ll store our protocol buffer sources in //proto, our java sources in //java, and go source in //go.

Note: go development within a bazel workspace is a little different than vanilla go. In particular, one does not have to adhere to a typical GOCODE layout having a src/, pkg/, bin/ subdirectories.

2.4.3: The GreeterTimer Server

The Java server’s main job is to accept requests and then connect to the requested Greeter service as a client. The implementation counts down the number of remaining messages and does a blocking sayHello(request) for each one. If the batchSize limit is met, the observer.onNext(response) message is invoked, streaming back a response to the client.

/* File //java/org/pubref/grpc/greetertimer:GreeterTimerServer.java */

 while (remaining-- > 0) {

 if (batchCount++ == batchSize) {
 BatchResponse response = BatchResponse.newBuilder()
 .setRemaining(remaining)
 .setBatchCount(batchCount)
 .setBatchTimeMillis(batchTime)
 .setErrCount(errCount)
 .build();
 observer.onNext(response);
 }

 blockingStub.sayHello(HelloRequest.newBuilder()
 .setName("#" + remaining)
 .build());
 }
}

2.4.4: The GreeterTimer Client

The Go client prepares a TimerRequest and gets back a stream interface from the client.TimeHello method. We call its Recv() method until EOF, at which point the call is complete. A summary of each BatchResponse is simply printed out to the terminal.

// File: //go:main.go

func submit(client greeterTimer.GreeterTimerClient, request *greeterTimer.TimerRequest) error {
 stream, err := client.TimeHello(context.Background(), request)
 if err != nil {
 log.Fatalf("could not submit request: %v", err)
 }
 for {
 batchResponse, err := stream.Recv()
 if err == io.EOF {
 return nil
 }
 if err != nil {
 log.Fatalf("error during batch recv: %v", err)
 return err
 }
 reportBatchResult(batchResponse)
 }
}

2.4.5: Generate the go protobuf+gRPC code

In our //proto:BUILD file, we have a go_proto_library rule loaded from the rules_protobuf repository. Internally, the rule declares to bazel that it is responsible for creating greetertimer.pb.go output file. This rule won’t actually do anything unless we depend on it somewhere else.

# File: //proto:BUILD
load("@org_pubref_rules_protobuf//go:rules.bzl", "go_proto_library")

go_proto_library(
 name = "go_default_library",
 protos = [
 "greetertimer.proto",
 ],
 with_grpc = True,
)

The go client implementation depends on the go_proto_library as source file provider to the go_binary rule. We also pass in some compile-time dependencies named in the GRPC_COMPILE_DEPS list.

load("@io_bazel_rules_go//go:def.bzl", "go_binary")
load("@org_pubref_rules_protobuf//go:rules.bzl", "GRPC_COMPILE_DEPS")

go_binary(
 name = "hello_client",
 srcs = [
 "main.go",
 ],
 deps = [
 "//proto:go_default_library",
 ] + GRPC_COMPILE_DEPS,
)

~/grpc_greetertimer$ bazel build //go:client

Here’s what happens when we invoke bazel to actually build the client binary:

Bazel checks to see if the inputs (files) that the binary depends on have changed (by content hash and filestamps). Bazel recognizes that the output files for the //proto:go_default_library have not been built.
Bazel checks to see if all the necessary inputs (including tools) for the go_proto_library are available. If not, download and build all the necessary tools. Then, invoke the rule.
1. Fetch the google/protobuf repository and build protoc from source (via a cc_binary rule).
2. Build the protoc-gen-go plugin from source (via a go_binary rule).
3. Invoke protoc with the protoc-gen-go plugin with the appropriate options and arguments.
4. Confirm that all the declared outputs of the go_proto_library where actually built (should be in bazel-bin/proto/greetertimer.pb.go).
Compile the generated greetertimer.pb.go with the client main.go file, creating the bazel-bin/go/client executable.

2.4.6: Generate the java protobuf libraries

The java_proto_library rule is functionally identical to the go_proto_library rule. However, instead of providing a *.pb.go file, it bundles up all the generated outputs into a *.srcjar file (which is then used as an input to the java_library rule). This an implementation detail of the java rule. Here is how we build the final java binary:

java_binary(
 name = "server",
 main_class = "org.pubref.grpc.greetertimer.GreeterTimerServer",
 srcs = [
 "GreeterTimerServer.java",
 ],
 deps = [
 ":timer_protos",
 "@org_pubref_rules_protobuf//examples/helloworld/proto:java",
 "@org_pubref_rules_protobuf//java:grpc_compiletime_deps",
 ],
 runtime_deps = [
 "@org_pubref_rules_protobuf//java:netty_runtime_deps",
 ],
)

The :timer_protos is a locally defined java_proto_library rule.
The @org_pubref_rules_protobuf//examples/helloworld/proto:java is an external java_proto_library rule that generates the greeter service client stub in our own workspace.
Finally, we name the compile-time and run-time dependencies for the executable jar. If these jar files have not yet been downloaded from maven central, they will be fetch as soon as we need them:

~/grpc_greetertimer$ bazel build java/org/pubref/grpc/greetertimer:server
~/grpc_greetertimer$ bazel build java/org/pubref/grpc/greetertimer:server_deploy.jar

This last form (having the extra _deploy.jar) is called an implicit target of the :server rule. When invoked this way, bazel will pack up all the required classes and generate a standalone executable jar that can be run independently in a jvm.

2.4.7: Run it!

First, we’ll start a greeter server (one at a time):

~/grpc_greetertimer$ cd ~/rules_protobuf
~/rules_protobuf$ bazel run examples/helloworld/go/server
~/rules_protobuf$ bazel run examples/helloworld/cpp/server
~/rules_protobuf$ bazel run examples/helloworld/java/org/pubref/rules_protobuf/examples/helloworld/server:netty
~/rules_protobuf$ bazel run examples/helloworld/csharp/GreeterServer
INFO: Server started, listening on 50051

In a separate terminal, start the greetertimer server:

~/grpc_greetertimer$ bazel build //java/org/pubref/grpc/greetertimer:server_deploy.jar
~/grpc_greetertimer$ java -jar bazel-bin/java/org/pubref/grpc/greetertimer/server_deploy.jar

Finally, in a third terminal, invoke the greetertimer client:

# Timings for the java server
~/rules_protobuf$ bazel run examples/helloworld/java/org/pubref/rules_protobuf/examples/helloworld/server:netty

~/grpc_greeterclient$ bazel run //go:client -- -total_size 10000 -batch_size 1000
17:31:04 1001 hellos (0 errs, 8999 remaining): 1.7 hellos/ms or ~590µs per hello
# ... plus a few runs to warm up the jvm...
17:31:13 1001 hellos (0 errs, 8999 remaining): 6.7 hellos/ms or ~149µs per hello
17:31:13 1001 hellos (0 errs, 7998 remaining): 9.0 hellos/ms or ~111µs per hello
17:31:13 1001 hellos (0 errs, 6997 remaining): 8.9 hellos/ms or ~112µs per hello
17:31:13 1001 hellos (0 errs, 5996 remaining): 9.2 hellos/ms or ~109µs per hello
17:31:13 1001 hellos (0 errs, 4995 remaining): 9.4 hellos/ms or ~106µs per hello
17:31:13 1001 hellos (0 errs, 3994 remaining): 9.0 hellos/ms or ~111µs per hello
17:31:13 1001 hellos (0 errs, 2993 remaining): 9.4 hellos/ms or ~107µs per hello
17:31:13 1001 hellos (0 errs, 1992 remaining): 9.4 hellos/ms or ~107µs per hello
17:31:13 1001 hellos (0 errs, 991 remaining): 9.1 hellos/ms or ~110µs per hello
17:31:14 991 hellos (0 errs, -1 remaining): 9.0 hellos/ms or ~111µs per hello```

```sh
# Timings for the go server
~/rules_protobuf$ bazel run examples/helloworld/go/server

~/grpc_greeterclient$ bazel run //go:client -- -total_size 10000 -batch_size 1000
17:32:33 1001 hellos (0 errs, 8999 remaining): 7.5 hellos/ms or ~134µs per hello
17:32:33 1001 hellos (0 errs, 7998 remaining): 7.9 hellos/ms or ~127µs per hello
17:32:34 1001 hellos (0 errs, 6997 remaining): 7.8 hellos/ms or ~128µs per hello
17:32:34 1001 hellos (0 errs, 5996 remaining): 7.7 hellos/ms or ~130µs per hello
17:32:34 1001 hellos (0 errs, 4995 remaining): 7.9 hellos/ms or ~126µs per hello
17:32:34 1001 hellos (0 errs, 3994 remaining): 8.0 hellos/ms or ~125µs per hello
17:32:34 1001 hellos (0 errs, 2993 remaining): 7.6 hellos/ms or ~132µs per hello
17:32:34 1001 hellos (0 errs, 1992 remaining): 7.9 hellos/ms or ~126µs per hello
17:32:34 1001 hellos (0 errs, 991 remaining): 7.9 hellos/ms or ~127µs per hello
17:32:34 991 hellos (0 errs, -1 remaining): 7.8 hellos/ms or ~128µs per hello

# Timings for the C++ server
~/rules_protobuf$ bazel run examples/helloworld/cpp:server

~/grpc_greeterclient$ bazel run //go:client -- -total_size 10000 -batch_size 1000
17:33:10 1001 hellos (0 errs, 8999 remaining): 9.1 hellos/ms or ~110µs per hello
17:33:10 1001 hellos (0 errs, 7998 remaining): 9.0 hellos/ms or ~111µs per hello
17:33:10 1001 hellos (0 errs, 6997 remaining): 9.1 hellos/ms or ~110µs per hello
17:33:10 1001 hellos (0 errs, 5996 remaining): 8.6 hellos/ms or ~116µs per hello
17:33:10 1001 hellos (0 errs, 4995 remaining): 9.0 hellos/ms or ~111µs per hello
17:33:10 1001 hellos (0 errs, 3994 remaining): 9.0 hellos/ms or ~111µs per hello
17:33:10 1001 hellos (0 errs, 2993 remaining): 9.1 hellos/ms or ~110µs per hello
17:33:10 1001 hellos (0 errs, 1992 remaining): 9.0 hellos/ms or ~111µs per hello
17:33:10 1001 hellos (0 errs, 991 remaining): 9.0 hellos/ms or ~111µs per hello
17:33:11 991 hellos (0 errs, -1 remaining): 9.0 hellos/ms or ~111µs per hello

# Timings for the C# server
~/rules_protobuf$ bazel run examples/helloworld/csharp/GreeterServer

~/grpc_greeterclient$ bazel run //go:client -- -total_size 10000 -batch_size 1000
17:34:37 1001 hellos (0 errs, 8999 remaining): 6.0 hellos/ms or ~166µs per hello
17:34:37 1001 hellos (0 errs, 7998 remaining): 6.7 hellos/ms or ~150µs per hello
17:34:37 1001 hellos (0 errs, 6997 remaining): 6.8 hellos/ms or ~148µs per hello
17:34:37 1001 hellos (0 errs, 5996 remaining): 6.8 hellos/ms or ~147µs per hello
17:34:37 1001 hellos (0 errs, 4995 remaining): 6.7 hellos/ms or ~150µs per hello
17:34:38 1001 hellos (0 errs, 3994 remaining): 6.7 hellos/ms or ~150µs per hello
17:34:38 1001 hellos (0 errs, 2993 remaining): 6.7 hellos/ms or ~149µs per hello
17:34:38 1001 hellos (0 errs, 1992 remaining): 6.7 hellos/ms or ~149µs per hello
17:34:38 1001 hellos (0 errs, 991 remaining): 6.8 hellos/ms or ~148µs per hello
17:34:38 991 hellos (0 errs, -1 remaining): 6.8 hellos/ms or ~147µs per hello

The informal analysis demonstrated comparable timings for c++, go, and java greeter service implementations. The c++ server had the overall fastest and most consistent performance. The go implementation was also very consistent, but slightly slower than C++. Java demonstrated some initial relative slowness likely due to the JVM warming up but soon converged on timings similar to the C++ implementation. C# has consistent performance but marginally slower.

2.5: Summary

Bazel assists in the construction of gRPC applications by providing a capable build environment for services built in a multitude of languages. rules_protobuf complements bazel by packaging up all the dependencies needed and abstracting away the need to call protoc directly.

In this workflow one does not need to check in the generated source code (it is always generated on-demand within your workspace). For projects that do require this, one can use the output_to_workspace option to place the generated files alongside the protobuf definitions.

Finally, rules_protobuf has full support for the grpc-gateway project via the grpc_gateway_proto_library and grpc_gateway_binary rules, so you can easily bridge your gRPC apps with HTTP/1.1 gateways.

Refer to the complete list of supported languages and gRPC versions for more information.

And… that’s a wrap. Happy procedure calling!

Paul Johnston is the principal at PubRef (@pub_ref), a solutions provider for scientific communications workflows. If you have an organizational need for assistance with Bazel, gRPC, or related technologies, please contact pcj@pubref.org. Thanks!

Blog: gRPC at VSCO

Tue, 06 Sep 2016 00:00:00 +0000

Our guest post today comes from Robert Sayre and Melinda Lu of VSCO.

Founded in 2011, VSCO is a community for expression—empowering people to create, discover and connect through images and words. VSCO is in the process of migrating their stack to gRPC.

In 2015, user growth forced VSCO down a familiar path. A monolithic PHP application in existence since the early days of the company was exhibiting performance problems and becoming difficult to maintain. We experimented with some smaller services in node.js, Go, and Java. At the same time, a larger messaging service for email, push messages, and in-app notifications was built in Go. Taking a first step away from JSON, we chose Protocol Buffers as the serialization format for this system.

Today, VSCO has largely settled on Go for new services. There are exceptions, particularly where a mature JVM solution is available for a given problem. Additionally, VSCO uses node.js for web applications, often with server-side React. Given that mix of languages, services, and some future data pipeline work detailed below, VSCO settled on gRPC and Protocol Buffers as the most practical solution for interprocess communication. A gradual migration from JSON over HTTP/1.1 APIs to gRPC over HTTP/2 is underway and going well. That said, there have been issues with the maturity of the PHP implementation relative to other languages.

Protocol buffers have been particularly valuable in building out our data ecosystem, where we rely on them to standardize and allow safe evolution of our data schemas in a language-agnostic way. As one example, we’ve built a Go service that feeds off our MySQL and MongoDB database replication logs and transforms backend database changes into a stream of immutable events in Kafka, with each row- or document-change event encoded as a protocol buffer. This database event stream allows us to add real-time data consumers as desired, without impacting production traffic and without having to coordinate with other systems. By processing all database events into protocol buffers en-route to Kafka, we can ensure that data is encoded in a uniform way that makes it easy to consume and use from multiple languages. Our implementation of MySQL-binary-log and Mongo-oplog tailers are available on GitHub.

Elsewhere in our data pipeline, we’ve begun using gRPC and protocol buffers to deliver behavioral events from our iOS and Android clients to a Go ingestion service, which then publishes these events to Kafka. To support this high-volume use case, we needed (1) a performant, fault-tolerant, language-agnostic RPC framework, (2) a way to ensure data compatibility as our product evolves, and (3) horizontally-scalable infrastructure. We’ve found gRPC, protocol buffers, and Go services running in Kubernetes a good fit for all three. As this was our first client-facing Go gRPC service, we did experience some new points of friction — in particular, load-balancer support and amenities like curl-like debugging have been lagging due to the youth of the HTTP/2 ecosystem. However, the ease of defining services with the gRPC IDL, using built-in architecture like interceptors, and scaling with Go have made the tradeoffs worthwhile.

As a first step in bringing gRPC to our mobile clients, we’ve shipped telemetry code in our iOS and Android apps. As of gRPC 1.0, this process is relatively straightforward. They only post events to our servers so far, and don’t do much with gRPC responses. The previous implementation was based on JSON, and our move to a single protocol buffer definition of our events uncovered a bunch of subtle bugs and differences between the clients.

One slight roadblock we ran into was the need for our clients to maintain compatibility with our JSON implementation as we ramp up, and for integration with vendor SDKs. This required a little bit of key-value coding on iOS, but it got more difficult on Android. We ended up having to write a protobuf compiler plugin to get the reflection features we needed while maintaining adequate performance. Drawing from that experience, we’ve made a concise example protoc plugin built with Bazel available on GitHub.

As more and more of our data becomes available in protocol buffer form, we plan to build upon this unified schema to expand our machine-learning and analytics systems. For example, we write our Kafka database replication streams to Amazon S3 as Apache Parquet, an efficient columnar disk-storage format. Parquet has low-level support for protocol buffers, so we can use our existing data definitions to write optimized tables and do partial deserializations where desired.

From S3, we run computations on our data using Apache Spark, which can use our protocol buffer definitions to define types. We’re also building new machine-learning applications with TensorFlow. It uses protocol buffers natively and allows us to serve our models as gRPC services with TensorFlow Serving.

So far, we’ve had good luck with gRPC and Protocol Buffers. They don’t eliminate every integration headache. However it’s easy to see how they help our engineers avoid writing a lot of boilerplate RPC code, while side-stepping the endless data-quality papercuts that come with looser serialization formats.

Blog: Why we have decided to move our APIs to gRPC

Mon, 29 Aug 2016 00:00:00 +0000

Vendasta started out 8 years ago as a point solution provider of products for small business. From the beginning we partnered with media companies and agencies who have armies of salespeople and existing relationships with those businesses to sell our software. It is estimated that over 30 million small businesses exist in the United States alone, so scalability of our SaaS solution was considered one of our top concerns from the beginning and it was the reason we started with Google App Engine and Datastore. This solution worked really well for us as our system scaled from hundreds to hundreds of thousands of end users. We also scaled our offering from a point solution to an entire platform with multiple products and the tools for partners to manage their sales of those products during this time.

All throughout this journey Python GAE served our needs well. We exposed a number of APIs via HTTP + JSON for our partners to automate tasks and integrate their other systems with our products and platform. However, in 2016 we introduced the Vendasta Marketplace. This marked a major change to our offering, which depended heavily on having 3rd party vendors use our APIs to deliver their own products in our platform. This was a major change because our public APIs provide an upper-bound on 3rd-party applications, and made us realize that we really needed to make APIs that were amazing, not just good.

The first optimization that we started with was to use the Go programming language to build endpoints that handled higher throughput with lower latency than we could get with Python. On some APIs this made an incredible difference: we saw 50th percentile response times to drop from 1200 ms to 4 ms, and even more spectacularly 99th percentile response times drop from 30,000 ms to 12 ms! On other APIs we saw a much smaller, but still significant difference.

The second optimization we used was to replicate large portions of our Datastore data into ElasticSearch. ElasticSearch is a fundamentally different storage technology to Datastore, and is not a managed service, so it was a big leap for us. But this change allowed us to migrate almost all of our overnight batch-processing APIs to real-time APIs. We had tried BigQuery, but it’s query processing times meant that we couldn’t display things in real time. We had tried cloudSQL, but there was too much data for it to easily scale. We had tried the appengine Search API, but it has limitations with result sets over 10,000. We instead scaled up our ElasticSearch cluster using Google Container Engine and with it’s powerful aggregations and facet processing our needs were easily met. So with these first two solutions in place, we had made meaningful changes to the performance of our APIs.

The last optimization we made was to move our APIs to gRPC. This change was much more extensive than the others as it affected our clients. Like ElasticSearch, it represents a fundamentally different model with differing performance characteristics, but unlike ElasticSearch we found it to be a true superset: all of our usage scenarios were impacted positively by it.

The first benefit we saw from gRPC was the ability to move from publishing APIs and asking developers to integrate with them, to releasing SDKs and asking developers to copy-paste example code written in their language. This represents a really big benefit for people looking to integrate with our products, while not requiring us to hand-roll entire SDKs in the 5+ languages our partners and vendors use. It is important to note that we still write light wrappers over the generated gRPC SDKs to make them package-manager friendly, and to provide wrappers over the generated protobuf structures.

The second benefit we saw from gRPC was the ability to break free from the call-and-response architecture necessitated by HTTP + JSON. gRPC is built on top of HTTP/2, which allows for client-side and/or server-side streaming. In our use cases, this means we can lower the time to first display by streaming results as they become ready on the server (server-side streaming). We have also been investigating the potential to offer very flexible create endpoints that easily support bulk ingestion with bi-directional streaming, this would mean we would allow the client to asynchronously stream results, while the server would stream back statuses allowing for easy checkpoint operations while not slowing upload speeds to wait for confirmations. We feel that we are just starting to see the benefits from this feature as it opens up a totally new model for client-server interactions that just wasn’t possible with HTTP.

The third benefit was the switch from JSON to protocol buffers, which works very well with gRPC. This improves serialization and deserialization times; which is very significant to some of our APIs, but appreciated on all of them. The more important benefit comes from the explicit format specification of proto, meaning that clients receive typed objects rather than free-form JSON. Because of this, our clients can reap the benefits of auto-completion in their IDEs, type-safety if their language supports it, and enforced compatibility between clients and servers with differing versions.

The final benefit of gRPC was our ability to quickly spec endpoints. The proto format for both data and service definition greatly simplifies defining new endpoints and finally allows the succinct definition of endpoint contracts. This means we are much better able to communicate endpoint specifications between our development teams. gRPC means that for the first time at our company we are able to simultaneously develop the client and the server side of our APIs! This means our latency to produce new APIs with the accompanying SDKs has dropped dramatically. Combined with code generation, it allows us to truly develop clients and servers in parallel.

Our experience with gRPC has been positive, even though it does not eliminate the difficulty of providing endpoints to partners and vendors, and address all of our performance issues. However, it does make improvements to our endpoint performance, integration with those endpoints, and even in delivery of SDKs.

Blog: gRPC Project is now 1.0 and ready for production deployments

Tue, 23 Aug 2016 00:00:00 +0000

Today, the gRPC project has reached a significant milestone with its 1.0 release. Languages moving to 1.0 include C++, Java, Go, Node, Ruby, Python and C# across Linux, Windows, and Mac. Objective-C and Android Java support on iOS and Android is also moving to 1.0. The 1.0 release means that the core protocol and API surface are now stable with measured performance, stress tested and developers can rely on these APIs and deploy in production, they will follow semantic versioning from here.

We are very excited about the progress we have made so far and would like to thank all our users and contributors. First announced in March 2015 with Square, gRPC is already being used in many open source projects like etcd from CoreOS, containerd from Docker, cockroachdb from Cockroach Labs, and by many other companies like Vendasta, Netflix, YikYak and Carbon 3d. Outside of microservices, telecom giants like Cisco, Juniper, Arista, and Ciena, are building support for streaming telemetry and network configuration from their network devices using gRPC, as part of OpenConfig effort.

From the beta release, we have made significant strides in the areas of usability, interoperability, and performance measurement on the road to 1.0. In most of the languages, the installation of the gRPC runtime as well as setup of a development environment is a single command. Beyond installation, we have set up automated tests for gRPC across languages and RPC types in order to stress test our APIs and ensure interoperability. There is now a performance dashboard available in the open to see latency and throughput for unary and streaming ping pong for various languages. Other measurements have shown significant gains from using gRPC/Protobuf instead of HTTP/JSON such as in CoreOS blogpost and in Google Cloud PubSub testing. In the coming months, we will invest a lot more in performance tuning.

Even within Google, we have seen Google cloud APIs like BigTable, PubSub, Speech, launch of a gRPC-based API surface leading to ease of use and performance benefits. Products like Tensorflow have effectively used gRPC for inter-process communication as well. Beyond usage, we are keen to see the contributor community grow with gRPC. We are already starting to see contributions around gRPC in meaningful ways in the grpc-ecosystem organization. We are very happy to see projects like grpc-gateway to enable users to serve REST clients with gRPC based services, Polyglot to have a CLI for gRPC, Prometheus monitoring of gRPC Services and work with OpenTracing. You can suggest and contribute projects to this organization here. We look forward to working with the community to take the gRPC project to new heights.

Blog: Mobile Benchmarks

Tue, 26 Jul 2016 00:00:00 +0000

As gRPC has become a better and faster RPC framework, we’ve consistently gotten the question, “How much faster is gRPC?” We already have comprehensive server-side benchmarks, but we don’t have mobile benchmarks. Benchmarking a client is a bit different than benchmarking a server. We care more about things such as latency and request size and less about things like queries per second (QPS) and number of concurrent threads. Thus we built an Android app in order to quantify these factors and provide solid numbers behind them.

Specifically what we want to benchmark is client side protobuf vs. JSON serialization/deserialization and gRPC vs. a RESTful HTTP JSON service. For the serialization benchmarks, we want to measure the size of messages and speed at which we serialize and deserialize. For the RPC benchmarks, we want to measure the latency of end-to-end requests and packet size.

Protobuf vs. JSON

In order to benchmark protobuf and JSON, we ran serializations and deserializations over and over on randomly generated protos, which can be seen here. These protos varied quite a bit in size and complexity, from just a few bytes to over 100kb. JSON equivalents were created and then also benchmarked. For the protobuf messages, we had three main methods of serializing and deserializing: simply using a byte array, CodedOutputStream/CodedInputStream which is protobuf’s own implementation of input and output streams, and Java’s ByteArrayOutputStream and ByteArrayInputStream. For JSON we used org.json’s JSONObject. This only had one method to serialize and deserialize, toString() and new JSONObject(), respectively.

In order to keep benchmarks as accurate as possible, we wrapped the code to be benchmarked in an interface and simply looped it for a set number of iterations. This way we discounted any time spent checking the system time.

interface Action {
 void execute();
}

// Sample benchmark of multiplication
Action a = new Action() {
 @Override
 public void execute() {
 int x = 1000 * 123456;
 }
}

for (int i = 0; i < 100; ++i) {
 a.execute();
}

Before running a benchmark, we ran a warmup in order to clean out any erratic behaviour by the JVM, and then calculated the number of iterations needed to run for a set time (10 seconds in the protobuf vs. JSON case). To do this, we started with 1 iteration, measured the time it took for that run, and compared it to a minimum sample time (2 seconds in our case). If the number of iterations took long enough, we estimated the number of iterations needed to run for 10 seconds by doing some math. Otherwise, we multiplied the number of iterations by 2 and repeated.

// This can be found in ProtobufBenchmarker.java benchmark()
int iterations = 1;
// Time action simply reports the time it takes to run a certain action for that number of iterations
long elapsed = timeAction(action, iterations);
while (elapsed < MIN_SAMPLE_TIME_MS) {
 iterations *= 2;
 elapsed = timeAction(action, iterations);
}
// Estimate number of iterations to run for 10 seconds
iterations = (int) ((TARGET_TIME_MS / (double) elapsed) * iterations);

Results

Benchmarks were run on protobuf, JSON, and gzipped JSON.

We found that regardless of the serialization/deserialization method used for protobuf, it was consistently about 3x faster for serializing than JSON. For deserialization, JSON is actually a bit faster for small messages (<1kb), around 1.5x, but for larger messages (>15kb) protobuf is 2x faster. For gzipped JSON, protobuf is well over 5x faster in serialization, regardless of size. For deserialization, both are about the same at small messages, but protobuf is about 3x faster for larger messages. Results can be explored in more depth and replicated in the README.

gRPC vs. HTTP JSON

To benchmark RPC calls, we want to measure end-to-end latency and bandwidth. To do this, we ping pong with a server for 60 seconds, using the same message each time, and measure the latency and message size. The message consists of some fields for the server to read, and a payload of bytes. We compared gRPC’s unary call to a simple RESTful HTTP JSON service. The gRPC benchmark creates a channel, and starts a unary call that repeats when it receives a response until 60 seconds have passed. The response contains a proto with the same payload sent.

Similarly for the HTTP JSON benchmarks, it sends a POST request to the server with an equivalent JSON object, and the server sends back a JSON object with the same payload.

// This can be found in AsyncClient.java doUnaryCalls()
// Make stub to send unary call
final BenchmarkServiceStub stub = BenchmarkServiceGrpc.newStub(channel);
stub.unaryCall(request, new StreamObserver<SimpleResponse>() {
 long lastCall = System.nanoTime();
 // Do nothing on next
 @Override
 public void onNext(SimpleResponse value) {
 }

 @Override
 public void onError(Throwable t) {
 Status status = Status.fromThrowable(t);
 System.err.println("Encountered an error in unaryCall. Status is " + status);
 t.printStackTrace();

 future.cancel(true);
 }
 // Repeat if time isn't reached
 @Override
 public void onCompleted() {
 long now = System.nanoTime();
 // Record the latencies in microseconds
 histogram.recordValue((now - lastCall) / 1000);
 lastCall = now;

 Context prevCtx = Context.ROOT.attach();
 try {
 if (endTime > now) {
 stub.unaryCall(request, this);
 } else {
 future.done();
 }
 } finally {
 Context.current().detach(prevCtx);
 }
 }
});

Both HttpUrlConnection and the OkHttp library were used.

Only gRPC’s unary calls were benchmarked against HTTP, since streaming calls were over 2x faster than the unary calls. Moreover, HTTP has no equivalent of streaming, which is an HTTP/2 specific feature.

Results

In terms of latency, gRPC is 5x-10x faster up to the 95th percentile, with averages of around 2 milliseconds for an end-to-end request. For bandwidth, gRPC is about 3x faster for small requests (100-1000 byte payload), and consistently 2x faster for large requests (10kb-100kb payload). To replicate these results or explore in more depth, check out our repository.

Blog: gRPC with REST and Open APIs

Mon, 09 May 2016 00:00:00 +0000

Our guest post today comes from Brandon Phillips of CoreOS. CoreOS builds open source projects and products for Linux Containers. Their flagship product for consensus and discovery etcd and their container engine rkt are early adopters of gRPC.

One of the key reasons CoreOS chose gRPC is because it uses HTTP/2, enabling applications to present both a HTTP 1.1 REST/JSON API and an efficient gRPC interface on a single TCP port (available for Go). This provides developers with compatibility with the REST web ecosystem, while advancing a new, high-efficiency RPC protocol. With the recent release of Go 1.6, Go ships with a stable net/http2 package by default.

Since many CoreOS clients speak HTTP 1.1 with JSON, gRPC’s easy interoperability with JSON and the Open API Specification (formerly Swagger) was extremely valuable. For their users who are more comfortable with HTTP/1.1+JSON-based and Open API Spec APIs they used a combination of open source libraries to make their gRPC services available in both gRPC and HTTP REST flavors, using API multiplexers to give users the best of both worlds. Let’s dive into the details and find out how they did it!

A gRPC application called EchoService

In this post we will build a small proof-of-concept gRPC application from a gRPC API definition, add a REST service gateway, and finally serve it all on a single TLS port. The application is called EchoService, and is the web equivalent of the shell command echo: the service returns, or “echoes”, whatever text is sent to it.

First, let’s define the arguments to EchoService in a protobuf message called EchoMessage, which includes a single field called value. We will define this message in a protobuf “.proto” file called service.proto. Here is our EchoMessage:

message EchoMessage {
 string value = 1;
}

In this same .proto file, we define a gRPC service that takes this data structure and returns it:

service EchoService {
 rpc Echo(EchoMessage) returns (EchoMessage) {
 }
}

Running this service.proto file “as is” through the Protocol Buffer compiler protoc generates a stub gRPC service in Go, along with clients in various languages. But gRPC alone isn’t as useful as a service that also exposes a REST interface, so we won’t stop with the gRPC service stub.

Next, we add the gRPC REST Gateway. This library will build a RESTful proxy on top of the gRPC EchoService. To build this gateway, we add metadata to the EchoService .proto to indicate that the Echo RPC maps to a RESTful POST method with all RPC parameters mapped to a JSON body. The gateway can map RPC parameters to URL paths and query parameters, but we omit those complications here for brevity.

service EchoService {
 rpc Echo(EchoMessage) returns (EchoMessage) {
 option (google.api.http) = {
 post: "/v1/echo"
 body: "*"
 };
 }
}

This means the gateway, once generated by protoc, can now accept a HTTP request from curl like this:

curl -X POST -k https://localhost:10000/v1/echo -d '{"value": "CoreOS is hiring!"}'

The whole system so far looks like this, with a single service.proto file generating both a gRPC server and a REST proxy:

To bring this all together, the echo service creates a Go http.Handler to detect if the protocol is HTTP/2 and the Content-Type is “application/grpc”, and sends such requests to the gRPC server. Everything else is routed to the REST gateway. The code looks something like this:

if r.ProtoMajor == 2 && strings.Contains(r.Header.Get("Content-Type"), "application/grpc") {
 grpcServer.ServeHTTP(w, r)
} else {
 otherHandler.ServeHTTP(w, r)
}

To try it out, all you need is a working Go 1.6 development environment and the following simple commands:

go get -u github.com/philips/grpc-gateway-example
grpc-gateway-example serve

With the server running you can try requests on both HTTP 1.1 and gRPC interfaces:

grpc-gateway-example echo Take a REST from REST with gRPC
curl -X POST -k https://localhost:10000/v1/echo -d '{"value": "CoreOS is hiring!"}'

One last bonus: because we have an Open API specification, you can browse the Open API UI running at https://localhost:10000/swagger-ui/#!/EchoService/Echo if you have the server above running on your laptop.

We’ve taken a look at how to use gRPC to bridge to the world of REST. If you want to take a look at the complete project, check out the repo on GitHub. We think this pattern of using a single protobuf to describe an API leads to an easy to consume, flexible API framework, and we’re excited to leverage it in more of our projects.

gRPC – gRPC Blog

Blog: gRPConf 2025 Announcement

Why Attend?

Blog: How robots talk: building distributed robots with gRPC and WebRTC

Bridging the gap between lab-grade robotics and real-world systems

Why gRPC makes sense for robots

Why WebRTC is also part of the picture

Flexible transport layers with gRPC

A real-world example: coordinating a claw game

Why this pattern matters for the broader community

Viam for real-time robot control

Blog: gRPC and AI: A Powerful Partnership

Blog: Highlights in gRPConf 2024:Customer Showcase, Developer Engagement, Birds of Feathers Discussions and more.

Blog: Announcing Our New Mailing List grpc-io-announce

Blog: Configuring proxyless gRPC with Kubernetes Gateway API and OpenTelemetry metrics for Cloud Service Mesh now available in preview

What’s new:

Benefits:

Get started:

Blog: gRPConf 2024 Schedule

Why Attend?

Blog: Celebrate gRPC Day

Blog: Can gRPC replace REST and WebSockets for Web Application Communication?

Understanding gRPC-Web and Its Place in Modern Web Development

The Mechanics of gRPC-Web: How It Works

The Theory of Transitioning from REST to gRPC-Web

Advantages of gRPC-Web Over REST

Setting Up a gRPC-Web Environment

Defining Service Methods with Protobuf

Replacing a Typical WebSocket Connection with gRPC-Web

Is It Time to Replace REST and WebSockets?

gRPC Support in Postman

Blog: Highlights in gRPConf 2023:Customer Showcase, Developer Engagement, Birds of Feathers Discussions and more.

Blog: Announcing gRPConf 2023!

Blog: gRPConf 2023 Schedule

Blog: gRPC performance benchmarks on GKE

Background

Benchmarking on Kubernetes

Cluster setup

Controller deployment

Continuous integration

Config generation

Prebuilt images

Test runner

Dashboard

Results

Running your own

Blog: Running gRPC and Protobuf on ARM64 (on Linux)

Overview of gRPC and Protobuf support on ARM64 Linux

ARM64 / aarch64 / ARMv8 terminology

Official ARM64 support is currently Linux-only

Appendix: list of changes/fixes/improvements

Blog: The future of gRPC in C# belongs to grpc-dotnet

What makes grpc-dotnet the preferred implementation

Why not keep Grpc.Core forever?

The plan

Q & A

Blog: Analyzing gRPC messages using Wireshark

Features

Capturing gRPC traffic

Note

Example

Address book .proto files

Setting protobuf search paths

Loading a capture file

Setting port traffic type

Decoding the search request message

Decoding the server-streamed response

History of gRPC and Protocol Buffers support

Learn more

Blog: Interceptors in gRPC-Web

Introduction

Note

What can I do with an interceptor?

Unary interceptor example

Stream interceptor example

Binding interceptors

Note

Feedback

Blog: Announcing gRPC-JS 1.0

Features

Address book `.proto` files

Add `.proto` files to the project