gRPC و Protocol BuffersgRPC & Protocol Buffers

Let's start with a confession: when two services in a system need to talk, most of us reach for REST over JSON by reflex. It works — but hidden behind that convenience is a pile of manual labor, string-juggling, and guesswork. Both sides must silently agree on field names, hand-parse JSON, and hope that a small change on one end doesn't quietly break the other. gRPC exists to turn that unwritten agreement into a formal, strongly-typed contract that compilers and tooling actually understand. In this chapter you won't just learn how to write a gRPC service — you'll understand why the technology exists, what happens behind the curtain, and where you should and shouldn't reach for it.

Part 0 — a few words you must feel before we start

Before code, a few terms recur throughout. Let me plant them in your mind now.

• Serialization: turning an in-memory object into a stream of bytes you can send over the network or store on disk. The reverse is deserialization. JSON is a text serialization format; Protocol Buffers is a binary one.
• IDL: Interface Definition Language — a language-independent file that says "here's the contract": which messages exist, with which fields, and which methods. A .proto file is an IDL.
• stub: generated code that, on the client, looks like a local object but actually dispatches the call over the network. The name means "the stand-in piece" for the real functions living on the other side.
• HTTP/2: the second version of HTTP that carries many requests concurrently over a single TCP connection (multiplexing), compresses headers, and supports bidirectional streaming. gRPC rides on top of it.

What RPC is: when we want the network to vanish

RPC stands for Remote Procedure Call. The central idea is to make a network call look like an ordinary function call. Instead of hand-building an HTTP request, constructing a URL, turning the body into JSON, sending it, and parsing the response, you just write:

BalanceResponse resp = accountService.getBalance(request);

as if it ran right here. But the illusion of "localness" is a double-edged sword. The network doesn't actually vanish; it just hides. Behind that one call there is still latency, packets can be lost, the server may be down, and the connection may drop mid-flight.

gRPC is one of today's most popular RPC frameworks, open-sourced by Google in 2015. The "g" officially stands for a different word in every release (the team's running joke), but its two pillars are fixed: Protocol Buffers for the contract and data, and HTTP/2 for transport. Let's crack open the first pillar.

Protocol Buffers: make the contract formal

Protocol Buffers (we'll shorten it to "protobuf") is two things that come together:

1. an Interface Definition Language (IDL) you write in .proto files that describes the shape of your data and services;
2. a binary serialization format that puts that data on the wire in the most compact form possible.

A sample .proto:

syntax = "proto3";

package bank.v1;

option java_package = "com.example.bank.v1";
option java_multiple_files = true;

// A message: a bundle of numbered fields
message BalanceRequest {
  string account_id = 1;
}

message BalanceResponse {
  string account_id = 1;
  int64  cents      = 2;   // balance in cents, to avoid float errors
  string currency   = 3;
}

// A service: a set of remote methods
service AccountService {
  rpc GetBalance(BalanceRequest) returns (BalanceResponse);
}

Let's unpack a few things:

• syntax = "proto3" says we use version 3 of the language (we'll get to "editions" later).
• Every field has a field number: the = 1, = 2... It's these numbers that travel on the wire, not the field names. This is the key to understanding everything.
• We chose int64 for money because floating-point (double) money is bug-prone; always keep money as an integer count of the smallest unit (cents).
• The service block defines remote methods. protobuf isn't only about data; it describes services too — and this is where gRPC enters.

The wire format: behind the bytes

Now for the compaction magic. When protobuf serializes a field, it writes a tag for that field followed by the value. The tag is built with this formula:

tag = (field_number << 3) | wire_type

That is, the field number is shifted left by 3 bits and the low 3 bits hold the wire type. The wire type tells the parser "how many bytes the next value is and how to read it." There are only a handful of wire types:

(Types 3 and 4 belonged to deprecated "groups.")

The heart of compaction is the varint — variable-width integer. Instead of every integer always taking 4 or 8 bytes, protobuf packs small numbers into fewer bytes. Each byte holds 7 bits of data, and the 8th (most-significant) bit is a continuation bit: if it's 1, "read the next byte too." So the number 1 is a single byte, but a billion takes several.

Let's decode a real message to feel how tiny it is. Suppose message T { int32 a = 1; } with a = 150. On the wire it becomes:

0x08 0x96 0x01

• 0x08 = tag: (1 << 3) | 0 → field 1, wire type VARINT.
• 0x96 0x01 = the varint for 150.

Just 3 bytes. The same data in JSON — {"a":150} — is nine bytes, and that's without counting whitespace.

proto3 and the new "editions"

For years we had two language versions: proto2 and proto3. Since 2023, the protobuf team introduced a new concept called Protobuf Editions, where instead of syntax = "proto3" you write edition = "2024" (the newest edition to date). The idea is that behaviors that used to differ between proto2 and proto3 (like "field presence" or how something is encoded) are now individually configurable features.

gRPC over HTTP/2: why transport matters

protobuf only solves the data and the contract. Now we must carry those bytes from A to B. gRPC does this over HTTP/2, and that choice is no accident.

Four things HTTP/2 gives gRPC:

• Multiplexing: many concurrent RPC calls over one TCP connection, no serial queue.
• Bidirectional streaming: because each stream can stay open independently, server and client can send and receive simultaneously.
• Header compression (HPACK): HTTP headers are compressed, so metadata is cheap.
• Binary framing: HTTP/2 is itself binary, matching protobuf's binary payload.

Code generation: from .proto to Java

Here's where the beauty shows. You write the .proto, and a compiler called protoc (with the protoc-gen-grpc-java plugin) generates Java classes from it: message classes (immutable, with a Builder pattern) and service stubs. In the Maven world you typically automate this with a plugin. The key dependencies:

<!-- dependencies (example version: gRPC-Java 1.82.x) -->
<dependency>
  <groupId>io.grpc</groupId>
  <artifactId>grpc-netty-shaded</artifactId>
  <version>1.82.1</version>
  <scope>runtime</scope>
</dependency>
<dependency>
  <groupId>io.grpc</groupId>
  <artifactId>grpc-protobuf</artifactId>
  <version>1.82.1</version>
</dependency>
<dependency>
  <groupId>io.grpc</groupId>
  <artifactId>grpc-stub</artifactId>
  <version>1.82.1</version>
</dependency>

From each service in .proto, three kinds of stub are generated for Java. Understanding their differences matters:

On the server side, protoc generates an abstract AccountServiceImplBase class that you extend and implement. Let's see a complete simple service.

The four call types

gRPC has four communication patterns, all coming from the same .proto file — distinguished only by the stream keyword.

service ChatService {
  // 1) unary: one request, one response
  rpc GetBalance(BalanceRequest) returns (BalanceResponse);

  // 2) server-streaming: one request, a stream of responses
  rpc WatchPrices(WatchRequest) returns (stream PriceTick);

  // 3) client-streaming: a stream of requests, one response
  rpc UploadPoints(stream Point) returns (UploadSummary);

  // 4) bidirectional: streams in both directions, independent
  rpc Chat(stream ChatMessage) returns (stream ChatMessage);
}

1) Unary — the backbone

Server side:

import io.grpc.stub.StreamObserver;
import io.grpc.Status;

public class AccountServiceImpl extends AccountServiceGrpc.AccountServiceImplBase {
    @Override
    public void getBalance(BalanceRequest request,
                           StreamObserver<BalanceResponse> responseObserver) {
        String id = request.getAccountId();
        if (id.isBlank()) {
            responseObserver.onError(Status.INVALID_ARGUMENT
                    .withDescription("account_id is required")
                    .asRuntimeException());
            return;
        }
        BalanceResponse resp = BalanceResponse.newBuilder()
                .setAccountId(id)
                .setCents(1_250_00)   // 1250.00
                .setCurrency("USD")
                .build();
        responseObserver.onNext(resp);   // send one response
        responseObserver.onCompleted();  // close the stream
    }
}

Notice that even in unary we use StreamObserver: onNext sends the value and onCompleted says "done." Starting the server:

import io.grpc.Server;
import io.grpc.ServerBuilder;

Server server = ServerBuilder.forPort(9090)
        .addService(new AccountServiceImpl())
        .build()
        .start();
server.awaitTermination();

Client side with a blocking stub — the cleanest form:

import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;

ManagedChannel channel = ManagedChannelBuilder
        .forAddress("localhost", 9090)
        .usePlaintext()   // dev only; use TLS in production
        .build();

AccountServiceGrpc.AccountServiceBlockingStub stub =
        AccountServiceGrpc.newBlockingStub(channel);

BalanceResponse resp = stub.getBalance(
        BalanceRequest.newBuilder().setAccountId("A-42").build());

System.out.println(resp.getCents() + " " + resp.getCurrency());
channel.shutdown();

2) Server-streaming

The server calls onNext several times and onCompleted at the end:

@Override
public void watchPrices(WatchRequest request,
                        StreamObserver<PriceTick> obs) {
    for (int i = 0; i < 5; i++) {
        obs.onNext(PriceTick.newBuilder()
                .setSymbol(request.getSymbol())
                .setPriceCents(10_000 + i * 25)
                .build());
    }
    obs.onCompleted();
}

The client with a blocking stub gets an Iterator:

Iterator<PriceTick> ticks = blockingStub.watchPrices(
        WatchRequest.newBuilder().setSymbol("ACME").build());
while (ticks.hasNext()) {
    System.out.println(ticks.next().getPriceCents());
}

3) Client-streaming

Here the client sends many messages. Because you must return a StreamObserver yourself, the async stub is required:

// server side
@Override
public StreamObserver<Point> uploadPoints(
        StreamObserver<UploadSummary> responseObs) {
    return new StreamObserver<>() {
        int count = 0;
        long sum = 0;
        @Override public void onNext(Point p) { count++; sum += p.getValue(); }
        @Override public void onError(Throwable t) { /* log */ }
        @Override public void onCompleted() {
            responseObs.onNext(UploadSummary.newBuilder()
                    .setCount(count).setTotal(sum).build());
            responseObs.onCompleted();   // one final response
        }
    };
}

Note that the server method returns a StreamObserver before receiving any message; it's that observer whose onNext fires with each message. The client's onCompleted tells the server "no more messages," and that's when the server sends its summary response.

4) Bidirectional streaming

The most powerful form: both sides send and receive independently.

@Override
public StreamObserver<ChatMessage> chat(
        StreamObserver<ChatMessage> responseObs) {
    return new StreamObserver<>() {
        @Override public void onNext(ChatMessage msg) {
            // echo immediately — no need to wait for the whole input
            responseObs.onNext(ChatMessage.newBuilder()
                    .setText("echo: " + msg.getText()).build());
        }
        @Override public void onError(Throwable t) { }
        @Override public void onCompleted() { responseObs.onCompleted(); }
    };
}

Deadlines and cancellation: controlling time

In Java you usually set it with withDeadlineAfter, which internally converts it into an absolute deadline:

BalanceResponse resp = blockingStub
        .withDeadlineAfter(500, TimeUnit.MILLISECONDS)
        .getBalance(request);

If the server doesn't respond by then, the call fails with status DEADLINE_EXCEEDED.

On the server side you can check whether anyone is still waiting for the answer — that is, whether the call was cancelled or its deadline passed:

if (Context.current().isCancelled()) {
    responseObserver.onError(Status.CANCELLED
            .withDescription("client gone").asRuntimeException());
    return;   // don't start the heavy work
}

Cancellation also happens when the client abandons a call before it finishes (e.g., the user closes the tab). gRPC propagates that cancellation to the server, and the server can stop the half-finished work. Context is the mechanism that carries the deadline and cancellation signal along the call chain.

Interceptors: cross-cutting interception

Interceptors solve cross-cutting concerns: logging, metrics, authentication, distributed tracing — things that repeat across all calls and shouldn't be tangled into business logic. There are two sides: ServerInterceptor and ClientInterceptor.

A simple server interceptor that logs each call's duration:

public class TimingInterceptor implements ServerInterceptor {
    @Override
    public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(
            ServerCall<ReqT, RespT> call,
            Metadata headers,
            ServerCallHandler<ReqT, RespT> next) {
        long start = System.nanoTime();
        String method = call.getMethodDescriptor().getFullMethodName();
        ServerCall<ReqT, RespT> wrapped =
            new ForwardingServerCall.SimpleForwardingServerCall<>(call) {
                @Override public void close(Status status, Metadata trailers) {
                    long ms = (System.nanoTime() - start) / 1_000_000;
                    System.out.printf("%s -> %s in %dms%n",
                            method, status.getCode(), ms);
                    super.close(status, trailers);
                }
            };
        return next.startCall(wrapped, headers);
    }
}

And installing it on the server:

ServerBuilder.forPort(9090)
        .addService(new AccountServiceImpl())
        .intercept(new TimingInterceptor())
        .build();

Metadata is the HTTP/2 headers — where an auth token (like Authorization: Bearer ...) or a trace id lives. A sample authentication: the interceptor reads the token from Metadata, validates it, and if invalid rejects the call with Status.UNAUTHENTICATED — all without adding a single line to the service code.

gRPC vs REST/JSON: which and when?

Now that we've seen both, let's compare honestly. gRPC isn't "better"; it's different, for different problems.

Schema evolution: change the contract without breaking

protobuf's biggest real-world strength is backward and forward compatibility. That means the server and client can run different versions of the .proto and still work correctly. This miracle comes from that same "field number" principle.

The golden rules of safe evolution:

• Adding a new field is safe — as long as it gets a fresh field number. Old parsers ignore it as an "unknown field" (and proto3 even keeps and re-emits it).
• Removing a field is safe — but you must reserve its number with reserved so nobody accidentally reuses it later.
• Never change a field number. Rename it, fine; but the number is sacred.
• Never give an incompatible type. Changing int32 to string corrupts data. Some changes are safe (like int32↔int64 within range) but most aren't.
• Don't make fields required. proto3 has no required at all, and that's correct: every field is optional with a default value, which is what makes evolution possible.

message BalanceResponse {
  reserved 4;                  // removed number is reserved
  reserved "old_field_name";   // removed name reserved too
  string account_id = 1;
  int64  cents      = 2;
  string currency   = 3;
  string bank_name  = 5;       // new field, fresh number — safe
}

When to use gRPC (and when not)

Reach for it when:

• you have internal microservice-to-microservice communication and performance matters.
• you need streaming (especially bidirectional): live feeds, telemetry, chat.
• you want a typed contract and generated code across multiple languages (polyglot).
• low latency and small messages are critical (mobile, IoT, high-traffic systems).

Don't reach for it when:

• you're building a public browser-facing API and don't want to tolerate a gRPC-Web proxy.
• human readability and easy debugging with curl matter more than speed.
• HTTP caching is a core need (like cacheable public content).
• your team is comfortable with REST and the scale doesn't justify gRPC's complexity.

Pitfalls and best practices

Let's gather the frequent traps in one place:

• Don't build a channel per call. Build one long-lived channel and reuse it; stubs are lightweight.
• Always set a deadline. A call with no deadline can hang forever and pin threads. There is no default deadline; you must set one explicitly.
• Retry only idempotent operations. After DEADLINE_EXCEEDED or UNAVAILABLE, don't blindly retry unless the operation is safe.
• Don't reuse field numbers; reserved the removed ones.
• Don't assume StreamObserver is thread-safe; synchronize writes from multiple threads.
• Don't send a giant message. gRPC has a default cap (~4 MB); for large files use chunked client-streaming.
• TLS in production. usePlaintext() is dev-only.
• Return errors with the right Status (like NOT_FOUND, INVALID_ARGUMENT, PERMISSION_DENIED) rather than a generic exception; the correct status code helps the client behave correctly.
• Set option java_multiple_files = true so each message gets its own class and your code is cleaner.

Interview Questions

Now it's time to drill. Answer each yourself first, then open the answer.