Escape from Protocol Buffer Hell: My 3-Week Battle Crying ‘It’s Not Encryption!’

“Wait, isn’t this encryption?” - The Initial Misconception

When a new microservices project started and the team leader said “Let’s use Protocol Buffer with gRPC,” I internally panicked thinking “Oh no, I don’t understand encryption technology…”

My first major misconception: “Protocol Buffer = Encryption technology”

When I actually saw Protocol Buffer binary data:

1

0A 05 41 6C 69 63 65 10 1A

“Wow, it’s completely encrypted! Can I decode this?”

This is the record of my 3-week investigation, failures, and discoveries.

Week 1: The “Why did JSON become 9 bytes!?” Incident

Failure Experience: The Initial Shock

Day 2 of the project. A senior said “Protocol Buffer is 3 times more efficient than JSON,” so I tried it out.

Original data (JSON):

1

{"name": "Alice", "age": 26}

Size: 26 bytes

Protocol Buffer binary:

1

0A 05 41 6C 69 63 65 10 1A

Size: 9 bytes

“Really? Actually one-third? But what language is this? Alien characters?”

Investigation Begins: Solving the Binary Mystery

I went into complete panic mode and started investigating until late at night.

First failure: Googled “Protocol Buffer encryption” → Nothing found, because it’s not encryption in the first place

Next failure: Opened with binary editor and stared at it → “What are 41 6C 69 63 65? ASCII? But what are 0A 05?”

Finally discovered: Looked up ASCII code table

• 41 = 65 = ‘A’
• 6C = 108 = ’l'
• 69 = 105 = ‘i’
• 63 = 99 = ‘c’
• 65 = 101 = ’e'

“Ah! It’s "Alice"! It’s not encryption, just encoding!”

Week 2: The Shocking Discovery of the “Field Number System”

Failure Experience: The Mystery of .proto Files

1
2
3
4
5

message User {
  int64 id = 1;      // What's this "= 1"?
  string name = 2;   // "= 2" too? Default value?
  int32 age = 3;     // "= 3" too?
}

“Huh? Is the default for id 1, and name 2? This makes no sense…”

Discovery of Truth: Shocking Facts

After 3 days of puzzling, I read the official documentation and was stunned.

Shocking facts:

• Field names are not transmitted!
• Only field numbers are transmitted!
• = 2 is not a default value but an identification number!

1
2
3
4
5

0A 05 41 6C 69 63 65 10 1A
↓
0A 05: Field 2 (name), length 5
41 6C 69 63 65: "Alice"
10 1A: Field 3 (age) = 26

“That’s why it’s small! It doesn’t send field names! Genius!”

Verification Experiment: Really 3x More Efficient?

JSON vs Protocol Buffer Showdown

“It’s true… about 3x more efficient for any data!”

Week 3: “gRPC vs tRPC, which one?” Selection Hell

Failure Experience: Lost in the RPC Technology Selection Maze

As the project reached its climax, the “So, do we use gRPC or tRPC?” problem erupted.

Initial shallow understanding:

• gRPC = Old technology?
• tRPC = New technology so it’s good?

After trying both…

gRPC Implementation Experience: “Why isn’t it type-safe?”

1
2
3
4
5
6
7
8

// gRPC client
const client = new UserServiceClient('localhost:50051');
const request = new GetUserRequest();
request.setId(123);

client.getUser(request, (err, response) => {
  // response.getName() ← This isn't type-checked!
});

“Huh? It’s TypeScript but not type-safe? What’s the point of gRPC?”

tRPC Implementation Experience: “This is too comfortable…”

1
2
3

// tRPC client
const user = await trpc.getUser.query({id: 123});
// ↑ Completely type-safe! IDE completion works too!

“This is amazing… the development experience is like night and day!”

Reality Check: “But what about inter-microservice communication?”

I noticed the tRPC trap after implementation.

tRPC limitations discovered:

• TypeScript only (no integration with other languages)
• Server-to-server communication uses JSON = Large data transfer
• Can’t benefit from Protocol Buffer advantages

“Ah, we’re using Go and Python for microservices too…”

Final Battle: The Truth of Technology Selection

Blood and Sweat Comparison Chart

My blood and sweat crystallized after 3 weeks of investigation:

Final Decision: The Importance of “Right Tool for Right Job”

Final project selection:

• Inter-microservice communication: gRPC + Protocol Buffer
  • Reason: Prioritized transfer efficiency and performance
• Frontend↔API: tRPC
  • Reason: Prioritized development experience and type safety

“In the end, there’s no silver bullet…”

What I Learned: Lessons from 3 Weeks of Struggle

Lesson 1: “Encryption” vs “Encoding”

Biggest misconception: I thought Protocol Buffer was encryption technology Truth: Just efficient binary encoding technology

Understanding the difference:

• Encryption: Security purpose, requires decryption key
• Encoding: Efficiency purpose, decodable if you know the rules

Lesson 2: The Trap of “New = Better”

Initial thinking: tRPC is new so it must be better Reality: Sometimes old gRPC is more suitable depending on use case

Truth of technology selection:

• Clarify requirements
• Understand trade-offs
• Select appropriate tools for specific purposes

Lesson 3: The Power of Binary Encoding

True value of Protocol Buffer:

• Not sending field names = Data size reduction
• Managing type information with schema = Type safety
• Multi-language support = Ecosystem breadth

Actual measurement data:

1
2
3
4
5
6
7

Regular REST API: 2.6KB per request
Same data with gRPC: 0.9KB per request

For 1 million requests per month:
- REST: 2.6GB
- gRPC: 0.9GB
- Reduction effect: 1.7GB (65% reduction)

Summary: The Obvious Conclusion That “Protocol Buffer Isn’t Encryption”

What I learned through 3 weeks of struggle:

1. Protocol Buffer is not encryption technology (should have been obvious…)
2. Binary encoding is truly efficient (proven by actual measurement)
3. RPC technologies are about right tool for right job (no silver bullet)
4. Getting your hands dirty is better than reading technical articles for understanding

Starting from the misconception “Protocol Buffer is encryption, right?”, the investigation resulted in understanding the full picture of modern RPC technologies.

For the next similar technical investigation:

• Read official documentation first
• Actually write code and try it out
• Question your assumptions
• Compare multiple technologies to understand

When asked “Is Protocol Buffer encryption?” I can now confidently answer “No! It’s efficient binary encoding!”

Everyone, when you encounter new technologies, don’t be afraid to get your hands dirty and investigate. You’ll surely discover more than you expect.