Protobuf IDL Support

This page explains how Apache Fory works with Protocol Buffers (.proto) schemas, how protobuf concepts map to Fory, and how to use protobuf-only Fory extension options.

What This Page Covers

• Choosing protobuf vs Fory for your use case
• Syntax and semantic differences that matter during adoption
• Supported Fory extension options in protobuf files
• Practical transition patterns from protobuf to Fory

Quick Decision Guide

Situation	Recommended Format
You are building gRPC APIs and rely on protobuf tooling	Protocol Buffers
You need maximum object-graph performance and ref tracking	Fory
You need circular/shared references in serialized data	Fory
You need strong unknown-field behavior for wire compatibility	Protocol Buffers
You need native structs/classes instead of protobuf wrappers	Fory

Protobuf vs Fory at a Glance

Aspect	Protocol Buffers	Fory
Primary purpose	RPC/message contracts	High-performance object serialization
Encoding model	Tag-length-value	Fory binary protocol
Reference tracking	Not built-in	First-class (ref)
Circular refs	Not supported	Supported
Unknown fields	Preserved	Not preserved
Generated types	Protobuf-specific model types	Native language constructs
gRPC ecosystem	Native	Java/Python service codegen

Fory can generate Java and Python gRPC service companions with --grpc. Those services use normal gRPC transports but serialize request and response payloads with Fory rather than protobuf. For broad gRPC ecosystem tooling, schema reflection, and protobuf-native interceptors, protobuf remains the mature/default choice.

Why Use Apache Fory

• Idiomatic generated code: Fory IDL generates language-idiomatic classes and structs that can be used directly as domain objects.
• Faster serialization: In Fory benchmarks, Fory can be around 10x faster than protobuf for object serialization workloads.
• Better graph modeling: Shared and circular references are first-class features instead of application-level ID-link workarounds.

See benchmark details under Performance References.

Syntax and Semantic Mapping

Package and File Options

Protocol Buffers

syntax = "proto3";
package example.models;
option java_package = "com.example.models";
option go_package = "example.com/models";

Fory

package example.models;

Fory uses one package namespace for cross-language registration. Language-specific package placement is still configurable in code generation.

Message and Enum Definitions

Protocol Buffers

message User {
  string id = 1;
  string name = 2;
  optional string email = 3;
  int32 age = 4;
  repeated string tags = 5;
  map<string, string> metadata = 6;
}

enum Status {
  STATUS_UNSPECIFIED = 0;
  STATUS_ACTIVE = 1;
}

Fory

message User [id=101] {
    string id = 1;
    string name = 2;
    optional string email = 3;
    int32 age = 4;
    list<string> tags = 5;
    map<string, string> metadata = 6;
}

enum Status [id=102] {
    UNKNOWN = 0;
    ACTIVE = 1;
}

Key differences:

• Fory can assign stable type IDs directly ([id=...]).
• Fory uses list<T> (with repeated T as alias).
• Enum naming conventions are language-driven instead of protobuf prefix style.

oneof to union

Protobuf oneof is translated to a nested Fory union plus an optional field referencing that union.

Protocol Buffers

message Event {
  oneof payload {
    string text = 1;
    int32 number = 2;
  }
}

Fory-style shape after translation

message Event {
    union payload {
        string text = 1;
        int32 number = 2;
    }
    optional payload payload = 1;
}

Notes:

• Union case IDs are derived from the original oneof field numbers.
• The synthetic union field uses the smallest oneof case number.

Imports and Well-Known Types

Protobuf imports are supported. Common well-known types map directly:

• google.protobuf.Timestamp -> timestamp
• google.protobuf.Duration -> duration
• google.protobuf.Any -> any

Type Mapping Highlights

Protobuf Type	Fory Mapping
bool	bool
int32, uint32	variable-length 32-bit integer kinds
sint32	zigzag 32-bit integer
int64, uint64	variable-length 64-bit integer kinds
sint64	zigzag 64-bit integer
fixed32, fixed64	fixed-width unsigned integer kinds
sfixed32, sfixed64	fixed-width signed integer kinds
float, double	float32, float64
string, bytes	string, bytes
repeated T	list<T>
map<K, V>	map<K, V>
optional T	optional T
oneof	union + optional union reference field
int64 [(fory).type = "tagged int64"]	tagged int64 encoding
uint64 [(fory).type = "tagged uint64"]	tagged uint64 encoding

Fory Extension Options (Protobuf)

Fory-specific options in .proto use the (fory). prefix.

option (fory).enable_auto_type_id = true;

message TreeNode {
  TreeNode parent = 1 [(fory).weak_ref = true];
  repeated TreeNode children = 2 [(fory).ref = true];
}

File-Level Options

Option	Type	Description
(fory).use_record_for_java_message	bool	Generate Java records for all messages in this file
(fory).polymorphism	bool	Enable polymorphic serialization metadata by default
(fory).enable_auto_type_id	bool	Auto-generate type IDs when omitted (compiler default is true)
(fory).evolving	bool	Default schema-evolution behavior for messages
(fory).go_nested_type_style	string	Go nested naming style: underscore (default) or camelcase
(fory).swift_namespace_style	string	Swift namespace style: enum (default) or flatten; applies only when package is non-empty

Message and Enum Options

Option	Applies To	Type	Description
(fory).id	message, enum	int	Explicit type ID for registration
(fory).alias	message, enum	string	Alternate name used for auto-ID hashing
(fory).evolving	message	bool	Override file-level evolution setting
(fory).use_record_for_java	message	bool	Generate Java record for this message
(fory).deprecated	message, enum	bool	Mark type as deprecated
(fory).namespace	message	string	Override default package-based namespace

Field-Level Options

Option	Type	Description
(fory).ref	bool	Enable reference tracking for this field
(fory).nullable	bool	Treat field as nullable (optional)
(fory).weak_ref	bool	Generate weak pointer semantics (C++/Rust codegen)
(fory).thread_safe_pointer	bool	Use Rust Arc/ArcWeak for ref fields; default false uses Rc/RcWeak
(fory).deprecated	bool	Mark field as deprecated
(fory).type	string	Primitive override for tagged 64-bit integer encoding

Reference option behavior:

• weak_ref = true implies ref tracking.
• For repeated fields, (fory).ref = true applies to list elements.
• For map<K, V> fields, (fory).ref = true applies to map values.
• weak_ref and thread_safe_pointer are codegen hints for C++/Rust.
• thread_safe_pointer defaults to false; it changes only the generated Rust pointer carrier and does not change the wire format.
• In Rust codegen, (fory).weak_ref = true uses RcWeak by default and switches to ArcWeak only when (fory).thread_safe_pointer = true.

Option Examples by Shape

message Graph {
  Node root = 1 [(fory).ref = true, (fory).thread_safe_pointer = true];
  repeated Node nodes = 2 [(fory).ref = true];
  map<string, Node> cache = 3 [(fory).ref = true];
  Node parent = 4 [(fory).weak_ref = true];
}

Reference Tracking vs Protobuf IDs

Protobuf itself does not preserve shared/cyclic object graphs. With Fory protobuf extensions, you can opt into graph semantics.

Without Fory ref options (protobuf-style IDs):

message TreeNode {
  string id = 1;
  string parent_id = 2;
  repeated string child_ids = 3;
}

With Fory ref options (object graph):

message TreeNode {
  TreeNode parent = 1 [(fory).weak_ref = true];
  repeated TreeNode children = 2 [(fory).ref = true];
}

Porting Protobuf Schemas To Fory

Step 1: Translate Schema Syntax

• Keep package names stable.
• Replace repeated T with list<T> (or keep repeated alias).
• Add explicit [id=...] where you need stable numeric registration.

Step 2: Convert oneof and Special Types

• oneof -> union + optional union field.
• Map protobuf well-known types to Fory primitives (timestamp, duration, any).

Step 3: Replace Protobuf Workarounds with ref

Where protobuf used manual ID links for object graphs, switch to Fory ref modifiers (and optional ref(weak=true) where needed).

Step 4: Update Build/Codegen

Replace protobuf generation steps with the Fory compiler invocation for target languages.

For Java and Python services, add --grpc to emit gRPC companion code:

foryc api.proto --java_out=./generated/java --python_out=./generated/python --grpc

Generated Java service files compile against grpc-java, and generated Python service modules import grpc. Add those dependencies in your application build; Fory runtime packages do not add gRPC as a hard dependency. Protobuf oneof fields are translated to Fory union fields inside request and response messages. Direct union RPC request or response types are not part of normal protobuf RPC syntax.

Step 5: Run Compatibility Checks

For staged transitions, keep both formats in parallel and verify payload-level parity with integration tests.

Coexistence Strategy

You can run protobuf and Fory in parallel during a staged transition:

public byte[] serialize(Object obj, Format format) {
    if (format == Format.PROTOBUF) {
        return ((MessageLite) obj).toByteArray();
    }
    return fory.serialize(obj);
}

Use translators at service boundaries while internal object-graph heavy paths migrate first.

Performance References

• Benchmarks: https://fory.apache.org/docs/introduction/benchmark
• Benchmark code: https://github.com/apache/fory/tree/main/benchmarks

Summary

Use protobuf when your primary concern is API contracts and gRPC ecosystem integration. Use Fory when object-graph performance, native models, and reference semantics are the primary concern.