Skip to main content
Version: dev

Cross-Language Serialization

Apache Fory™ supports seamless data exchange between Java and other languages (Python, Rust, Go, JavaScript, etc.) through the xlang serialization format. This enables multi-language microservices, polyglot data pipelines, and cross-platform data sharing.

Enable Cross-Language Mode​

To serialize data for consumption by other languages, enable xlang mode:

import org.apache.fory.*;
import org.apache.fory.config.*;

// Create Fory instance with XLANG mode
Fory fory = Fory.builder()
    .withXlang(true).withCompatible(true)
    .withRefTracking(true)  // Enable reference tracking for complex graphs
    .build();

Register Types for Cross-Language Compatibility​

Types must be registered with consistent IDs or names across all languages. Fory supports two registration methods:

public record Person(String name, int age) {}

// Register with numeric ID - faster and more compact
fory.register(Person.class, 1);

Person person = new Person("Alice", 30);
byte[] bytes = fory.serialize(person);
// bytes can be deserialized by Python, Rust, Go, etc.

Benefits: Faster serialization, smaller binary size Trade-offs: Requires coordination to avoid ID conflicts across teams/services

public record Person(String name, int age) {}

// Register with string name - more flexible
fory.register(Person.class, "example.Person");

Person person = new Person("Alice", 30);
byte[] bytes = fory.serialize(person);
// bytes can be deserialized by Python, Rust, Go, etc.

Benefits: Less prone to conflicts, easier management across teams, no coordination needed Trade-offs: Slightly larger binary size due to string encoding

Cross-Language Example: Java ↔ Python​

Java (Serializer)​

import org.apache.fory.*;
import org.apache.fory.config.*;

public record Person(String name, int age) {}

public class Example {
    public static void main(String[] args) {
        Fory fory = Fory.builder()
            .withXlang(true).withCompatible(true)
            .withRefTracking(true)
            .build();

        // Register with consistent name
        fory.register(Person.class, "example.Person");

        Person person = new Person("Bob", 25);
        byte[] bytes = fory.serialize(person);

        // Send bytes to Python service via network/file/queue
    }
}

Python (Deserializer)​

import pyfory
from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: pyfory.Int32

# Create Fory in xlang mode
fory = pyfory.Fory(xlang=True, compatible=True, ref=True)

# Register with the SAME name as Java
fory.register_type(Person, typename="example.Person")

# Deserialize bytes from Java
person = fory.deserialize(bytes_from_java)
print(f"{person.name}, {person.age}")  # Output: Bob, 25

Handling Circular and Shared References​

Cross-language mode supports circular and shared references when reference tracking is enabled:

public class Node {
    public String value;
    public Node next;
    public Node parent;
}

Fory fory = Fory.builder()
    .withXlang(true).withCompatible(true)
    .withRefTracking(true)  // Required for circular references
    .build();

fory.register(Node.class, "example.Node");

// Create circular reference
Node node1 = new Node();
node1.value = "A";
Node node2 = new Node();
node2.value = "B";
node1.next = node2;
node2.parent = node1;  // Circular reference

byte[] bytes = fory.serialize(node1);
// Python/Rust/Go can correctly deserialize this with circular references preserved

Type Mapping Considerations​

Not all Java types have equivalents in other languages. When using xlang mode:

  • Use primitive types (int, long, double, String) for maximum compatibility
  • Use standard collections (List, Map, Set) instead of language-specific ones
  • Use reduced-precision carriers (Float16, BFloat16, Float16List, BFloat16List) for 16-bit float payloads
  • Treat Float16[], BFloat16[], Float16List, and BFloat16List as list<T> carriers by default; use @ArrayType when the schema must be array<float16> or array<bfloat16>
  • Avoid Java-specific types like Optional, BigDecimal (unless the target language supports them)
  • See Type Mapping Guide for complete compatibility matrix

Lists and Dense Arrays​

Java primitive arrays are dense array<T> carriers, except plain byte[], which defaults to bytes. General Java collections and Fory primitive-list carriers such as Int32List, Float16List, and BFloat16List use list<T> unless the field has explicit @ArrayType metadata.

Fory schemaJava field shape
list<int32>List<Integer> or Int32List
array<bool>boolean[]
array<int8>@Int8Type byte[] type-use
array<int16>short[]
array<int32>int[]
array<int64>long[]
array<uint8>@UInt8Type byte[] type-use
array<uint16>@UInt16Type short[] type-use
array<uint32>@UInt32Type int[] type-use
array<uint64>@UInt64Type long[] type-use
array<float16>Float16Array or @Float16Type short[]
array<bfloat16>BFloat16Array or @BFloat16Type short[]
array<float32>float[]
array<float64>double[]

Prefer type-use syntax for primitive-array annotations:

private @UInt32Type int[] ids;
private @BFloat16Type short[] values;

Compatible Types​

public record UserData(
    String name,           // ✅ Compatible
    int age,               // ✅ Compatible
    List<String> tags,     // ✅ Compatible
    Map<String, Integer> scores  // ✅ Compatible
) {}

Problematic Types​

public record UserData(
    Optional<String> name,    // ❌ Not cross-language compatible
    BigDecimal balance,       // ❌ Limited support
    EnumSet<Status> statuses  // ❌ Java-specific collection
) {}

Performance Considerations​

Cross-language mode has additional overhead compared to Java-only mode:

  • Type metadata encoding: Adds extra bytes per type
  • Type resolution: Requires name/ID lookup during deserialization

For best performance:

  • Use ID-based registration when possible (smaller encoding)
  • Disable reference tracking if you don't need circular references (withRefTracking(false))
  • Use Java mode (withXlang(false)) when only Java serialization is needed

Cross-Language Best Practices​

  1. Consistent Registration: Ensure all services register types with identical IDs/names
  2. Version Compatibility: Use compatible mode for schema evolution across services

Troubleshooting Cross-Language Serialization​

"Type not registered" errors​

  • Verify type is registered with same ID/name on both sides
  • Check if type name has typos or case differences

"Type mismatch" errors​

Data corruption or unexpected values​

  • Verify both sides enable xlang mode
  • Ensure both sides have compatible Fory versions

See Also​