Skip to main content
Version: dev

Schema Evolution

Schema evolution allows your data structures to change over time while maintaining compatibility with previously serialized data. Fory Go supports this through compatible mode.

Enabling Compatible Mode​

Enable compatible mode when creating a Fory instance:

f := fory.New(fory.WithCompatible(true))

How It Works​

Without Compatible Mode (Default)​

  • Compact serialization without metadata
  • Struct hash is checked during deserialization
  • Any schema change causes ErrKindHashMismatch

With Compatible Mode​

  • Type metadata is written to serialized data
  • Supports adding, removing, and reordering fields
  • Enables forward and backward compatibility

Supported Schema Changes​

Adding Fields​

New fields can be added; they receive zero values when deserializing old data:

// Version 1
type UserV1 struct {
    ID   int64
    Name string
}

// Version 2 (added Email)
type UserV2 struct {
    ID    int64
    Name  string
    Email string  // New field
}

f := fory.New(fory.WithCompatible(true))
f.RegisterStruct(UserV1{}, 1)

// Serialize with V1
userV1 := &UserV1{ID: 1, Name: "Alice"}
data, _ := f.Serialize(userV1)

// Deserialize with V2
f2 := fory.New(fory.WithCompatible(true))
f2.RegisterStruct(UserV2{}, 1)

var userV2 UserV2
f2.Deserialize(data, &userV2)
// userV2.Email = "" (zero value)

Removing Fields​

Removed fields are skipped during deserialization:

// Version 1
type ConfigV1 struct {
    Host     string
    Port     int32
    Timeout  int64
    Debug    bool  // Will be removed
}

// Version 2 (removed Debug)
type ConfigV2 struct {
    Host    string
    Port    int32
    Timeout int64
    // Debug field removed
}

f := fory.New(fory.WithCompatible(true))
f.RegisterStruct(ConfigV1{}, 1)

// Serialize with V1
config := &ConfigV1{Host: "localhost", Port: 8080, Timeout: 30, Debug: true}
data, _ := f.Serialize(config)

// Deserialize with V2
f2 := fory.New(fory.WithCompatible(true))
f2.RegisterStruct(ConfigV2{}, 1)

var configV2 ConfigV2
f2.Deserialize(data, &configV2)
// Debug field data is skipped

Reordering Fields​

Field order can change between versions:

// Version 1
type PersonV1 struct {
    FirstName string
    LastName  string
    Age       int32
}

// Version 2 (reordered)
type PersonV2 struct {
    Age       int32   // Moved up
    LastName  string
    FirstName string  // Moved down
}

Compatible mode handles this automatically by matching fields by name.

Incompatible Changes​

Some changes are NOT supported, even in compatible mode:

Type Changes​

// NOT SUPPORTED
type V1 struct {
    Value int32  // int32
}

type V2 struct {
    Value string  // Changed to string - INCOMPATIBLE
}

Renaming Fields​

// NOT SUPPORTED (treated as remove + add)
type V1 struct {
    UserName string
}

type V2 struct {
    Username string  // Different name - NOT a rename
}

This is treated as removing UserName and adding Username, resulting in data loss.

Best Practices​

1. Use Compatible Mode for Persistent Data​

// For data stored in databases, files, or caches
f := fory.New(fory.WithCompatible(true))

2. Provide Default Values​

type ConfigV2 struct {
    Host    string
    Port    int32
    Timeout int64
    Retries int32  // New field
}

func NewConfigV2() *ConfigV2 {
    return &ConfigV2{
        Retries: 3,  // Default value
    }
}

// After deserialize, apply defaults
if config.Retries == 0 {
    config.Retries = 3
}

Cross-Language Schema Evolution​

Schema evolution works across languages:

Go (Producer)​

type MessageV1 struct {
    ID      int64
    Content string
}

f := fory.New(fory.WithCompatible(true))
f.RegisterStruct(MessageV1{}, 1)
data, _ := f.Serialize(&MessageV1{ID: 1, Content: "Hello"})

Java (Consumer with newer schema)​

public class Message {
    long id;
    String content;
    String author;  // New field in Java
}

Fory fory = Fory.builder()
    .withXlang(true)
    .withCompatibleMode(true)
    .build();
fory.register(Message.class, 1);
Message msg = fory.deserialize(data, Message.class);
// msg.author will be null

Performance Considerations​

Compatible mode mainly affects serialized size:

AspectSchema ConsistentCompatible Mode
Serialized SizeSmallerLarger (includes metadata, especially without field IDs)
SpeedFastSimilar (metadata is just memcpy)
Schema FlexibilityNoneFull

Note: Using field IDs (fory:"id=N") reduces metadata size in compatible mode.

Recommendation: Use compatible mode for:

  • Persistent storage
  • Cross-service communication
  • Long-lived caches

Use schema consistent mode for:

  • In-memory operations
  • Same-version communication
  • Minimum serialized size

Error Handling​

Hash Mismatch (Schema Consistent Mode)​

f := fory.New()  // Compatible mode disabled

// Schema changed without compatible mode
err := f.Deserialize(oldData, &newStruct)
// Error: ErrKindHashMismatch

Unknown Fields​

In compatible mode, unknown fields are skipped silently. To detect them:

// Currently, Fory skips unknown fields automatically
// No explicit API for detecting unknown fields

Complete Example​

package main

import (
    "fmt"
    "github.com/apache/fory/go/fory"
)

// V1: Initial schema
type ProductV1 struct {
    ID    int64
    Name  string
    Price float64
}

// V2: Added fields
type ProductV2 struct {
    ID          int64
    Name        string
    Price       float64
    Description string  // New
    InStock     bool    // New
}

func main() {
    // Serialize with V1
    f1 := fory.New(fory.WithCompatible(true))
    f1.RegisterStruct(ProductV1{}, 1)

    product := &ProductV1{ID: 1, Name: "Widget", Price: 9.99}
    data, _ := f1.Serialize(product)
    fmt.Printf("V1 serialized: %d bytes\n", len(data))

    // Deserialize with V2
    f2 := fory.New(fory.WithCompatible(true))
    f2.RegisterStruct(ProductV2{}, 1)

    var productV2 ProductV2
    if err := f2.Deserialize(data, &productV2); err != nil {
        panic(err)
    }

    fmt.Printf("ID: %d\n", productV2.ID)
    fmt.Printf("Name: %s\n", productV2.Name)
    fmt.Printf("Price: %.2f\n", productV2.Price)
    fmt.Printf("Description: %q (zero value)\n", productV2.Description)
    fmt.Printf("InStock: %v (zero value)\n", productV2.InStock)
}